What is this episode about?
1. Minute 06:46 – Ayush shares with us his amazing experience in the gaming industry, working as an analyst in places and games such as Call of Duty, Ubisoft, RedHill Games, and Sciplay.
2. Minute 10:41 – Ayush discusses the types of metrics you should look into in every phase of the game.
3. Minute 13:11 – We asked Ayush what is the insight that brings his team the biggest impact and he said that for free-to-play games, the most important insight that you would like to know is what actually converts a player into a payer.
4. Minute 15:20 – Ayush tells us how as soon as you start asking the right questions, you get an insight. As an analyst, you come up with a lot of hypotheses that constantly need to be approved or disproved and that’s how you come up with insights.
Audio
Video
Transcript
00:05 Natalie Markovits | Affogata’s Host
Hello, and welcome to Affogata’s podcast, Let’s Talk Customer Feedback. Today we have the pleasure of hosting, Ayush Gupta, Analyst Team Lead at Sciplay Games. First, let’s say hi to Ayush. How are you doing today?
01:13 Ayush Gupta | Analyst Team Lead at Sciplay Games
Hi, Natalie. Thank you for inviting me here and I’m doing really good. How are you?
01:20 Natalie Markovits | Affogata’s Host
I’m very good, thank you and we’re very excited to have you here. So, before we dive into more player insights, we would love to hear more about you. So, tell us who are you? What three words, would you describe yourself?
01:34 Ayush Gupta | Analyst Team Lead at Sciplay Games
So, I’m Ayush. I’m from India, and have been an analyst for over 10 years now and out of which six years have been in gaming and I started my career in a consultancy, working for different clients and different industries and one of those clients happened to be Call of Duty and that’s how I was compelled to, you know, make a career in gaming and since then, I’ve worked in Ubisoft and a few more companies working for different free to play games and games on PC and mobile and console as well. If there are three words that would define me, I think the first one is passionate. The second one would be spiritual and the third would be that I’m a bit crazy in the sense that if I’m all in on something, either I don’t do it at all. So, if I’m doing something, I like to do it with 100% of my ability and my motivation on it.
02:32 Natalie Markovits | Affogata’s Host
That’s awesome to hear and yeah, I was gonna say something before. You have an amazing experience in the gaming industry and that’s why we’re so excited to have you here. So, thanks for sharing that, and since you’ve been in this industry for so long, what is your favorite video game of all time?
02:51 Ayush Gupta | Analyst Team Lead at Sciplay Games
I mean, it’s a tough question. I guess anyone who answers this question cannot have one answer to it. So, in my mind, there are like three games I cannot differentiate between because they have a huge impact on me and even on my career. The first game is FIFA because that’s the only game I played like competitively. I used to go into competitions when I’m in college and won a few in my college days as well. The second one is GTA Vice City, because that’s the first foray into open-world games, where you could interact with the entire environment and they became like a landmark in terms of open-world games and the last game that I really, really like is Sims because it’s the future of simulation of real-life events and I was really excited to play it when I played it for the first time and I was really inspired by it.
03:54 Natalie Markovits | Affogata’s Host
I agree. I was obsessed with Sims and building your house and your friends. I was obsessed with it. So, I completely relate to your answer and the last question that I would love to ask you to get to know you a bit more is, if you have a time machine, would you travel to the past or the future and to when?
04:15 Ayush Gupta | Analyst Team Lead at Sciplay Games
Okay, this is a question that I’ve always thought about a lot in my life is not just for this podcast. So, in terms of going into the future, it seems really exciting knowing what will happen in the future in the next 40 or 50 years. But I feel that if I go to the future, I’ll be like a caveman, not able to understand the technology of that type. So, it’s better to you know, go to an ancient civilization and learn about how they lived, like maybe 2000, 2500 years back, and understand more about the history because history is always, you know, disorienting. It’s not never linear. So, I would really love to understand the world of that time.
04:57 Natalie Markovits | Affogata’s Host
I love that answer and I think that would be so interesting. You will definitely look like a crazy person and now before we dive deep into today’s subject, let’s listen to a short story from the tales from our feedback crypt.
05:16 STORY
When the long-awaited Duke Nukem Forever finally was released in June 2011, it was met with an avalanche of negative reviews and widespread disappointment. Certainly not a good sign for the game or the Duke Nukem serious. The situation was made one extra bit worse by the interference of marketing representative Jim Ratner. Even in famous Twitter outburst, Ratner stated his intention to limit review copies of future games he represented by blacklisting any vocal critics of Duke Nukem Forever. Even after 2k games formally cut ties with Rartner’s firm, the troubling implications of his statement still lingered. Thank you for joining me and see you next week on tales from the feedback crypt.
06:23 Natalie Markovits | Affogata’s Host
And with that story in mind, we can’t wait to hear what Ayush has to say about all of this today, so let’s start. So, again, as we discussed before Ayush, we saw that you have a lot of experience in the games industry working at places like Ubisoft, RedHill games, and now Sciplay. Can you tell us about your experience at these companies and can you introduce us a bit about your role now?
06:46 Ayush Gupta | Analyst Team Lead at Sciplay Games
Yeah, definitely. So, like, as mentioned, across my experiences with working on Call of Duty, that’s the first game that I ever worked on, and at that time, I was just learning about the gaming industry and how things actually work for any game. Moving on to Ubisoft, I was part of a lot of mobile games, live operations of a few mobile games and at the same time, I was able to create a small analytics team for the QA team as well. They have a huge testing studio in Pune where they test these games, test out the issues that these games may have and I was also part of building the analyst QA team, so that they can help these testers understand what parts of the game they have tested, what maps they have tested, and so on and this is something which is very unique that I did for Ubisoft and moving on to RedHill games, I was part of building a new game from scratch, which is, again, a brand new experience for me, because I’ve never done that before. My role incorporated building, data pipelines, and data architecture from scratch. Also making sure what are the data points that we need to track for different players and it’s the kind of the same insight play, we are, you know, building a new game, and I’m part of the data pipeline, data architecture processes and then soon when we you know, go into different kinds of launches, we’ll do a lot of live operations and A/B test, which will use this data to, you know, help the designers, the producer and the product team to understand the different KPIs of the game, and how we can improve the game, give them brand new insights about it.
08:40 Natalie Markovits | Affogata’s Host
That sounds extremely interesting, and you mentioned that you work with a lot of data and how does player feedback influence your day-to-day job?
08:50 Ayush Gupta | Analyst Team Lead at Sciplay Games
I mean, it influences the entire core of my job. According to me, even as an analyst the player data that we get is the gameplay data that is the player feedback, in my opinion, and we use it on a daily basis. For our reports, for our insights, we have to use it all the time and as for player feedback in general, I feel it is important for a game when you have a certain player set maintain. So, if your retention and DU is at a certain level, that’s the time where player feedback is the most important. You have a certain number of players in the game. Now, you can understand what feedback they can give you. You can take the game from let’s say it was at level 3. You can take it to level 10 or level and beyond. But to take a game, a new brand-new game from level zero to level three, that’s where the player feedback would have the minimum impact. You can do a lot of player research, we know what players like, what kind of games players formed, and even from a design perspective, you understand what kind of game you are building and for what audience. But still, it’s a creative process on the design team, from the art team, from the UI teams and they build a game, and they need to put it out there and it needs to go to a certain level before the player feedback can come into the picture and take it to the next level.
10:30 Natalie Markovits | Affogata’s Host
Right, and what is the type of data you search for when doing player research? What are the metrics you look to optimize the games’ performance, for example?
10:41 Ayush Gupta | Analyst Team Lead at Sciplay Games
For player research, I feel it totally depends on the kind of problem I’m trying to solve, when like, what’s the question? And what’s the problem that we are facing right now? If the problem may start, okay, we have a game in mind, and we need to find the right audience for it, then we try to find, okay, if we are making, let’s say, an RPG game, what kind of audience we can target and in which countries we can target. This is the kind of player research we would like to do and let’s say if we are looking for a different mechanic in itself that we are trying to build. In that scenario, there is a different kind of player research we will do, we will look at what’s working in the market, what has worked previously and what has made the most money in the last few years, what was the highest numbered, highest types of games that have come up in the last few years. So, those are the kinds of things we’ll look at and that’s where we need to understand the question and then start doing player research.
In terms of the different metrics that we look at, again, it depends on the type of the game, and the phase the game is in right now. So, let’s say a new game that is launched into the market and it’s in a stick launch phase. At that time, we want to make sure that the players have a good experience technically. There are no FPS drops that happen, players are able to play a load the entire game without issues, the loading time of the game is as minimum as possible. So, all these things come into the picture at that time. Now, let’s say we have already done that, and we are in the soft launch phase of the game. Here, you will look at the REM metrics, basically, the retention engagement and monetization metrics, right? What is happening, how many players are we retaining, and D-1 compared to D-7 and D-30 things like that. So, in like a game for Call of Duty, which is launched every year, you don’t need to actually, you know, focus on the FPS drops and all those metrics, you directly go into engagement. If you increase engagement, revenue and attention will definitely follow.
12:58 Natalie Markovits | Affogata’s Host
Yeah, it makes total sense, and out of everything that you are mentioning now in all of this data that you utilize, in your opinion, what are the top three player insights that bring your team the biggest impact?
13:11 Ayush Gupta | Analyst Team Lead at Sciplay Games
So, if we are talking about, you know, free-to-play games, I think what most important kind of insight that you would like to know is what actually converts a player into a payer. As soon as a player is converted, I guess they are more engaged in the game, more likely to be in the game for a longer period of time, and which in turn increases the LTV of the entire game. So, that’s a very good insight to have if we really want that. Another insight that I can think of is how we increase the engagement and stickiness of the game. How do we make sure that players will want to play our game more so that we understand that at least for an hour, if you’re playing the game every day, what makes you stick to that? What are the different aspects of the game that makes you stick to the game? So, these are the kinds of insights that we would like to know, and the third insight that is, you know, I mean, some of the most important things that we like to know is, during the first-time user experience, what makes the player leave the game. We can understand the trigger points. Okay, this is the player where players are dropping, but according to me, even before the trigger point comes in, the player has a certain perception about the game and the trigger point may lead him to quit from the game. But there are other things as well, which have made the player quit and these points you know, it’s very hard to understand from the data.
14:50 Natalie Markovits | Affogata’s Host
Yeah, I agree with that. It’s actually it’s going to come later in the questions as well. But because you’re in the analyst team, you usually get all of this quantitative data, and then you’re missing kind of like the qualitative data to complete the big picture, so I’m gonna get there in a bit. But my next question is, what determines how quickly you get an insight? What usually is the most time-consuming element of the work of getting an actual insight?
15:20 Ayush Gupta | Analyst Team Lead at Sciplay Games
So, as soon as you start asking the right questions, you get to the insight and as an analyst, you come up with a lot of hypotheses. If this is happening, maybe because of X, or Y, or Z reasons, and you go through all of these three, and you might end up disapproving all of them and then you need to ask another question and this is the kind of process that you go through. Let’s say, one, you have a hypothesis, you are able to prove it. Now, there’ll be a few more hypotheses that will come in response to that hypothesis. So, this is the kind of, you know, funnel you need to follow and as soon as you, you know, start asking the right question you’re able to prove it or disprove it. That’s how you get your insight. This whole entire process can be cumbersome sometimes and it can also lead you to nowhere, you know, initially, but eventually, it will make sure that you find the right path.
16:24 Natalie Markovits | Affogata’s Host
Yeah, I agree. Being an analyst, I sometimes it’s hard to understand, but really, the asking the question and knowing how to formulate the question is kind of like the biggest part of it and it can be the hardest one as well. So, yeah, I can completely relate to that and now that we’ve talked about the quantitative data a lot, we would like to know, because it’s always super interesting to ask, if you being in the analyst team, if you receive qualitative feedback reports from player-facing teams in the company to complement your quantitative data. If this happens, how does it work?
17:06 Ayush Gupta | Analyst Team Lead at Sciplay Games
So, again, it really depends on the type of company, the type of game you’re working for at the moment and if it’s a new game, you don’t really have any player feedback coming in. But for big companies, big games, where you need a community manager handling the requests of different players, then they do you know, kind of provide us the player sentiment for the game for a particular feature as well and they give us, you know, PowerPoint reports, okay, what’s happening in the community right now? What are they feeling? What are they saying? So, these reports do come in. But in terms of numbers, I wouldn’t say that they are the quantity of these values can be determined, from the reports that are provided by the player-facing teams. [Cross talk]. Yeah, so I was saying that, for me, qualitative and quantitative data always goes hand in hand. Like if we have to understand where there’s a drop off of players, that will only happen if there’s a certain quantity of players actually did. So, to identify a point at the drop-off point, the quantity has to be quite huge compared to the other drop-off points.
18:33 Natalie Markovits | Affogata’s Host
Definitely and I know that you just talked about it a bit now. But can you tell us a bit about how you intertwine player insights, qualitative and quantitative data? Maybe you have an example that you had a question of like what happened? But then you had the what in terms of the data, but then you didn’t have the why? How do you intertwine these types of data points?
18:59 Ayush Gupta | Analyst Team Lead at Sciplay Games
So, like I mentioned, they do go hand in hand and there was a time we tried to do player surveys. After a certain number of matches that a player plays, we try to do player surveys and try to find answers, like, what did you didn’t like, what do you like about the game and so on. So, in terms of this, they are able to tell us the right answer sometimes, but the quantity of players actually answering those surveys is so low, and also the sample size may not be the right sample, because very few people actually try to answer the surveys and only those players, answer these surveys who are really, really engaged into the game and you never know what the non-engaged players actually want to say. So, it’s in those terms, it becomes hard, but in terms of from a numbers perspective, that you look at whatever we can get from the player data, they always go hand in hand.
19:54 Natalie Markovits | Affogata’s Host
These have been some amazing insights. So, thank you for that and for the last question. I know it can be super hard. But we would love to know if there would be a dream insight you could get from your players, what would it be?
20:11 Ayush Gupta | Analyst Team Lead at Sciplay Games
I guess I’ve already talked about it and that’s what makes a player leave the game in the FTP. So, this is like one of the hardest questions, a lot of games phase. Even games that are in their live operations phase and they see that okay, the D-1 retention is dropping off. So, why is it dropping off? So, that question is the hardest and like I said, identifying the trigger points. You can still do that, you can identify the trigger points at what point the players are dropping off. But what led them there is something that is very hard from the data because you don’t have any other data for these players except for okay, they dropped at this point and they played the previous four, five steps. But yeah, that’s my biggest dream insight if I ever can get that.
21:01 Natalie Markovits | Affogata’s Host
Definitely and that’s an awesome way to end the podcast. So, Ayush, I want to thank you so much for taking the time to participate in today’s podcast. It was super interesting to learn about your role as an analyst team lead, and how player feedback influences your position.
21:18 Ayush Gupta | Analyst Team Lead at Sciplay Games
Thank you so much, Natalie.