Joining us in conversation today is Xiaoyang Yang, Head of Data AI Security and IT over at Second Dinner Studios. Tuning in, you’ll hear about his recently launched video game, MARVEL SNAP, and how he uses data as a tool to listen to what players want. Find out how Xiaoyang moved from a background in theoretical physics to working in video games and AI and hear how different players can experience the same feature in an entirely different way.
Xiaoyang Yang, Head of Data AI Security and IT over at Second Dinner Studios, explains how Second Dinner navigates the issue of excess data with intention and discover the metrics that go deeper than the surface to measure the quality of competition, balance, and fairness within gaming. Xiaoyang also describes the difference between AI and gaming AI and shows us how each can be used to enhance the other. Listen to today’s episode for a careful look at how AI can be used to improve player experience and how gaming can act as a testing ground to improve AI in everyday life.
Key Points From This Episode:
Tweetables:
“We try to really listen to what our players are saying. One way to do that is through data. We use data as a tool.” — Xiaoyang Yang [0:02:28]
“When you see the scale, you begin to really understand that different players have different desires. Sometimes, different players see the same feature or the same experience in a very different type of way.” — Xiaoyang Yang [0:04:46]
“We see a lot of opportunities to use technology data AI to make MARVEL SNAP approachable to a wide audience of players and, hopefully, some players who have never tried the genre of collectible card games.” — Xiaoyang Yang [0:11:25]
“We want to make sure that there are different sets of cards you can use to have fun and still be competitive in the game. That's not an easy task.” — Xiaoyang Yang [0:19:25]
Links Mentioned in Today’s Episode:
EPISODE 54
[00:00:03] RS: Welcome to How AI Happens, a podcast where experts explain their work at the cutting-edge of artificial intelligence. You'll hear from AI researchers, data scientists, and machine learning engineers as they get technical about the most exciting developments in their field and the challenges they're facing along the way. I'm your host, Rob Stevenson. We're about to learn How AI Happens.
[00:00:31] RS: Joining me today on How AI Happens is the Head of Data AI Security and IT over at Second Dinner Studios, Xiaoyang. Xiaoyang, welcome to the podcast. How are you today?
[00:00:40] XY: I'm doing great. Thank you, Rob, for having me here.
[00:00:43] RS: So pleased to have you, lots to get into in terms of the AI and ML being injected into video game development, which is what Second Dinner primarily concerns itself with. First of all, congratulations are in order. Congrats on launching Marvel snap your most recent game that is in the Marvel Universe. Congrats on getting that out the door.
[00:01:02] XY: Thank you. Thank you. We recently just announced it. Right now we are getting player’s feedback with our soft launch in a few countries. Really great feedback and awesome things, our players are consuming our game. So I’m really glad. Thank you, Rob.
[00:01:17] RS: The video game launch is an interesting one, because it's related to this constant problem I come up against when I'm speaking with folks, which is how do when the product is good enough to move from testing to production? In some cases, I feel you can't ever traverse that final gap until it's out there in the world. Now I'm sure you're getting all this user feedback, things you didn't think of. Is that the case?
[00:01:40] XY: Yeah. That's a great question. We think this quite a lot and different games and different game studios have very different philosophy. You probably have heard one end of the spectrum, people talk about, hey, it's very much data driven approach. So that we define certain types of KPIs and if we hit them, okay, ready to go to the next milestone. The other end of the spectrum, there often is more creative and design driven. My gut feeling tells me as a game designer, I think this is good enough. I feel my players are happy enough, time to go to the next gate, to go through the next milestone.
That's a pretty wide spectrum and in between there are a lots of different ways to do it and think about it. I think our philosophy is instead of thinking about, hey, is data driven, or gut feeling driven, which I do really listen to with our players are saying. One way to do it is through data, not surprisingly. We use data as a tool instead of saying, “Hey, data is driving our decisions.” We use data as a tool to really listen to, what our players are saying? How they're using our product? What's our pain point? What's their joys and everything?
Then we translate their voices using data into insights so that we can make informed decisions to see if we're ready to go to the next step and what are the iterations and improvement we need to do over there. Sometimes it means big changes, right? We're not shy away from making big changes, because we want to launch a game. It's a live service game. The players will be enjoying for many, many years, and a lot of players will enjoy it. We want to make sure that we constantly and keep iterating on the game. Even if it means there's a huge change that needs to be made to the game.
[00:03:19] RS: Yeah, yeah, of course. Is some of the insight you glean, because of the scale, you suddenly realize when you go from however many testers that have been working on it to all of these players, 1000s, hundreds of thousands, hopefully millions, now all in the game at the same time. Is the scale part of the equation there, too?
[00:03:39] XY: Absolutely, absolutely. That's a really great point. Traditional game development usually starts with prototyping and very small limited scale of play tests. There is a limitation to that there is a huge value from that, because you can get really fast iteration, you get a lot of feedback from those players. The limitation over there is the very selected type of players. Very often, there are folks who are extremely excited about this genre, and who has tons of experience in this genre. When you go out to a mass audience, and you usually realize, hey, actually, there are a lot of different types of players. There are players who is super hardcore into the genre, there are players who are very interested, and are players who are just super new, as they've never tried this before, as a could be a potential opportunity for this product. So we really go out and you realize the player sentiment, their motivation and their fantasy for the game. It's a big spectrum over there.
When you see the scale, you actually start to understand the different players have different desire. Sometimes different players see the same feature or the same type of experience in a very different way. Some people could be very passionate about some really hardcore competitive features. Some casual players are like that's too much, it's much more me, I think that's something I would encourage game developer to focus on for now, because I want something that's for me.
That's a really great point, Rob. I think when we hit the scale, and now we're seeing it, we see more players coming in, not just hardcore players, but more other players, more casual, mid core and opportunities base players coming in. We're really listening to pay what all of these players are talking to and data plays a great role there, because then it’s no longer scalable to talk to everyone, and sit down and watch how everyone is playing.
[00:05:32] RS: Yeah, yeah, of course. We jumped in at the deep end here with video game shipping and receiving user feedback. I want to get into more of that as we go along, but first, I'd love it to just introduce you to the folks out there in podcast land, because you've had this career in video games, working at Blizzard and then Riot Games. I'm just curious if you could walk us through your career a little bit, maybe share with us how you came to Second Dinner Studios. Then at what point in your career, AI and ML started to play a factor?
[00:06:02] XY: Yeah. So I came from a background that is a little bit interesting to start with. My undergraduate major was in theoretical physics, which probably has nothing to do with video games, and has limited connection with machine learning and AI. It’s a very theoretical mathematical subfield of physics. When I was doing my undergrad, because I went to a very small college and who is a close connection with some of the national labs doing physics research, I started doing some research in the particle of physics space, which of the folks who are familiar with this actually involves a huge amount of data, it’s pretty much like, at least back in the day. You can actually observe what's actually happening there in the particles. You need to interpret what's happening by getting a lot of data and do modeling and understand what's happening deep down there in the very microscope.
I did a bunch of work over there. It was a very fun opportunity, I went to a lecture, the lecture was given by a professor who is doing vision research, and the professor was from a physics background. He started talking about how the mindset and numerical computing and the modeling that is derived from a lot of more fundamental physics space are widely used in a space of computer vision and vision research in general. That was super inspiring to me. After the lecture, I went to talk with a professor and I was like, “Hey, I have no idea what I can do in the vision space. I’m a theoretical physics major and some of the topics you mentioned, and you just talk about the super interesting, I want to learn more.”
He was like, “Oh, come to my lab, do some research here. I think physics major is great. Just come over here and I will try. So when there [inaudible 00:07:51] some research, we started doing some the early days, machine learning, AI stuff, it's machine learning modeling to understand the vision perception. It's very fun there. We happen to know a professor from graduate school from UCLA actually, was doing research in the very similar space and collaborating with that professor in my undergrad school. I ended up talking to that Professor as well, we started working together. Also he’s my professor in the college, awesome vision with the machine learning projects.
Then I end up going to the grad school doing machine learning and vision research. So that opens the door for me thinking about how to use my background in mathematics, theoretical physics and modeling to help understanding how the human vision works. What are the assumptions, what are the optimizations that our human visual system is doing? By the way, our system, our visual system is fascinating. It’s probably one of the most powerful systems that you can imagine, in any of the things we have built and experience. It's just fascinating there.
At that point, my focus was pretty much on machine learning theoretical part and modeling part of the vision. I have no idea that you can do anything with this background in gaming. It was very fun. I realized one day, hey, Blizzard is actually in California, the headquarter is there. I have been playing the game for many, many, many years. I applied for some random positions over there. I just search keywords and saying, “Hey, what are these things? What are those things?” I applied for those, first year I even got a rejection letter. Second year, I got a rejection letter, which I'm like, “Oh, that's progress, it's awesome. Probably someone's reading my resume.”
Year three, the head of back in the days, they call it business intelligence. The head of essentially the data department over there gave me a call and said, “Hey, your background seems interesting. There are some fun things we're trying to do over here and probably you can help us.” So I got an internship at Blizzard, so that opens the door to go into game industry. When I walk into Blizzard I have no idea actually what I can do there. I'm like, “It’s Blizzard, I’m going to do it.” I went there. I started working on some projects related to Battle.net, related to security. Eventually they gave me a full time offer and eventually also moved into the game side of game studio at Blizzard, primarily working on Warcraft.
I was really fascinating. It was my first time actually seeing how massive amount of data is generated in this virtual work. It's literally a work, right? It's World of Warcraft and there are so much information about player behavior. There are so much information about clear motivation, what they like, what they don't like, how they interact with each other. It's a whole new universe to me. Got me so excited, I started doing more fun stuff in the machine learning and data space with World of Warcraft. Eventually, I got opportunity to join Riot Games. Riot Games is growing League of legends. I started to think about how to make article games.
I was very fortunate to be part of that journey and going there and start building the data organization as Riot Games is growing. So about two years ago, guys this amazing opportunity working with some of the old friends and colleagues in this new game studio called Second Dinner. We're very grateful we have this partnership with Marvel. We're making this, MARVEL SNAP is our next game. Yeah. We see a lot of opportunities to use technology data AI to make the game really approachable to a wide audience of players, and hopefully some players who probably never tried the genre of collectible card games. Yeah, that's where we are. We announced the game, launching the game. Now we're iterating on that. It's how my journey coming from a theoretical physics major working on particle physics stuff, all the way to in gaming, making some fun data and AI works here.
[00:11:50] RS: Yeah. Thank you for setting that context. You mentioned something really interesting, related to World of Warcraft, which was just the sheer scope of data there was. Of course, if you're going to collect data in the physical world, you need sensors everywhere, but in a digital video game world sensors are everywhere, they just are in the ether of the world no matter what. So you capture every single action, every single equipment change, every single step, every motion, every jump, every bullet fired by a gun or whatever item in the game. It's a huge amount. It's almost like, is it too much? How do you even separate the signal from the noise when you've captured data on every single possible thing you can have data for?
[00:12:34] XY: That's a really great question, Rob. Talking about collecting data from the game, the gaming industry also are going through quite a bit of iteration was the best way to do it. Back in the days, World of Warcraft, when I was working on that, there is a combination of hey, let's just get data from whatever operational system that is supporting the game, operating. If there is a record in operational database, the replicated record and then we interpret what it means. The awesome thing is Rob, you just mentioned, hey, we pretty much can capture everything, because in order to do anything the backhand, the system needs to know what you're doing and what you have been done in the past.
The tricky part is those systems are not designed for analytics or AI. Sometimes you end up on this operational database, how do you actually make sense of the information over there? It's so tricky. The next iteration of a lot of the game data collection starts to add in telemetry hooks, or analytics hooks in specific events or specific actions the players are doing. This is a much more intentional way of collecting data. The data you collected a much more ready for consumption in the purpose of AI analytics. We always see a combination of those. I think the industry is moving more and more towards, “Hey, be very intentional add-ins, analytics hooks at specific places and actions so that you get really high quality and usable data from the game, instead of relying heavily on operational database.
Back to your question, Rob, there are a huge amount of data, no matter which way you go, collecting the data. One of the shooting game I worked on in the past, you can two point can collect every single bullet coming out of the weapons. It's huge amount of data. In some games, we can basically track the location of the character of the players character on the map, and was high granularity. The huge amount of data and a lot of those data probably won't be relevant to some of the questions we're trying to answer. A lot of the data could even generate certain type of noise. Just imagine if you record an intersection of the traffic light intersection for two months. Now you want to find some pattern about who's walking by, what are they doing over there by just watching all of those videos, is probably impossible. It's really hard to do that. There is indeed an issue of too much data, both from a signal to noise ratio perspective, as well as from another big perspective, which is collecting data, cost money. It is an investment, processing, storing the data, actually, it could be very expensive if you actually start doing that. What we're trying to do when we think about too much amount of data is think many of the games I worked on, and a lot of the friends and connections I've been talking to regarding collecting data is what becoming more and more intentional. We're more and more thinking about, how we're going to be using the data to start? Then start being very intentional in collecting the right amount of data and then use the data for what we need. Instead of saying, “Hey, let's collect everything. Then let's figure out if we're going to be using it.” I think, that's also an evolution of how we think about collecting data and using data in gaming.
[00:15:58] RS: Yeah. It's important to call out how much noise there is and the example of the intersection, say between 1am and 4am, there's largely nothing happening, right? So you're like, “Oh, more data is better, but a huge chunk of that data is just not really relevant to what you need. Prioritizing feels really important. Is that what is meant by the telemetry hook you mentioned? Is that like, okay, we are going to hone in on only specific actions and only collect data on these specific areas?
[00:16:26] XY: Yap, absolutely. That's the purpose. We want to be very, very intentional about what are the actions that will help us to best understand the player behavior and motivation. What information will best represent how the players are interacting with our game, so we go through this design process and essentially design those telemetry events and flesh out what type of information at what granularity is going to be most helpful for us to answer those questions. Then we go in and designs those telemetry hooks. Then harvest those information, to help us understanding those questions we have.
I think one beauty about working in gaming data is, I think, compared to some of potentially, E-commerce or advertisement companies. We can get a lot of information and we don't necessarily need to know who you are. We don't have to collect for most of the use cases, regarding helping game design, game iteration. We don't really need to know if this data is from Xiaoyang, from Rob, or from whoever. So what we really want to understand is the type of players and what are the behaviors, and what are the motivations. Then we can iterate our product to improve our product for those types of players.
I think that gives us a lot of great opportunity to really respect the privacy of our players. We don't need to know who you are. We don't need to collect any of the personal information. We get your behavior. We know it's a player, whatever ABCD, hash number, whatever thing were there. Then we can have the analysis and we can build machine learning and data science AI models to understand what it means when this player do this behavior and what this motivation over there. I think that is a really fun part of working with game data.
[00:18:17] RS: Yeah. You don't have the challenge of PII, right? I'm not particularly concerned, what you do with the data of how my avatar runs around a Call of Duty map, it's not identifiable to me, I risk nothing by letting you have that. Yeah, that's a nice little advantage. I like that you mentioned you have been deliberate about what are the player behaviors that will be illuminating for you and making the game better.
For someone who maybe believes data is an arms race that more is always better, we'll sort it out later, we'll figure out what to do with it. Let's just get our hands on as much as possible for a quarter as much as possible. I would love to hear what those behaviors are. I'm imagining you and your teams are sitting around a whiteboard and is writing down. “Okay, what do we care about?” What do we care about learning from this data? What are the behaviors associated with that? Could you maybe give some examples of what those things are?
[00:19:11] XY: Yeah, absolutely. I can give you one example about the genre. We're building a collectible card game, and I worked on some collectible card games in the past. One of the really helpful way that you can use data in this type of game is balancing the game. We want to make sure that the game has a pretty diverse matter. Instead of hey, there's just this one way of this several cards you can use to win the game. We want to make sure that there are different sets of cards you can use to still have fun, still be competitive in the game. That's not an easy task.
In order to do that, we need to really understand what the manual looks like, how the balance look like, especially card game usually release new cards pretty frequently. Each new release means a potential change in the matter and a potential change in the balance. So when we think about that, we start to think about, okay, in order to understand that what type of information we want to collect? There are definitely information about, “Hey, we should be collecting, winning and losing.” That is basically indicating the strength of the cards and deck.
We can go deeper there. We can go deeper than, “Hey, this type of cards or this set of cards tend to be stronger earlier game, or mid game, or tend to dominate, as the end game.” It’s a lot of strength related metrics we can add to help the designers, to help the game team to understand the balance of the game. There is another part, which is, are they fun? A card of that has a super powerful, not necessarily means, it's fun to play. Sometimes they've connected very much, but sometimes, we do see a wide range of players, they choose cards and decks or sets of cards that are not necessarily the strongest. They still play it. We also want to start looking into, “Okay, what are the cards on sets of cards the players are building or collecting, using the frequency and things like that?”
There are metrics on that front as well, so just take this as example. If there are very specific questions about the balance, the matter, we can be very intentional about what type of data we want to collect. There are definitely a lot of other information we can collect, for example, we can say, “Hey, we can just collect every time when you start building a set of cards, how often you click on certain things? How often you use some cards that you can save it or forward it?” Sometimes not relevant to the questions we want to answer over here. Probably we want to make some trade-offs and get the data that provides the most a signal instead of just collecting everything.
Right? So that is under the Northstar of is it fun, right? Because is it balanced that you want – I need to be too overpowered. You want people to be able to remain competitive, which makes it more fun. Then, oh, this isn't the best card, but people like it, because it's fun. I’m just putting out from a product development standpoint, what are the drivers that you care about? It seems it's important to figure out, “Okay, we want this to be really fun. Therefore, let's put in these telemetry hooks so that we can tweak the game in ways that make it more fun.” Right? Just as that is the priority, does that data we're going to pay attention to?
[00:22:24] XY: Yap, absolutely, absolutely. I would say the ultimate goal of making a game is make it really fun for the players, and making it competitive, making it fair and balanced is a way and one of the very important pillar to make some PvP and competitive games fun. But at the end of the day, it's about making it really fun for the players to play.
[00:22:45] RS: Yeah, yeah of course. The other thing I really wanted to get into with you was this explosion of possible use cases and opportunities that just result from adding AI and ML to video games. When you think about the use of synthetic data, the industrial Metaverse and what can be gleaned from a simulation, really. How do you see video game development fitting into the bigger picture of maybe more widespread AI use cases?
[00:23:15] XY: Yeah. That's a really great question, Rob. I think if we look back a few years, the moment AlphaGo shocked the world. AlphaGo also shocked the gaming industry. It's very interesting coincidence, Move 37 when Lee Sedol was playing AlphaGo. I think that was such a profound moment when Lee Sedol himself and some of the folks in the GO community realize, I think that's creativity. The AI is doing something so innovative. I think that can be informative and helping how we humans think about problem solving and think about many, many things.
It's actually very interesting, I think for the audience, if they are fighting game fans, Move 37 is also profound moment when they're expecting the day's people realize how powerful human are as a mechanics that fighting game was never designed for humans can do it. I think it's such a fun coincidence. Move 30 sevens is a great example there. Back to AlphaGo, I think that moment shocked game industry, because we start to see a lot of opportunities we can bring in AI and help facilitate creativity in gaming. We just go back to one of the examples we just mentioned, helping the balance in design, helping to understand if the game is fun.
Thinking about how we do balance in the past, whenever there is a PvP game, no matter it’s the first person shooting or card game or MOBA. How you do balancing is, the designers they think about, okay, I think this is balance. This new characters, new weapons new champion, new card should be balanced. Let's give it to the players so they can help us to test and then we observe. Okay, is this indeed balanced or is actually off. Every time you do that it's a long process and it also means you deliver sub optimal experience to the players. They're essentially helping you to test out, sometimes they got pissed off, why is this thing so OP? Why this card is so broken? Why this weapon is just not usable? There's a sub optimal experience, there are trade-offs over there.
The moment about AlphaGo and when we start thinking about gaming, we realize, “Hey, AI, they can play games in a very competitive level. They can play games just very similar to how humans are going to be approaching it. They can even play games. You can make them play games at different levels.” When you start thinking about that you realize, “Hey, wait a minute, what if we asked the AI to help us and tell us if the game is balanced before we gave the game to the players.” So that AI can play the game, we can ask the AI, “Hey, play like Esports competitive player. Play a hardcore player. Play like a mid-core player. Play like a casual player. Then go play millions of AI bots, will be playing the game overnight. Next morning, the designer come back to his desk or her desk, they will see the balance report over there, generated by the AI. That's a such a powerful tool for the designers. It's such a powerful tool for game development, I mean, start thinking about balance and matter and everything like that.
We start thinking about, “Hey, if AI can do and play games like that at a human level, it unlocks a lot of opportunities for us.” If you think more about it, when AI can really play at a competitive level and at different levels, that opens a new door about the PvE game. That opens a new door about how the NPCs could be interacting with the players to provide really engaging experience. In the past, the NPC, the AI, you talked about game AI versus AI, actually very different, right? You talk about game AI, you talked about behavior tree, finite state machine, the goal oriented action planning. Those type of traditional game AI methods that are widely used.
They're not necessarily AI is that I learned from experience, not necessarily AI that improving as you get more experienced. Now we think about, wow, AI can learn to such a level. There is new research coming out that means AI can collaborate with each other, and they can learn to collaborate with each other. That just opens a new door about how we think about PvE games, how we think about creating a word that is so engaging and so fun to play. I think the development and progress in AI over the past 10 years, I feel so fortunate and lucky to be here and being able to experience and see what's happening with there and what it means to gaming. Actually a lot of adoptions, some of the ideas, I just mentioned, are happening in the gaming industry pretty widely.
Rob, I think another part of the question mentioned was, what do you think about the gaming industry potentially can facilitate or in gaming help the data and machine learning that's development in general? Gaming is just such a great playground and training ground for AI and machine learning. I'm sure you have been following a lot of the progress for many years, and many audience as well. Some of the most prominent playground for AI the games, they could be some of the older games, it could be some ambitious places actually trying some new games like Dota, some of the shooting games as a playground for AI.
The games creates such a both controlled as well as reaching environment for the AI to go learn, and for the researchers and developers to observe and tweak and understand what's happening over there. Game creates such a great space for the AI to learn and for the people who are interested in and doing research in AI to understand what's going on. I think, there are a lot of great progress recently to create a parity of the work in a certain environment. It could be for driving. It could even be for some surgical or healthcare use cases. You can create very much a parity, environment or virtual world for the AI to tinker on, which is very manageable, very effective learning environment and very safe compared to really put a car on the streets. There are a lot of progresses happening, super exciting and I think the connection is just going to get really much, much stronger moving forward.
[00:29:29] RS: Yeah. Loads to unpack there, I have to say. I want to quickly just define a couple of terms in case someone out there isn't necessarily a video game fan, although I do suspect that the AI practitioner video game fan Venn diagram has a lot of overlap, but an NPC is a non-playable character, right? Anything that is a bot or an AI as you said, PvE Player versus the Environment. You're playing against the game itself as opposed to other players in a one-on-one human versus human capacity. But with regards to the NPC I thought that was so interesting in that game AI versus AI, you made a point to differentiate that, because a game AI has a decision tree, whereas a pure form of AI would involve learning. Could you speak a little bit more about the difference there? Is it that a game AI is more finite or how would you characterize the two?
[00:30:21] XY: Yeah. That's a good question, Rob. If we talk about a finite state machine, or a behavior tree type of AI, the basically very similar to the very old days of AI, it's a human crafted expert system, probably, it's one way to think about it. All the rules of logics are there, you can tweak it a little bit to add a little bit stochastic components there to make it a little bit more fun. Still, it's a human crafted system. Today, when we more generally talk about AI, we're talking about learning as a huge part, AI is learning either directly from the signals or for example, reinforcement learning is learning through interacting and trial and error in the environment. So they're very different.
If you think about moving forward, a lot of researchers and actually a lot of game companies start to think about how do we really embrace the development in the general AI space, and create agents or create AI in the game that are learning and very engaging, and instead of have to handcraft every rule, handcraft every probability in the old AI system. We want the AI to learn, to adapt, to how the players are consuming it. I think the distinction was so clear back in the days. I guess, at least today as well for most of the games, but we start to see, the development starts with blurring the line, because more and more game developers they realize, by adopting the new AI technology, by adopting the mindset and technology of making the AI agents to be able to learn, there are so much more enriched experience that we can provide from the endgame NPC, you just mentioned.
[00:32:07] RS: Yeah, yeah, that makes sense. Could you give an example of AI in a video game, maybe going beyond the traditional view of game AI that you find truly impressive or that is just a great leap forward from your point of view?
[00:32:19] XY: Yeah, absolutely. One thing I want to mention is that I think there is always two parts that are developing. If two parts are imbalanced, I think that's the perfect case for progression. One part is, we would love to understand what's the awesome use case that a certain technology will be able to unlock. The other part is, we would love to understand if the current progress of technology would actually be able to support that type of use case. I think, right now, AI is in this phase of finding a match of those two arms. There are a lot of great progress in AI technology. We're still trying to figure out what is the killer app of using those new advanced AI in gaming.
Yeah, we'd love to hear what you think, Rob, and I think a lot of conversations in this space right now. One of the use case, I want to mention is there is a game developer in China, Tencent Games, and one of their very popular game. It's a mobile game, which is a multiple players, five players on each side fighting each other. That game developer, they built AI based on reinforcement learning and a lot of other advanced technologies and they open up this feature to players to go try, to play with those AI to see if you can beat them. They made it into a game mode over there. The players are really impressed, because in the past, if a professional player goes in to play this type of game, no matter how well you handcraft it the AI system, they're just no match to human.
It's like, a professional player go there totally crush all the opponents from AI, but this game is different, this AI is different, the professional Esports player coming in, they will have a very competitive experience versus AI and different level, skill level players coming in. They can choose to play with different types of skill level of AI in the game. They can feel really engaging. They don't feel they're playing bots. When they're playing, they could feel, oh, wow, this is engaging. This is like giving me really great experience and competitiveness that make me sometimes forgot I’m playing AI. Sometimes they do remind me, because it's just like, they're just too good. That still happens.
I think that is a really profound example about when you enable this experience with this type of technology there are some new experience you can deliver to the players. It was just not possible in the past. For sure, I think the game industry overall, a lot of the friends connections, a lot of folks in the space, having a lot of discussion thinking about what's the killer app of actually leveraging this technology and making it so meaningful to player experience. It's still an ongoing journey. I think we're in a very good position and getting closer to really figure it out.
[00:34:59] RS: Xiaoyang, this has been a fascinating conversation. Thank you for walking us through your company, your background and all of your takes on AI in the video game space. Awesome, awesome conversation. Thanks for doing this with me.
[00:35:11] XY: Thank you, Rob. Again, thank you so much for having me. I had a lot of fun.
[00:35:17] RS: How AI Happens is brought to you by Sama. Sama provides accurate data for ambitious AI, specializing in image, video and sensor data annotation and validation for machine learning algorithms in industries such as transportation, retail, e-commerce, media MedTech, robotics and agriculture. For more information, head to sama.com.