 |
Forty years ago, PAC-MAN appeared in arcades in Japan and munched a path to global stardom. Now the retro classic has been reborn and delivered courtesy of AI.
Trained on 50,000 episodes of the game, a new AI model created by Nvidia Research called GameGAN, can generate a fully functional version of PAC-MAN—without an underlying game engine. That means that even without understanding a game’s fundamental rules, AI can recreate the game with convincing results.
A generative adversarial network (GAN) is a machine learning (ML) model in which two neural networks compete with each other to become more accurate in their predictions.
Nvidia says their GameGAN is the first neural network model that mimics a computer game engine by harnessing GANs. Made up of two competing neural networks, a generator, and a discriminator, GAN-based models learn to create new content that’s convincing enough to pass for the original.
As an artificial agent plays the GAN-generated game, GameGAN responds to the agent’s actions, generating new frames of the game environment in realtime. GameGAN can even generate game layouts it’s never seen before if trained on screenplays from games with multiple levels or versions.
 |
| Nvidia’s GameGan created this layout. (Source: Nvidia) |
This capability could be used by game developers to automatically generate layouts for new game levels, as well as by AI researchers to more easily develop simulator systems for training autonomous machines.
The GameGAN edition relies on neural networks, instead of a traditional game engine, to generate PAC-MAN’s environment. The AI keeps track of the virtual world, remembering what’s already been generated to maintain visual consistency from frame to frame.
No matter the game, the GAN can learn its rules simply by ingesting screen recordings and agent keystrokes from past gameplay. Game developers could use such a tool to automatically design new level layouts for existing games, using screenplay from the original levels as training data.
It’s not just about games
Autonomous robots are typically trained in a simulator, where the AI can learn the rules of an environment before interacting with objects in the real world. Creating a simulator is a time-consuming process for developers, who must code rules about how objects interact with one another and how light works within the environment.
Simulators are used to develop autonomous machines of all kinds, such as warehouse robots learning how to grasp and move objects around, or delivery robots that must navigate sidewalks to transport food or medicine. BMW recently demonstrated how they are training autonomous factory robots using the Nvidia Isaac Robotics platform for simulation. GameGAN introduces the possibility that the work of writing a simulator for tasks like these could one day be replaced by simply training a neural network.
GameGAN is authored by Sanja Fidler, Seung-Wook Kim, Jonah Philion, Yuhao Zhou, and MIT professor Antonio Torralba. The paper will be presented at the prestigious Conference on Computer Vision and Pattern Recognition in June.
What do we think?
 |
As an AI training exercise, this is an interesting experiment. One can see similarities in its use for robots in a controlled environment. I could see GameGAN being useful for game developers as well. Think about it: How about a game mash-up? Using this technology, one could interpolate between different games and create a new one! Talk about time to market! I could also see this technology being used to transfer a game from one operating system to a different one.
Another potential use-case is for gamers. Gamers love to mod (e.g., Minecraft). With GameGAN, a gamer could potentially modify a game that is impossible now without access to source code. I dug into the research and the implication is that one could interactively replace one environment with another, or swap out a character with another.
I’m sure there are plenty of other use-cases both game developers and gamers will discover. That might actually be even more interesting than GameGAN itself!