AI developed by Nvidia recreated the game Pac-Man from scratch by just watching the gameplay

Photo by Giorgio Trovato on Unsplash

World’s best graphics processing company Nvidia is developing some futuristic AIs. Nvidia has been working on several Artificial Intelligence projects and has been carrying out major research in this field for a long time now. This time the company has extended its boundaries and has developed an astonishing Artificial Intelligence which recreated the retro classic Japanese game Pac-Man on its 40th anniversary from scratch by just watching the gameplay. The name of this AI is NVIDIA GameGan which is basically a Neural Game Engine.

How does this AI recreate a game by just watching the gameplay?

Well, the researchers have said that the basic principle used here is ‘Model-Based Learning’ in which the entire logic of the game including the controller inputs is stored on the neural networks from where this information is further developed into that game from scratch and frame by frame. Hence there is no rendering of coding or images required for the AI.

The AI though was not able to re-capture the images of the ‘Ghost characters’ of the game which are meant to chase and kill Pac-Man and hence resulted in a blurry image of these characters. This happened because the movement of the Ghosts in the game is the result of a complex algorithm and each of these ghosts is programmed with these complex and unique algorithms to determine their movement across the maze. Albeit the programming algorithms of the Pac-Man are a lot less complex than that of the ghosts as its movements are fixed to the controller inputs.

The basic architecture of the GameGan is divided into three parts which are the Dynamic Engine, The Rendering Model, and The Memory Storage or the ‘memory module’, and this architecture works in two halves. In the first half, the Neural Game Engine or the GameGan replicates or tries to copy the input data visually through the game, and in the second half, this set of logical input data is then compared with the data of the original game. If the generated data matches the original data of the source, the game is then generated by the AI else if it does not match the data is rejected and again sent in the process of correct data generation. This goes in a perpetual loop until the data matches accurately.

Sanja Fidler who is Nvidia’s director of AI in Toronto Research Labs told that GameGan had to be trained on 50,000 episodes of Pac-Man to generate that fully functional game which requires no underlying game engine. Since it was impossible for a human being to generate this humongous data of 50,000 episodes the help of an AI agent was taken to generate the data. The initial challenges included the invincibility of the AI agent as it was so good in the game that it hardly died in any round. This resulted in the creation of the game in which the Ghosts just followed the Pac-Man arbitrarily and are unable to reach it ever.

The memory storage or the ‘memory module’ of the GameGan AI generated a new aspect to it according to the researchers. The memory module allowed the storage of the internal map of this game world which is actually the static element of the game rather than the dynamic elements like the Pac-Man and the Ghosts. This will allow the AI to create new maps, levels, and worlds by itself without any human interventions. Thereby the gamers and the users will be gifted with uncountable new maps and game worlds. This will enhance the dynamics of the game exponentially.

Advantages and Future aspects of the GameGan AI

There have been several advantages predicted by the researchers as well as the gamers themselves on these new characteristics of the AI.

  • The biggest advantage of this AI will be the speeding of game development and creation. The creators will need not have to code from scratch for new layouts and levels of a certain game, the AI will eventually create new game worlds visually.
  • The AI will simplify the development and creation of new simulation systems for training autonomous machines. This will allow the AI to learn the rules of the actual working environment even before interacting with any other real object of the world.
  • By just visual data in the near future, the machines will be able to drive a car, go for grocery shopping, play a sport, learn laws of physics in the real world, etc. which will be a humongous achievement for development purposes.
  • The AI will help in a very easy transfer of the game from a particular Operating System to another Operating System. Hence the game will not have to be developed again with the codes for various Operating Systems, the AI will automatically do it.
  • The game can be compressed by the AI in the memory module of the Neural links or the Neural Networks and can be stored there permanently allowing the continuous development of the statics and dynamics of the game totally by the AI.
  • This AI in the near future will allow automated machines to outperform humans in dangerous and catastrophic situations carrying out experiments and rescue operations.


Experiments, researches, implementation, and development on new characteristic AIs are being performed by various other scientists, researchers, and engineers from all across the world. The new age of machines and Artificial Intelligence is commencing and soon all of us will be equipped or will be aided with various efficient AI robots with different capabilities which will curb time wastage and will accelerate human development to novel extents. GameGan is an exquisite example of development in machine learning and deep learning of the computers where the possibility of developing a machine visually is now made real. This AI will be used extensively in the near future to generate new simulators without a set of codes to ponder over and will allow it by just training the complex neural networks. We hope to watch new and amazing AIs by Nvidia and other organizations.

Leverage machine learning in your organization with Tipstat. Contact us here

Interested in working with Tipstat on AI? Check out our open positions here