Unlike bots playing Unreal Tournament or StarCraft, Google’s DeepMind division did not provide all the information to the AI when it was tasked with playing the notoriously hard Atari game Montezuma’s Revenge. Instead, it learned how to play the game the same way humans do, by looking at the screen, pushing buttons and seeing what works.
f this setup sounds familiar, it’s because last February DeepMind unveiled an earlier iteration of the same agent, but when that bot tried to take on Montezuma’s Revenge, it couldn’t score a single point. But as the video below shows, now DeepMind’s AI is dodging skulls, grabbing keys, and scoring points like a pro. The difference? Curiosity.
Essentially, this meant creating a reward system for the AI agent, something comparable to the human brain’s “pleasure chemicals.” DeepMind’s scientists then connected these digital rewards to the agent’s exploration system, giving the desire to look around its surroundings and be curious about it.
As the research paper explains, the new agent is much more effective than previous iterations, exploring far more of the trap-filled rooms of Montezuma’s Revenge. “After 50 million frames, the agent using exploration bonuses has seen a total of 15 rooms, while the no-bonus agent has seen two,” write the researchers.
Watch an AI beat a game most puny humans couldn’t below.