How Microsoft is open-sourcing Minecraft AI experiments

Alan Martin September 23, 2016

When people say “X is so much more than a simple game”, it’s usually just marketing spiel: a vain attempt to make something sound more interesting than it actually is, but Minecraft is definitely an exception to this rule. Not only is it being used effectively for educational purposes, but its new owners at Microsoft want to push it to the heart of research into artificial intelligence.

How Microsoft is open-sourcing Minecraft AI experiments

Speaking at New Scientist Live, Microsoft Research’s Katja Hofmann was on hand to explain how and why the game is the perfect petri dish for artificial-intelligence experimentation. “With artificial intelligence, it’s actually very hard to experiment. If we wanted to build a robot that can climb stairs, walk and talk to us – building that robot is currently extremely expensive,” Hofmann explained, citing million-pound budgets and hundreds of researchers. “However, if we look at computer games, they allow us to very rapidly iterate. We can quickly come up with new tasks, test new ideas and see whether machines can solve them or not.” project_malmo_ai_learning_minecraft

Minecraft, with its huge variety and even larger player base, is a surprisingly effective alternative. The game, as players will know, is a huge open sandbox where players can build, create and adventure on their own or collaboratively. “It’s this variety that makes Minecraft such a fascinating platform for artificial intelligence research,” explains Hofmann. And so Project Malmo was born earlier this year. It’s a free download that allows anyone (not just the hundreds of academics Hofmann cites as running their own experiments) to test the fascinating waters of machine learning and artificial intelligence.

“What Project Malmo does is create a layer around Minecraft to make it as easy as possible to focus just on implementing an agent and to start experimenting as quickly as possible.” You just need a little coding knowledge to get started, and Hofmann demonstrated a simple python script where the AI agent was able to learn how to run around in a circle jumping in “just a few lines of code”.

Getting a little more advanced, Hofmann demonstrated the “cliff walking” reinforcement learning problem, where the AI has to learn how to navigate successfully from A to B without falling into pitfalls – in this case, blocks of lava. “Initially it will just try and interact with the environment, and it will jump into the lava a lot. But it will learn from that experience,” explains Hofmann, adding that the AI will learn to solve the problem within about six minutes.

And Project Malmo agents aren’t just limited to learning about lava the hard way, of course. “It could learn to climb up ladders, it could learn to jump and do complicated parkour challenges.”

“In the long term we think that combining those ideas will allow us to develop agents that not only avoid lava, but eventually collaborate and interact with us using natural human language. One area I’m particularly excited about is the ability to collaborate with others and have several AI agents – or AI and human agents – solving tasks together within Minecraft.”

Crucially, this is very different from the kind of AI training that goes into, say, the facial recognition software in your camera, where machines are fed thousands and thousands of pictures of faces until they learn what telltale markers to look for, and it’s this kind of thing that Microsoft is hoping to provide an alternative to within the Minecraft sandbox. “It just gets really annoying – we don’t have time to label every little thing,” explains Hofmann. “We need to move to more interactive learning, through trial and error. If it gets stuck, maybe it could ask for help, or maybe it could try something new and see if it works.”

This is all well and good, but is still essentially machines learning in quite an unsophisticated way: trial and error. As humans, we do that too, but we also accompany that with our own past experiences, and that’s something Hofmann hopes can be exploited. Better artificial agents, she says, would combine reinforcement learning with reasoning. Then they won’t need to “jump into lava 100 times before they do something sensible.”

It genuinely does feel like an exciting time in artificial intelligence – advanced enough for pace to be rapid, but early enough for researchers to make a clear mark in the world. Hofmann echoed these thoughts at the end of her talk, encouraging students unsure of their career prospects to consider dipping a toe into the fledgling industry. With Project Malmo free to install on top of Minecraft, it certainly wouldn’t hurt to try.

New Scientist Live runs until 25 September at the ExCeL Centre in London. Tickets are available here.

Images: o0N0o and brownpau used under Creative Commons