After beating humans at Go, AI turns its attentions to poker

Alan Martin March 31, 2016

You win this round, AI. And the next round. And the round after that.

After beating humans at Go, AI turns its attentions to poker

DeepMind crushed the finest human Go player on the planet last month, but Artificial Intelligence isn’t content to sit on its laurels, as a University College London team is pushing a new game for humans to lose dominance in – poker.

The team – which includes David Silver from DeepMind – has developed an AI that is able to learn Texas Hold’em and Leduc, and develop strategies by playing matches against itself. This “Neural Fictitious Self-Play” learning method allows the AI “

to learn directly from their experience of interacting in the game,” meaning it learns from its mistakes and successes to become the poker master you really would prefer not to be taking on for the pot.

Indeed, the strategy that the AI learned for Texas Hold’em “approached the performance of human experts and state-of-the-art methods”. In Leduc, a simple six-card version of the game, the machine is even more affecting, approaching the “Nash equilibrium“. ai_learning_poker

In terms of AI’s dominance at games, things are getting increasingly complex over time. It is now 19 years since IBM’s Deep Blue beat Kasparov, and less than a month since Google’s DeepMind beat Lee Se-dol – that may seem like slow progress, but while Chess has around 20 moves available, Go has 200 or so. Poker complicates things further by providing unknowable data – the AI cannot know what cards its opponents have in any given hand.

This presents a unique problem to researchers, as Johannes Heinrich told The Guardian. “Games of imperfect information do pose a challenge to deep reinforcement learning, such as used in Go. I think it is an important problem to address, as most real-world applications do require decision making with imperfect information.”

“The key aspect of our result is that the algorithm is very general and learned a game of poker from scratch without having any prior knowledge about the game. This makes it conceivable that it is also applicable to other real-world problems that are strategic in nature,” he added.

Chess, Go and – soon – poker. Don’t worry guys, we’ve probably got football for a while longer.

Images: Morgan and Sharat Ganapati used under Creative Commons