Artificial intelligence has now pretty much conquered poker

Artificial intelligence has passed the last major milestone in mastering poker: six-player no-limit Texas Hold’em.

Games like poker, with hidden cards and players who bluff, present a greater challenge to AI than games where every player can see the whole board. Over the last few years, computers have become aces at increasingly complicated forms of one-on-one poker, but multiplayer games take that complexity to the next level (SN Online: 5/13/15).

Now, a card shark AI dubbed Pluribus has outplayed more than a dozen elite professionals at six-player Texas Hold’em, researchers report online July 11 in Science. Algorithms that can plot against several adversaries using such spotty information could make savvy business negotiators, political strategists or cybersecurity watchdogs.

Pluribus honed its initial strategy by playing against copies of itself, starting from scratch and gradually learning which actions helped to win. Then, the AI used that intuition for when to hold and when to fold during the first betting round of each hand against five human players.

During subsequent betting rounds, Pluribus fine-tuned its strategy by imagining how the game might play out if it took different actions. Unlike artificial intelligence trained for two-player poker, Pluribus didn’t speculate all the way to the end of the game — which would require too many computations when dealing with so many players (SN: 4/1/17, p. 12). Instead, the AI imagined several moves ahead and decided what to do based on those hypothetical futures and different strategies that players could adopt.

In 10,000 hands of Texas Hold’em, Pluribus competed against five contestants from a pool of 13 professionals, all of whom had won more than $1 million playing poker. Every 100 hands, Pluribus raked in, on average, about $480 from its human competitors.

“This is roughly the amount that elite human professionals aspire to beat weaker players by,” implying that Pluribus was a savvier player than its human opponents, says Noam Brown of Facebook AI Research in New York City. Brown, along with Tuomas Sandholm of Carnegie Mellon University in Pittsburgh, created Pluribus.

Now that AI has poker in the bag, algorithms could test their strategic reasoning in games with more complex hidden information, says computer scientist Viliam Lisý of the Czech Technical University in Prague, who was not involved in the work. In games like Kriegspiel — a chess spin-off where players can’t see each other’s pieces — the unknowns can become far more complicated than a few cards held close to opponents’ chests, Lisý says.

Video games like StarCraft, which allow many more types of moves and free players from rigid, turn-based play, could also serve as new tests of AI cleverness (SN: 5/11/19, p. 34).