Poker Coverage: Poker Legislation Poker Tournaments Daily Fantasy Sports

Championship Poker Bot: 'A Nuclear Weapon For Poker'

Professor Tuomas Sandholm Discusses His Heads-Up No-Limit Hold'em Bot 'Tartanian7' And What It Means For The Future Of Poker

Print-icon
 

Tuomas Sandholm is Professor at Carnegie Mellon University in the Computer Science Department with more than 450 published papers. Over the past decade, he has applied his computational game theory knowledge and optimization algorithms to develop a computer program, or “bot.”

Sandholm’s objective was to design the greatest heads-up no-limit Texas hold’em player in the world and, possibly, the ultimate training tool for that particular form of poker.

When asked if he thought, in the future, if learning from what his program has discovered would be mandatory for all those wanting to compete at the top of the game, Sandholm didn’t mince words.

“I think so. It’s a bit of a nuclear weapon for poker. You don’t want to be bringing a knife to a gun fight.”

Certainly a strong statement, but Sandholm’s success with his program lends weight to the words. This year his bot won both categories of competition for heads-up no-limit at the Association for the Advancement of Artificial Intelligence (AAAI) Annual Computer Poker Competition, beating each opponent along the way with statistical significance.

The idea of a ‘poker bot’ has captivated the collective consciousness of the poker world since the inception of online poker. For the dubious, the idea that they could be playing poker against a machine was disconcerting. For futurists the possibility of applying computer science to the game was exciting. The truth is that people have been creating bots and running them in online games for at least a decade, with the first commercially available program, called “WinHoldEm,” even being featured prominently in tech magazine Wired.

Prof. Tuomas SandholmThe New York Times even wrote a feature on bots in 2011, with the conclusion of the article being that, for now, most human players should not fear playing against bots. One reason is because the top online poker sites have sophisticated methods of identifying robotic players and actively hunt for them.

But Prof. Sandholm also notes that while the bots out there might be sophisticated enough to be winning players, they aren’t nearly as strong as the top bots that he has created and competed against. “They aren’t very good. The current ones that are out there now, some of them have participated in the AAAI competition and turned out to be not very good.“

So, bots are currently out in the marketplace, but for now researchers at university computer science programs mostly develop the very good ones. Card Player recently spoke to Sandholm at length about the competition, the development of his bot, and what’s in store for this particular program in the future.

Designing ‘Tartanian7’ To Dominate

The ninth annual Computer Poker Competition consisted of six categories, two of which featuring no-limit hold’em. Approximately 50 million hands were played. The categories all used duplicate matches, which in the case of two-player games meant that they played an ordinary heads-up match of a set number of hands between two bots, with the specific cards dealt being recorded. At the end of the specified number of hands the bot’s memories are reset, they switch positions and play the set of hands over with the other player’s cards.

“The competition has two categories in heads-up no-limit Texas hold’em,” elaborated Prof. Sandholm. “One is called ‘total bankroll,’ in which the bots play against each other and whichever bot wins the most virtual money wins. The other category is called ‘instant runoff,’ in which you again have all of the bots play each other and throw out the one that loses the most and then play another round and so on until there is one bot remaining.”

Sandholm said these two categories measure different strengths. “The total bankroll measures mostly how well you do against the weak opponents and the elimination measures mostly how well you do against the strong opponents,” he explained.

'Tartanian7' Playing Instant Runoff 2nd-Place Finisher 'Prelude'

Sandholm’s program, officially entered by him and his Ph.D students Noam Brown and Sam Ganzfried, was named “Tartanian7”. Sometimes teams make variations of their program for the two separate hold’em categories, but the same version of Tartanian7 won both categories and, impressively, was the best against every single one of the opponents with statistical significance.

When asked if the success of his program had anything to do with his own experience playing strategy games and poker Sandholm quickly noted that, “we did not program in any human poker playing knowledge. Our approach was purely algorithmic and our algorithms created the program.”

“They key was really our superior knowledge of computational game theory and optimization algorithms,” said Sandholm. “More specifically, all of the top bots were created with the following kind of framework that my student Andrew Gilpin and I put together in 2005. You first run an abstraction algorithm that generates a smaller, but strategically similar, game so that we can then use an equilibrium-finding algorithm that finds a Nash equilibrium or an approximate Nash equilibrium for the smaller game. The original game (in this case, heads-up no-limit hold’em) is far too large to solve for Nash equilibrium. So we solve the smaller, abstract game, and then we use a reverse-mapping algorithm to map the answer from the abstract game back to the original game.”

Proposed by John Forbes Nash, the Nobel Prize winning mathematician who was the subject of the Hollywood movie A Beautiful Mind, the concept of a Nash equilibrium essentially is a solution in a non-cooperative game in which the player has no reason to diverge from their strategy given an opponent’s approach. In poker, it would be a game-theoretic strategy that would work against any approach your opponent presents.

Having found what he considers to at least be an approximation of the Nash equilibrium for heads-up no-limit hold’em, Sandholm entered the same bot into both categories.

“We did not do any opponent exploitation this year. We have done research on the subject and have used it in some previous years, specifically in the total bankroll competition, because there you can do well using opponent exploitation by taking advantage of the weakest opponents. But this year we didn’t do any of that. We just had an approximately game-theoretic solution and we used that to win both categories, playing with a static strategy throughout. That you could win both competitions with the same program, with the same exact program that plays superbly against both weak and strong opponents, is something I find interesting.”

Make sure to check back on CardPlayer.com for the second part of this article, in which Sandholm discusses the possibility of man vs. machine match with Tartanian7, how the bot could be used as a teaching tool and more.

 
 
 
 

Comments

nanonerd
2 years ago

blah blah blah. maybe the other computer programs weren't that good. why not let the bot player a live pro, now that would be interesting.

i'd also be curious if he ever pitted his program against a copy of the same program. It'd be interesting to see the results after wiping out the memory and letting them switch sides to play the same set of hands.

when will tartar7 play a live pro?

 
Reply
 

Marv1
2 years ago

The CMU bots have disappointed in the AAAI contest for many years. The University of Alberta has owned that competition since it began, and even independent programmers have won more gold medals than CMU.

The method he says "he" came up with is exactly the method used in an award-winning paper by the U of A in 2003. What a pompous blowhard. He should tell the truth, instead of making stuff up.

 
Reply
 

ThePlayer108
6 months ago

When it comes on creating a poker bot using 'just' algorithms I think they are going to the wrong directions. There is so much hidden information in the game that only a real poker player can detect certain situations. I'm sure that most bots that works with only algorithms can be easily beaten by normal players since you can bluff them.

However, for me it make more sense that some commercial bots can play really good poker, for instance, Abz Poker bot let you create your own bot by doing real strategies as humans with the advantages that can remember players history and using HUD on the go. This means that you can make a bot playing like a great poker player and in the same time it remembers everything. But I assume that using 3rd party softwares is not allowed in AAAI..

 
Reply