Skip to content(if available)orjump to list(if available)

How A.I. Conquered Poker

How A.I. Conquered Poker

187 comments

·January 18, 2022

splonk

(Former pro and high stakes player, occasional solver developer)

This is actually one of the best poker articles I've ever seen in generalist media. Not too clickbaity, reasonable high level overview of game theory, a (very accurate IMO) quote from old pro Erik Seidel about the state of the game just 15 years ago, a discussion on variance vs. EV and results, and most importantly, an emphasis on math, randomization techniques and emotional control rather than the TV image of staring in someone's eyes and reading their soul. Probably the one biggest misconception people have is that pros have sick reading abilities since TV likes to emphasize staredowns, when the actual single biggest skill long term pros have is the ability to lose hand after hand for hours and still play their best game.

Incidentally, the anecdote at the top of the article is pretty intuitive game theoretically. Basically on the river you need to bluff with some portion of your hands, or else nobody will ever call when you have a good hand. The natural portion of your hands to bluff with is the absolute worst ones - you don't want to bluff with your middling hands because you have some small chance of just winning a showdown when it checks around. On a board of Kc4c5c2d2c, 7d6d is quite likely the absolute worst hand you can hold given the action that's taken place, therefore it's the one you bluff with. (For some pot/stack sizes it's possible that you bluff so rarely that you have to choose between 7d6d/7h6h/7s6s, but that's getting into details.)

ramoz

Similar story here. I’ve always played, mostly online, before Black Friday (pretty sure I was a minor when it happened) but played all the offshore sites through college. Professional software career took off and bounced out, finally ~3y ago I decided to get back into it and I would only play challenging games (only thing challenging about a 1/3 or tourneys is the grind). So, started with 2/5, within a week was playing the 10/25, 25/50.

1. Emotional control and pushing through the variance are key, I would play all nighters and it was mostly a subtle rise up vs big pots (except the one that did me in)

2. You absolutely do read players, I didn’t need to be a pro to learn how to observe the degenerates. Even decent players signaled things here or there (obv not how they eat an oreo , but I had a knack for knowing when someone was off their style). And it’s not even close wit a normal group of friends.

3. the sharks and their meta game… I had run up to about 50K within 3 weeks from my starting 2-3K. Had the pipe dream. I thought I had a growing bond with the pros. When I say pros I’m not talking grinders, I’m talking consistent big winners. About a handful of them. None of them tv personalities. - There were few times we’d run into each other, but I picked off a few donk bets here or there. However their persistence is what did me in. Went the entire month observing their advanced play style, which the meta was harsh, there were few rivers that mattered more than pre flop and continuation/donk betting. This style of play was like nothing I was seeing on tv. But I was winning with them. The last time I played, about a month into my run, I was on a 12h 25/50 game winding down, turned 30k into 70k (had 20k sitting back) pulled QQ and went up against a pre-flop donker. I face a 3 bet and get that gut feeling but put on lower pair or AK. Flop rainbows, large betting action. Turn, junk, I face an insane Re raise after testing a raise of my own, I know it’s off but it’s either AA/KK or AK at this point. My guts off but I’m too far in to quit. On the river I’m defeated, check, face bet, ultimately end up throwing 50K into the pot, lose to aces.

I never had a gut feeling so intense though. Like I was screaming at myself to get out. But I was caught.

Such a draining hand I never went back. I don’t have the discipline to grind games, and the tech industry pays more.

Stasis5001

When you say tech industry pays more, I'm assuming the effective long-run hourly earnings of poker players is way less glamorous than it looks from an outsider seeing highlights? Doing a basic google search led me to https://pokerdb.thehendonmob.com/ranking/6737/, which seems to suggest that 81 players earned over 1mm USD in 2021. But presumably those players don't earn that YoY, and even if they did, plenty more software engineers earn > 1mm USD per annum, so this seems to support your conclusion. OTOH, I have no insight to the home games or whatever sources are not included in these rankings, so I'd love to hear a little more.

splonk

FanaHOVA mentioned the main thing you missed. One other thing I'd mention is that a lot of pros have staking arrangements (the article briefly mentions some versions of them). For example, I know one pro (who's won one of the more prestigious WSOP bracelets) who gets 30% of his tournament winnings in exchange for having all his entry fees paid by his backer. I suspect he's traded off more upside in favor of stability than most, but there's a lot of deals out there where the players aren't taking home anywhere close to 100% of their public winnings.

FanaHOVA

Those aren’t net of buy ins. Negreanu for example won $3.1M, but spent $2.56M in buy ins, so net profit was around $600k (doesn’t account for all his content ofc, but that’s not every player)

ramoz

The other commenters said it for tourneys. I can’t stand tournaments.

For cash games, I was only playing on the casino. For the 1/3 (1$/3$ blinds) the earnings rate seems absolutely abysmal and you have to account for rake. There are grinders who play these games, along with 2/5. My assumption is any grinder playing here is down under or not making enough to compete with a professional career in the tech industry.

Next is 2/5. Here I could’ve sustained a decent salary. A pro at these stakes showed me his past year earnings which was 180k.

It’s hard to tell with the 10/25, 25/50 pros. There were definitely dudes sitting on 500k+, maybe way more, chips that they kept in the casino and I assume similar amounts of cash at home. There were guys leaving with big nights but still losing 90k in a night. I’d hear about some being down under 200K at any point. There were probably some money launders. There was the occasional private investment fund guy that came in, was short with slick back hair, very loud mouth, would brag and show off one of his 17M personal banking account. Then there were the very quiet, calculated guys. It seemed that they were operating on some formula I never cared to figure out. They’d avoid big hands for the most part. Some form of grind I didn’t have the discipline for. I also didn’t have the discipline to handle large swings at those stakes, nor the bankroll, or desire to play lower stakes to build a bankroll.

burntoutfire

You lost $50k in a 25/50 online game? Was the pot 2000 BBs??? I've never even heard of anything close to it.

ramoz

Casino, there was no max. A lot of players sitting on 100k+. There were far bigger hands than this one.

bloodyplonker22

> Probably the one biggest misconception people have is that pros have sick reading abilities since TV likes to emphasize staredowns, when the actual single biggest skill long term pros have is the ability to lose hand after hand for hours and still play their best game.

This is absolutely true, but let me add to this: live pros like Phil Ivey, who have played the game in person virtually their entire lives, do have sick reading ability. However, it mostly works against amateurs. Actual pros know how to hide their tells better. If you take an online pro who has been clicking at the computer all day on multiple tables, that pro will not have anywhere close to the reading ability of Phil Ivey.

dkjaudyeqooe

I think old school pros like Ivey and Dwan are very good at reading the entire table. Dwan emphatically and correctly states who had the best hand against his bluff [1]. Ivey emphatically and correctly basically calls a guy a liar at the end of a hand over what the guy said he folded [2]. They have amazing "situational awareness".

[1] https://youtu.be/dS_uv88YuPs?t=179 [2] https://youtu.be/IN1bW4jo4i4?t=72

MaxfordAndSons

Indeed, but it should be made clear that these kinds of reads aren't necessarily (or likely) coming from physical appearances alone - there's all sorts of metadata to consider. For example, in the Dwan hand, the timing of Eastgate's call on the flop tipped his relative hand strength, showing that he had a very strong hand but also capping his range. A hand weaker than trips would have deliberated between folding and calling, and a hand stronger than trips with a weak kicker would have likely paused to consider raising. Of course, savvy players can fake these timing tells, but Dwan also knew that Eastgate was out of his depth (a tournament player making a charity appearance on a televised cash game because he'd just won the WSOPME and had yolo money), so he was unlikely to be doing this - more situational metadata.

sillysaurusx

So how much have you won and lost in total? I don’t have any knowledge of poker, so I don’t know how to ballpark this. And the impersonal questions are boring after seven lifetimes, so I thought I’d ask.

It would be interesting to see a GitHub style graph of poker activity, with won and lost corresponding to added lines and removed lines.

splonk

6 figures. Turns out there's a lot more (and less risky) money in software development than there is in poker. I gave up the professional poker life pretty early and actually made quite a lot more as a talented "amateur" with a Silicon Valley software dev salary that took away the risk of ruin constraints I had a pro. Plus I didn't really enjoy playing professionally anyways - it's pretty mindnumbingly boring to play the number of hands you need to in order to get the EV/variance tradeoff to some acceptable point, at least for me.

My highest and lowest sessions were in the 5 figure range (cash game player, not tournament), so I was never up in the really nosebleed stakes, but I was high enough up that a lot of the nosebleed players would play in my games. Some of them have an impressive ability to play their A-game even when switching between games an order of magnitude in stakes apart. Some of them, uh, don't. People you've seen on TV are overwhelmingly more likely to be in the second category and were generally welcome in my games, at least for monetary reasons. (There was an old screenshot of Phil Hellmuth sitting at a full 9- or 10-handed 300/600 limit hold'em table online. He's out of chips and every other player is sitting out, waiting for him to reload. I have a pretty similar live story about him.)

magixx

I imagine that for pro players to play at their best can be taxing and wouldn't be surprised if some just wanted to play less costly games for more what they could deem to be more fun.

VirusNewbie

Are you Paul P?

vincentmarle

> The natural portion of your hands to bluff with is the absolute worst ones - you don't want to bluff with your middling hands because you have some small chance of just winning a showdown when it checks around.

I don't understand this logic, can you elaborate a bit more? What do you suggest to do with middling hands then?

splonk

One very simplified model is that on the river, all your hands fall into certain buckets, from strongest to weakest. (I'm ignoring a ton of nuance here and there are actually many more buckets in a real game)

- Worth betting, because you have the best hand

- Worth calling, because your hand is good enough to beat a bluff (and maybe some value bets as well).

- Intending to fold, because your hand is bad, but maybe it's good enough to win if the opponent's hand is worse and they don't bluff.

- Worth bluffing, because your hand is so bad it can't win otherwise.

The middling hands would literally be the middle two buckets in this example. Call with the better ones, fold with the worse ones. (To complicate this more, in a real world situation the worst part of the "intending to fold" bucket might become a "planning to bluff-raise" bucket, due to similar logic as to why you bluff with your worst hands.)

bluecalm

Yeah but that only works in the very simplified model. In the real game you pretty much always choose bluffing hands based on card removal (the ones that makes your opponent holding a calling hand less likely).

jliptzin

I think he’s saying you don’t want to “waste” a bluff on a hand you might end up winning anyway just by checking it down.

tziki

> What do you suggest to do with middling hands then?

You try to get to a showdown. With middling hands there's a chance the opponent has a worse hand. It's when your hand is so bad your opponent has you almost certainly beat that you get to the bluffing territory, as that's the only way for you to win.

null

[deleted]

null

[deleted]

DennisP

Interesting, I read some Sklansky books about 15 years ago and he talked about semibluffs, where you're partly bluffing but you've also got some chance to win. Sounds like strategy has evolved into doing the exact opposite.

GoatOfAplomb

The distinction is that semi-bluffs are for when there are still cards to be dealt.

beardedetim

Would we call that thin value betting on the river?

DennisP

Ahh, ok thanks.

EGreg

Good to see poker theory approaching the way I naturally thought when I played on PokerStars (before Chuck Schumer and co shut it all down for us).

People used to make fun of me for playing limits and sucking out on people. And no limit I would have some crazy strategies to fake out a table and make them anger-bet into my connected straight flush draws, when I was “representing” a two pair etc.

On large tables, often you want many people to stay in, until the river, because then the pot is big enough that multiple people will go all-in. You just have to be very sure you probably have the best hand (such as a flush that you were drawing for). I woul actually “goad” people with raising small amounts, whenever I had suited connectors for example, and if I hit a draw or the nuts on the flop, I would act as if I am “protecting” a top pair or so. On the other hand, when you are on a draw and if you want to see the next hand on the turn, you have to bet the flop much harder, because then it signals to most “basic” people that you’re “protecting” a hand — therefore gaining you an informational advantage when you do hit — as well as the “intimidation” factor that lets you “check” the turn to see the river — because they often think you’re checkraising them and also check, giving you another free card. You pay more on the flop but have two free cards after. Also If you don’t hit your draw, sometimes they didn’t hit either, and you can take it down with an aggressive bet since you were “representing” that you flopped a strong hand. To summarize:

  1) play large tables
  2) play suited connectors, or suited Ace-something
  3) limp in or if you are late-position, double the bet to make people call and grow he pot and be more excited to bet later, you also represent a reasonably high pair
  4) after a flop - if you are one card away from hitting a very strong draw (eg ace high flush) then bet hard on the flop — late position is always best for this… otherwise you have to bet less to keep people in
  5) regardless of whether the turn card makes your draw, check on the turn … you may have to call any bet if multiple people stay in, and play the odds
  6) if you made the nuts on the river, checkraise all in (since you represented that you are protecting a flooped hand).
  7) there were also timing factors, people get more annoyed if you take too long. And you can play multiple tables online until you hit these kinds of starting hands
People said I was playing wrong but I intuitively felt like this would give a lot higher payoffs than just straight play with no bluffs.

PS: Against a really intelligent table where people don’t churn, you’d have to leave after a couple big wins and do the same thing elsewhere. The whole idea in poker is to take advantage of most people just executing some common strategy or mindset.

MaxfordAndSons

> People said I was playing wrong but I intuitively felt like this would give a lot higher payoffs than just straight play with no bluffs.

No winning players nowadays would advocate for a strategy that doesn't involve bluffing.

Not to be harsh, but just to point it out for any novices who might be seduced by the simplicity of what you propose: the "strategy" you outline is simply nonsense and would not be winning in the long run even vs a fairly weak field. Maybe vs the level of play in the pre-solver world it did okay (though I doubt it would win except vs the very weakest fields), but today it would simply be burning money, even at microstakes.

Some of the heuristics you describe work in some situations, but you make no mention of accounting for other players positions, other players ranges, or the cards on the board; it is simply based on your hand and your perception of the player populations tendencies (which have changed dramatically since the pre-solver days). Ignoring the majority of the publicly available information in a given hand, in this game of incomplete information, is a grave strategic mistake.

jliptzin

Cool tip thx. How did you do with this strategy?

bluecalm

I am the solver programmer mentioned in the article. I think it came out pretty decently. Maybe not all the details are right (I was late answering fact checking email) but I think it's a decent read and the author has done a good job!

im3w1l

How perfect is the play that piosolver calculates? Do you make any simplifying assumptions?

I noticed in the article that piosolver can incorporate a mixed strategy with multiple bet sizes. Pretty neat, is that a recent feature? As I rememember it, a few years back people would discuss what the optimal bet size was in various situations with the implicit assumption that there is just one.

bluecalm

Adding multiple betsizes was one of the first features we added in 2015 - so not very recent, no. The one true correct bet size is a seducing idea but that just doesn't exist for most situations. You can try finding the best one with the assumption you can only use one but it's not the best idea in my opinion. Main reasons are that that's the sizing most people will be most ready to play against and EV difference between possible bet sizes is minimal.

What happened in chess is now happening in poker: at first people were all about the best theoretical move and then gradual shift to "not necessary the best just sound enough but less likely to be analyzed by the opponents" started happening.

As to the first part of your question: the solutions are pretty much perfect. The only assumption is about possible bet sizes. As to accuracy we measure it in theoretical exploitability (how much a perfect adversary who knows our strategy exactly could win against us). You can easily go so low that even a theoretical perfect adversary wouldn't get close to beating the rake vs the solution, even at high stakes.

null

[deleted]

michalsustr

Congrats on putting this together, it's not easy! I'm curious, what papers have you used for your implementation? I know there are a couple of reimplementations of DeepStack out there.

Disclaimer: I'm doing phd in this area, generalizing to harder games than poker.

bluecalm

The solver was purely based on our (mine and my partner's) ideas. We tried reading some papers to find ideas to improve our solver but we had found that they are not very helpful - written in opaque language, using wrong tree representation and not focusing on practical implementation aspect. Maybe things changed since then but I haven't found anything useful at all in published papers about poker with the exception of two page one by Oskari Tammelin describing CFR+ algorithm (which we haven't used in our software because of its significant memory requirements).

We considered it very unlucky that the science paper was published as we thought more people will implement solvers then (ours was ready in mid 2014 and we released and a working solver with a GUI in early 2015, 3 months after the Limit Holdem paper). As it turned out though it didn't really matter much.

I had an idea to code a poker solver around 2008. Unfortunately I was yet to learn to code back then. The biggest challenge was to overcome implementation issues. Once I got the crucial part to be fast enough I knew the solver is possible. I think not having much background in the field allowed me to come up with more natural (and better) tree representation and memory efficient (even if not the fastest) algorithm as I was thinking more as poker player than a computer scientist back then.

michalsustr

Interesting! We have very recently wrote a paper on the various tree representations [1]. Maybe it is the same distinction you found on your own? I judge this based on the screenshots on your website. Public state trees are amenable to nice factorization over the hands, and indeed most implementations use it, but formally they write they use the history tree notation.

[1] https://arxiv.org/abs/2112.10890

ewhauser421

Congrats - PIOSolver is an amazing program. What are the future plans for it? It would be great if you didn't need a PhD to set it up though. I keep wondering if I'm actually training myself using the right solutions or not based on the setup :)

bluecalm

The plans are to start turning it into real company which hopefully means easier to use software, more learning material and better communication. Till now it was mostly two guys working from home and overwhelmed by it all.

muldvarp

2016 you said "It's a very small niche and getting less popular but we are still doing very well." Did this turn out to be true or has PioSOLVER become more popular again?

bluecalm

I think poker in general is slowly dwindling but somehow the solver in fact got more popular over the years. I think the pandemics helped us a bit and maybe general switch to nie analytical play. To be honest I don't understand it very well. We have never done any marketing, our website is terrible (put together on a free Shopify template and never really updated) but somehow it keeps getting more traction.

sillysaurusx

How did you end up working on this?

It’s an interesting history.

bluecalm

I was always interested in math and encountered game theory concepts early in life. It's a perfect mix of being not a very good programmer, not a top math mind, not an exceptional poker player but still being all those things at the right time and place.

sillysaurusx

I really like that you got to do this, and that you turned your weaknesses into such a great strength. Thank you for showing that you don’t need to be exceptional in a specific area to do exceptional work. Best of luck with whatever you decide to pursue next.

dabei

Great job! What makes PioSolver much more popular than the other solvers?

tromp

I like how the article gives examples of the sort of entropy players draw on to make their random choices:

> Koon will often randomly select which of the solver’s tactics to employ in a given hand. He’ll glance down at the second hand on his watch, or at a poker chip to note the orientation of the casino logo as if it were a clock face, in order to generate a percentage between 1 and 100.

splonk

The watch face is the classic one, the chip orientation is new to me. I like it, it's less obvious that you're randomizing something, and given how much poker players like to fidget with their chips the orientation is probably reasonably random.

Another classic is to use the suit of your cards, but that has problems with being correlated with the state of the board. It works fine preflop.

jstx1

But it hasn't conquered it! I kept searching the article for some new recent breakthrough that I've missed but it's not there. Yes, solvers like Pio have been around for years and limit holdem has been essentially solved for a while but nobody plays limit holdem anyway.

The two most popular games (no-limit Texas holdem and pot-limit Omaha) are still unsolved.

noambrown

Bots are superhuman in no-limit Texas hold'em. Libratus beat top humans in two-player in 2017 and Pluribus beat top humans in six-player in 2019: https://www.science.org/doi/abs/10.1126/science.aao1733 https://www.science.org/doi/abs/10.1126/science.aay2400

It's shocking that the reporter didn't mention these results or anything else more recent than 2015.

hooande

Have to say, Pluribus beating top humans for 10,000 hands isn't the same thing as being super human. It's just too small of a sample to make that claim.

Further, thousands of the hands that Pluribus played against the human pros are available online in an easy to parse format [0]. I've analyzed them. Pluribus has multiple obvious deficiencies in its play that I can describe in detail.

It seems like it's very difficult to set up any kind of proper repeatable and controlled experiment involving something as random as poker. Personally, I would be much more convinced if Pluribus played against real humans online and was highly profitable over a period of several months. This violates the terms of service / rules of many online poker sites, but it seems like the most definitive way to claim terms like "solved" or "superhuman"

[0] http://kevinwang.us/lets-analyze-pluribuss-hands/

noambrown

Normally 10,000 hands would be too small a sample size but we used variance-reduction techniques to reduce the luck factor. Think things like all-in EV but much more powerful. It's described in the paper.

lawrenceyan

Noam is being too humble, but he’s one of the primary creators of Libratus/Pluribus as an fyi.

bko

The article was fascinating but honestly read like an ad for PioSolver

bluecalm

The journalist who contacted me told me he did so because the software keeps coming up when he talked to pro players. While it's certainly not one to advance the science the most (talk to Noam Brown if you want that), not the fastest (talk to Oskari Tammelin about that), it's still very popular and the first to get big following. It changed the game and got into the online poker culture. I am quite proud of that and I think it's deserved to be mentioned a lot in an article about how computers changed poker.

howdydoo

I know that PioSolver is not a "poker AI" per se, but the article seems to say it can tell you what to do based on the table situation. Has anyone tried pitting pro players against PioSolver?

jeffreyrogers

PioSolver requires putting in the hand range of the opponent, so the quality of PioSolver's solution is largely down to how accurate the guess at that hand range is. But if a pro knows he is playing against PioSolver configured with a certain hand range he can just change his strategy to adapt. In theory though if PioSolver knows the correct hand range then it shouldn't be possible to any better than tie given enough hands.

evrydayhustling

There is still a lively academic community and major progress! Check out CMU's no limit results [1]. (I realize articles like this have to pick some angles to make it interesting, but it was weird to see only dated research mentioned.)

But if you are rooting against the machines, don't worry: it is almost certainly impossible to calculate a full equilibrium policy for no limit multiplayer, so we will instead be debating over the virtues of various types of imperfection for a long time. And even if an Oracle gave us convenient access to equilibrium strategy, it would still not be the optimum at a table full of imperfect players. Your poker game is safe for a while!

[1] https://www.nature.com/articles/d41586-019-02156-9

bluecalm

It doesn't even matter if you can calculate multiplayer equilibrium. It's not the solution the same way it is in heads-up. You can still lose if you employ the equilibrium in multiplayer unlike in HU.

noambrown

That's not true in practice for poker. Pluribus showed that if you run CFR in multiplayer poker you get a solution that works great in practice. Multiple equilibria are certainly a theoretical issue for many games, but poker conveniently isn't one of them.

tziki

No limit holdem has been essentially solved. Pluribus & co not withstanding, you just haven't heard of it because the people who have solved it are busy printing money in online poker (yes, I know they try to detect bots, and no, they can't detect them all). With stakes this high, academic progress lags the 'actual' state-of-the-art by years.

throwaway_Aef8

IIRC it's "solved" for heads up but not really multiway like 3+ to the flop. I believe in a recent Bart Hanson Youtube he points out that mutltiway is not solved.

throwaway_Aef8

oh wait, Google showed me this:

> Machines have raised the stakes once again. A superhuman poker-playing bot called Pluribus has beaten top human professionals at six-player no-limit Texas hold’em poker, the most popular variant of the game. It is the first time that an artificial-intelligence (AI) program has beaten elite human players at a game with more than two players

https://www.nature.com/articles/d41586-019-02156-9#ref-CR1

jeffreyrogers

It's not solved for multiway in the sense that the optimal move in each situation isn't known, but there are AIs like Pluribus that have superhuman performance.

protoax

Hasn't Facebook's Pluribus come pretty close to "solving" no-limit holdem? I don't know a lot about poker so I can't really assess their claim's validity.

ironrabbit

Not even mentioning Pluribus is just bad journalism

kuboble

The article is mostly about a change in the industry and adaptation of solver strategies by human players of all levels.

Pluribus had approximately zero impact on that.

vvarp

Worth to note that PioSolver isn’t the only tool of its kind - there’s also Jesolver (http://jeskola.net/jesolver_beta/), which is speed optimized variant, and cloud-based Deepsolver (https://deepsolver.com) which leverages neural nets to do the job

achenatx

<<But for Doug Polk, who largely retired from poker in 2017 after winning tens of millions of dollars, the change solvers have wrought is more existential. “I feel like it kind of killed the soul of the game,” Polk says, changing poker “from who can be the most creative problem-solver to who can memorize the most stuff and apply it.”>>

I wrote probably the first online texas holdem game, played on IRC, around 1994. Back then in rec.gambling blackjack teams were forming and instead of joining a blackjack team I started to play poker, because it was more fun than counting cards. I wrote the game so I could play against other people and was immediately a winning player in low limit games of the time in lake charles and vegas. However even in the 90s, it was about statistics.

I do regret not allowing people to play for money (people were constantly asking).

Even back then the primitive bots were mostly killing human players.

mikemcd

I'm pretty sure it was Vanessa Selbst who said that, not Doug Polk

smallerfish

I think there's room for a "battlebots meets online poker" product. Imagine buying your bot a real money budget and setting it loose in a tournament to combat other bots.

Wingman4l7

This has happened before, but only once that I know of. It was hosted by the creator of the (apparently now defunct) WinHoldEm "poker botting" software, Ray E. Bornert II. It never ran again, only a small handful of the more experienced bot authors attended, and turnout was poor, with the highest max buy in at $100 (http://robopoker.blogspot.com/2007/12/pokerbot-world-champio...). It was called the 2007 Poker Bot World Championship (PBWC).

The online poker community is extremely hostile towards poker bot software in general; you would be very hard-pressed to find an existing poker website that would be willing to encourage and host such tournaments.

bloodyplonker22

Online poker has already been evolving in this direction. Any serious player who plays online will use player assistance tools (such as PIOsolver). Top level poker AI such as Pluribus has already proven that it can hold its own and beat the best online players such as linuslove. It is well known that online poker sites are full of poker bots now. There are programs to help users try to tell if they are playing against a bot by seeing how fast it reacts on each hand, for example if it always takes a fixed amount of time on each hand or if there is a pattern.

smallerfish

Yeah, that was the evolution of my thought process. Online poker is already full of bots, which will get more and more sophisticated over time and defeat bot detection hurdles -- so why not embrace it and have bot-only zones? You could have entire tournaments play out in less than a minute if all participants were software.

bloodyplonker22

because humans are the worst players in most cases now. Especially at the lower limit tables. Obviously, you want to play the worst players to make the most money.

cortesoft

I would expect bots to add random delays? They could even determine real player delay distributions and emulate that.

bloodyplonker22

yes, that is the obvious step that bot makers have taken. When bots were barely at the online poker scene, nobody cared to even check. Of course, there are still other ways to check for bots such as a user playing for an unreasonable amount of time or an extraordinary amount of tables, or simply not answering to chat.

conductr

I'm actually working on this as a side project. I've been dragging my feet as I wasn't sure anyone would want to "code to play" and had no real marketing plan.

It's a fun project though, especially trying to design in a way that proves I'm not manipulating the deck behind the scenes.

Email in profile if anyone is up for being an alpha player.

MaxfordAndSons

I've been thinking about a similar project. Do you intend to have it be only bots? I think it would be interesting to also allow human players to play against the bots (perhaps incentivized by getting preferential rakeback deals compared to the Bot players?). Many players fret about bots on other sites; it would be cool to enable those of us who want the challenge play them knowingly and with a slight financial handicap :)

smallerfish

I sketched out plans for the idea a while back (and have been too busy to do anything about it.) I'd have bot only games, and mixed games. I'd also have tools built into the game to let you pull up full stats on any player (there's add on software right now for major platforms that gives asynchronous advantage to players who use it, so to mitigate that it'd need to be adopted.) I think if the stats were good enough it'd be relatively easy to spot bots, which might make human-only tables possible also. The system would be punitive against bots which didn't declare themselves as such, but it would fully supportive of bots being part of the game on the online sphere.

conductr

Both could work. I’m particularly interested in the all bot variety because the pace can be very accelerated. For example, an idea I’m kicking around is having timeout be under 1 second.

I see it like HFT. You may have multiple bots running multiple playing strategies and an entire tournament could occur in minutes/hours. A large weekly tournament might even be like a sporting event for hackers.

The human vs bot scenario you’re interested in would be possible on the same platform with a longer timeout period.

smallerfish

I think the real money aspect would obviously get you some traction, but that's difficult to deal with (since you might run into online gambling laws, although you could very credibly argue that this isn't really any different from algorithmic trading). Maybe you could get in on the hype train and launch it as a crypto product?

tgarv

There's a "Pokerbots" class/competition at MIT (during January aka Independent Activities Period), along with a robotics competition and a real time strategy video game bot competition. I did both the robotics and poker ones, which were a ton of fun!

russellbeattie

But hasn't the problem been solved? Two bots would play perfectly and walk away with roughly 50% of the pot.

This article is about how humans have been memorizing the variations - akin to chess openings - in order to play as perfectly as possible.

smallerfish

No, because you have the randomness of the deck, and you have bluffing. Bots could be tuned to be aggressive or defensive, or switch strategies depending on conditions in the game.

Nextgrid

Isn't that just algorithmic trading?

meetups323

Classic games tend to be more fun than New York horoscopes. For me at least, there's something fundamentally profound about solving a logical puzzle better than anyone in history, that just isn't there in guessing which way some lines are going to go.

soferio

What teaching or training tools are out there for a very average player at no limit Texas hold’em who just wants to get at bit better to a respectable level at a modest time commitment , and does not need to be a pro-level player?

tunesmith

Yeah, really I just want an open-source poker solitaire that asks me what I should do and then shows me the odds after I answer.

mdavis6890

The odds are a small part of what you have to consider.

If you want to quickly improve to a proficient level there's no quicker way than to read Theory of Poker and Hold 'em for Advanced Players. Both of these books focus primarily on limit poker, but the concepts are critical for no limit as well. And you'll realize there's a lot more strategy and nuance in limit poker than you thought.

wallawe

ask and you shall receive: https://gtowizard.com/ (go to the "practice" section for what you are looking for)

jstx1

Books (there's very few good ones)

Training videos

1-on-1 coaching

PioSolver (and other similar software)

For any of these to stick, you need to spend some amount of time studying by yourself; just consuming learning material and playing isn't enough.

Reflecticon

Could you recommend a couple of books for a start?

jdm386

Modern Poker Theory as mentioned by the other reply is good. I'd also suggest Play Optimal Poker by Andrew Brokos. The former outlines a lot of strategy in the game and talks a little about game theory and its implications. The latter gives less direct poker advice but uses "toy games" that share some similarities with poker but are simplified to demonstrate some of the implications that game theory has on poker. It does contain some direct poker advice but it's more of a book to teach you how to think properly about how game theory applies to poker and how you can use that to study with something like a solver and actually understand what you're seeing. I would suggest you pick up both, I think that the Brokos book assumes you know certain poker terms and doesn't contain a glossary whereas modern poker theory is almost like a textbook and has a section where it defines all of the terms it uses. Modern poker theory will have more actionable advice but the Brokos book is excellent to teach you how to think about game theory and requires more self-reflection and has more questions to the reader.

Applications of no limit by Janda or mathematics of poker are recommended by a different reply here. I would caution that these are extremely academic and extremely dense texts that would be a very tough read for a newer player. Mathematics is less practical and more of a math book than a poker book and applications is Janda showing how to work out solver-like solutions before solvers existed and also contains a lot of math. I think the other two above are more practical and aren't going to lead you to put the book down a 10th of the way through.

jstx1

The Mathematics of Poker, by Chen & Ankenman - this one doesn't age because it's really fundamental if you want to understand things from the ground up. But it won't seem that practical at first.

Applications of No-Limit Hold 'em, by Matthew Janda - this one goes into specifics about how to think on each street, how to build a good strategy etc. Some of the details will be different from the latest theory because the book was released in 2013 but it's still a very solid read if you want to level up.

MaxfordAndSons

Modern Poker Theory by Michael Acevedo is considered a contemporary gold standard.

alasdair_

If you like video content, the Daniel Negranu Masterclass is extremely well done. Phil Ivey also has content on that site but I found it much less comprehensive.

After that, perhaps watch some Doug Polk on Youtube to see how he uses the concept of putting people on ranges of hands and what his thinking is in a specific spot.

speedyapoc

Training sites are probably going to be your best bet to improve quickly. I see Run it Once was mentioned. I've personally used Upswing Poker in the past and they had some great resources. Bart Hanson's Crush Live Poker videos are great too but I can't vouch for the training course.

treme

www.runitonce.com 25$/mo essential membership will give you enough from beginner to semi pro easily.

knicholes

Lol, awesome. I built the Stripe integration for this site in 2013. The first day we launched, we landed over $100k in subscriptions.

marcodiego

Is there any remaining logic game which AI hasn't become superhuman? IIRC Monte Carlo tree search has done wonders in this area.

isaacg

As far as I know, in Magic: the Gathering, the best bots are far worse than most players. Part of the difficulty is that the rules are so complicated that there are only a couple of complete rules implementations. Beyond that, it's an imperfect information game with far more actions per game than poker, so optimal-solver techniques haven't seen success.

dwohnitmok

I think this is purely a resource issue, e.g. if Google Brain decided to make an MtG bot I would be fairly confident it would be superhuman. Even real time strategy games like Starcraft are looking like they're on the cusp of superhuman bots (Alphastar was competitive as Protoss against elite players, but did not consistently beat them).

archontes

I doubt very highly it would be able to sit at a game of commander. Four players with 100 card singleton decks would be an absolutely enormous space to operate in.

runnerup

Alphastar also didn't play with the same limitations that a human has. Even after removing its ability to see the entire map and finally forcing it to scroll around, alphastar never misclicks (so its APM==EPM) and can still blast nearly unlimited APM for short bursts as long as its "average APM" over an x-second period matched human's APM.

I believe Alphastar would generate more interesting strategies if we limited alphastar to a bit below human APM and forced it to emulate USB K+M to click (instead of using an API, which it currently does) and adding a progressively increasing random fuzzing layer against its inputs so that as it clicks faster the precision/accuracy goes down.

By "interesting strategies" I mean strategies that humans could learn to adopt. Currently its main strategy is "perfectly juggle stalkers" which is a neat party trick, but that particular strategy is about as interesting to me as 2011-era SC AI[0]. Obviously how it arrived at that strategy is quite interesting, but the style of play is not relevant to humans, and may in fact even get beaten by hardcoded AI's.

I'm also very curious what Alphastar could come up with if it were truly unsupervised learning. AIUI, the first many rounds of training were supervised based on high level human replays -- so it would have gotten stuck in a local minima near what has already been invented by humans.

This may be relevant if Microsoft reboots Blizzard's IP. I would love to have an alphastar in SC3 to play against off-line, or have as a teammate, archon mode, etc. I think all RTS' are kind of "archon mode with AI teammate" already. The AI currently handles unit pathing, selection of units to attack, etc. With an alphastar powering the internal AI instead, more tactics/micro can be offloaded to AI and allow humans to focus more on strategy. That seems like it would be super cool.

Examples: "Here AI, I made two drop ships of marines. Take these to the main base and find an optimal place to drop them. If you encounter strong resistance or lots of static defense, just leave and come back home"

"Here AI, use these two drop ships of marines to distract while I use the main army to push the left flank. Take them into the main, natural, or 4th base -- goal is to keep them alive for as long as possible. Focus on critical infrastructure/workers where possible but mostly just keep them alive and moving around to distract the opponent."

0: Automaton 2000 AI perfectly controls 50-supply zerglings (2.5k mineral) vs. 60-supply (3k mineral, 2.5k gas) siege tanks: https://www.youtube.com/watch?v=IKVFZ28ybQs

jdmoreira

The search tree is huge in mtg. It has to be the largest of any game. You can take actions all the time. There are triggers all the time, you can stack your actions on top of your opponent actions. Huge space really.

And then of course it's also imperfect information both in the sense of your opponent hand but also his deck. The cardpool is also very large for some formats.

I actually don't think it's solvable just by throwing MCTS at it with todays hardware but would love to know more about this, if someone else has more insight please reply.

EDIT: Oh and there is also the meta-game / deck building aspect. If you are going to win a tournament you have to have favorable matchups against most players in the room.

perfect_wave

An MTG limited player named Ryan Saxe created an AI that drafts and builds deck to a highly successfully degree: https://github.com/RyanSaxe/mtg

He was able to reach Mythic, the highest ranking tier on Magic Arena. Of course this is a different problem to actually playing the game (and probably significantly easier). That being said, this is one guy doing it as a side project with restricted resources.

I imagine that MTG could played quite successfully by an AI if someone where to dedicate the resources. Imo much of the difficulty is in laying the groundwork. Large amounts of data don't exist publicly and laying the framework for a bot to play itself would be quite difficult (and then the computation costs would be extremely expensive).

cortesoft

Wouldn’t the fact that the game is constantly changing, with new cards being added and old ones being removed, also make it harder to solve?

jdmoreira

That's harder on humans than on computers. Computers have perfect information on what the legal cardpool is at game they are playing

golergka

Have people tried using GPT-3 for this?

splonk

Here's a recent paper from DeepMind about their recent efforts towards generalized game solving (caveat: I haven't actually read it yet).

https://arxiv.org/pdf/2112.03178.pdf

Abstract:

"Games have a long history of serving as a benchmark for progress in artificial intelligence. Recently, approaches using search and learning have shown strong performance across a set of perfect information games, and approaches using game-theoretic reasoning and learning have shown strong performance for specific imperfect information poker variants. We introduce Player of Games, a general-purpose algorithm that unifies previous approaches, combining guided search, self-play learning, and game-theoretic reasoning. Player of Games is the first algorithm to achieve strong empirical performance in large perfect and imperfect information games — an important step towards truly general algorithms for arbitrary environments. We prove that Player of Games is sound, converging to perfect play as available computation time and approximation capacity increases. Player of Games reaches strong performance in chess and Go, beats the strongest openly available agent in heads-up no-limit Texas hold’em poker (Slumbot), and defeats the state-of-the-art agent in Scotland Yard, an imperfect information game that illustrates the value of guided search, learning, and game-theoretic reasoning"

btheshoe

Imperfect information games that require agents to cooperate with each other haven't been solved yet. (multiagent cooperation is still a hard problem) Hanabi is a big(ish) research area. Contract bridge has had some work done on it.

HWR_14

I'm surprised that Hanabi is such a huge research area. Compared to bridge or poker, it seems like a far more simple game.

ska

> I'm surprised ... it seems like a far more simple game.

This probably shouldn't surprise you. If you are researching how to attack a new/difficult class of problems, typically you look for the simplest versions of that class first.

pclmulqdq

Contract Bridge is still untouched. I'm not sure if it's because of lack of resources, but it has a lot of difficult elements: cooperation, huge search trees due to imperfect information, and information theoretic aspects of bidding.

RivieraKid

For some time I've been pondering about creating a multiplayer poker AI that would (1) play reasonably well but not at a superhuman level (2) not require expensive hardware.

Frustratingly, this seems to be an unsolved problem. Pluribus is only 6-max, requires fixed starting stack sizes and requires a lot of hardware.

MaxfordAndSons

I mean, it may not have been publicly solved, but there are definitely bots out there in the wild that are beating human fields on online poker sites.

But also I think Pokersnowie is a bot made with more traditional ML methods (i.e. training on hand histories rather than solving an entire game tree with CFR), which basically fits the description.

RivieraKid

Is there enough public info to reimplement PokerSnowie? I only know it uses neural nets in some way.

torginus

Does this mean that online poker is effectively over?

jstx1

It's kind of dead-ish because there aren't big edges and most people who play online are decent enough. Online poker has been deteriorating for years (with maybe a small uptick due to the pandemic). But this article could have been written a few years ago, there's nothing new in it.

redisman

Ever since the US outlawed it it’s been dead-ish

paxys

Chess has been "solved" by computers for a long time now, yet the online Chess community is bigger than ever. I don't see why it would be any different for Poker.

fdgsdfogijq

Big difference with poker is that its all about money, the game doesnt work if people arent risking something. And if computers are superior, the game breaks down.

robrenaud

You are worried that people will cheat in online poker, or what?

For poker, people can still play face to face, and it being a game of mixed strategies means it's probably a lot less important that poker bots are better than people than in chess. Top chess players use the computers to set up preparation bombs on their opponents, with rebuttals to moves for specific positions created by the computers. That doesn't work in a game of imperfect information and mixed strategies.

throwaway287391

> Big difference with poker is that its all about money, the game doesnt work if people arent risking something.

I don't see this, I used to really enjoy online poker tournaments for play-money. It's a fun and strategic game in and of itself with nothing more than minor bragging rights on the line, to me at least. I honestly don't see why it should be any different from chess in terms of purely recreational play. (You could say chess is a deeper game, and that's probably true in some ways, but that's not a positive feature for casuals like me with no desire to invest many hours/years into mastering the game.)

rainonmoon

Poker is not really all about money. Plenty of games might only ask you to stake like $5 but the joy and aim of the game is really to outplay the table, not take home $25.

abaldwin99

The difference for poker is it takes a lot more play time to sort players by skill. There's no ELO to climb and thus you don't get to play against more challenging opponents.

e.g. I'll lose 100% of my matches against a Chess GM but against the best poker player in the world I could go all in preflop every hand and still win the match ~20% of the time.

Only way to counter this variance is to play lots and lots of hands (Law of large numbers) It could take weeks/months in a single matchup to determine the better poker player.

In terms of playing for real money, in general you'll find more challenging competition at higher stakes. But then there's an incentive for bots.

vanusa

Unlike with chess, in the poker world there's a huge financial incentive to run bots (or provide bot assistance to live players).

null

[deleted]