Game Theory and Poker

pkr_ennis

Howdy
I've recently become interested in Game Theory in relation to poker, and was wondering if anyone here studies this or something close to it. I've been watching videos on running gambit, and would be interested in some discussion on the topic.
C

gerry87

pkr_ennis wrote: »

Howdy
I've recently become interested in Game Theory in relation to poker, and was wondering if anyone here studies this or something close to it. I've been watching videos on running gambit, and would be interested in some discussion on the topic.
C

What sort of stuff are you interested in, ring games/tournaments? You probably came across him already, but if not look for books by David Slansky.

pkr_ennis

I know David Sklansky thanks.

There really aren't any hard facts and figures in his books. I was looking to discuss actual game trees and how the results there relate to poker startegy.
FYI there's a good book on the subject "Mathematics of poker" bill chen and another guy I forgot the name of.

fergalr

pkr_ennis wrote: »

I know David Sklansky thanks.

There really aren't any hard facts and figures in his books. I was looking to discuss actual game trees and how the results there relate to poker startegy.
FYI there's a good book on the subject "Mathematics of poker" bill chen and another guy I forgot the name of.

Hey there,

I'd be interested in this subject too.

I'm not much of a poker player. I read Dan Harringtons books, and Slansky, but I've only played a little casually.

I was reading a book recently called 'the logic of life'. Its a popular science economics book by Tim Harford. In it he talks about Chris ('Jesus') Ferguson using game theory to conduct analysis (monte carlo based? - he doesnt say) of poker playing technique, which he used to discover some insights about flaws in pro level play (afair something about the pros raising too much post flop with good hands to push out opposition, avoiding showdown).

I'd be very curious to know what sort of a form this analysis would have taken. How would he go about doing this? Casually googling for it, I find plenty of people saying that Ferguson knows lots about game theory, but little in the way of detail! I'd almost think it was just part of the mystique, if I hadn't read it in a reputable book.

I was interested in games when I did my CS undergrad, and just did a course on ai for games recently (minimax, AB, various tree search strategies). We had some lectures on poker, but they were didn't go into huge detail, and wasn't really GT based. (The slides for the lecture are here: http://www.csi.ucd.ie/Staff/acater/2009/comp30260/30260_L16.pdf I'm going to hope that as they are publicly accessible the lecturer (whos a decent guy) won't mind me linking).

I don't know a huge amount of game theory (read a bit of a book by Gibbons) but would like to know more about what sort of approach someone would use for poker.


On a separate, but related, topic, I was at a talk by a guy called Bruce Bueno de Mesquita where he talked about applying game theory to make real world political predictions. It didn't go into technical detail on the models he used. I'd be very interested if anyone knew of details or resources on how to go about applying game theory techniques to real world problems - implementation details on the techniques used.

pkr_ennis

fergalr wrote: »

I was reading a book recently called 'the logic of life'. Its a popular science economics book by Tim Harford. In it he talks about Chris ('Jesus') Ferguson using game theory to conduct analysis (monte carlo based? - he doesnt say) of poker playing technique, which he used to discover some insights about flaws in pro level play (afair something about the pros raising too much post flop with good hands to push out opposition, avoiding showdown).

I'd be very curious to know what sort of a form this analysis would have taken. How would he go about doing this? Casually googling for it, I find plenty of people saying that Ferguson knows lots about game theory, but little in the way of detail! I'd almost think it was just part of the mystique, if I hadn't read it in a reputable book.


I don't know a huge amount of game theory (read a bit of a book by Gibbons) but would like to know more about what sort of approach someone would use for poker.

Chris Ferguson is one of the best poker players in the world. He was World Champion in 1999 I think. He uses GT for all his decisions. He says he never uses 'tells', which are physical givaways to the strength of a players hand, that had become a large part of the best pro's strategies. They had some Math down but it was mostly psycological based poker, and still is today, even in the biggest public game.

Now everyone is starting to use GT. What they are doing is making models of poker situations and figuring optimal play using Nash's Equilibrium and then skewing the results in relation to a playing style or a particular player's tendencies to create the best possible EV decisions.Seemingly these formula's can take pages of algebra to figure out.

Online poker has many pro players using GT. These are some of the biggest winners in the game. Ferguson said the poker world now, is like 1900 america when everyone was happy with the horse and carrige, but Henry Ford was clincking away in his garage trying to perfect his machine.

He's saying that the fututre of poker is going down this route and I wanna be a part of that. I've recently watched some training videos on using a programe called 'gambit' which it was shown to create optimal solutions to simple games, but I'm not an expert with numbers and am struggling to even construct a tree there. However, I am good at poker strategy.
C

cavedave

Total poker has a chapter on game theory that really only deals with the cuban missile crisis. Still it is worth a read.

RoundTower

here's a simple poker game. If you have studied some basic game theory, you can solve this problem.

we play with a 3-card deck. Ace beats King beats Queen.

We both put in 1 euro to the pot. You act first and your choices are to bet nothing or bet 1 euro.

If you check (bet nothing), your opponent checks too. Whoever has the best hand wins the pot of €2.

Your opponent's choices are to call the bet or fold. If he folds, you win the pot. If he calls, he puts in €1 and whoever has the best hand wins the pot of €4.

What is the equilibrium ("optimal") strategy for this game? How much money do you win (or lose) each hand if both players play the optimal strategy? How, if at all, would you change your strategy if you knew your opponent was not following his optimal strategy? Once you can answer these questions precisely, you can attempt to generalise it to a more complicated poker game.

gerry87

The trouble with the poker/game theory connection is that it starts to break down pretty rapidly when you start to relax assumptions.

Most gt problems assume all agents are rational and have equal/perfect knowledge of game theory.

I remember a discussion over in the poker forum ages ago about one of the game theory problems. The Travellers Dilemmait was essentially this:

You and a friend traveling on an airline, each with a vase (the only two of their type in the world). The airline looses both and comes to each of you separately asking how much it was worth to refund you. The rules were,
1) Max price 100.
2) Min price 2.
3) If you say the same value of your friend you both get that amount.
4) Whoever says the lower price gets that price +1 and whoever says the higher price gets the low price -1.

Game theory has a logical 'right' answer (Nash Equilibrium is 2). But almost nobody in the poker forum answered that, even after taking a few steps along the GT way of thinking most said 99 or 98.

If game theory could work for poker you probably need to have some idea of the utility of your opponents (even just the average player you find at whatever level). Even that changes between hands, like what stage of a tournament they're at.

Edit: Does anyone know how many permutations of actions there are in say a round of limit poker? For the sake of how many branches would be on a decision tree.

fergalr

RoundTower wrote: »

here's a simple poker game. If you have studied some basic game theory, you can solve this problem.

we play with a 3-card deck. Ace beats King beats Queen.

We both put in 1 euro to the pot. You act first and your choices are to bet nothing or bet 1 euro.

If you check (bet nothing), your opponent checks too. Whoever has the best hand wins the pot of €2.

Your opponent's choices are to call the bet or fold. If he folds, you win the pot. If he calls, he puts in €1 and whoever has the best hand wins the pot of €4.

What is the equilibrium ("optimal") strategy for this game? How much money do you win (or lose) each hand if both players play the optimal strategy? How, if at all, would you change your strategy if you knew your opponent was not following his optimal strategy? Once you can answer these questions precisely, you can attempt to generalise it to a more complicated poker game.

Roundtower:
I'd be interested to know how to go about the analysis of this game.
Like I said, I only know what I do about game theory from reading a bit of a book. (the games we saw doing the AI course were of perfect information, so it was mostly about minimax and search)


Could you clarify a few things about the setup?

* I presume players get dealt one card each, and can't see what the remaining card in the deck is.
* Are the players' cards dealt face up or face down?
I'm assuming that each players cards are dealt so that only the player can see them (like the hole cards in holdem)


In your setup, you've removed the second players ability to decide anything in the event that the first player checks.
This would seem to make the payoffs of checking for player 1 to be 1 for an ace, 0 for a king, and -1 for a queen. If the first player can't see their card, I guess that makes the payoff/ev for checking blind to be 0 - fair enough, thats pretty obvious.

Working it out what happens when the first player raises is a little trickier though.
If the first player can see their card, then whether the player raises or checks gives the second player some information. The fact that the first player acts first, but the second player doesn't know what the first players hole card is after the action, while the first player does, makes it a dynamic game of imperfect information.
I'd need to read up on this a little before I'd be able to figure out what the value is. Am I looking to find the perfect bayesian equilibrium here - if its a dynamic game of imperfect info and you are talking about an equilibrium, I presume thats what you mean?

Am I on the right track, or have I seen it as more complex than it is, or did you mean it to be the simpler case where the cards are dealt face up?

fergalr

gerry87 wrote: »

Edit: Does anyone know how many permutations of actions there are in say a round of limit poker? For the sake of how many branches would be on a decision tree.

I think you are asking about the branching factor of a poker game here, right? How many unique choices there are at each 'ply' or move in the game?
I guess that for limit poker, its actually quite small in terms of the actions available to each player. I'd say NL poker could also be suitably 'discretised' into a relatively small number actions at each ply. Certainly compared to something like Go, or even chess.

But I would guess that where the computational complexity would occur in doing some sort of search through the outcomes isn't in the number of actions available to each player at each turn - its in the number of actions available to nature (ie chance) as each extra card comes down - that's probably what increases the complexity a lot.

In other words, a players may have only a small number of actions each (say, check, fold, bet) but then the flop comes and you draw 3 cards from 50 unknowns and the branching factor shoots up - so trying to calculate the potential outcomes of a giving action must get complex computationally.

(Haven't really thought about this, but thats what I'd guess the problem is)

Edit: when you say a 'round of poker' I'm assuming you mean a hand, rather than a round of betting, or round in a tournament etc

pkr_ennis

RoundTower wrote: »

here's a simple poker game. If you have studied some basic game theory, you can solve this problem.

we play with a 3-card deck. Ace beats King beats Queen.

We both put in 1 euro to the pot. You act first and your choices are to bet nothing or bet 1 euro.

If you check (bet nothing), your opponent checks too. Whoever has the best hand wins the pot of €2.

Your opponent's choices are to call the bet or fold. If he folds, you win the pot. If he calls, he puts in €1 and whoever has the best hand wins the pot of €4.

What is the equilibrium ("optimal") strategy for this game? How much money do you win (or lose) each hand if both players play the optimal strategy? How, if at all, would you change your strategy if you knew your opponent was not following his optimal strategy? Once you can answer these questions precisely, you can attempt to generalise it to a more complicated poker game.

I have zero experience working out gt solutions, however I have watched several instructional videos using the programme gambit. You build the trees and press compute 1 nash equilibirum and hey presto you get an answer.
I am having trouble working the software and haven't got it down yet. I've posted questions about it on the stox poker forums which is where I viewed the vids and hope to be working this software soon. I tried to solve the game posted here, but I suspect I've something done wrong e.g. the answer I got never bluffs and only value bets half the time with the nuts.

GT and poker mix perfectly because we can get perfect information on our opponents through a poker tracking app. Therefore being able to derive exact solutions vs a range of different opponents or their tendencies.

pkr_ennis

fergalr wrote: »

Could you clarify a few things about the setup?

* I presume players get dealt one card each, and can't see what the remaining card in the deck is.
* Are the players' cards dealt face up or face down?
I'm assuming that each players cards are dealt so that only the player can see them (like the hole cards in holdem)

I saw this problem before and those assumptions are correct.

pkr_ennis

fergalr wrote: »

Edit: when you say a 'round of poker' I'm assuming you mean a hand, rather than a round of betting, or round in a tournament etc

They mean a round of betting.
Are you asking how many decision points there would be in a round of betting in Limit Hold'em poker?
Lots and lots is the answer to that question. I mean there would be 1 infinite branch, if they had never ending stacks and kept re-raising each other.
Each player has 3 options once a bet has been placed or 2 options if facing no action. If player b re-opens the betting after player a's initial bet then this would be another 3 options player a is faced with. If they only had enough chips for 1 bet on each street the model would be smaller.

fergalr

pkr_ennis wrote: »

They mean a round of betting.
Are you asking how many decision points there would be in a round of betting in Limit Hold'em poker?
Lots and lots is the answer to that question. I mean there would be 1 infinite branch, if they had never ending stacks and kept re-raising each other.
Each player has 3 options once a bet has been placed or 2 options if facing no action. If player b re-opens the betting after player a's initial bet then this would be another 3 options player a is faced with. If they only had enough chips for 1 bet on each street the model would be smaller.

Just had a quick read of this paper:
http://www.cs.cmu.edu/~sandholm/tartanian.AAMAS08.pdf
which is extremely interesting, and deals with a lot of these issues.

They are only talking about a heads up poker player which is obviously a much simpler game, but its interesting to see what they've done.

They quote a size of 10^18 nodes for the game tree for limit poker, which might answer gerry87s earlier question about the search space. I presume from the context they mean heads up poker, but its not crystal clear to me from the context. Either way, its a lot of nodes.

They do some interesting things to cut the tree size down a bit.
Firstly, they do in fact discretise the NL bets into a much smaller space of actions, to cut down the search space. That's quite interesting. Their discretisation makes sense to me, but I'm not a very good poker player - interesting to hear what others think of it. Most poker books I've read have typically mentioned only a few separate bet sizes (typically expressed as a fraction of the pot) anyway (maybe we some random noise thrown in).

The next thing they did which I thought was interesting was look for strategic symmetries in the space of cards that can be dealt, to cut down the size of natures moves that have to be considered distinctly. Thats quite interesting, be interested to read more about how they did that. They used a lossy version as well to cut the space down, grouping together similar cards using a k-means clustering (standard machine learning way of grouping together similar items). Cool stuff.

Then they use some autogenerated c++ code to calculate nash equilibrium for the imperfect game. Have to say, didn't understand that part of the paper, same thing thats stopping me from solving round towers problem, haven't seen equilibrium models for dynamic imperfect info games. Have to read up on that - anyone recommend a good tutorial?

fergalr

gerry87 wrote: »

The trouble with the poker/game theory connection is that it starts to break down pretty rapidly when you start to relax assumptions.

Most gt problems assume all agents are rational and have equal/perfect knowledge of game theory.

I remember a discussion over in the poker forum ages ago about one of the game theory problems. The Travellers Dilemmait was essentially this:

You and a friend traveling on an airline, each with a vase (the only two of their type in the world). The airline looses both and comes to each of you separately asking how much it was worth to refund you. The rules were,
1) Max price 100.
2) Min price 2.
3) If you say the same value of your friend you both get that amount.
4) Whoever says the lower price gets that price +1 and whoever says the higher price gets the low price -1.

Game theory has a logical 'right' answer (Nash Equilibrium is 2). But almost nobody in the poker forum answered that, even after taking a few steps along the GT way of thinking most said 99 or 98.

I know that the relevance of game theory to real life is something people argue back and forth over.

I would note though that the game you gave isn't a zero sum game. I think (perhaps without much justification) that GT results seem to me to make more real life sense when the game is zero sum.


I also think our intuition is also different from the GT results because in real life such interactions or 'games' rarely occur in a once off situation where you'll never see the other actors again. A lot of the time we only look at a single stage in a complex game and apply GT to it, ignoring the larger context which typically involves iteration or reputation which changes things a lot.

For example, while the 'correct' thing to do in a prisoners dilemma from a GT point of view is to both defect (betray the other prisoner) this conflicts with most peoples real life intuition, which is much closer to the iterated game solution - probably because if one prisoner betrayed the other in a similar real life situation there would be later reprisals of some sort - we are conditioned to assume that word will get out of the actions and to act more like its always an iterated game.

gerry87

10^18 does seem a lot for heads up if it means what i think it does. Small blind folds (first outcome). Small blind calls, big blind folds (second outcome). Small blind raises... etc could there be 10^18 outcomes? Am i getting this wrong? Is it be taking into account the variations in cards as well?

I'd say no-limit would probably be even less than this, that seems to be what they're going on with the betting model.

Just skimmed, but seems interesting. Wonder if they ever put those bots into action.

fergalr

gerry87 wrote: »

10^18 does seem a lot for heads up if it means what i think it does. Small blind folds (first outcome). Small blind calls, big blind folds (second outcome). Small blind raises... etc could there be 10^18 outcomes? Am i getting this wrong? Is it be taking into account the variations in cards as well?

I dont know what exactly the number refers to - they don't say, nor do they seem to give a reference.

I presume they are taking into account the variations in the cards.
Perhaps they are also considering the situation where players keep reraising each other until they've exhausted their stacks (something they deal with later in their betting model by capping the number of repeated bets to 3 - on the basis that this is more than would occur in practice the vast majority of times anyway).

gerry87 wrote: »

I'd say no-limit would probably be even less than this, that seems to be what they're going on with the betting model.

Just skimmed, but seems interesting. Wonder if they ever put those bots into action.

Hah - I'd say that's the question on everyone's mind :-)
I'm sure they wouldn't be adverse to running it as a bot if they could - came across another instance where they said that they had built something that played one of the party poker games very well (near optimally), but the game structure was altered before they could make money - hmm! That would imply they actually went through the hassle of wiring it up to party poker (screen scraping or something similar - I presume there isn't an exposed api ! )

I'd wonder how good a bot would actually need to be before it could start making a profit on the small stakes cash tables - probably wouldn't need to even be that smart. You mightn't even need to do such sophisticated GT analysis to produce a bot with positive expectation for easier games - simpler learning approaches might work etc - I've no idea whats out there, but I'm sure a lot of people have put time and effort into it.
With the level of analysis thats in the published work I've come across in a quick google, it'd make me quite wary about what might be out there.

gerry87

fergalr wrote: »

I dont know what exactly the number refers to - they don't say, nor do they seem to give a reference.

I presume they are taking into account the variations in the cards.
Perhaps they are also considering the situation where players keep reraising each other until they've exhausted their stacks (something they deal with later in their betting model by capping the number of repeated bets to 3 - on the basis that this is more than would occur in practice the vast majority of times anyway).



Hah - I'd say that's the question on everyone's mind :-)
I'm sure they wouldn't be adverse to running it as a bot if they could - came across another instance where they said that they had built something that played one of the party poker games very well (near optimally), but the game structure was altered before they could make money - hmm! That would imply they actually went through the hassle of wiring it up to party poker (screen scraping or something similar - I presume there isn't an exposed api ! )

I'd wonder how good a bot would actually need to be before it could start making a profit on the small stakes cash tables - probably wouldn't need to even be that smart. You mightn't even need to do such sophisticated GT analysis to produce a bot with positive expectation for easier games - simpler learning approaches might work etc - I've no idea whats out there, but I'm sure a lot of people have put time and effort into it.
With the level of analysis thats in the published work I've come across in a quick google, it'd make me quite wary about what might be out there.

On micro-limits a basic strategy would probably do alright.

if (Aces or Kings)
    All-Inski
else
    Check/Fold

Wonder if a tournament bot would be easier, if you just made it play tight premium hands until its in the money, then hand it over to you... sure its basically how i play.

Think we're getting away from OP's topic a little!

pkr_ennis

gerry87 wrote: »

Think we're getting away from OP's topic a little!

Not at all. I just wish I was up on all this math stuff. From what I understand of bots and the sort, it's obvious there has to be some profitable activity, especially in the small stakes, but poker sites have software to catch them out so I think their activity has been limited for a while, although some of the top minds have to be doing this. It's money for nothing, well almost. There are bots that can almost beat top proffesional players. I heard there is a school in Canada that has the best, it had several matches at the WSOP with top pro's and beat at least 1 of them. Here's a link to some chat and the like.
http://www.stoxpoker.com/blogentry_more.php?blogid=3740&langid=1&memberblog=#MORE
By the way, playing just aces and kings is obv a terrible strategy as you'll get pwned over over by the man who has the GT down and knows your super tight tendencies lol.

gerry87

pkr_ennis wrote: »

Not at all. I just wish I was up on all this math stuff. From what I understand of bots and the sort, it's obvious there has to be some profitable activity, especially in the small stakes, but poker sites have software to catch them out so I think their activity has been limited for a while, although some of the top minds have to be doing this. It's money for nothing, well almost. There are bots that can almost beat top proffesional players. I heard there is a school in Canada that has the best, it had several matches at the WSOP with top pro's and beat at least 1 of them. Here's a link to some chat and the like.
http://www.stoxpoker.com/blogentry_more.php?blogid=3740&langid=1&memberblog=#MORE
By the way, playing just aces and kings is obv a terrible strategy as you'll get pwned over over by the man who has the GT down and knows your super tight tendencies lol.

For aces you'd need about 40 big blinds in the pot whenever you hit, so its probably not ideal, still more likely to happen at microstakes, at .01/.02 its only 80cent!

At .01/.02 you aren't likely to come across folk that have thought about game theory! Generally people either won't notice or won't stay at the table long enough to notice. The rake might eat you up tho. As fergal said, a simple expected value game might work but not at the higher stakes.

I'm sure you'd have to throw a random element into the algo to stop them picking up, i've heard them banning people for things like not taking any breaks for 24+ hours straight.

But as you say clearly the amount of research people put into this, there's probably a little more too it. But... i've got a great idea for a roulette bot!

pkr_ennis

gerry87 wrote: »

For aces you'd need about 40 big blinds in the pot whenever you hit, so its probably not ideal, still more likely to happen at microstakes, at .01/.02 its only 80cent!

At .01/.02 you aren't likely to come across folk that have thought about game theory! Generally people either won't notice or won't stay at the table long enough to notice. The rake might eat you up tho. As fergal said, a simple expected value game might work but not at the higher stakes.

I'm sure you'd have to throw a random element into the algo to stop them picking up, i've heard them banning people for things like not taking any breaks for 24+ hours straight.

But as you say clearly the amount of research people put into this, there's probably a little more too it. But... i've got a great idea for a roulette bot!

I guess there is a good chance to beat those micro games. As you said there is no-one watching how well the other is playing.

Roulette bot is not gonna work as the game isn't zero sum, and the outcome is completely random. A backgammon bot is easy I imagine as there is a programme that plays the game perfectly.

Bots are against the rules of the game so. . .

RoundTower

fergalr wrote: »

Roundtower:
I'd be interested to know how to go about the analysis of this game.
Like I said, I only know what I do about game theory from reading a bit of a book. (the games we saw doing the AI course were of perfect information, so it was mostly about minimax and search)


Could you clarify a few things about the setup?

* I presume players get dealt one card each, and can't see what the remaining card in the deck is.
* Are the players' cards dealt face up or face down?
I'm assuming that each players cards are dealt so that only the player can see them (like the hole cards in holdem)

yes you are right in your assumptions. there is no complicated number-crunching involved, you should need a pen and paper at most.

fergalr

pkr_ennis wrote: »

Not at all. I just wish I was up on all this math stuff. From what I understand of bots and the sort, it's obvious there has to be some profitable activity, especially in the small stakes, but poker sites have software to catch them out so I think their activity has been limited for a while, although some of the top minds have to be doing this. It's money for nothing, well almost.

I imagine that most of the countermeasures to stop bots involve making it really hard to write something that programmatically plays the game, and making it hard from a software engineering point of view to maintain a working bot.
For example, if you change the client interface subtly but frequently, you'd make it so that a bot based on screen scraping would need constant updating to keep running. You could also make your poker client watch out for other programs that are running while it runs etc, but someone would work around that by running it in a VM. Ultimately though, if you own a poker website, by changing your protocol or client you could probably frustrate someone thats trying to maintain a bot to the point where they'd make their money easier by writing software.
I imagine another thing you could do is simply track the amount of games played by each client - if it exceeds a certain threshold per month, they are probably a bot.

I guess someone running a poker website could do more sophisticated things such as looking for patterns in their players play to look for people that play deterministically, as an attempt to catch bots, but I'd imagine this wouldn't be worth their while - probably easy to write a bot with sufficient randomness to throw this stuff off.

Still, just by making the bot<->game communication awkward enough, changing it frequently enough, and watching to make sure no one plays too many games, you'd probably make the whole exercise less profitable for the bot writers than if they just wrote software for a living.

Or maybe there's a lot of people living large from poker bots, I dunno :-)

pkr_ennis wrote: »

There are bots that can almost beat top proffesional players. I heard there is a school in Canada that has the best, it had several matches at the WSOP with top pro's and beat at least 1 of them.

Are you sure about that? Is this for a limited heads-up game, or for the full game?

Whatever about playing simple strategies, I'd be very surprised to hear poker bots are good enough to beat top pros.
Thats not what I took from the link you gave - and from the stuff I linked to previously, I'd have thought really good poker play was still quite a bit away.

Its a very hard problem to solve. Perfect play for full NL holdem (as opposed to heads up) even hand by hand, sounds like its still well intractable; and even if you have AI thats finding some approximation of hand by hand equilibrium strategies, theres a lot of other hard stuff they would ideally be doing too that a real world player would (such as opponent modelling to maximise the winnings, or strategising about the overall state of the game - reasoning about blinds going up, the changing game of their opponents etc things that its very hard to account for in a bot, and things that might be necessary to be competitive with the best players)

pkr_ennis wrote: »

Here's a link to some chat and the like.
http://www.stoxpoker.com/blogentry_more.php?blogid=3740&langid=1&memberblog=#MORE
By the way, playing just aces and kings is obv a terrible strategy as you'll get pwned over over by the man who has the GT down and knows your super tight tendencies lol.

Well - from what I understand of the game theory approaches as discussed so far, they are more concerned with figuring out what the best mixed strategy is to play - so whats the mathematically soundest thing to do every time - and not so much considering what your opponent does.

I understand the idea of modifying the probabilities of the model on account of the opponents behaviour to exploit your opponent too, but from what I can see, thats a whole other can of worms?

fergalr

pkr_ennis wrote: »

I guess there is a good chance to beat those micro games. As you said there is no-one watching how well the other is playing.

Roulette bot is not gonna work as the game isn't zero sum, and the outcome is completely random. A backgammon bot is easy I imagine as there is a programme that plays the game perfectly.

Bots are against the rules of the game so. . .

I'm pretty sure the thing about a roulette bot was a joke :-)
Being a game where perfect play is negative expectation, its very easy to write a bot... but impossible to write one that makes money :-)

Unless you can find some flaw in the random number generator.
Or, as the legend goes, do some clever physics modelling to use the fact that bets don't close until some time after the ball has being launched, but before it stops, to get a numerical edge.

fergalr

RoundTower wrote: »

yes you are right in your assumptions. there is no complicated number-crunching involved, you should need a pen and paper at most.

Cool - Ill just have to read up on how to solve games of that type (dynamic, imperfect information).
Could you recommend a good tutorial, or source of information?

gerry87

RoundTower wrote: »

here's a simple poker game. If you have studied some basic game theory, you can solve this problem.

we play with a 3-card deck. Ace beats King beats Queen.

We both put in 1 euro to the pot. You act first and your choices are to bet nothing or bet 1 euro.

If you check (bet nothing), your opponent checks too. Whoever has the best hand wins the pot of €2.

Your opponent's choices are to call the bet or fold. If he folds, you win the pot. If he calls, he puts in €1 and whoever has the best hand wins the pot of €4.

What is the equilibrium ("optimal") strategy for this game? How much money do you win (or lose) each hand if both players play the optimal strategy? How, if at all, would you change your strategy if you knew your opponent was not following his optimal strategy? Once you can answer these questions precisely, you can attempt to generalise it to a more complicated poker game.

My Guess (I took it from the first euro was already in the pot.):

I have a queen I check-
EV(checking) = 0
EV(betting) = P(he has a ace)(-1) + P(he has a king & thinks i have a queen)(-1) + P(he has a king & thinks i have a ace)(+2)
EV(betting) = .5(-1) + (.5*.5)(-1) + P(.5*.5)(+2) = -.25

I have ace I bet-
EV(checking) = +2
EV(Betting) = P(he has a queen)(+2) + P(he has a king & thinks i have a queen)(+4) + P(he has a king & thinks i have a ace)(+2)
EV(Betting) = .5(+2) + (.5*.5)(+3) + (.5*.5)(+2) = 2.25

I have a king I check-
EV(checking) = P(he has a queen)(2) + P(he has a ace)(0)
EV(checking) = .5(2) + .5(0) = 1
EV(betting) = P(he has a queen)(2) + P(he has a ace)(-1)
EV(betting) = .5(2) + .5(-1) = 0

---

Ace - EV(2.25)
King - EV(1)
Queen - EV(0)

EV(Game) = (1/3)*2.25 + (1/3)*1 + (1/3)*0 = 1.0833

Anywhere near?

pkr_ennis

fergalr wrote: »

Well - from what I understand of the game theory approaches as discussed so far, they are more concerned with figuring out what the best mixed strategy is to play - so whats the mathematically soundest thing to do every time - and not so much considering what your opponent does.

I understand the idea of modifying the probabilities of the model on account of the opponents behaviour to exploit your opponent too, but from what I can see, thats a whole other can of worms?

I think you got exactly what I'm looking for here anyway. Mixed strategy coupled with the correct adjustments is a killer strategy. Not sure how possible it is though. It looks like the bot guys replicate complex poker situations and simplify to come to an approximate answer.

fergalr

gerry87 wrote: »

My Guess (I took it from the first euro was already in the pot.):

I have ace I bet-
EV(checking) = +2
EV(Betting) = P(he has a queen)(+2) + P(he has a king & thinks i have a queen)(+4) + P(he has a king & thinks i have a ace)(+2)
EV(Betting) = .5(+2) + (.5*.5)(+3) + (.5*.5)(+2) = 2.25

Just looking at the situation where player 1 has an Ace here.

The only thing that I don't understand is why you assign '(.5*.5)' as the value for 'P(he has a king & thinks I have an ace)'
I understand that the probability is .5 that player2 has a king, if player1 has either the ace or queen.
But I don't understand why the value for 'thinks I have an ace' (ie: 'player2 thinks player1 has an ace') is .5


Doesn't player1 provide information to player2 when player1 chooses to either raise or to check? Given that player1 knows what player1s card is, and that knowing what player1s card is probably influences player1s actions, how can player2 assign .5 to 'thinks player1 has an ace' after player1 has acted? It would have been .5 if all player2 knew was that player2 had a king, but given that player2 knows player2 has a king AND that player2 knows player1 decided to raise, as opposed to check, surely that value of 'thinks player1 has an ace' must change from .5?
I would have thought there was a feedback effect here in the model that would have to be analysed (and that would hopefully converge to a value that could be used)?
How does that work?

Again, I'm not familiar with this type of analysis, so maybe you know more than me.

pkr_ennis

RoundTower wrote: »

What is the equilibrium ("optimal") strategy for this game?

Not taking money into account-

Player 1 bets ace everytime
Player 1 checks king everytime
Player 1 bets queen 50% of the time
Player 2 calls ace everytime
Player 2 calls king 50% of the time and
Player 2 folds queen everytime.

gerry87

fergalr wrote: »

Just looking at the situation where player 1 has an Ace here.

The only thing that I don't understand is why you assign '(.5*.5)' as the value for 'P(he has a king & thinks I have an ace)'
I understand that the probability is .5 that player2 has a king, if player1 has either the ace or queen.
But I don't understand why the value for 'thinks I have an ace' (ie: 'player2 thinks player1 has an ace') is .5


Doesn't player1 provide information to player2 when player1 chooses to either raise or to check? Given that player1 knows what player1s card is, and that knowing what player1s card is probably influences player1s actions, how can player2 assign .5 to 'thinks player1 has an ace' after player1 has acted? It would have been .5 if all player2 knew was that player2 had a king, but given that player2 knows player2 has a king AND that player2 knows player1 decided to raise, as opposed to check, surely that value of 'thinks player1 has an ace' must change from .5?
I would have thought there was a feedback effect here in the model that would have to be analysed (and that would hopefully converge to a value that could be used)?
How does that work?

Again, I'm not familiar with this type of analysis, so maybe you know more than me.

You're probably right, first pass I was taking it that player 2 wasn't getting any signals. So there's the .5 chance he has a king. Then from his point of view you either have an ace or a queen, each P=.5, so half the time he'll call and half the time he'll fold. Similar to in pkr_ennis's strategy above where he says 'player 2 calls king 50% of the time' - hell call when he thinks i have a queen and fold when he thinks i have an ace.

RoundTower

here's how you solve this problem:

there is nothing to be gained from checking with an Ace, so you should always bet.
there is nothing to be gained from betting with a King, so you should always check.
with a Queen you should adpot a mixed strategy, either check or bet. let's say you bet with probability x, 0 <= x <= 1

for him, he should always call with an Ace and fold with a Queen. He should call with a King some of the time - enough that you don't get to win the pot every time with a Queen, but he also doesn't want to pay you off every time you have an ace. So say he calls with probability y.

Then there are 6 possibilities for how the cards get dealt, all equrobable. When you have an Ace and he has a Queen, for example, you make €1. When you have a Queen and he has an Ace, you lose €2 x of the time and lose €1 (1-x) of the time, so on average you make €(-2x -1(1-x)) which is €(-x-1). You can also find the outcomes for the other four possibilities too, and average them together to get the expectation of the game. You should get (x + y - 3xy)/6.

Now to find the optimal x, you need to know that you should choose x such that it you are indifferent to his choice of y, in other words, the terms containing y should sum to 0. So y - 3xy = 0 for all y, therefore x = 1/3. Similarly you should get y = 1/3, and the expectation of the game should be that you, on average, make one eighteenth of a euro. He expects to lose one eighteenth of a euro on average

Now you could see how the game changes if you have a different choice, for example if the bet amount was €2 instead of €1, or more interestingly, if you had a choice to check, bet €1 or bet €2.

pkr_ennis

Wow, thats genius!
I think that this shows a basic optimal betting strategy for poker. Care to share the frequency player 1 should bluff with a queen roundtower? Please.
It would be even more interesting to add player 2 raising options in there as this would create a basic betting/bluffing/calling strategy in a real poker situation. Wouldn't it?
I'm starting to understand your location roundtower lol.

fergalr

Lets see if I've understood this... apologies if I'm a bit too verbose here, RoundTower, want to see if I understood it properly.
I also have a question or two about the applying this to games in general:

RoundTower wrote: »

here's how you solve this problem:
there is nothing to be gained from checking with an Ace, so you should always bet.

With an Ace, the payoff/profit for a check is 1. No matter what player 2 has or does, in response to a raise, the minimum payoff is still 1, and could be as high as 2, if the other player calls the raise. So rationally, you'd always bet.

RoundTower wrote: »

there is nothing to be gained from betting with a King, so you should always check.

If you call with a king, the EV of the profit is 0.
But, if you raise, the EV of the profit falls to -.5 (half the time player2 will have an ace, and you'll lose 2, half the time player2 will have a queen, and you'll only gain 1, because they will rationally fold, knowing they are beaten, hence -.5 EV)
-.5 EV being less than 0 EV, it's rational to always check, with a King.

RoundTower wrote: »

with a Queen you should adpot a mixed strategy, either check or bet. let's say you bet with probability x, 0 <= x <= 1

So, this is where it gets a little trickier.
When you have a queen, and player2 has an ace, you are going to lose money. Nothing can be done about that. But half the time player1 has a queen, player2 has a King, and not an Ace.
When player1 has a queen, and player2 has that king, player2 doesn't know if player1 has a queen or an ace. Therefore there's a chance of being able to bluff player2 out of the pot, by raising.
Hence, you can't say that player1 should either always check or always bet here.
The game theoretic thing to do will be to adopt a mixed strategy, which means that you don't always take a single action, instead you choose which action according to a probability distribution. As you can only either check or bet, the distribution here can be represented by a single number - essentially the ratio of checks to bets.

So, the only thing to do to figure out optimal play for player1 is to figure out what that number is.

RoundTower wrote: »

for him, he should always call with an Ace and fold with a Queen.

Simple enough - he's ahead with an ace, and behind with a queen. The only time he'll have an ace and have a decision to make is when you are trying to bluff him with your queen, so he obviously calls. The only time he has a queen and has a decision to make is when you have raised with your ace, so he obviously folds.

RoundTower wrote: »

He should call with a King some of the time - enough that you don't get to win the pot every time with a Queen, but he also doesn't want to pay you off every time you have an ace. So say he calls with probability y.

So, thats what player2 does when player2 has an ace or a queen.
There are two situations in which player2 can have a King, and have a decision to make about whether to call or fold.
One is when player1 has an Ace, and has made a bet, because it made no sense to just check with an ace, and the other is when player1 has a queen and has made a bet as a bluff (with frequency X).
So, when player2 has the King, and sees a raise, he has to call a certain amount of the time - ie, as you say, with probability Y.

What the Y is to maximise player2's EV obviously depends on what X player 1 is using, given that player1 moves first each time.
And also, if player1 wants to maximise player1s winnings, then player1s X should be chosen mindful of player2s Y. For example, if Y was 1, player1 would make X to be 0 - if player2 always calls with the King, player1 will always check the queen, (and maximise profits by getting paid for his Ace).

But if Y was 0, player 1 would make X to be 1 - if player2 never calls with the King, player1 will always bet with the queen, and while player1 won't get paid for his ace, he'll take down the pots when he only has a queen instead.

But we're trying to find the equilibrium strategy, which is the best X given that player2 will chose the best Y...

RoundTower wrote: »

Then there are 6 possibilities for how the cards get dealt, all equrobable. When you have an Ace and he has a Queen, for example, you make €1. When you have a Queen and he has an Ace, you lose €2 x of the time and lose €1 (1-x) of the time, so on average you make €(-2x -1(1-x)) which is €(-x-1). You can also find the outcomes for the other four possibilities too, and average them together to get the expectation of the game. You should get (x + y - 3xy)/6.

So, the expectation are:
[player1card,player2card]:[expectation]
AQ:1
AK:y+1
KA:-1
KQ:1
QA:-x-1
QK:2x-3xy-1

And these are all equally likely, the expectation of the game is the sum of them all divided by their number, so as you said, (x+y-3xy)/6, again, where x is the probability of player1 to bluff with a queen, and y is the probability of palyer2 to call with a king.

RoundTower wrote: »

Now to find the optimal x, you need to know that you should choose x such that it you are indifferent to his choice of y, in other words, the terms containing y should sum to 0.

So we now have the expectation of the game (from the perspective of player1, and remembering the game is zero sum), expressed as a function of X and Y.
What player1 wants to do is choose X such that the value of the expectation, given the worst value of Y player2 can choose for that X, is at a maximum.
This happens when player1 chooses 1/3 as the value. (see pic)

I'm not sure I understand what you say that the player 'should choose x such that you are indifferent to his choice of y, in other words, the terms containing y should sum to 0' - can you explain this part in more detail?
I understand that 1/3,1/3 is the equilibrium from the function, and theres plenty of ways to calculate that, but I don't fully see how you calculated that?

RoundTower wrote: »

So y - 3xy = 0 for all y, therefore x = 1/3. Similarly you should get y = 1/3, and the expectation of the game should be that you, on average, make one eighteenth of a euro. He expects to lose one eighteenth of a euro on average
Now you could see how the game changes if you have a different choice, for example if the bet amount was €2 instead of €1, or more interestingly, if you had a choice to check, bet €1 or bet €2.

I'm pretty sure I understand the analysis of this game that you gave.

I'm wondering though, how well does the method used generalise? Working this stuff out by hand, like you did there, could get pretty complex as the game complexity grew. How would you go about analysing more complex games? Is it possible to work these things out algorithmically, or to represent the game rules in some standardised format, and apply something to calculate the equilibrium positions in a general way? (or do some sort of stochastic approximation?)

fergalr

pkr_ennis wrote: »

Wow, thats genius!
I think that this shows a basic optimal betting strategy for poker. Care to share the frequency player 1 should bluff with a queen roundtower? Please.

AFAIK he's saying that X is the probability that a player bets with a queen (a bet with a queen is always a bluff here) so thats the number you want. For the equilibrium strategy, its .333... so, bluff one third of the time, with the queen.

pkr_ennis wrote: »

It would be even more interesting to add player 2 raising options in there as this would create a basic betting/bluffing/calling strategy in a real poker situation. Wouldn't it?
I'm starting to understand your location roundtower lol.

I'd be very wary about extending the results from this analysis - or even a slightly more complex one - to a more complex poker game, like real poker. Small changes in the game setup could make the correct strategies very different.

Mellor

pkr_ennis wrote: »

Chris Ferguson is one of the best poker players in the world.

I'm pretty sure his father is (or was) a leading GT professor, in UCLA IIRC



When I look at game theory problems, I tend to approach it as if I was playing a reasonably rational opponent , but not one that plays with perfect rationality

Example, Nash's say that the answer to the travelers dilemma is 2, yet we all know that picking say 94 would have a much higher EV against most rational people.

This is the huge flaw in nash's equilibrium, it assumes everyone has perfect rationality, which they don't.

A perfect example is the guess 2/3's of the average puzzle.
In this, players attempt to guess a number that is 2/3s the average all of all players guesses. The solution is simple, yet hugely flawed.



Applying this to the AKQ problem, I'd assume that player B calls with a K 50% of the time. Now what is our EV, and how often should we bluff.

pkr_ennis

Mellor wrote: »

A perfect example is the guess 2/3's of the average puzzle.
In this, players attempt to guess a number that is 2/3s the average all of all players guesses. The solution is simple, yet hugely flawed.



Applying this to the AKQ problem, I'd assume that player B calls with a K 50% of the time. Now what is our EV, and how often should we bluff.

I understood the solution like this, That B calls 100% of the time with a K b/c A's bets with Q's 33% of the time. This is where A's edge lies in the puzzle, and it's b/c B never gets a chance to raise etc.

GT is flawed so, jeepers. Just wondering if you could quantify that and add it to a puzzle, like this-

A is on the river with the 2nd nuts and bets,
B has the third nuts and is thinking something along these lines,
1, A bets a hand that is beating me x% of the time
2, A bets a hand that is losing to me x% of the time
mmm, stuck here, b/c A is considering the times in which he's beat, which would/could take into consideration A's tilting tendencies as well as when he's playing solidly.
Maybe you have to come up with 2 different numbers. . .

I found this article very intersting and is totally relevant to this thread
http://www.twoplustwo.com/magazine/issue60/Christenson-commentary-approximating-game-theoretic-optimal-strategies-full-scale-poker.php

C

REFLINE1

Mellor wrote: »

I'm pretty sure his father is (or was) a leading GT professor, in UCLA IIRC



When I look at game theory problems, I tend to approach it as if I was playing a reasonably rational opponent , but not one that plays with perfect rationality

Example, Nash's say that the answer to the travelers dilemma is 2, yet we all know that picking say 94 would have a much higher EV against most rational people.

This is the huge flaw in nash's equilibrium, it assumes everyone has perfect rationality, which they don't.





A perfect example is the guess 2/3's of the average puzzle.
In this, players attempt to guess a number that is 2/3s the average all of all players guesses. The solution is simple, yet hugely flawed.



Applying this to the AKQ problem, I'd assume that player B calls with a K 50% of the time. Now what is our EV, and how often should we bluff.

Your right Mellor-His father is Thomas.S. Ferguson.
Some really good examples at the link below.
http://www.math.ucla.edu/~tom/Game_Theory/mat.pdf

RoundTower

fergalr wrote: »

I'm not sure I understand what you say that the player 'should choose x such that you are indifferent to his choice of y, in other words, the terms containing y should sum to 0' - can you explain this part in more detail?
I understand that 1/3,1/3 is the equilibrium from the function, and theres plenty of ways to calculate that, but I don't fully see how you calculated that?

I'm not sure what the proof is that this works as a method to find the equilibrium of the game, I just know it does.

As for the argument that "game theory is flawed because it assumes rational players" - this isn't really true. Examples like the Travellers Dilemma are interesting because they are atypical and provide unexpected results. If you could find optimal play for poker against rational players, firstly, you would probably be able to crush almost any game in the world, and secondly, it would then be a relatively small step to move towards a strategy that exploits weaker players (for example, in the example game, if the guy always called with a King you would never bluff with a Queen).

You can't solve any "real" poker game by breaking down all the trillions of possibilities, like we did for the one-card game, but you can still apply some of the conclusions from it. For example, you can generalise the result to say that your optimum bluffing frequency should be such that your bluffs:value bets are in the ratio bet size:pot size+bet size (which was 1:3 in the example).

fergalr

Mellor wrote: »

I'm pretty sure his father is (or was) a leading GT professor, in UCLA IIRC

When I look at game theory problems, I tend to approach it as if I was playing a reasonably rational opponent , but not one that plays with perfect rationality

This makes sense to me, in many real world scenarios, especially where it isn't a fully correct model of the real world, and where it isn't a zero sum game, and where the other players are acting intuitively.
For example, the prisoners dilemma as mentioned earlier.

Mellor wrote: »

Example, Nash's say that the answer to the travelers dilemma is 2, yet we all know that picking say 94 would have a much higher EV against most rational people.

This is the huge flaw in nash's equilibrium, it assumes everyone has perfect rationality, which they don't.

A perfect example is the guess 2/3's of the average puzzle.
In this, players attempt to guess a number that is 2/3s the average all of all players guesses. The solution is simple, yet hugely flawed.

Applying this to the AKQ problem, I'd assume that player B calls with a K 50% of the time. Now what is our EV, and how often should we bluff.

I think the equilibrium analysis here is pretty useful though.
Like, as player1 I now know the correct way to play this game, and if we player and you are player2, I'll always beat you - in the end, I'll always come away profitable (over the long run, expected value, etc).
Without the game theory analysis, its possible that as I played I might wander either 1) make a silly mistake (not betting with my Ace) or 2) without making silly mistakes, still wander into that blue corner of the surface where I have -EV.


If B calls with a K 50% of the time, we should never bluff, and our EV is about .08 - which means the game is worth more to player1 than it was before. In other words, B is now losing more money from deviating from the best strategy.

So yes, if you have an opponent thats deviating from the best strategy like this, then it does make sense to change from the game theory best strategy in order to maximise how much money you take off your opponent.

However, we must remember that the deviation the opponent did didn't improve the game from their perspecitive - by deviating from .333 player2 did not improve his chances to take money from player1 - only gave player1 an opportunity to take more money from him.

If player1 still just played .3333 then he'd still beat player2 always.
Where as if player1 instead deviates to never bluffing, in an attempt to maximise the amount taken from a player2 who we think is calling half the time, and player2 starts to never call, suddenly the EV for player1 goes to zero - no longer making any money at all.
You get into this cycle of trying to figure out what your opponent is actually playing, having to second guess them, and repeat that again and again - whereas with the GT optimum player1 can just sit there and take their earnings.

pkr_ennis wrote: »

I understood the solution like this, That B calls 100% of the time with a K b/c A's bets with Q's 33% of the time. This is where A's edge lies in the puzzle, and it's b/c B never gets a chance to raise etc.

GT is flawed so, jeepers.

Its not a perfect model of the real world and how the real world works.
But the analysis of this poker game, as done here, isn't flawed as such - you need to be careful about what claims are made, but the claims that are made are accurate - .333333 is the way to go if you are player1, and you'll take home the best average profit over time, even against an infinitely rational and smart opponent.

pkr_ennis wrote: »

Just wondering if you could quantify that and add it to a puzzle, like this-

A is on the river with the 2nd nuts and bets,
B has the third nuts and is thinking something along these lines,
1, A bets a hand that is beating me x% of the time
2, A bets a hand that is losing to me x% of the time
mmm, stuck here, b/c A is considering the times in which he's beat, which would/could take into consideration A's tilting tendencies as well as when he's playing solidly.
Maybe you have to come up with 2 different numbers. . .

I found this article very intersting and is totally relevant to this thread
http://www.twoplustwo.com/magazine/issue60/Christenson-commentary-approximating-game-theoretic-optimal-strategies-full-scale-poker.php

C

RoundTower wrote: »

I'm not sure what the proof is that this works as a method to find the equilibrium of the game, I just know it does.

Will have to look at that in more detail later, be interested to read more about this.

RoundTower wrote: »

As for the argument that "game theory is flawed because it assumes rational players" - this isn't really true. Examples like the Travellers Dilemma are interesting because they are atypical and provide unexpected results. If you could find optimal play for poker against rational players, firstly, you would probably be able to crush almost any game in the world, and secondly, it would then be a relatively small step to move towards a strategy that exploits weaker players (for example, in the example game, if the guy always called with a King you would never bluff with a Queen).

You can't solve any "real" poker game by breaking down all the trillions of possibilities, like we did for the one-card game, but you can still apply some of the conclusions from it. For example, you can generalise the result to say that your optimum bluffing frequency should be such that your bluffs:value bets are in the ratio bet size:pot size+bet size (which was 1:3 in the example).

I should also read more about mechanically trying to solve bigger games that better approximate the real world, to trying to find partial solutions to them. Interesting stuff...

Thanks for the example, and the solutions/discussion about it, RoundTower.

Mellor

fergalr wrote: »

I think the equilibrium analysis here is pretty useful though.
Like, as player1 I now know the correct way to play this game, and if we player and you are player2, I'll always beat you - in the end, I'll always come away profitable (over the long run, expected value, etc).

In the original problem (Kuhn poker), it's player two that has the edge.
The one as posted left out the re-raise option for P2, but copied the edge straight to P1.
Maybe it was a typo, or maybe this version has the same EV just reversed, be a bit coincidence though

RoundTower

Mellor wrote: »

In the original problem (Kuhn poker), it's player two that has the edge.
The one as posted left out the re-raise option for P2, but copied the edge straight to P1.
Maybe it was a typo, or maybe this version has the same EV just reversed, be a bit coincidence though

I just picked it as the simplest possible poker game I could imagine, it isn't copied from anywhere although I have seen similar writings before.

I find it hard to believe Player 2 has exactly a .08 edge if he always has the option to bet or raise (not sure what you mean by "reraise" here), but you could be right.

cavedave

I saw that "total poker" book cheap I mentioned earlier in Chapters Dublin today. They also have "The Education of a Poker Player" which is not a great book on how to play holdem but is a great autobiography with a large poker elements.

pkr_ennis

I was thinking about player 2's calling frequency and was wondering why his optimal number wasn't 33% if player 1's betting frequency with a Q was known to be 33%. To my mind, this would save player 2 money or am I getting into second level thinking here, where 2 is adjusting already?

A more complete poker puzzle would be a JQKA game with player 2 being able to bet and raise.

Thanks y'all for the thread,
C

Mellor

RoundTower wrote: »

I just picked it as the simplest possible poker game I could imagine, it isn't copied from anywhere although I have seen similar writings before.

Yeah, I figured you just made up a sample problem.

I find it hard to believe Player 2 has exactly a .08 edge if he always has the option to bet or raise (not sure what you mean by "reraise" here), but you could be right.

I was a little of in my last post. See problem below for accurate details

pkr_ennis wrote: »

A more complete poker puzzle would be a JQKA game with player 2 being able to bet and raise.

Try solving it with the re-rasie without introducing the 4th card first.


3 cards, AKQ (or KQJ)
Each player antes $1 and is dealt a card at random
Player 1 can either check or bet $1
If P1 checks, P2 can check or bet $1 (and P1 must call or fold)
if P1 bets, P2 can call or fold

Basically, each player has the option to raise the pot to $3, and the other then can call for a total of $4. The pot can't be bigger than $4.



What are optimal strategies for player 1 and player 2.
Who has the advantage, or is there one?

RoundTower

this seems like a much more difficult problem despite having changed it only slightly.

fergalr

pkr_ennis wrote: »

I was thinking about player 2's calling frequency and was wondering why his optimal number wasn't 33% if player 1's betting frequency with a Q was known to be 33%. To my mind, this would save player 2 money or am I getting into second level thinking here, where 2 is adjusting already?

Not sure what you are saying here.

The poker problem, as stated by RoundTower here, has equilibrium at 1/3,1/3, as he said, and we worked out - so player2s calling frequency is 33.33..%.
Maybe you misread something there?

pkr_ennis wrote: »

A more complete poker puzzle would be a JQKA game with player 2 being able to bet and raise.

Thanks y'all for the thread,
C

That'd be a more interesting problem all right, although could be a lot harder.
I'll take a look at the version with the reraise at some stage, and then maybe an extra card, but each extra decision grows the tree a lot.

pkr_ennis

fergalr wrote: »

The poker problem, as stated by RoundTower here, has equilibrium at 1/3,1/3, as he said, and we worked out - so player2s calling frequency is 33.33..%.
Maybe you misread something there?

I miss read and have troubles with understanding the math speak. Thanks for clarifying that for me. I seem to be enjoying poker more (winning more) recently, and disguising my hands better since getting into this... Life's Good lol, C

Mellor

RoundTower wrote: »

this seems like a much more difficult problem despite having changed it only slightly.

Yeah, I had a quick attempt at it and soon noticed i'd need a lot more time.
I'll try to work it out though.

NewApproach

This is an interesting concept, one which I thought about myself a few months back when studying GT in college, but my initial conclusion after thinking about it is that it would only be of use in limit poker, rather than no limit, and to a lesser extent pot limit.

Its easy to say an ace has no reason to check, so should bet etc, but AFAIS the model you propose suggests that each player can only put money into the pot once, and this is not realistic.

Even with all the tracking software in the world, we will never know a players exact tendencies at any given moment, and each 'move' in poker is very dependent on both the other player(s) and their previous actions in the hand.

While obviously a very interesting concept, I somehow have my doubts as to how useful it would be in a real life situation.

fergalr

NewApproach wrote: »

This is an interesting concept, one which I thought about myself a few months back when studying GT in college, but my initial conclusion after thinking about it is that it would only be of use in limit poker, rather than no limit, and to a lesser extent pot limit.

Why do you say that?
We've discussed earlier in the thread the idea that you can discretise no limit games into a much smaller set of actions. Obviously, NL increases the already large search space, but aside from that, once you have the idea of discretising the actions, how does it change things?

NewApproach wrote: »

Its easy to say an ace has no reason to check, so should bet etc, but AFAIS the model you propose suggests that each player can only put money into the pot once, and this is not realistic.

Its not supposed to be a 'model' of real poker. Its a cut down toy version of the game to allow a proof of concept of the GT reasoning. The results from it aren't expected to apply to real poker in any way.

NewApproach wrote: »

Even with all the tracking software in the world, we will never know a players exact tendencies at any given moment,

The idea with a GT approach is that you don't need to know the other players exact tenancies, instead, you play rationally optimally, given that they are playing rationally optimally too.

NewApproach wrote: »

and each 'move' in poker is very dependent on both the other player(s) and their previous actions in the hand.

While obviously a very interesting concept, I somehow have my doubts as to how useful it would be in a real life situation.

I certainly agree with you that what we've looked at here isn't at all useful in a real game of NL texas hold-em.
It'd be very useful if you were playing this simple game though.
And it does show that a GT approach can apply to some simple variants.

I would be hesitant to rule out someone 'solving' texas hold-em, at some stage, given the amount of work thats gone into some of the research papers mentioned - and more compute is always becoming available.

Mellor

NewApproach wrote: »

Its easy to say an ace has no reason to check, so should bet etc, but AFAIS the model you propose suggests that each player can only put money into the pot once, and this is not realistic.

You completely missed the point. And actually studied GT?????
It's a GT problem, not a poker hand. And there are applications of this in poker. But it's not a case a copy and paste. Its basic GT strategy.