Welcome to my Texas Hold'em poker neural network! I created this using the Keras Sequential API and it plays a very basic game of poker. It takes the last few round states and returns a decision to either fold, call, or raise. I took on this project because poker, at its core, is a psychological game with a trinary output, similar to Rock, Paper, Scissors. 

To make the game more manageable for my neural network, I simplified it by representing each player as their stack (the amount of chips they have remaining) and representing the cards in play as two values. The change in a player's stack over the course of a few rounds of betting allows the network to understand their betting patterns without having to know their exact move. On top of that, the cards in the networks' hand and on the table can be reduced to two values. The first value indicates the networks' current hand strength, calculated by the built-in functions provided by PyPokerEngine. The other value represents all the possible hands that could be created given the cards on the table - so if three cards of the same suit appear, the network will know that a flush could be created with what is on the table. While this simplification is not perfect, it is enough for the network to function properly. 

PyPokerEngine is a library on github that provides a Texas Hold'em environment in python that is relatively easy to use. It also has the added benefit of having a GUI built in, so I can play against my network once training is complete. 

My network plays in games with six players and takes in an array of five lists as input. Each list represents the round state of a single round of betting, and five rounds make up one full hand. Each list has a length of eight, with the first six values representing the stacks of each player and the last two values representing the network's hand rank and the strongest hands that can be made with the cards on the table. In total, this results in 40 inputs for the network to consider before making a decision. These inputs are put through a Conv 1D layer and downsampled to a single value that is mapped to a poker decision using a Leaky ReLU curve. The leaky rectified linear unit curve was chosen because it can be easily mapped to poker values: any value below 0 is a fold, any value between 0 and the minimum bet is a call, and any value above the minimum bet is a raise of that amount exactly. 

I originally chose to use the Leaky ReLU curve instead of the standard ReLU because of my loss function: the absolute value of my network's moves minus the amount won. Since my network attempts to minimize the output of my loss function, I  realized that choosing standard ReLU may cause my network to always output 0, folding every time. This lack of play is not what I want to encourage, so I switched to Leaky ReLU, causing it to prefer calls over folds. This is because the absolute value of a negative number is positive, so a strong fold is discouraged compared to a passive call. While this isn't the most profitable poker strategy, I had decided I would rather a network that plays poker somewhat poorly, than one that doesn't play at all. 

After initial testing and further research, I realized it would be best to again output using tanh: the network will simply output a value corresponding to a percent of its remaining stack. However, this poses a few new issues, namely, that the inputs must be normalized along a 0 to 1 curve, as standard numbers will negatively affect training using tanh. 

In the future, I plan to switch to the Functional API in order to pass a sum of past predictions into my custom loss function. I also plan to include the network's distance from the dealer button, which should provide a significant amount of useful information for higher-level gameplay. 

This text has been enhanced with AI (ChatGPT) to make my notes more legible and digestible for the average reader