What Does It Mean for a Poker Bot to Be Good?

Abstract

This note examines a deceptively difficult question in poker AI: what should it mean for one agent to be better than another? Tournament profit is intuitive, but it is field-dependent and can conceal severe exploitability. Equilibrium-based notions offer stronger guarantees in heads-up poker, yet become less decisive in multiplayer settings. Drawing on the Annual Computer Poker Competition, Local Best Response analysis and population-evaluation work, the note argues that combining these properties into a single weighted score merely relocates subjective judgement. A more interpretable approach is to evaluate expected value over a deliberately designed distribution of strategic environments, while making the remaining benchmark choices explicit and defensible.

Keywords

• poker AI
• agent evaluation
• exploitability
• benchmark design
• Nash equilibrium