May the Best Bot Win

Introduction

Every poker player has watched a fish run good, and every player has watched skill prevail over time, hence the old adage: “Nobody runs better than the pro”. When faced with the problem of ranking players, or poker bots, this raises a vital question: how long must we run a simulation to be confident enough to declare one participant the strongest? And how does this vary across different configurations, game modes and table sizes? Especially when large prizes, tournaments or scientific frontiers are at stake, we must be reasonably certain of our ranking, without spending unnecessary time and compute once the precision we need has been reached.

Keywords

• poker AI
• agent evaluation
• variance reduction
• sample complexity
• AIVAT