Smooth UCT Search in Computer Poker
Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 554-559.
Abstract
Self-play Monte Carlo Tree Search (MCTS) has been successful in many perfect-information two-player games. Although these methods have been extended to imperfect-information games, so far they have not achieved the same level of practical success or theoretical convergence guarantees as competing methods. In this paper we introduce Smooth UCT, a variant of the established Upper Confidence Bounds Applied to Trees (UCT) algorithm. Smooth UCT agents mix in their average policy during self-play and the resulting planning process resembles game-theoretic fictitious play. When applied to Kuhn and Leduc poker, Smooth UCT approached a Nash equilibrium, whereas UCT diverged. In addition, Smooth UCT outperformed UCT in Limit Texas Hold'em and won 3 silver medals in the 2014 Annual Computer Poker Competition.
@inproceedings{ijcai2015p84,
title = {Smooth UCT Search in Computer Poker},
author = {Heinrich, Johannes and Silver, David},
booktitle = {Proceedings of the Twenty-Fourth International Joint Conference on
Artificial Intelligence, {IJCAI-15}},
publisher = {International Joint Conferences on Artificial Intelligence},
editor = {Qiang Yang and Michael Wooldridge},
pages = {554--559},
year = {2015},
month = {7},
note = {Main Track},
url = {https://ijcai.org/proceedings/2015/84},
}
TY - CPAPER TI - Smooth UCT Search in Computer Poker AU - Heinrich, Johannes AU - Silver, David T2 - Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15 PB - International Joint Conferences on Artificial Intelligence PY - 2015 SP - 554 EP - 559 UR - https://ijcai.org/proceedings/2015/84 ER -Copied
Machine-readable versions of this paper, for tools and AI agents:
