{"type":"ScholarlyArticle","title":"Smooth UCT Search in Computer Poker","authors":[{"name":"Johannes Heinrich","slug":"johannes-heinrich","given":"Johannes","family":"Heinrich","affiliation":""},{"name":"David Silver","slug":"david-silver","given":"David","family":"Silver","affiliation":""}],"abstract":"Self-play Monte Carlo Tree Search (MCTS) has been successful in many perfect-information two-player games. Although these methods have been extended to imperfect-information games, so far they have not achieved the same level of practical success or theoretical convergence guarantees as competing methods. In this paper we introduce Smooth UCT, a variant of the established Upper Confidence Bounds Applied to Trees (UCT) algorithm. Smooth UCT agents mix in their average policy during self-play and the resulting planning process resembles game-theoretic fictitious play. When applied to Kuhn and Leduc poker, Smooth UCT approached a Nash equilibrium, whereas UCT diverged. In addition, Smooth UCT outperformed UCT in Limit Texas Hold'em and won 3 silver medals in the 2014 Annual Computer Poker Competition.","doi":null,"url":"https://ijcai.org/proceedings/2015/84","pdf":"https://ijcai.org/proceedings/2015/0084.pdf","markdown":"https://ijcai.org/proceedings/2015/84.md","bibtex":"https://ijcai.org/proceedings/2015/bibtex/84","volume":{"slug":"2015","title":"Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2015","url":"https://ijcai.org/proceedings/2015/","year":2015,"location":"Buenos Aires, Argentina","dates":"25–31 July 2015","editors":"Qiang Yang and Michael Wooldridge","isbn":"978-1-57735-738-4","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":"Game Theory","track":"Main Track","number":84,"pages":{"first":554,"last":559},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}