{"type":"ScholarlyArticle","title":"Libratus: The Superhuman AI for No-Limit Poker","authors":[{"name":"Noam Brown","slug":"noam-brown","given":"Noam","family":"Brown","affiliation":"Computer Science Department Carnegie Mellon University"},{"name":"Tuomas Sandholm","slug":"tuomas-sandholm","given":"Tuomas","family":"Sandholm","affiliation":"Computer Science Department Carnegie Mellon University and Strategic Machine, Inc."}],"abstract":"No-limit Texas Hold'em is the most popular variant of poker in the world.  Heads-up no-limit Texas Hold'em is the main benchmark challenge for AI in imperfect-information games. We present Libratus, the first - and so far only - AI to defeat top human professionals in that game. Libratus's architecture features three main modules, each of which has new algorithms: pre-computing a solution to an abstraction of the game which provides a high-level blueprint for the strategy of the AI, a new nested subgame-solving algorithm which repeatedly calculates a more detailed strategy as play progresses, and a self-improving module which augments the pre-computed blueprint over time.","doi":"10.24963/ijcai.2017/772","url":"https://ijcai.org/proceedings/2017/772","pdf":"https://ijcai.org/proceedings/2017/0772.pdf","markdown":"https://ijcai.org/proceedings/2017/772.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/772","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Demos","track":"Demos","number":772,"pages":{"first":5226,"last":5228},"keywords":["Uncertainty in AI: Sequential Decision Making","Knowledge Representation, Reasoning, and Logic: Game Theory","Agent-based and Multi-agent Systems: Noncooperative Games"],"videos":[],"references":[{"position":1,"raw":"[Bard et al., 2014] Nolan Bard, Michael Johanson, and Michael Bowling. Asymmetric abstractions for adversar- ial settings. In International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), pages 501- 508, 2014.","title":"Asymmetric abstractions for adversarial settings","authors":"Bard","year":"2014","venue":"International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bowling et al., 2015] Michael Bowling, Neil Burch, Michael Johanson, and Oskari Tammelin. Heads-up limit hold'em poker is solved. Science, 347(6218):145-149, January 2015.","title":"Heads-up limit hold'em poker is solved","authors":"Michael Bowling, Neil Burch, Michael Johanson, Oskari Tammelin","year":"2015","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brown and Sandholm, 2014] Noam Brown and Tuomas Sandholm. Regret transfer and parameter optimization. In Proceedings of the Twenty-Eighth AAAI Conference on Ar- tificial Intelligence, pages 594-601. AAAI Press, 2014.","title":"Regret transfer and parameter optimization","authors":"Sandholm Brown, Noam Brown, Tuomas Sandholm","year":"2014","venue":"Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brown and Sandholm, 2015] Noam Brown and Tuomas Sandholm. Regret-based pruning in extensive-form games. In Advances in Neural Information Processing Sys- tems, pages 1972-1980, 2015.","title":"Regret-based pruning in extensive-form games","authors":"Sandholm Brown, Noam Brown, Tuomas Sandholm","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Brown and Sandholm, 2016a] Noam Brown and Tuomas Sandholm. Baby Tartanian8: Winning agent from the 2016 annual computer poker competition. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16), pages 4238-4239, 2016.","title":"Baby Tartanian8: Winning agent from the 2016 annual computer poker competition","authors":"Sandholm Brown, Noam Brown, Tuomas Sandholm","year":"2016","venue":"Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16)","doi":null,"url":null,"internal_url":"/proceedings/2016/639"},{"position":6,"raw":"[Brown and Sandholm, 2016b] Noam Brown and Tuomas Sandholm. Strategy-based warm starting for regret min- imization in games. In AAAI Conference on Artificial In- telligence (AAAI), 2016.","title":"Strategy-based warm starting for regret minimization in games","authors":"Sandholm Brown, Noam Brown, Tuomas Sandholm","year":"2016","venue":"AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Brown and Sandholm, 2017a] Noam Brown and Tuomas Sandholm. Reduced space and faster convergence in imperfect-information games via pruning. In International Conference on Machine Learning, 2017.","title":"Reduced space and faster convergence in imperfect-information games via pruning","authors":"Sandholm Brown, Noam Brown, Tuomas Sandholm","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Brown and Sandholm, 2017b] Noam Brown and Tuomas Sandholm. Safe and nested endgame solving for imperfect-information games. In AAAI-17 Workshop on Computer Poker and Imperfect Information Games, 2017.","title":"Safe and nested endgame solving for imperfect-information games","authors":"Sandholm Brown, Noam Brown, Tuomas Sandholm","year":"2017","venue":"AAAI-17 Workshop on Computer Poker and Imperfect Information Games","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Brown et al., 2015] Noam Brown, Sam Ganzfried, and Tuo- mas Sandholm. Hierarchical abstraction, distributed equi- librium computation, and post-processing, with applica- tion to a champion no-limit texas hold'em agent. In Proceedings of the 2015 International Conference on Au- tonomous Agents and Multiagent Systems, pages 7-15. In- ternational Foundation for Autonomous Agents and Mul- tiagent Systems, 2015.","title":"Hierarchical abstraction, distributed equilibrium computation, and post-processing, with application to a champion no-limit texas hold'em agent","authors":"Brown","year":"2015","venue":"Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Burch et al., 2014] Neil Burch, Michael Johanson, and Michael Bowling. Solving imperfect information games using decomposition. In AAAI Conference on Artificial In- telligence (AAAI), pages 602-608, 2014.","title":"Solving imperfect information games using decomposition","authors":"Neil Burch, Michael Johanson, Michael Bowling","year":"2014","venue":"AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Campbell et al., 2002] Murray Campbell, A Joseph Hoane, and Feng-Hsiung Hsu. Deep Blue. Artificial intelligence, 134(1-2):57-83, 2002.","title":"Deep Blue","authors":"Murray Campbell, Feng-Hsiung Joseph Hoane, Hsu","year":"2002","venue":"Artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ganzfried and Sandholm, 2015] Sam Ganzfried and Tuo- mas Sandholm. Endgame solving in large imperfect- information games. In International Conference on Au- tonomous Agents and Multi-Agent Systems (AAMAS), pages 37-45, 2015.","title":"Endgame solving in large imperfectinformation games","authors":"Sandholm ; Ganzfried, Tuomas Sandholm","year":"2015","venue":"International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jackson, 2014] Eric Jackson. A time and space efficient al- gorithm for approximately solving large imperfect infor- mation games. In AAAI Workshop on Computer Poker and Imperfect Information, 2014.","title":"A time and space efficient algorithm for approximately solving large imperfect information games","authors":"Eric Jackson","year":"2014","venue":"AAAI Workshop on Computer Poker and Imperfect Information","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Johanson et al., 2011] Michael Johanson, Kevin Waugh, Michael Bowling, and Martin Zinkevich. Accelerating best response calculation in large extensive games. In Pro- ceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pages 258-265, 2011.","title":"Accelerating best response calculation in large extensive games","authors":"Michael Johanson, Kevin Waugh, Michael Bowling, Martin Zinkevich","year":"2011","venue":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2011/54"},{"position":15,"raw":"[Kroer et al., 2017] Christian Kroer, Kevin Waugh, Fatma Kılınc ¸-Karzan, and Tuomas Sandholm. Theoretical and practical advances on smoothing for extensive-form games. In Proceedings of the ACM Conference on Eco- nomics and Computation (EC), 2017.","title":"Theoretical and practical advances on smoothing for extensive-form games","authors":"Kroer","year":"2017","venue":"Proceedings of the ACM Conference on Economics and Computation (EC)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lanctot et al., 2009] Marc Lanctot, Kevin Waugh, Martin Zinkevich, and Michael Bowling. Monte Carlo sampling for regret minimization in extensive games. In Proceed- ings of the Annual Conference on Neural Information Pro- cessing Systems (NIPS), pages 1078-1086, 2009.","title":"Monte Carlo sampling for regret minimization in extensive games","authors":"Marc Lanctot, Kevin Waugh, Martin Zinkevich, Michael Bowling","year":"2009","venue":"Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Moravcik et al., 2016] Matej Moravcik, Martin Schmid, Karel Ha, Milan Hladik, and Stephen Gaukrodger. Re- fining subgames in large imperfect information games. In AAAI Conference on Artificial Intelligence (AAAI), 2016.","title":"Refining subgames in large imperfect information games","authors":"Matej Moravcik, Martin Schmid, Karel Ha, Milan Hladik, Stephen Gaukrodger","year":"2016","venue":"AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nesterov, 2005] Yurii Nesterov. Excessive gap technique in nonsmooth convex minimization. SIAM Journal of Opti- mization, 16(1):235-249, 2005.","title":"Excessive gap technique in nonsmooth convex minimization","authors":"Yurii Nesterov","year":"2005","venue":"SIAM Journal of Optimization","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Schaeffer et al., 2007] Jonathan Schaeffer, Neil Burch, Yn- gvi Björnsson, Akihiro Kishimoto, Martin Müller, Robert Lake, Paul Lu, and Steve Sutphen. Checkers is solved. Science, 317(5844):1518-1522, 2007.","title":"Checkers is solved","authors":"Jonathan Schaeffer, Neil Burch, Yngvi Björnsson, Akihiro Kishimoto, Martin Müller, Robert Lake, Paul Lu, Steve Sutphen","year":"2007","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Silver et al., 2016] David Silver, Aja Huang, Chris J Maddi- son, Arthur Guez, Laurent Sifre, George Van Den Driess- che, Julian Schrittwieser, Ioannis Antonoglou, Veda Pan- neershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484-489, 2016.","title":"Mastering the game of go with deep neural networks and tree search","authors":"David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den, Marc Driessche, Lanctot","year":"2016","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tammelin et al., 2015] Oskari Tammelin, Neil Burch, Michael Johanson, and Michael Bowling. Solving heads-up limit texas hold'em. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pages 645-652, 2015.","title":"Solving heads-up limit texas hold","authors":"Neil Oskari Tammelin, Michael Burch, Michael Johanson, Bowling","year":"2015","venue":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zinkevich et al., 2007] Martin Zinkevich, Michael Johan- son, Michael H Bowling, and Carmelo Piccione. Regret minimization in games with incomplete information. In Proceedings of the Annual Conference on Neural Informa- tion Processing Systems (NIPS), pages 1729-1736, 2007.","title":"Regret minimization in games with incomplete information","authors":"Martin Zinkevich, Michael Johanson, Michael H Bowling, Carmelo Piccione","year":"2007","venue":"Proceedings of the Annual Conference on Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}