{"type":"ScholarlyArticle","title":"An Algorithm for Constructing and Solving Imperfect Recall Abstractions of Large Extensive-Form Games","authors":[{"name":"Jiri Cermak","slug":"jiri-cermak","given":"Jiri","family":"Cermak","affiliation":"Department of Computer Science , Czech Technical University in Prague"},{"name":"Branislav Bošanský","slug":"branislav-bosansky","given":"Branislav","family":"Bošanský","affiliation":"Department of Computer Science , Czech Technical University in Prague"},{"name":"Viliam Lisý","slug":"viliam-lisy","given":"Viliam","family":"Lisý","affiliation":"Department of Computer Science , Czech Technical University in Prague"}],"abstract":"We solve large two-player zero-sum extensive-form games with perfect recall. We propose a new algorithm based on fictitious play that significantly reduces memory requirements for storing average strategies. The key feature is exploiting imperfect recall abstractions while preserving the convergence rate and guarantees of fictitious play applied directly to the perfect recall game. The algorithm creates a coarse imperfect recall abstraction of the perfect recall game and automatically refines its information set structure only where the imperfect recall might cause problems. Experimental evaluation shows that our novel algorithm is able to solve a simplified poker game with 7.10^5 information sets using an abstracted game with only 1.8% of information sets of the original game. Additional experiments on poker and randomly generated games suggest that the relative size of the abstraction decreases as the size of the solved games increases.","doi":"10.24963/ijcai.2017/130","url":"https://ijcai.org/proceedings/2017/130","pdf":"https://ijcai.org/proceedings/2017/0130.pdf","markdown":"https://ijcai.org/proceedings/2017/130.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/130","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Knowledge Representation, Reasoning, and Logic","track":"Main track","number":130,"pages":{"first":936,"last":942},"keywords":["Knowledge Representation, Reasoning, and Logic: Game Theory","Agent-based and Multi-agent Systems: Noncooperative Games"],"videos":[],"references":[{"position":1,"raw":"[Bošanský et al., 2015] Branislav Bošanský, Albert Xin Jiang, Milind Tambe, and Christopher Kiekintveld. Com- bining compact representation and incremental generation in large games with sequential strategies. In AAAI, pages 812-818, 2015.","title":"Combining compact representation and incremental generation in large games with sequential strategies","authors":"Branislav Bošanský, Albert Xin Jiang, Milind Tambe, Christopher Kiekintveld","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bošanský et al., 2016] Branislav Bošanský, Viliam Lisý, Marc Lanctot, Jiří Čermák, and Mark H.M. Winands. Al- gorithms for Computing Strategies in Two-Player Simul- taneous Move Games. Artificial Intelligence, 237:1-40, 2016.","title":"Algorithms for Computing Strategies in Two-Player Simultaneous Move Games","authors":"Branislav Bošanský, Viliam Lisý, Marc Lanctot, Jiří Čermák, H M Mark, Winands","year":"2016","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bošanský et al., 2014] Branislav Bošanský, Christopher Kiekintveld, Viliam Lisý, and Michal Pěchouček. An Exact Double-Oracle Algorithm for Zero-Sum Extensive- Form Games with Imperfect Information. Journal of Arti- ficial Intelligence Research, 51:829-866, 2014.","title":"An Exact Double-Oracle Algorithm for Zero-Sum Extensive-Form Games with Imperfect Information","authors":"Bošanský","year":"2014","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brown and Sandholm, 2015] Noam Brown and Tuomas W Sandholm. Simultaneous abstraction and equilibrium find- ing in games. In AAAI, 2015.","title":"Simultaneous abstraction and equilibrium finding in games","authors":"Sandholm Brown, Noam Brown, Tuomas W Sandholm","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":"/proceedings/2015/75"},{"position":5,"raw":"[Brown, 1949] George W Brown. Some notes on computa- tion of games solutions. Technical report, DTIC Docu- ment, 1949.","title":"Some notes on computation of games solutions","authors":"George W Brown","year":"1949","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Čermák et al., 2017] Jiři Čermák, Branislav Bošanskỳ, and Michal Pěchouček. Combining incremental strategy gen- eration and branch and bound search for computing maxmin strategies in imperfect recall games. In AAMAS, pages 902-910, 2017.","title":"Combining incremental strategy generation and branch and bound search for computing maxmin strategies in imperfect recall games","authors":"Čermák","year":"2017","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen and Bowling, 2012] Katherine Chen and Michael Bowling. Tractable objectives for robust policy optimiza- tion. In NIPS, pages 2078-2086, 2012.","title":"Tractable objectives for robust policy optimization","authors":"Katherine Chen, Michael Bowling","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Daskalakis and Pan, 2014] Constantinos Daskalakis and Qinxuan Pan. A counter-example to karlin's strong conjecture for fictitious play. In Annual Symposium on Foundations of Computer Science, pages 11-20. IEEE, 2014.","title":"A counter-example to karlin's strong conjecture for fictitious play","authors":"Constantinos Daskalakis, Qinxuan Pan","year":"2014","venue":"Annual Symposium on Foundations of Computer Science","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gilpin and Sandholm, 2007] Andrew Gilpin and Tuomas Sandholm. Lossless Abstraction of Imperfect Information Games. Journal of the ACM, 54(5), 2007.","title":"Lossless Abstraction of Imperfect Information Games","authors":"Sandholm Gilpin, Andrew Gilpin, Tuomas Sandholm","year":"2007","venue":"Journal of the ACM","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gilpin et al., 2007] Andrew Gilpin, Tuomas Sandholm, and Troels Bjerre Sørensen. Potential-aware automated ab- straction of sequential games, and holistic equilibrium analysis of texas hold'em poker. In AAAI, pages 50-57, 2007.","title":"Potential-aware automated abstraction of sequential games, and holistic equilibrium analysis of texas hold'em poker","authors":"Andrew Gilpin, Tuomas Sandholm, Troels Bjerre Sørensen","year":"2007","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Heinrich et al., 2015] Johannes Heinrich, Marc Lanctot, and David Silver. Fictitious self-play in extensive-form games. In ICML, pages 805-813, 2015.","title":"Fictitious self-play in extensive-form games","authors":"Johannes Heinrich, Marc Lanctot, David Silver","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Johanson et al., 2011] Michael Johanson, Kevin Waugh, Michael Bowling, and Martin Zinkevich. Accelerating best response calculation in large extensive games. In IJ- CAI, volume 11, pages 258-265, 2011.","title":"Accelerating best response calculation in large extensive games","authors":"Michael Johanson, Kevin Waugh, Michael Bowling, Martin Zinkevich","year":"2011","venue":"IJ-CAI","doi":null,"url":null,"internal_url":"/proceedings/2011/54"},{"position":13,"raw":"[Karlin, 2003] Samuel Karlin. Mathematical methods and theory in games, programming, and economics, volume 2. Courier Corporation, 2003.","title":"Samuel Karlin","authors":"Karlin","year":"2003","venue":"Mathematical methods and theory in games, programming, and economics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kroer and Sandholm, 2014] Christian Kroer and Tuomas Sandholm. Extensive-Form Game Abstraction with Bounds. In EC, pages 621-638. ACM, 2014.","title":"Christian Kroer and Tuomas Sandholm. Extensive-Form Game Abstraction with Bounds","authors":"Sandholm Kroer","year":"2014","venue":"EC","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kroer and Sandholm, 2016] Christian Kroer and Tuomas Sandholm. Imperfect-Recall Abstractions with Bounds in Games. In EC, pages 459-476. ACM, 2016.","title":"Christian Kroer and Tuomas Sandholm. Imperfect-Recall Abstractions with Bounds in Games","authors":"Sandholm Kroer","year":"2016","venue":"EC","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kuhn, 1953] Harold W. Kuhn. Extensive Games and the Problem of Information. Contributions to the Theory of Games, II:193-216, 1953.","title":"Extensive Games and the Problem of Information","authors":"Harold W Kuhn","year":"1953","venue":"Contributions to the Theory of Games","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lanctot et al., 2012] Marc Lanctot, Richard Gibson, Neil Burch, Martin Zinkevich, and Michael Bowling. No- Regret Learning in Extensive-Form Games with Imperfect Recall. In ICML, pages 1-21, 2012.","title":"No-Regret Learning in Extensive-Form Games with Imperfect Recall","authors":"Marc Lanctot, Richard Gibson, Neil Burch, Martin Zinkevich, Michael Bowling","year":"2012","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lisý et al., 2016] Viliam Lisý, Trevor Davis, and Michael Bowling. Counterfactual Regret Minimization in Sequen- tial Security Games. In AAAI, 2016.","title":"Counterfactual Regret Minimization in Sequential Security Games","authors":"Viliam Lisý, Trevor Davis, Michael Bowling","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Moravčík et al., 2017] Matej Moravčík, Martin Schmid, Neil Burch, Viliam Lisý, Dustin Morrill, Nolan Bard, Trevor Davis, Kevin Waugh, Michael Johanson, and Michael Bowling. Deepstack: Expert-level artificial in- telligence in heads-up no-limit poker. Science, 2017.","title":"Deepstack: Expert-level artificial intelligence in heads-up no-limit poker","authors":"Matej Moravčík, Martin Schmid, Neil Burch, Viliam Lisý, Dustin Morrill, Nolan Bard, Trevor Davis, Kevin Waugh, Michael Johanson, Michael Bowling","year":"2017","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Robinson, 1951] Julia Robinson. An iterative method of solving a game. Annals of mathematics, pages 296-301, 1951.","title":"An iterative method of solving a game","authors":"Julia Robinson","year":"1951","venue":"Annals of mathematics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Schmid et al., 2014] Martin Schmid, Matej Moravcik, and Milan Hladik. Bounding the support size in extensive form games with imperfect information. In AAAI, pages 784- 790, 2014.","title":"Bounding the support size in extensive form games with imperfect information","authors":"Martin Schmid, Matej Moravcik, Milan Hladik","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[von Stengel, 1996] Bernhard von Stengel. Efficient Com- putation of Behavior Strategies. Games and Economic Be- havior, 14:220-246, 1996.","title":"Efficient Computation of Behavior Strategies","authors":"Von Stengel, ; Bernhard, Von Stengel","year":"1996","venue":"Games and Economic Behavior","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Waugh et al., 2009] Kevin Waugh, Martin Zinkevich, Michael Johanson, Morgan Kan, David Schnizlein, and Michael H Bowling. A Practical Use of Imperfect Recall. In Symposium on Abstraction, Reformulation and Approximation (SARA), 2009.","title":"A Practical Use of Imperfect Recall","authors":"Kevin Waugh, Martin Zinkevich, Michael Johanson, Morgan Kan, David Schnizlein, Michael H Bowling","year":"2009","venue":"Symposium on Abstraction, Reformulation and Approximation (SARA)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zinkevich et al., 2008] M. Zinkevich, M. Johanson, M. Bowling, and C. Piccione. Regret Minimization in Games with Incomplete Information. In NIPS, pages 1729-1736, 2008.","title":"Regret Minimization in Games with Incomplete Information","authors":"M Zinkevich, M Johanson, M Bowling, C Piccione","year":"2008","venue":"NIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}