{"type":"ScholarlyArticle","title":"Automated Construction of Bounded-Loss Imperfect-Recall Abstractions in Extensive-Form Games (Extended Abstract)","authors":[{"name":"Jiří Čermák","slug":"jiri-cermak","given":"Jiri","family":"Cermak","affiliation":"Faculty of Electrical Engineering , Czech Technical University in Prague"},{"name":"Viliam Lisý","slug":"viliam-lisy","given":"Viliam","family":"Lisý","affiliation":"Faculty of Electrical Engineering , Czech Technical University in Prague"},{"name":"Branislav Bošanský","slug":"branislav-bosansky","given":"Branislav","family":"Bošanský","affiliation":"Artificial Intelligence Center , Department of Computer Science ,"}],"abstract":"Information abstraction is one of the methods for tackling large extensive-form games (EFGs). Removing some information available to players reduces the memory required for computing and storing strategies. We present novel domain-independent abstraction methods for creating very coarse abstractions of EFGs that still compute strategies that are (near) optimal in the original game. First, the methods start with an arbitrary abstraction of the original game (domain-specific or the coarsest possible). Next, they iteratively detect which information is required in the abstract game so that a (near) optimal strategy in the original game can be found and include this information into the abstract game. Moreover, the methods are able to exploit imperfect-recall abstractions where players can even forget the history of their own actions. We present two algorithms that follow these steps -- FPIRA, based on fictitious play, and CFR+IRA, based on counterfactual regret minimization. \r\nThe experimental evaluation confirms that our methods can closely approximate Nash equilibrium of large games using abstraction with only 0.9% of information sets of the original game.","doi":"10.24963/ijcai.2020/701","url":"https://ijcai.org/proceedings/2020/701","pdf":"https://ijcai.org/proceedings/2020/0701.pdf","markdown":"https://ijcai.org/proceedings/2020/701.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/701","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Journal track","track":"Journal track","number":701,"pages":{"first":5030,"last":5034},"keywords":["Uncertainty in AI: Sequential Decision Making","Agent-based and Multi-agent Systems: Noncooperative Games"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25408/file_25408.m3u8","length_sec":283,"provider":"hls"}],"references":[{"position":1,"raw":"[Bard et al., 2013] Nolan Bard, Michael Johanson, Neil Burch, and Michael Bowling. Online implicit agent mod- elling. In Proceedings of the 2013 international con- ference on Autonomous agents and multi-agent systems, pages 255-262. International Foundation for Autonomous Agents and Multiagent Systems, 2013.","title":"Online implicit agent modelling","authors":"Bard","year":"2013","venue":"Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Billings et al., 2003] Darse Billings, Neil Burch, Aaron Davidson, Robert Holte, Jonathan Schaeffer, Terence Schauenberg, and Duane Szafron. Approximating game- theoretic optimal strategies for full-scale poker. In IJCAI, pages 661-668, 2003.","title":"Approximating gametheoretic optimal strategies for full-scale poker","authors":"Darse Billings, Neil Burch, Aaron Davidson, Robert Holte, Jonathan Schaeffer, Terence Schauenberg, Duane Szafron","year":"2003","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bowling et al., 2015] Michael Bowling, Neil Burch, Michael Johanson, and Oskari Tammelin. Heads-up limit hold'em poker is solved. Science, 347(6218):145-149, 2015.","title":"Heads-up limit hold'em poker is solved","authors":"Michael Bowling, Neil Burch, Michael Johanson, Oskari Tammelin","year":"2015","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bošanský et al., 2014] Branislav Bošanský, Christopher Kiekintveld, Viliam Lisý, and Michal Pěchouček. An Exact Double-Oracle Algorithm for Zero-Sum Extensive- Form Games with Imperfect Information. Journal of Arti- ficial Intelligence Research, 51:829-866, 2014.","title":"An Exact Double-Oracle Algorithm for Zero-Sum Extensive-Form Games with Imperfect Information","authors":"Bošanský","year":"2014","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Brown and Sandholm, 2015] Noam Brown and Tuomas Sandholm. Simultaneous abstraction and equilibrium finding in games. In Proceedings of the 24th Interna- tional Conference on Artificial Intelligence, pages 489- 496. AAAI Press, 2015.","title":"Simultaneous abstraction and equilibrium finding in games","authors":"Sandholm Brown, Noam Brown, Tuomas Sandholm","year":"2015","venue":"Proceedings of the 24th International Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2015/75"},{"position":6,"raw":"[Brown and Sandholm, 2017] Noam Brown and Tuomas Sandholm. Superhuman AI for heads-up no-limit poker: Libratus beats top professionals. Science, page eaao1733, 2017.","title":"Superhuman AI for heads-up no-limit poker: Libratus beats top professionals","authors":"Sandholm Brown, Noam Brown, Tuomas Sandholm","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Brown et al., 2018] Noam Brown, Tuomas Sandholm, and Brandon Amos. Depth-limited solving for imperfect- information games. arXiv preprint arXiv:1805.08195, 2018.","title":"Depth-limited solving for imperfectinformation games","authors":"Brown","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Burch et al., 2014] Neil Burch, Michael Johanson, and Michael Bowling. Solving imperfect information games using decomposition. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, pages 602- 608. AAAI Press, 2014.","title":"Solving imperfect information games using decomposition","authors":"Neil Burch, Michael Johanson, Michael Bowling","year":"2014","venue":"Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Čermák et al., 2017] Jiří Čermák, Branislav Bošansky, and Viliam Lisy. An algorithm for constructing and solv- ing imperfect recall abstractions of large extensive-form games. In Proceedings of the 26th International Joint Con- ference on Artificial Intelligence, pages 936-942. AAAI Press, 2017.","title":"An algorithm for constructing and solving imperfect recall abstractions of large extensive-form games","authors":"Čermák","year":"2017","venue":"Proceedings of the 26th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/130"},{"position":10,"raw":"[Čermák et al., 2020] Jiří Čermák, Viliam Lisý, and Branislav Bošanský. Automated construction of bounded- loss imperfect-recall abstractions in extensive-form games. Artificial Intelligence, 282(103248), 2020.","title":"Automated construction of boundedloss imperfect-recall abstractions in extensive-form games","authors":"Čermák","year":"2020","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gilpin and Sandholm, 2007] Andrew Gilpin and Tuomas Sandholm. Lossless abstraction of imperfect information games. Journal of the ACM (JACM), 54(5):25, 2007.","title":"Lossless abstraction of imperfect information games","authors":"Sandholm Gilpin, Andrew Gilpin, Tuomas Sandholm","year":"2007","venue":"Journal of the ACM (JACM)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gilpin et al., 2007] Andrew Gilpin, Tuomas Sandholm, and Troels Bjerre Sørensen. Potential-aware automated ab- straction of sequential games, and holistic equilibrium analysis of texas hold'em poker. In Proceedings of the National Conference on Artificial Intelligence, volume 22, page 50, 2007.","title":"Potential-aware automated abstraction of sequential games, and holistic equilibrium analysis of texas hold'em poker","authors":"Andrew Gilpin, Tuomas Sandholm, Troels Bjerre Sørensen","year":"2007","venue":"Proceedings of the National Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hoda et al., 2010] Samid Hoda, Andrew Gilpin, Javier Peña, and Tuomas Sandholm. Smoothing Techniques for Computing Nash Equilibria of Sequential Games. Math- ematics of Operations Research, 35(2):494-512, May 2010.","title":"Smoothing Techniques for Computing Nash Equilibria of Sequential Games","authors":"Samid Hoda, Andrew Gilpin, Javier Peña, Tuomas Sandholm","year":"2010","venue":"Mathematics of Operations Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Johanson et al., 2012] Michael Johanson, Nolan Bard, Neil Burch, and Michael Bowling. Finding optimal abstract strategies in extensive-form games. In Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, pages 1371-1379. AAAI Press, 2012.","title":"Finding optimal abstract strategies in extensive-form games","authors":"Michael Johanson, Nolan Bard, Neil Burch, Michael Bowling","year":"2012","venue":"Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kroer and Sandholm, 2014] Christian Kroer and Tuomas Sandholm. Extensive-Form Game Abstraction with Bounds. In Proceedings of the fifteenth ACM conference on Economics and computation, pages 621-638. ACM, 2014.","title":"Christian Kroer and Tuomas Sandholm. Extensive-Form Game Abstraction with Bounds","authors":"Sandholm Kroer","year":"2014","venue":"Proceedings of the fifteenth ACM conference on Economics and computation","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kroer and Sandholm, 2016] Christian Kroer and Tuomas Sandholm. Imperfect-Recall Abstractions with Bounds in Games. In Proceedings of the seventeenth ACM con- ference on Economics and computation, pages 459-476. ACM, 2016.","title":"Christian Kroer and Tuomas Sandholm. Imperfect-Recall Abstractions with Bounds in Games","authors":"Sandholm Kroer","year":"2016","venue":"Proceedings of the seventeenth ACM conference on Economics and computation","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lisý et al., 2015] Viliam Lisý, Marc Lanctot, and Michael Bowling. Online monte carlo counterfactual regret min- imization for search in imperfect information games. In Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems, pages 27- 36. International Foundation for Autonomous Agents and Multiagent Systems, 2015.","title":"Online monte carlo counterfactual regret minimization for search in imperfect information games","authors":"Viliam Lisý, Marc Lanctot, Michael Bowling","year":"2015","venue":"Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Moravčík et al., 2017] Matej Moravčík, Martin Schmid, Neil Burch, Viliam Lisý, Dustin Morrill, Nolan Bard, Trevor Davis, Kevin Waugh, Michael Johanson, and Michael Bowling. Deepstack: Expert-level artificial in- telligence in heads-up no-limit poker. Science, 2017. Rehmeyer et al., 2008]","title":"Deepstack: Expert-level artificial intelligence in heads-up no-limit poker","authors":"Matej Moravčík, Martin Schmid, Neil Burch, Viliam Lisý, Dustin Morrill, Nolan Bard, Trevor Davis, Kevin Waugh, Michael Johanson, Michael Bowling","year":"2008","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"Julie Rehmeyer, Nathan Fox, and Renzi Rico. Ante up, human: The adventures of polaris the poker-playing robot. Wired, 16.12:186-191, Decem- ber 2008.","title":"Ante up, human: The adventures of polaris the poker-playing robot","authors":"Julie Rehmeyer, Nathan Fox, Renzi Rico","year":"2008","venue":"Wired","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shi and Littman, 2000] Jiefu Shi and Michael L Littman. Abstraction methods for game theoretic poker. Comput- ers and Games, 2063:333-345, 2000.","title":"Abstraction methods for game theoretic poker","authors":"Littman Shi, Jiefu Shi, Michael L Littman","year":"2000","venue":"Computers and Games","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tammelin, 2014] Oskari Tammelin. Cfr+. CoRR, abs/1407.5042, 2014.","title":"Cfr+. CoRR","authors":"Oskari Tammelin","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[von Stengel, 1996] Bernhard von Stengel. Efficient Com- putation of Behavior Strategies. Games and Economic Be- havior, 14:220-246, 1996.","title":"Efficient Computation of Behavior Strategies","authors":"Von Stengel, ; Bernhard, Von Stengel","year":"1996","venue":"Games and Economic Behavior","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zinkevich et al., 2007] Martin Zinkevich, Michael Johan- son, Michael H Bowling, and Carmelo Piccione. Re- gret minimization in games with incomplete information. In Advances in Neural Information Processing Systems, pages 1729-1736, 2007.","title":"Regret minimization in games with incomplete information","authors":"Martin Zinkevich, Michael Johanson, Michael H Bowling, Carmelo Piccione","year":"2007","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}