{"type":"ScholarlyArticle","title":"Look-ahead Search on Top of Policy Networks in Imperfect Information Games","authors":[{"name":"Ondřej Kubíček","slug":"ondrej-kubicek","given":"Ondřej","family":"Kubíček","affiliation":"Artificial Intelligence Center , Department of Computer Science , Faculty of Electrical Engineering , Czech Technical University in Prague"},{"name":"Neil Burch","slug":"neil-burch","given":"Neil","family":"Burch","affiliation":"Sony AI; Alberta Machine Intelligence Institute , University of Alberta"},{"name":"Viliam Lisý","slug":"viliam-lisy","given":"Viliam","family":"Lisý","affiliation":"Artificial Intelligence Center , Department of Computer Science , Faculty of Electrical Engineering , Czech Technical University in Prague"}],"abstract":"Search in test time is often used to improve the performance of reinforcement learning algorithms. Performing theoretically sound search in fully adversarial two-player games with imperfect information is notoriously difficult and requires a complicated training process. We present a method for adding test-time search to an arbitrary policy-gradient algorithm that learns from sampled trajectories. Besides the policy network, the algorithm trains an additional critic network, which estimates the expected values of players following various transformations of the policies given by the policy network. These values are then used for depth-limited search. We show how the values from this critic can create a value function for imperfect information games. Moreover, they can be used to compute the summary statistics necessary to start the search from an arbitrary decision point in the game. The presented algorithm is scalable to very large games since it does not require any search during train time. We evaluate the algorithm's performance when trained along Regularized Nash Dynamics, and we evaluate the benefit of using the search in the standard benchmark game of Leduc hold'em, multiple variants of imperfect information Goofspiel, and Battleships.","doi":"10.24963/ijcai.2024/480","url":"https://ijcai.org/proceedings/2024/480","pdf":"https://ijcai.org/proceedings/2024/0480.pdf","markdown":"https://ijcai.org/proceedings/2024/480.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/480","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":480,"pages":{"first":4344,"last":4352},"keywords":["Machine Learning: ML: Multiagent Reinforcement Learning","Agent-based and Multi-agent Systems: MAS: Multi-agent learning","Search: S: Game playing","Search: S: Local search"],"videos":[],"references":[{"position":1,"raw":"[Bard et al., 2014] Nolan Bard, Michael Johanson, and Michael Bowling. Asymmetric abstractions for adversar- ial settings. In Proceedings of the 2014 international con- ference on Autonomous agents and multi-agent systems, pages 501-508, 2014.","title":"Asymmetric abstractions for adversarial settings","authors":"Bard","year":"2014","venue":"Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bowling et al., 2009] Michael Bowling, Nicholas Risk, Nolan Bard, Darse Billings, Neil Burch, Joshua Davidson, John Hawkin, Robert Holte, Michael Johanson, Morgan Kan, Bryce Paradis, Jonathan Schaeffer, David Schnizlein, Duane Szafron, Kevin Waugh, and Martin Zinkevich. A demonstration of the polaris poker system. In Adaptive Agents and Multi-Agent Systems, pages 1391-1392, 01 2009.","title":"A demonstration of the polaris poker system","authors":"Michael Bowling, Nicholas Risk, Nolan Bard, Darse Billings, Neil Burch, Joshua Davidson, John Hawkin, Robert Holte, Michael Johanson, Morgan Kan, Bryce Paradis, Jonathan Schaeffer, David Schnizlein, Duane Szafron, Kevin Waugh, Martin Zinkevich","year":"2009","venue":"Adaptive Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brown and Sandholm, 2017] Noam Brown and Tuomas Sandholm. Safe and nested subgame solving for imperfect-information games. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vish- wanathan, and R. Garnett, editors, Advances in Neural In- formation Processing Systems, volume 30.","title":"Safe and nested subgame solving for imperfect-information games","authors":"Sandholm Brown, Noam Brown, Tuomas Sandholm","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brown and Sandholm, 2018] Noam Brown and Tuomas Sandholm. Superhuman ai for heads-up no-limit poker: Libratus beats top professionals. Science, 359(6374):418- 424, 2018.","title":"Noam Brown and Tuomas Sandholm. Superhuman ai for heads-up no-limit poker: Libratus beats top professionals","authors":"Sandholm Brown","year":"2018","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Brown and Sandholm, 2019] Noam Brown and Tuomas Sandholm. Superhuman ai for multiplayer poker. Science, 365(6456):885-890, 2019.","title":"Noam Brown and Tuomas Sandholm. Superhuman ai for multiplayer poker","authors":"Sandholm Brown","year":"2019","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Brown et al., 2018] Noam Brown, Tuomas Sandholm, and Brandon Amos. Depth-limited solving for imperfect- information games.","title":"Depth-limited solving for imperfectinformation games","authors":"Brown","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Gar- nett, editors, Advances in Neural Information Processing Systems, volume 31.","title":"Advances in Neural Information Processing Systems","authors":"S Bengio, H Wallach, H Larochelle, K Grauman, N","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"Curran Associates, Inc., 2018.","title":null,"authors":"Curran Associates, Inc","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Brown et al., 2020] Noam Brown, Anton Bakhtin, Adam Lerer, and Qucheng Gong. Combining deep reinforcement learning and search for imperfect-information games. In","title":"Combining deep reinforcement learning and search for imperfect-information games","authors":"Brown","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Process- ing Systems, volume 33, pages 17057-17069.","title":"Advances in Neural Information Processing Systems","authors":"H Larochelle, M Ranzato, R","year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Burch et al., 2014] Neil Burch, Michael Johanson, and Michael Bowling. Solving imperfect information games using decomposition. Proceedings of the AAAI Confer- ence on Artificial Intelligence, 28(1), Jun. 2014.","title":"Solving imperfect information games using decomposition","authors":"Neil Burch, Michael Johanson, Michael Bowling","year":"2014","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Espeholt et al., 2018] Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Vlad Mnih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, Iain Dunning, Shane Legg, and Koray Kavukcuoglu. IMPALA: Scalable dis- tributed deep-RL with importance weighted actor-learner architectures. In Jennifer Dy and Andreas Krause, edi- tors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 1407-1416. PMLR, 10-15 Jul 2018.","title":"IMPALA: Scalable distributed deep-RL with importance weighted actor-learner architectures","authors":"Espeholt","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gruslys et al., 2020] Audrunas Gruslys, Marc Lanctot, Rémi Munos, Finbarr Timbers, Martin Schmid, Julien Pérolat, Dustin Morrill, Vinícius Flores Zambaldi, Jean- Baptiste Lespiau, John Schultz, Mohammad Gheshlaghi Azar, Michael Bowling, and Karl Tuyls. The advan- tage regret-matching actor-critic. CoRR, abs/2008.12234, 2020.","title":"The advantage regret-matching actor-critic","authors":"Gruslys","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hennes et al., 2020] Daniel Hennes, Dustin Morrill, Shayegan Omidshafiei, Rémi Munos, Julien Perolat, Marc Lanctot, Audrunas Gruslys, Jean-Baptiste Lespiau, Paavo Parmas, Edgar Duèñez Guzmán, and Karl Tuyls. Neural replicator dynamics: Multiagent learning via hedging policy gradients. In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS '20, page 492-501, Richland, SC, 2020. International Foundation for Autonomous Agents and Multiagent Systems.","title":"Neural replicator dynamics: Multiagent learning via hedging policy gradients","authors":"Daniel Hennes, Dustin Morrill, Shayegan Omidshafiei, Rémi Munos, Julien Perolat, Marc Lanctot, Audrunas Gruslys, Jean-Baptiste Lespiau, Paavo Parmas, Edgar Duèñez Guzmán, Karl Tuyls","year":"2020","venue":"Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS '20","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kovařík et al., 2022] Vojtěch Kovařík, Martin Schmid, Neil Burch, Michael Bowling, and Viliam Lisý. Rethinking formal models of partially observable multiagent decision making. Artificial Intelligence, 303:103645, 2022.","title":"Rethinking formal models of partially observable multiagent decision making","authors":"Vojtěch Kovařík, Martin Schmid, Neil Burch, Michael Bowling, Viliam Lisý","year":"2022","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kovařík et al., 2023] Vojtěch Kovařík, Dominik Seitz, Vil- iam Lisý, Jan Rudolf, Shuo Sun, and Karel Ha. Value functions for depth-limited solving in zero-sum imperfect- information games. Artificial Intelligence, 314:103805, 2023.","title":"Value functions for depth-limited solving in zero-sum imperfectinformation games","authors":"Vojtěch Kovařík, Dominik Seitz, Viliam Lisý, Jan Rudolf, Shuo Sun, Karel Ha","year":"2023","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lanctot et al., 2017] Marc Lanctot, Vinicius Zambaldi, Au- drunas Gruslys, Angeliki Lazaridou, Karl Tuyls, Julien Perolat, David Silver, and Thore Graepel. A unified game- theoretic approach to multiagent reinforcement learning. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Ad- vances in Neural Information Processing Systems, vol- ume 30.","title":"A unified gametheoretic approach to multiagent reinforcement learning","authors":"Marc Lanctot, Vinicius Zambaldi, Audrunas Gruslys, Angeliki Lazaridou, Karl Tuyls, Julien Perolat, David Silver, Thore Graepel","year":null,"venue":"Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"Curran Associates, Inc., 2017.","title":null,"authors":"Curran Associates, Inc","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lanctot et al., 2019] Marc Lanctot, Edward Lockhart, Jean- Baptiste Lespiau, Vinicius Zambaldi, Satyaki Upadhyay, Julien Pérolat, Sriram Srinivasan, Finbarr Timbers, Karl Tuyls, Shayegan Omidshafiei, Daniel Hennes, Dustin Morrill, Paul Muller, Timo Ewalds, Ryan Faulkner, János Kramár, Bart De Vylder, Brennan Saeta, James Brad- bury, David Ding, Sebastian Borgeaud, Matthew Lai, Julian Schrittwieser, Thomas Anthony, Edward Hughes, Ivo Danihelka, and Jonah Ryan-Davis. OpenSpiel: A framework for reinforcement learning in games. CoRR, abs/1908.09453, 2019.","title":"OpenSpiel: A framework for reinforcement learning in games","authors":"Marc Lanctot, Edward Lockhart, Jean-Baptiste Lespiau, Vinicius Zambaldi, Satyaki Upadhyay, Julien Pérolat, Sriram Srinivasan, Finbarr Timbers, Karl Tuyls, Shayegan Omidshafiei, Daniel Hennes, Dustin Morrill, Paul Muller, Timo Ewalds, Ryan Faulkner, János Kramár, Bart De Vylder, Brennan Saeta, James Bradbury, David Ding, Sebastian Borgeaud, Matthew Lai, Julian Schrittwieser, Thomas Anthony, Edward Hughes, Ivo Danihelka, Jonah Ryan-Davis","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[McAleer et al., 2023] Stephen Marcus McAleer, Gabriele Farina, Marc Lanctot, and Tuomas Sandholm. ESCHER: Eschewing importance sampling in games by comput- ing a history value function to estimate regret. In The Eleventh International Conference on Learning Represen- tations, 2023.","title":"ESCHER: Eschewing importance sampling in games by computing a history value function to estimate regret","authors":"Stephen Marcus Mcaleer, Gabriele Farina, Marc Lanctot, Tuomas Sandholm","year":"2023","venue":"The Eleventh International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[McMahan, 2011] Brendan McMahan. Follow-the- regularized-leader and mirror descent: Equivalence theorems and l1 regularization. In Geoffrey Gordon, David Dunson, and Miroslav Dudík, editors, Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15 of Proceedings of Machine Learning Research, pages 525-533, Fort Lauderdale, FL, USA, 11-13 Apr 2011. PMLR. Moravcik et al., 2016]","title":"Follow-theregularized-leader and mirror descent: Equivalence theorems and l1 regularization","authors":"Brendan Mcmahan","year":"2011","venue":"Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"Matej Moravcik, Martin Schmid, Karel Ha, Milan Hladik, and Stephen Gaukrodger. Refin- ing subgames in large imperfect information games. Pro- ceedings of the AAAI Conference on Artificial Intelligence, 30(1), Feb. 2016.","title":"Refining subgames in large imperfect information games","authors":"Matej Moravcik, Martin Schmid, Karel Ha, Milan Hladik, Stephen Gaukrodger","year":"2016","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Moravčík et al., 2017] Matej Moravčík, Martin Schmid, Neil Burch, Viliam Lisý, Dustin Morrill, Nolan Bard, Trevor Davis, Kevin Waugh, Michael Johanson, and Michael Bowling. Deepstack: Expert-level artifi- cial intelligence in heads-up no-limit poker. Science, 356(6337):508-513, 2017.","title":"Deepstack: Expert-level artificial intelligence in heads-up no-limit poker","authors":"Matej Moravčík, Martin Schmid, Neil Burch, Viliam Lisý, Dustin Morrill, Nolan Bard, Trevor Davis, Kevin Waugh, Michael Johanson, Michael Bowling","year":"2017","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Perolat et al., 2021] Julien Perolat, Remi Munos, Jean- Baptiste Lespiau, Shayegan Omidshafiei, Mark Rowland, Pedro Ortega, Neil Burch, Thomas Anthony, David Bal- duzzi, Bart De Vylder, Georgios Piliouras, Marc Lanctot, and Karl Tuyls. From poincaré recurrence to convergence in imperfect information games: Finding equilibrium via regularization. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Ma- chine Learning, volume 139 of Proceedings of Machine Learning Research, pages 8525-8535. PMLR, 18-24 Jul 2021.","title":"From poincaré recurrence to convergence in imperfect information games: Finding equilibrium via regularization","authors":"Julien Perolat, Remi Munos, Jean-Baptiste Lespiau, Shayegan Omidshafiei, Mark Rowland, Pedro Ortega, Neil Burch, Thomas Anthony, David Balduzzi, Bart De Vylder, Georgios Piliouras, Marc Lanctot, Karl Tuyls","year":"2021","venue":"Proceedings of the 38th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Perolat et al., 2022] Julien Perolat, Bart De Vylder, Daniel Hennes, Eugene Tarassov, Florian Strub, Vincent de Boer, Paul Muller, Jerome T Connor, Neil Burch, Thomas Anthony, et al. Mastering the game of stratego with model-free multiagent reinforcement learning. Science, 378(6623):990-996, 2022.","title":"Mastering the game of stratego with model-free multiagent reinforcement learning","authors":"Julien Perolat, Bart De Vylder, Daniel Hennes, Eugene Tarassov, Florian Strub, Paul Vincent De Boer, Jerome T Muller, Neil Connor, Thomas Burch, Anthony","year":"2022","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Russell and Norvig, 2009] Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall Press, USA, 3rd edition, 2009.","title":"Artificial Intelligence: A Modern Approach","authors":"Stuart Russell, Peter Norvig","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Schmid et al., 2023] Martin Schmid, Matej Moravčík, Neil Burch, Rudolf Kadlec, Josh Davidson, Kevin Waugh, Nolan Bard, Finbarr Timbers, Marc Lanctot, G. Zacharias Holland, Elnaz Davoodi, Alden Christianson, and Michael Bowling. Student of games: A unified learning algorithm for both perfect and imperfect information games. Science Advances, 9(46):eadg3256, 2023.","title":"Student of games: A unified learning algorithm for both perfect and imperfect information games","authors":"Martin Schmid, Matej Moravčík, Neil Burch, Rudolf Kadlec, Josh Davidson, Kevin Waugh, Nolan Bard, Finbarr Timbers, Marc Lanctot, G Zacharias Holland, Elnaz Davoodi, Alden Christianson, Michael Bowling","year":"2023","venue":"Science Advances","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Seitz et al., 2021] Dominik Seitz, Nikita Milyukov, and Vil- iam Lisý. Learning to guess opponent's information in large partially observable games. In Proc. AAAI Workshop Reinforcement Learning in Games, 2021.","title":"Learning to guess opponent's information in large partially observable games","authors":"Dominik Seitz, Nikita Milyukov, Viliam Lisý","year":"2021","venue":"Proc. AAAI Workshop Reinforcement Learning in Games","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Silver et al., 2018] David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. A general reinforcement learning algo- rithm that masters chess, shogi, and go through self-play. Science, 362(6419):1140-1144, 2018.","title":"A general reinforcement learning algorithm that masters chess, shogi, and go through self-play","authors":"David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, Demis Hassabis","year":"2018","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Sokota et al., 2023] Samuel Sokota, Ryan D'Orazio, J. Zico Kolter, Nicolas Loizou, Marc Lanctot, Ioannis Mitliagkas, Noam Brown, and Christian Kroer. A unified approach to reinforcement learning, quantal response equilibria, and two-player zero-sum games, 2023.","title":"A unified approach to reinforcement learning, quantal response equilibria, and two-player zero-sum games","authors":"Samuel Sokota, D' Ryan, J Zico Orazio, Nicolas Kolter, Marc Loizou, Ioannis Lanctot, Noam Mitliagkas, Christian Brown, Kroer","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Steinberger et al., 2020] Eric Steinberger, Adam Lerer, and Noam Brown. DREAM: deep regret minimization with advantage baselines and model-free learning. CoRR, abs/2006.10410, 2020.","title":"DREAM: deep regret minimization with advantage baselines and model-free learning","authors":"Eric Steinberger, Adam Lerer, Noam Brown","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Tammelin et al., 2015] Oskari Tammelin, Neil Burch, Michael Johanson, and Michael Bowling. Solving heads- up limit texas hold'em. In Qiang Yang and Michael J. Wooldridge, editors, Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, July 25-31, 2015, pages 645-652. AAAI Press, 2015.","title":"Solving headsup limit texas hold","authors":"Neil Oskari Tammelin, Michael Burch, Michael Johanson, Bowling","year":"2015","venue":"Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Šustr et al., 2021] Michal Šustr, Martin Schmid, Matej Moravčík, Neil Burch, Marc Lanctot, and Michael Bowl- ing. Sound algorithms in imperfect information games. In Proceedings of the 20th International Conference on Au- tonomous Agents and MultiAgent Systems, AAMAS '21, page 1674-1676, Richland, SC, 2021. International Foun- dation for Autonomous Agents and Multiagent Systems.","title":"Sound algorithms in imperfect information games","authors":"Michal Šustr, Martin Schmid, Matej Moravčík, Neil Burch, Marc Lanctot, Michael Bowling","year":"2021","venue":"Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS '21","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wang et al., 2023] Tony Tong Wang, Adam Gleave, Nora Belrose, Tom Tseng, Michael D Dennis, Yawen Duan, Viktor Pogrebniak, Sergey Levine, and Stuart Russell. Ad- versarial policies beat professional-level go AIs, 2023.","title":"Adversarial policies beat professional-level go AIs","authors":"Tony Tong, Wang, Adam Gleave, Nora Belrose, Tom Tseng, D Michael, Yawen Dennis, Viktor Duan, Sergey Pogrebniak, Stuart Levine, Russell","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhang and Sandholm, 2021] Brian Zhang and Tuomas Sandholm. Subgame solving without common knowl- edge. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 23993-24004.","title":"Subgame solving without common knowledge","authors":"Brian Zhang, Tuomas Sandholm","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"Curran Associates, Inc., 2021.","title":null,"authors":"Curran Associates, Inc","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zinkevich et al., 2007] Martin Zinkevich, Michael Johan- son, Michael Bowling, and Carmelo Piccione. Regret minimization in games with incomplete information. In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Ad- vances in Neural Information Processing Systems, vol- ume 20.","title":"Regret minimization in games with incomplete information","authors":"Martin Zinkevich, Michael Johanson, Michael Bowling, Carmelo Piccione","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"Curran Associates, Inc., 2007.","title":null,"authors":"Curran Associates, Inc","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}