{"type":"ScholarlyArticle","title":"A Meta-Game Evaluation Framework for Deep Multiagent Reinforcement Learning","authors":[{"name":"Zun Li","slug":"zun-li","given":"Zun","family":"Li","affiliation":"University of Michigan"},{"name":"Michael P. Wellman","slug":"michael-p-wellman","given":"Michael P.","family":"Wellman","affiliation":"University of Michigan"}],"abstract":"Evaluating deep multiagent reinforcement learning (MARL) algorithms is complicated by stochasticity in training and sensitivity of agent performance to the behavior of other agents. We propose a meta-game evaluation framework for deep MARL, by framing each MARL algorithm as a meta-strategy, and repeatedly sampling normal-form empirical games over combinations of meta-strategies resulting from different random seeds. Each empirical game captures both self-play and cross-play factors across seeds. These empirical games provide the basis for constructing a sampling distribution, using bootstrapping, over a variety of game analysis statistics. We use this approach to evaluate state-of-the-art deep MARL algorithms on a class of negotiation games. From statistics on individual payoffs, social welfare, and empirical best-response graphs, we uncover strategic relationships among self-play, population-based, model-free, and model-based MARL methods. We also investigate the effect of run-time search as a meta-strategy operator, and find via meta-game analysis that the search version of a meta-strategy generally leads to improved performance.","doi":"10.24963/ijcai.2024/17","url":"https://ijcai.org/proceedings/2024/17","pdf":"https://ijcai.org/proceedings/2024/0017.pdf","markdown":"https://ijcai.org/proceedings/2024/17.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/17","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":17,"pages":{"first":148,"last":156},"keywords":["Game Theory and Economic Paradigms: GTEP: Noncooperative games","Agent-based and Multi-agent Systems: MAS: Multi-agent learning"],"videos":[],"references":[{"position":1,"raw":"[Agarwal et al., 2021] Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, et al. Deep reinforcement learning at the edge of the statistical precipice. In 35th Int'l Conf. on Neural Information Processing Systems, pages 29304- 29320, 2021.","title":"Deep reinforcement learning at the edge of the statistical precipice","authors":"Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro","year":"2021","venue":"35th Int'l Conf. on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aumann, 1964] Robert J. Aumann. Mixed and behavior strategies in infinite extensive game. In M. Dresher, L. S. Shapley, and A. W. Tucker, editors, Advances in Game Theory, Annals of Mathematics Studies, pages 627-650. Princeton University Press, 1964.","title":"Mixed and behavior strategies in infinite extensive game","authors":"Robert J Aumann","year":"1964","venue":"Advances in Game Theory","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Balduzzi et al., 2018] David Balduzzi, Karl Tuyls, Julien Perolat, and Thore Graepel. Re-evaluating evaluation. In 32nd Int'l Conf. on Neural Information Processing Sys- tems, 2018.","title":"Re-evaluating evaluation","authors":"David Balduzzi, Karl Tuyls, Julien Perolat, Thore Graepel","year":"2018","venue":"32nd Int'l Conf. on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bellemare et al., 2013] Marc G. Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47:253-279, 2013.","title":"The arcade learning environment: An evaluation platform for general agents","authors":"G Marc, Yavar Bellemare, Joel Naddaf, Michael Veness, Bowling","year":"2013","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Breiman, 1996] Leo Breiman. Bagging predictors. Machine Learning, 24:123-140, 1996.","title":"Bagging predictors","authors":"Breiman","year":"1996","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Brown et al., 2020] Noam Brown, Anton Bakhtin, Adam Lerer, and Qucheng Gong. Combining deep reinforcement learning and search for imperfect-information games. In 34th Int'l Conf. on Neural Information Processing Sys- tems, pages 17057-17069, 2020.","title":"Combining deep reinforcement learning and search for imperfect-information games","authors":"Brown","year":"2020","venue":"34th Int'l Conf. on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bubeck et al., 2009] Sébastien Bubeck, Rémi Munos, and Gilles Stoltz. Pure exploration in multi-armed bandits problems. In 20th Int'l Conf. on Algorithmic Learning Theory, pages 23-37, 2009.","title":"Pure exploration in multi-armed bandits problems","authors":"Sébastien Bubeck, Rémi Munos, Gilles Stoltz","year":"2009","venue":"20th Int'l Conf. on Algorithmic Learning Theory","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cheng et al., 2004] Shih-Fen Cheng, Daniel M. Reeves, Yevgeniy Vorobeychik, and Michael P. Wellman. Notes on equilibria in symmetric games. In AAMAS-04 Work- shop on Game-Theoretic and Decision-Theoretic Agents, 2004.","title":"Notes on equilibria in symmetric games","authors":"Shih-Fen Cheng, Daniel M Reeves, Yevgeniy Vorobeychik, Michael P Wellman","year":"2004","venue":"AAMAS-04 Workshop on Game-Theoretic and Decision-Theoretic Agents","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cowling et al., 2012] Peter I. Cowling, Edward J. Powley, and Daniel Whitehouse. Information set Monte Carlo tree search. IEEE Transactions on Computational Intelligence and AI in Games, 4:120-143, 2012.","title":"Information set Monte Carlo tree search","authors":"I Peter, Edward J Cowling, Daniel Powley, Whitehouse","year":"2012","venue":"IEEE Transactions on Computational Intelligence and AI in Games","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Danihelka et al., 2022] Ivo Danihelka, Arthur Guez, Julian Schrittwieser, and David Silver. Policy improvement by planning with Gumbel. In 10th Int'l Conf. on Learning Representations, 2022.","title":"Policy improvement by planning with Gumbel","authors":"Danihelka","year":"2022","venue":"10th Int'l Conf. on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Davison and Hinkley, 1997] Anthony Christopher Davison and David Victor Hinkley. Bootstrap Methods and Their Application. Cambridge University Press, 1997.","title":"Bootstrap Methods and Their Application","authors":"Anthony Christopher, Davison, David Victor, Hinkley","year":"1997","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Espeholt et al., 2018] Lasse Espeholt, Hubert Soyer, Remi Munos, et al. IMPALA: Scalable distributed deep-RL with importance weighted actor-learner architectures. In 35th Int'l Conf. on Machine Learning, pages 1407-1416, 2018.","title":"IMPALA: Scalable distributed deep-RL with importance weighted actor-learner architectures","authors":"Espeholt","year":"2018","venue":"35th Int'l Conf. on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Foerster et al., 2018] Jakob Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. Counterfactual multi-agent policy gradients. In 32nd AAAI Conf. on Artificial Intelligence, 2018. [Gorsane et al., 2022] Rihab Gorsane, Omayma Mahjoub, Ruan John de Kock, Roland Dubb, Siddarth Singh, and Arnu Pretorius. Towards a standardised performance eval- uation protocol for cooperative MARL. In 36th Int'l Conf. on Neural Information Processing Systems, pages 5510- 5521, 2022.","title":"Towards a standardised performance evaluation protocol for cooperative MARL","authors":"Foerster","year":"2018","venue":"th Int'l Conf. on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Gronauer and Diepold, 2022] Sven Gronauer and Klaus Diepold. Multi-agent deep reinforcement learning: A sur- vey. Artificial Intelligence Review, 55:895-943, 2022.","title":"Multi-agent deep reinforcement learning: A survey","authors":"Diepold Gronauer, Sven Gronauer, Klaus Diepold","year":"2022","venue":"Artificial Intelligence Review","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hefti, 2017] Andreas Hefti. Equilibria in symmetric games: Theory and applications. Theoretical Economics, 12(3):979-1002, 2017.","title":"Equilibria in symmetric games: Theory and applications","authors":"Andreas Hefti","year":"2017","venue":"Theoretical Economics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Heinrich and Silver, 2016] Johannes Heinrich and David Silver. Deep reinforcement learning from self- play in imperfect-information games. arXiv preprint arXiv:1603.01121, 2016.","title":"Deep reinforcement learning from selfplay in imperfect-information games","authors":"Silver Heinrich, Johannes Heinrich, David Silver","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Henderson et al., 2018] Peter Henderson, Riashat Islam, Philip Bachman, et al. Deep reinforcement learning that matters. In 32nd AAAI Conf. on Artificial Intelligence, 2018.","title":"Deep reinforcement learning that matters","authors":"Henderson","year":"2018","venue":"32nd AAAI Conf. on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Hennes et al., 2020] Daniel Hennes, Dustin Morrill, Shayegan Omidshafiei, et al. Neural replicator dynamics: Multiagent learning via hedging policy gradients. In 19th Int'l Conf. on Autonomous Agents and Multi-Agent Systems, pages 492-501, 2020.","title":"Neural replicator dynamics: Multiagent learning via hedging policy gradients","authors":"Daniel Hennes, Dustin Morrill, Shayegan Omidshafiei","year":"2020","venue":"19th Int'l Conf. on Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Hu et al., 2020] Hengyuan Hu, Adam Lerer, Alex Peysakhovich, and Jakob Foerster. \"Other-play\" for zero-shot coordination. In 37th Int'l Conf. on Machine Learning, pages 4399-4410, 2020.","title":"Other-play\" for zero-shot coordination","authors":"Hengyuan Hu, Adam Lerer, Alex Peysakhovich, Jakob Foerster","year":"2020","venue":"37th Int'l Conf. on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Hu et al., 2021] Hengyuan Hu, Adam Lerer, Noam Brown, and Jakob Foerster. Learned belief search: Efficiently im- proving policies in partially observable settings. arXiv preprint arXiv:2106.09086, 2021.","title":"Learned belief search: Efficiently improving policies in partially observable settings","authors":"Hengyuan Hu, Adam Lerer, Noam Brown, Jakob Foerster","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Huijben et al., 2022] Iris A. M. Huijben, Wouter Kool, Max B. Paulus, and Ruud J. G. Van Sloun. A review of the Gumbel-max trick and its extensions for discrete stochasticity in machine learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2):1353- 1371, 2022.","title":"A review of the Gumbel-max trick and its extensions for discrete stochasticity in machine learning","authors":"A M Iris, Wouter Huijben, Max B Kool, Paulus, J G Ruud, Van Sloun","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Jordan et al., 2007] Patrick R. Jordan, Christopher Kiek- intveld, and Michael P. Wellman. Empirical game- theoretic analysis of the TAC supply chain game. In 6th Int'l Conf. on Autonomous Agents and Multi-Agent Sys- tems, pages 1188-1195, 2007.","title":"Empirical gametheoretic analysis of the TAC supply chain game","authors":"Patrick R Jordan, Christopher Kiekintveld, Michael P Wellman","year":"2007","venue":"6th Int'l Conf. on Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Kiekintveld and Wellman, 2008] Christopher Kiekintveld and Michael P. Wellman. Selecting strategies using empirical game models: An experimental analysis of meta-strategies. In 7th Int'l Conf. on Autonomous Agents and Multi-Agent Systems, pages 1095-1101, 2008.","title":"Selecting strategies using empirical game models: An experimental analysis of meta-strategies","authors":"Christopher Kiekintveld, Michael P Wellman","year":"2008","venue":"7th Int'l Conf. on Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Kreps, 1990] David M. Kreps. Game Theory and Economic Modelling. Oxford University Press, 1990.","title":"Game Theory and Economic Modelling","authors":"M David, Kreps","year":"1990","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Lanctot et al., 2017] Marc Lanctot, Vinicius Zambaldi, Au- drunas Gruslys, et al. A unified game-theoretic approach to multiagent reinforcement learning. In 31st Int'l Conf. on Neural Information Processing Systems, pages 4190- 4203, 2017.","title":"A unified game-theoretic approach to multiagent reinforcement learning","authors":"Marc Lanctot, Vinicius Zambaldi, Audrunas Gruslys","year":"2017","venue":"31st Int'l Conf. on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Lanctot et al., 2019] Marc Lanctot, Edward Lockhart, Jean- Baptiste Lespiau, et al. Openspiel: A framework for reinforcement learning in games. arXiv preprint arXiv:1908.09453, 2019.","title":"Openspiel: A framework for reinforcement learning in games","authors":"Marc Lanctot, Edward Lockhart, Jean-Baptiste Lespiau","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Leibo et al., 2021] Joel Z. Leibo, Edgar A. Dueñez- Guzman, Alexander Vezhnevets, et al. Scalable evaluation of multi-agent reinforcement learning with Melting Pot. In 38th Int'l Conf. on Machine Learning, pages 6187-6199, 2021.","title":"Scalable evaluation of multi-agent reinforcement learning with Melting Pot","authors":"Joel Z Leibo, Edgar A Dueñez-Guzman, Alexander Vezhnevets","year":"2021","venue":"38th Int'l Conf. on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Lewis et al., 2017] Mike Lewis, Denis Yarats, Yann N. Dauphin, Devi Parikh, and Dhruv Batra. Deal or no deal? End-to-end learning for negotiation dialogues. In Confer- ence on Empirical Methods in Natural Language Process- ing, 2017.","title":"Deal or no deal? End-to-end learning for negotiation dialogues","authors":"Mike Lewis, Denis Yarats, Yann N Dauphin, Devi Parikh, Dhruv Batra","year":"2017","venue":"Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Li and Wellman, 2024] Zun Li and Michael P. Wellman. A meta-game evaluation framework for deep multiagent re- inforcement learning. arXiv preprint arXiv:2405.00243, 2024.","title":"A meta-game evaluation framework for deep multiagent reinforcement learning","authors":"Li, Wellman ; ] Zun Li, Michael P Wellman","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Li et al., 2023] Zun Li, Marc Lanctot, Kevin R. McKee, et al. Combining tree-search, generative models, and Nash bargaining concepts in game-theoretic reinforcement learning. arXiv preprint arXiv:2302.00797, 2023.","title":"Combining tree-search, generative models, and Nash bargaining concepts in game-theoretic reinforcement learning","authors":"Zun Li, Marc Lanctot, Kevin R Mckee","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Lowe et al., 2017] Ryan Lowe, Yi I. Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environ- ments. In 31st Int'l Conf. on Neural Information Process- ing Systems, 2017.","title":"Multi-agent actor-critic for mixed cooperative-competitive environments","authors":"Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, Igor Mordatch","year":"2017","venue":"31st Int'l Conf. on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Nash, 1951] John Nash. Non-cooperative games. Annals of Mathematics, pages 286-295, 1951.","title":"John Nash. Non-cooperative games","authors":"Nash","year":"1951","venue":"Annals of Mathematics","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Papoudakis et al., 2020] Georgios Papoudakis, Filippos Christianos, Lukas Schäfer, and Stefano V. Albrecht. Comparative evaluation of cooperative multi-agent deep reinforcement learning algorithms. arXiv preprint arXiv:2006.07869, 2020.","title":"Comparative evaluation of cooperative multi-agent deep reinforcement learning algorithms","authors":"Georgios Papoudakis, Filippos Christianos, Lukas Schäfer, Stefano V Albrecht","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Pepels et al., 2014] Tom Pepels, Tristan Cazenave, Mark H. M. Winands, and Marc Lanctot. Minimizing simple and cumulative regret in monte-carlo tree search. In ECAI-14 Workshop on Computer Games, pages 1-15, 2014.","title":"Minimizing simple and cumulative regret in monte-carlo tree search","authors":"Tom Pepels, Tristan Cazenave, H M Mark, Marc Winands, Lanctot","year":"2014","venue":"ECAI-14 Workshop on Computer Games","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Perolat et al., 2021] Julien Perolat, Remi Munos, Jean- Baptiste Lespiau, Shayegan Omidshafiei, et al. From Poincaré recurrence to convergence in imperfect informa- tion games: Finding equilibrium via regularization. In 38th Int'l Conf. on Machine Learning, pages 8525-8535, 2021.","title":"From Poincaré recurrence to convergence in imperfect information games: Finding equilibrium via regularization","authors":"Julien Perolat, Remi Munos, Jean-Baptiste Lespiau, Shayegan Omidshafiei","year":"2021","venue":"38th Int'l Conf. on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Perolat et al., 2022] Julien Perolat, Bart De Vylder, Daniel Hennes, et al. Mastering the game of Stratego with model-free multiagent reinforcement learning. Science, 378(6623):990-996, 2022.","title":"Mastering the game of Stratego with model-free multiagent reinforcement learning","authors":"Julien Perolat, Bart De Vylder, Daniel Hennes","year":"2022","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Sandholm et al., 2005] Tuomas Sandholm, Andrew Gilpin, and Vincent Conitzer. Mixed-integer programming meth- ods for finding Nash equilibria. In 19th AAAI Conf. on Artificial Intelligence, pages 495-501, 2005.","title":"Mixed-integer programming methods for finding Nash equilibria","authors":"Sandholm","year":"2005","venue":"19th AAAI Conf. on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Schmid et al., 2023] Martin Schmid, Matej Moravcik, Neil Burch, and Rudolf Kadlec others. Student of games: A unified learning algorithm for both perfect and imperfect information games. Science Advances, 2023.","title":"Student of games: A unified learning algorithm for both perfect and imperfect information games","authors":"Schmid","year":"2023","venue":"Science Advances","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.","title":"Proximal policy optimization algorithms","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Silver and Veness, 2010] David Silver and Joel Veness. Monte-Carlo planning in large POMDPs. In 24th Int'l Conf. on Neural Information Processing Systems, vol- ume 23, 2010.","title":"Monte-Carlo planning in large POMDPs","authors":"Veness Silver, David Silver, Joel Veness","year":"2010","venue":"24th Int'l Conf. on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Silver et al., 2016] David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, et al. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587):484- 489, 2016.","title":"Mastering the game of Go with deep neural networks and tree search","authors":"David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den, Driessche","year":"2016","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Silver et al., 2018] David Silver, Thomas Hubert, Julian Schrittwieser, et al. A general reinforcement learning algo- rithm that masters chess, shogi, and Go through self-play. Science, 362(6419):1140-1144, 2018.","title":"A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play","authors":"David Silver, Thomas Hubert, Julian Schrittwieser","year":"2018","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Sokota et al., 2024] Samuel Sokota, Gabriele Farina, David J Wu, Hengyuan Hu, Kevin A Wang, J. Zico Kolter, and Noam Brown. The update equivalence framework for decision-time planning. In Thirteenth International Conference on Learning Representation, 2024.","title":"The update equivalence framework for decision-time planning","authors":"Samuel Sokota, Gabriele Farina, David J Wu, Hengyuan Hu, Kevin A Wang, J Zico Kolter, Noam Brown","year":"2024","venue":"Thirteenth International Conference on Learning Representation","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Strouse et al., 2021] D. J. Strouse, Kevin McKee, Matt Botvinick, Edward Hughes, and Richard Everett. Collabo- rating with humans without human data. In 35th Int'l Conf. on Neural Information Processing Systems, pages 14502- 14515, 2021.","title":"Collaborating with humans without human data","authors":"D J Strouse, Kevin Mckee, Matt Botvinick, Edward Hughes, Richard Everett","year":"2021","venue":"35th Int'l Conf. on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Treutlein et al., 2021] Johannes Treutlein, Michael Dennis, Caspar Oesterheld, and Jakob Foerster. A new formal- ism, method and open issues for zero-shot coordination. In 38th Int'l Conf. on Machine Learning, pages 10413- 10423, 2021.","title":"A new formalism, method and open issues for zero-shot coordination","authors":"Treutlein","year":"2021","venue":"38th Int'l Conf. on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Vinyals et al., 2019] Oriol Vinyals, Igor Babuschkin, Woj- ciech M. Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H. Choi, et al. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature, 575(7782):350-354, 2019.","title":"Grandmaster level in StarCraft II using multi-agent reinforcement learning","authors":"Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi","year":"2019","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Wellman et al., 2024] Michael P. Wellman, Karl Tuyls, and Amy Greenwald. Empirical game-theoretic analysis: A survey. Technical Report 2403.04018, arXiv, 2024.","title":"Empirical game-theoretic analysis: A survey","authors":"P Michael, Karl Wellman, Amy Tuyls, Greenwald","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Wiedenbeck et al., 2014] Bryce Wiedenbeck, Ben- Alexander Cassell, and Michael P. Wellman. Bootstrap statistics for empirical games. In 13th Int'l Conf. on Autonomous Agents and Multi-Agent Systems, pages 597-604, 2014.","title":"Bootstrap statistics for empirical games","authors":"Bryce Wiedenbeck, Ben-Alexander Cassell, Michael P Wellman","year":"2014","venue":"13th Int'l Conf. on Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Yu et al., 2022] Chao Yu, Akash Velu, Eugene Vinitsky, Ji- axuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effectiveness of PPO in cooperative multi-agent games. In 36th Int'l Conf. on Neural Information Process- ing Systems, pages 24611-24624, 2022.","title":"The surprising effectiveness of PPO in cooperative multi-agent games","authors":"Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, Yi Wu","year":"2022","venue":"36th Int'l Conf. on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}