{"type":"ScholarlyArticle","title":"Policy Space Response Oracles: A Survey","authors":[{"name":"Ariyan Bighashdel","slug":"ariyan-bighashdel","given":"Ariyan","family":"Bighashdel","affiliation":"Eindhoven University of Technology , NL"},{"name":"Yongzhao Wang","slug":"yongzhao-wang","given":"Yongzhao","family":"Wang","affiliation":"The Alan Turing Institute , UK"},{"name":"Stephen McAleer","slug":"stephen-mcaleer","given":"Stephen","family":"McAleer","affiliation":"Carnegie Mellon University , USA"},{"name":"Rahul Savani","slug":"rahul-savani","given":"Rahul","family":"Savani","affiliation":"Eindhoven University of Technology , NL"},{"name":"Frans A. Oliehoek","slug":"frans-a-oliehoek","given":"Frans A.","family":"Oliehoek","affiliation":"Delft University of Technology , NL; University of Liverpool , UK"}],"abstract":"Game theory provides a mathematical way to study the interaction between multiple decision makers. However, classical game-theoretic analysis is limited in scalability due to the large number of strategies, precluding direct application to more complex scenarios. This survey provides a comprehensive overview of a framework for large games, known as Policy Space Response Oracles (PSRO), which holds promise to improve scalability by focusing attention on sufficient subsets of strategies. We first motivate PSRO and provide historical context. We then focus on the strategy exploration problem for PSRO: the challenge of assembling effective subsets of strategies that still represent the original game well with minimum computational cost. We survey current research directions for enhancing the efficiency of PSRO, and explore the applications of PSRO across various domains. We conclude by discussing open questions and future research.","doi":"10.24963/ijcai.2024/880","url":"https://ijcai.org/proceedings/2024/880","pdf":"https://ijcai.org/proceedings/2024/0880.pdf","markdown":"https://ijcai.org/proceedings/2024/880.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/880","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Survey Track","track":"Survey Track","number":880,"pages":{"first":7951,"last":7961},"keywords":["Game Theory and Economic Paradigms: General","Game Theory and Economic Paradigms: GTEP: Noncooperative games","Agent-based and Multi-agent Systems: General"],"videos":[],"references":[{"position":1,"raw":"[Albrecht et al., 2024] S. V. Albrecht, F. Christianos, and L. Schäfer. Multi-Agent Reinforcement Learning: Foun- dations and Modern Approaches. MIT Press, 2024.","title":"Multi-Agent Reinforcement Learning: Foundations and Modern Approaches","authors":"S V Albrecht, F Christianos, L Schäfer","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[An and Zhou, 2023] Z. An and L. Zhou. Double Oracle Al- gorithm for Game-Theoretic Robot Allocation on Graphs. arXiv:2312.11791, 2023.","title":"Double Oracle Algorithm for Game-Theoretic Robot Allocation on Graphs","authors":"Z An, L Zhou","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Angeline et al., 1993] P. J. Angeline, J. B. Pollack, and Oth- ers. Competitive Environments Evolve Better Solutions for Complex Tasks. In ICGA, 1993.","title":"Competitive Environments Evolve Better Solutions for Complex Tasks","authors":"P J Angeline, J B Pollack, Others","year":"1993","venue":"ICGA","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ansari et al., 2019] A. Ansari, M. Dadgar, A. Hamzeh, J. Schlötterer, and M. Granitzer. Competitive influence maximization: integrating budget allocation and seed se- lection. arXiv:1912.12283, 2019.","title":"Competitive influence maximization: integrating budget allocation and seed selection","authors":"A Ansari, M Dadgar, A Hamzeh, J Schlötterer, M Granitzer","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Aung et al., 2022] A. P. P. Aung, X. Wang, R. Yu, B. An, S. Jayavelu, and X. Li. DO-GAN: A Double Oracle Framework for Generative Adversarial Networks. In CVRP, 2022.","title":"DO-GAN: A Double Oracle Framework for Generative Adversarial Networks","authors":"A P P Aung, X Wang, R Yu, B An, S Jayavelu, X Li","year":"2022","venue":"CVRP","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Avis et al., 2010] D. Avis, G. D. Rosenberg, R. Savani, and B. Von Stengel. Enumeration of nash equilibria for two- player games. Economic theory, 42, 2010.","title":"Enumeration of nash equilibria for twoplayer games","authors":"D Avis, G D Rosenberg, R Savani, B Von, Stengel","year":"2010","venue":"Economic theory","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Balduzzi et al., 2018] D. Balduzzi, K. Tuyls, J. Pérolat, and T. Graepel. Re-evaluating evaluation. In NIPS, 2018.","title":"Re-evaluating evaluation","authors":"D Balduzzi, K Tuyls, J Pérolat, T Graepel","year":"2018","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Balduzzi et al., 2019] D. Balduzzi, M. Garnelo, Y. Bachrach, W. Czarnecki, J. Perolat, M. Jaderberg, and T. Graepel. Open-ended learning in symmetric zero-sum games. In ICML, 2019.","title":"Open-ended learning in symmetric zero-sum games","authors":"D Balduzzi, M Garnelo, Y Bachrach, W Czarnecki, J Perolat, M Jaderberg, T Graepel","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Behpour et al., 2019a] S. Behpour, K. M. Kitani, and B. D. Ziebart. ADA: Adversarial data augmentation for object detection. In WACV, 2019.","title":"ADA: Adversarial data augmentation for object detection","authors":"Behpour","year":"2019","venue":"WACV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Behpour et al., 2019b] S. Behpour, A. Liu, and B. Ziebart. Active learning for probabilistic structured prediction of cuts and matchings. In ICML, 2019.","title":"Active learning for probabilistic structured prediction of cuts and matchings","authors":"Behpour","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Behpour, 2018] S. Behpour. Arc: Adversarial robust cuts for semi-supervised and multi-label classification. In CVPR, 2018.","title":"Arc: Adversarial robust cuts for semi-supervised and multi-label classification","authors":"S Behpour","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Benders, 1962] J. F. Benders. Partitioning procedures for solving mixed-variables programming problems. Nu- merische Mathematik, 4, 1962.","title":"Partitioning procedures for solving mixed-variables programming problems","authors":"J F Benders","year":"1962","venue":"Numerische Mathematik","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Brown, 1951] G. W. Brown. Iterative solution of games by fictitious play. Act. Anal. Prod Allocation, 13, 1951.","title":"Iterative solution of games by fictitious play","authors":"G W Brown","year":"1951","venue":"Act. Anal. Prod Allocation","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Celli and Gatti, 2018] A. Celli and N. Gatti. Computational results for extensive-form adversarial team games. In AAAI, 2018.","title":"Computational results for extensive-form adversarial team games","authors":"Gatti Celli, A Celli, N Gatti","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Cui and Yang, 2023] J. Cui and X. Yang. Macta: A multi- agent reinforcement learning approach for cache timing at- tacks and detection. ICLR, 2023.","title":"Macta: A multiagent reinforcement learning approach for cache timing attacks and detection","authors":"Yang ; Cui, ] J Cui, X Yang","year":"2023","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Czarnecki et al., 2020] W. M. Czarnecki, G. Gidel, B. Tracey, K. Tuyls, S. Omidshafiei, D. Balduzzi, and M. Jaderberg. Real world games look like spinning tops. In NeurIPS, 2020.","title":"Real world games look like spinning tops","authors":"W M Czarnecki, G Gidel, B Tracey, K Tuyls, S Omidshafiei, D Balduzzi, M Jaderberg","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Dinh et al., 2022] L. C. Dinh, S. M. McAleer, Z. Tian, N. Perez-Nieves, O. Slumbers, D. H. Mguni, J. Wang, H. B. Ammar, and Y. Yang. Online double oracle. TMLR, 2022.","title":"Online double oracle","authors":"L C Dinh, S M Mcaleer, Z Tian, N Perez-Nieves, O Slumbers, D H Mguni, J Wang, H B Ammar, Y Yang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Fang, 2019] F. Fang. Integrate learning with game theory for societal challenges. In IJCAI, 2019.","title":"Integrate learning with game theory for societal challenges","authors":"F Fang","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Fathony et al., 2018] R. Fathony, S. Behpour, X. Zhang, and B. Ziebart. Efficient and consistent adversarial bipartite matching. In ICML, 2018.","title":"Efficient and consistent adversarial bipartite matching","authors":"R Fathony, S Behpour, X Zhang, B Ziebart","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Feng et al., 2021] X. Feng, O. Slumbers, Z. Wan, B. Liu, S. McAleer, Y. Wen, J. Wang, and Y. Yang. Neural auto- curricula in two-player zero-sum games. In NeurIPS, 2021.","title":"Neural autocurricula in two-player zero-sum games","authors":"X Feng, O Slumbers, Z Wan, B Liu, S Mcaleer, Y Wen, J Wang, Y Yang","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ficici and Pollack, 2003] S. G. Ficici and J. B. Pollack. A game-theoretic memory mechanism for coevolution. In GECCO, 2003.","title":"A game-theoretic memory mechanism for coevolution","authors":"S G Ficici, J B Pollack","year":"2003","venue":"GECCO","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Gemp et al., 2022] I. Gemp, R. Savani, M. Lanctot, Y. Bachrach, T. W. Anthony, R. Everett, A. Tacchetti, T. Eccles, and J. Kramár. Sample-based approximation of nash in large many-player games via gradient descent. In AAMAS, 2022.","title":"Sample-based approximation of nash in large many-player games via gradient descent","authors":"Gemp","year":"2022","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Giboulot et al., 2023] Q. Giboulot, T. Pevnỳ, and A. D. Ker. The non-zero-sum game of steganography in heteroge- neous environments. IEEE Transactions on Information Forensics and Security, 2023.","title":"The non-zero-sum game of steganography in heterogeneous environments","authors":"Q Giboulot, T Pevnỳ, A D Ker","year":"2023","venue":"IEEE Transactions on Information Forensics and Security","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Johanson et al., 2012] M. Johanson, N. Bard, N. Burch, and M. Bowling. Finding optimal abstract strategies in extensive-form games. In AAAI, 2012.","title":"Finding optimal abstract strategies in extensive-form games","authors":"M Johanson, N Bard, N Burch, M Bowling","year":"2012","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Jordan et al., 2010] P. R. Jordan, L. J. Schvartzman, and M. P. Wellman. Strategy exploration in empirical games. In AAMAS, 2010.","title":"Strategy exploration in empirical games","authors":"P R Jordan, L J Schvartzman, M P Wellman","year":"2010","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Killian et al., 2023] J. A. Killian, A. Biswas, L. Xu, S. Verma, V. Nair, A. Taneja, A. Hegde, N. Madhiwalla, P. R. Diaz, S. Johnson-Yu, and Others. Robust planning over restless groups: engagement interventions for a large- scale maternal telehealth program. In AAAI, 2023.","title":"Robust planning over restless groups: engagement interventions for a largescale maternal telehealth program","authors":"J A Killian, A Biswas, L Xu, S Verma, V Nair, A Taneja, A Hegde, N Madhiwalla, P R Diaz, S Johnson-Yu, Others","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Konicki et al., 2022] C. Konicki, M. Chakraborty, and M. P. Wellman. Exploiting extensive-form structure in empirical game-theoretic analysis. In WINE, 2022.","title":"Exploiting extensive-form structure in empirical game-theoretic analysis","authors":"C Konicki, M Chakraborty, M P Wellman","year":"2022","venue":"WINE","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Lanctot et al., 2017] M. Lanctot, V. Zambaldi, A. Gruslys, A. Lazaridou, K. Tuyls, J. Pérolat, D. Silver, and T. Grae- pel. A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning. NIPS, 2017.","title":"A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning","authors":"M Lanctot, V Zambaldi, A Gruslys, A Lazaridou, K Tuyls, J Pérolat, D Silver, T Graepel","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Lanctot et al., 2019] M. Lanctot, E. Lockhart, J.-B. Lespiau, V. Zambaldi, S. Upadhyay, J. Pérolat, S. Srini- vasan, F. Timbers, K. Tuyls, S. Omidshafiei, et al. Openspiel: A framework for reinforcement learning in games. arXiv preprint arXiv:1908.09453, 2019.","title":"Openspiel: A framework for reinforcement learning in games","authors":"M Lanctot, E Lockhart, J.-B Lespiau, V Zambaldi, S Upadhyay, J Pérolat, S Srinivasan, F Timbers, K Tuyls, S Omidshafiei","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Li and Wellman, 2021] Z. Li and M. P. Wellman. Evolution strategies for approximate solution of Bayesian games. In AAAI, 2021.","title":"Evolution strategies for approximate solution of Bayesian games","authors":"Z Li, M P Wellman","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Li and Wellman, 2024] Z. Li and M. P. Wellman. A meta- game evaluation framework for deep multiagent reinforce- ment learning. In IJCAI, 2024.","title":"A metagame evaluation framework for deep multiagent reinforcement learning","authors":"Z Li, M P Wellman","year":"2024","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Li et al., 2023a] S. Li, X. Wang, Y. Zhang, W. Xue, J. Černỳ, and B. An. Solving large-scale pursuit-evasion games using pre-trained strategies. In AAAI, 2023.","title":"Solving large-scale pursuit-evasion games using pre-trained strategies","authors":"Li","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Li et al., 2023b] Y. Li, K. Xiong, Y. Zhang, J. Zhu, S. Mcaleer, W. Pan, J. Wang, Z. Dai, and Y. Yang. Jiangjun: Mastering Xiangqi by tackling non-transitivity in two-player zero-sum games. TMLR, 2023.","title":"Jiangjun: Mastering Xiangqi by tackling non-transitivity in two-player zero-sum games","authors":"Li","year":"2023","venue":"TMLR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Li et al., 2023c] Z. Li, M. Lanctot, K. R. McKee, L. Marris, I. Gemp, D. Hennes, P. Muller, K. Larson, Y. Bachrach, and M. P. Wellman. Combining Tree-Search, Genera- tive Models, and Nash Bargaining Concepts in Game- Theoretic Reinforcement Learning. arXiv:2302.00797, 2023.","title":"Combining Tree-Search, Generative Models, and Nash Bargaining Concepts in Game-Theoretic Reinforcement Learning","authors":"Li","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Li et al., 2024a] P. Li, S. Li, X. Wang, J. Cerny, Y. Zhang, S. McAleer, H. Chan, and B. An. Grasper: A generalist pursuer for pursuit-evasion problems. In AAMAS, 2024.","title":"Grasper: A generalist pursuer for pursuit-evasion problems","authors":"Li","year":"2024","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Li et al., 2024b] P. Li, S. Li, C. Yang, X. Wang, X. Huang, H. Chan, and B. An. Self-adaptive psro: Towards an auto- matic population-based game solver. In IJCAI, 2024.","title":"Self-adaptive psro: Towards an automatic population-based game solver","authors":"Li","year":"2024","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2024/16"},{"position":37,"raw":"[Liang et al., 2023] Y. Liang, Y. Sun, R. Zheng, X. Liu, T. Sandholm, F. Huang, and S. McAleer. Game-theoretic robust reinforcement learning handles temporally-coupled perturbations. arXiv:2307.12062, 2023.","title":"Game-theoretic robust reinforcement learning handles temporally-coupled perturbations","authors":"Y Liang, Y Sun, R Zheng, X Liu, T Sandholm, F Huang, S Mcaleer","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Liu et al., 2021] X. Liu, H. Jia, Y. Wen, Y. Hu, Y. Chen, C. Fan, Z. Hu, and Y. Yang. Towards unifying behavioral and response diversity for open-ended learning in zero- sum games. In NeurIPS, 2021.","title":"Towards unifying behavioral and response diversity for open-ended learning in zerosum games","authors":"Liu","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Liu et al., 2022a] S. Liu, M. Lanctot, L. Marris, and N. Heess. Simplex neural population learning: Any- mixture bayes-optimality in symmetric zero-sum games. In ICML, 2022.","title":"Simplex neural population learning: Anymixture bayes-optimality in symmetric zero-sum games","authors":"Liu","year":"2022","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Liu et al., 2022b] S. Liu, L. Marris, D. Hennes, J. Merel, N. Heess, and T. Graepel. NeuPL: Neural population learning. arXiv:2202.07415, 2022.","title":"NeuPL: Neural population learning","authors":"Liu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Liu et al., 2022c] Z. Liu, C. Yu, Y. Yang, Z. Wu, Y. Li, and Others. A Unified Diversity Measure for Multiagent Rein- forcement Learning. In NeurIPS, 2022.","title":"A Unified Diversity Measure for Multiagent Reinforcement Learning","authors":"Liu","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Liu et al., 2024] S. Liu, L. Marris, M. Lanctot, G. Piliouras, J. Z. Leibo, and N. Heess. Neural population learning be- yond symmetric zero-sum games. In AAMAS, 2024.","title":"Neural population learning beyond symmetric zero-sum games","authors":"S Liu, L Marris, M Lanctot, G Piliouras, J Z Leibo, N Heess","year":"2024","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Long et al., 2023] W. Long, T. Hou, X. Wei, S. Yan, P. Zhai, and L. Zhang. A Survey on Population-Based Deep Rein- forcement Learning. Mathematics, 11, 2023.","title":"A Survey on Population-Based Deep Reinforcement Learning","authors":"W Long, T Hou, X Wei, S Yan, P Zhai, L Zhang","year":"2023","venue":"Mathematics","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Ma et al., 2019] Y. Ma, G. Wang, X. Hu, H. Luo, and X. Lei. Cooperative occupancy decision making of Multi- UAV in Beyond-Visual-Range air combat: A game theory approach. IEEE Access, 8, 2019.","title":"Cooperative occupancy decision making of Multi-UAV in Beyond-Visual-Range air combat: A game theory approach","authors":"Y Ma, G Wang, X Hu, H Luo, X Lei","year":"2019","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Ma et al., 2023] C. Ma, Z. Yang, M. Gao, H. Ci, J. Gao, X. Pan, and Y. Yang. Red teaming game: A game- theoretic framework for red teaming language models. arXiv:2310.00322, 2023.","title":"Red teaming game: A gametheoretic framework for red teaming language models","authors":"C Ma, Z Yang, M Gao, H Ci, J Gao, X Pan, Y Yang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Marris et al., 2021] L. Marris, P. Muller, M. Lanctot, K. Tuyls, and T. Graepel. Multi-agent training beyond zero-sum with correlated equilibrium meta-solvers. In ICML, 2021.","title":"Multi-agent training beyond zero-sum with correlated equilibrium meta-solvers","authors":"L Marris, P Muller, M Lanctot, K Tuyls, T Graepel","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[McAleer et al., 2020] S. McAleer, J. B. Lanier, R. Fox, and P. Baldi. Pipeline PSRO: A scalable approach for finding approximate Nash equilibria in large games. In NeurIPS, 2020.","title":"Pipeline PSRO: A scalable approach for finding approximate Nash equilibria in large games","authors":"S Mcaleer, J B Lanier, R Fox, P Baldi","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[McAleer et al., 2021] S. McAleer, J. B. Lanier, K. A. Wang, P. Baldi, and R. Fox. XDO: A double oracle algorithm for extensive-form games. In NeurIPS, 2021.","title":"XDO: A double oracle algorithm for extensive-form games","authors":"S Mcaleer, J B Lanier, K A Wang, P Baldi, R Fox","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[McAleer et al., 2022a] S. McAleer, J. B. Lanier, K. Wang, P. Baldi, R. Fox, and T. Sandholm. Self-play psro: To- ward optimal populations in two-player zero-sum games. arXiv:2207.06541, 2022.","title":"Self-play psro: Toward optimal populations in two-player zero-sum games","authors":"Mcaleer","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[McAleer et al., 2022b] S. McAleer, K. Wang, M. Lanctot, J. Lanier, P. Baldi, and R. Fox. Anytime optimal psro for two-player zero-sum games. arXiv:2201.07700, 2022.","title":"Anytime optimal psro for two-player zero-sum games","authors":"Mcaleer","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[McAleer et al., 2023] S. M. McAleer, G. Farina, G. Zhou, M. Wang, Y. Yang, and T. Sandholm. Team-PSRO for learning approximate TMECor in large team games via co- operative reinforcement learning. In NeurIPS, 2023.","title":"Team-PSRO for learning approximate TMECor in large team games via cooperative reinforcement learning","authors":"S M Mcaleer, G Farina, G Zhou, M Wang, Y Yang, T Sandholm","year":"2023","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[McKelvey and Palfrey, 1995] R. D. McKelvey and T. R. Palfrey. Quantal response equilibria for normal form games. Games and Economic Behavior, 10(1):6-38, 1995.","title":"Quantal response equilibria for normal form games","authors":"R D Mckelvey, T R Palfrey","year":"1995","venue":"Games and Economic Behavior","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[McMahan et al., 2003] H. B. McMahan, G. J. Gordon, and A. Blum. Planning in the presence of cost functions con- trolled by an adversary. In ICML, 2003.","title":"Planning in the presence of cost functions controlled by an adversary","authors":"H B Mcmahan, G J Gordon, A Blum","year":"2003","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Muller et al., 2020] P. Muller, S. Omidshafiei, M. Row- land, K. Tuyls, J. Pérolat, S. Liu, D. Hennes, L. Marris, M. Lanctot, E. Hughes, and Others. A Generalized Train- ing Approach for Multiagent Learning. In ICLR, 2020.","title":"A Generalized Training Approach for Multiagent Learning","authors":"P Muller, S Omidshafiei, M Rowland, K Tuyls, J Pérolat, S Liu, D Hennes, L Marris, M Lanctot, E Hughes, Others","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Muller et al., 2022] P. Muller, M. Rowland, R. Elie, G. Pil- iouras, J. Perolat, M. Lauriere, R. Marinier, O. Pietquin, and K. Tuyls. Learning Equilibria in Mean-Field Games: Introducing Mean-Field PSRO. In AAMAS, 2022.","title":"Learning Equilibria in Mean-Field Games: Introducing Mean-Field PSRO","authors":"P Muller, M Rowland, R Elie, G Piliouras, J Perolat, M Lauriere, R Marinier, O Pietquin, K Tuyls","year":"2022","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":56,"raw":"[Niu et al., 2021] L. Niu, D. Sahabandu, A. Clark, and R. Poovendran. A game-theoretic framework for con- trolled islanding in the presence of adversaries. In GameSec, 2021.","title":"A game-theoretic framework for controlled islanding in the presence of adversaries","authors":"L Niu, D Sahabandu, A Clark, R Poovendran","year":"2021","venue":"GameSec","doi":null,"url":null,"internal_url":null},{"position":57,"raw":"[Oliehoek et al., 2006] F. A. Oliehoek, E. D. De Jong, and N. Vlassis. The parallel Nash memory for asymmetric games. In GECCO, 2006.","title":"The parallel Nash memory for asymmetric games","authors":"F A Oliehoek, E D De, N Jong, Vlassis","year":"2006","venue":"GECCO","doi":null,"url":null,"internal_url":null},{"position":58,"raw":"[Oliehoek et al., 2019] F. A. Oliehoek, R. Savani, J. Gallego, E. van der Pol, and R. Groß. Beyond local Nash equilibria for adversarial networks. In BNAIC, 2019.","title":"Beyond local Nash equilibria for adversarial networks","authors":"F A Oliehoek, R Savani, J Gallego, E Van Der Pol, R Groß","year":"2019","venue":"BNAIC","doi":null,"url":null,"internal_url":null},{"position":59,"raw":"[Omidshafiei et al., 2019] S. Omidshafiei, C. Papadimitriou, G. Piliouras, K. Tuyls, M. Rowland, J.-B. Lespiau, W. M. Czarnecki, M. Lanctot, J. Perolat, and R. Munos. α-rank: Multi-agent evaluation by evolution. Scientific reports, 9, 2019.","title":"α-rank: Multi-agent evaluation by evolution","authors":"S Omidshafiei, C Papadimitriou, G Piliouras, K Tuyls, M Rowland, J.-B Lespiau, W M Czarnecki, M Lanctot, J Perolat, R Munos","year":"2019","venue":"Scientific reports","doi":null,"url":null,"internal_url":null},{"position":60,"raw":"[Perez-Nieves et al., 2021] N. Perez-Nieves, Y. Yang, O. Slumbers, D. H. Mguni, Y. Wen, and J. Wang. Mod- elling behavioural diversity for learning in open-ended games. In ICML, 2021.","title":"Modelling behavioural diversity for learning in open-ended games","authors":"N Perez-Nieves, Y Yang, O Slumbers, D H Mguni, Y Wen, J Wang","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":61,"raw":"[Popovici et al., 2012] E. Popovici, A. Bucci, R. P. Wiegand, and E. D. De Jong. Coevolutionary Principles, 2012.","title":"Coevolutionary Principles","authors":"E Popovici, A Bucci, R P Wiegand, E D De, Jong","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":62,"raw":"[Sanjaya et al., 2022] R. Sanjaya, J. Wang, and Y. Yang. Measuring the non-transitivity in chess. Algorithms, 15(5):152, 2022.","title":"Measuring the non-transitivity in chess","authors":"R Sanjaya, J Wang, Y Yang","year":"2022","venue":"Algorithms","doi":null,"url":null,"internal_url":null},{"position":63,"raw":"[Savani and Turocy, 2024] R. Savani and T. L. Turocy. Gam- bit: The Package for doing Computation in Game Theory, version 16.2.0., 2024.","title":"Gambit: The Package for doing Computation in Game Theory","authors":"R Savani, T L Turocy","year":"2024","venue":"version","doi":null,"url":null,"internal_url":null},{"position":64,"raw":"[Schvartzman and Wellman, 2009] L. J. Schvartzman and M. P. Wellman. Exploring large strategy spaces in empiri- cal game modeling. In AAMAS-AMEC Workshop, 2009.","title":"Exploring large strategy spaces in empirical game modeling","authors":"L J Schvartzman, M P Wellman","year":"2009","venue":"AAMAS-AMEC Workshop","doi":null,"url":null,"internal_url":null},{"position":65,"raw":"[Shoham et al., 2007] Y. Shoham, R. Powers, and T. Grenager. If multi-agent learning is the answer, what is the question? Artificial intelligence, 171, 2007.","title":"If multi-agent learning is the answer, what is the question?","authors":"Y Shoham, R Powers, T Grenager","year":"2007","venue":"Artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":66,"raw":"[Slumbers et al., 2023] O. Slumbers, D. H. Mguni, S. B. Blumberg, S. M. Mcaleer, Y. Yang, and J. Wang. A game- theoretic framework for managing risk in multi-agent sys- tems. In ICML, 2023.","title":"A gametheoretic framework for managing risk in multi-agent systems","authors":"O Slumbers, D H Mguni, S B Blumberg, S M Mcaleer, Y Yang, J Wang","year":"2023","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":67,"raw":"[Smith and Wellman, 2023] M. O. Smith and M. P. Well- man. Co-Learning Empirical Games and World Models. arXiv:2305.14223, 2023.","title":"Co-Learning Empirical Games and World Models","authors":"M O Smith, M P","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":68,"raw":"[Smith et al., 2023] M. O. Smith, T. Anthony, and M. P. Wellman. Strategic knowledge transfer. JMLR, 24, 2023.","title":"Strategic knowledge transfer","authors":"M O Smith, T Anthony, M P Wellman","year":"2023","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":69,"raw":"[Sokota et al., 2019] S. Sokota, C. Ho, and B. Wiedenbeck. Learning deviation payoffs in simulation-based games. In AAAI, 2019.","title":"Learning deviation payoffs in simulation-based games","authors":"S Sokota, C Ho, B Wiedenbeck","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":70,"raw":"[Song et al., 2024] Y. Song, H. Jiang, Z. Tian, H. Zhang, Y. Zhang, J. Zhu, Z. Dai, W. Zhang, and J. Wang. An em- pirical study on google research football multi-agent sce- narios. Machine Intelligence Research, pages 1-22, 2024.","title":"An empirical study on google research football multi-agent scenarios","authors":"Y Song, H Jiang, Z Tian, H Zhang, Y Zhang, J Zhu, Z Dai, W Zhang, J Wang","year":"2024","venue":"Machine Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":71,"raw":"[Tang et al., 2023] X. Tang, L. C. Dinh, S. M. Mcaleer, and Y. Yang. Regret-minimizing double oracle for extensive- form games. In ICML, 2023.","title":"Regret-minimizing double oracle for extensiveform games","authors":"X Tang, L C Dinh, S M Mcaleer, Y Yang","year":"2023","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":72,"raw":"[Taylor and Jonker, 1978] P. D. Taylor and L. B. Jonker. Evolutionary stable strategies and game dynamics. Math- ematical biosciences, 40, 1978.","title":"Evolutionary stable strategies and game dynamics","authors":"P D Taylor, L B Jonker","year":"1978","venue":"Mathematical biosciences","doi":null,"url":null,"internal_url":null},{"position":73,"raw":"[Tong et al., 2020] L. Tong, A. Laszka, C. Yan, N. Zhang, and Y. Vorobeychik. Finding needles in a moving haystack: Prioritizing alerts with adversarial reinforce- ment learning. In AAAI, 2020.","title":"Finding needles in a moving haystack: Prioritizing alerts with adversarial reinforcement learning","authors":"L Tong, A Laszka, C Yan, N Zhang, Y Vorobeychik","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":74,"raw":"[Tuyls and Weiss, 2012] K. Tuyls and G. Weiss. Multiagent learning: Basics, challenges, and prospects. Ai Magazine, 33(3):41-41, 2012.","title":"Multiagent learning: Basics, challenges, and prospects","authors":"K Tuyls, G Weiss","year":"2012","venue":"Ai Magazine","doi":null,"url":null,"internal_url":null},{"position":75,"raw":"[Vinyals et al., 2019] O. Vinyals, I. Babuschkin, et al. Grandmaster level in StarCraft II using multi-agent rein- forcement learning. Nature, 575, 2019.","title":"Grandmaster level in StarCraft II using multi-agent reinforcement learning","authors":"O Vinyals, I Babuschkin","year":"2019","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":76,"raw":"[von Stengel, 2002] B. von Stengel. Computing equilibria for two-person games. In Handbook of game theory with economic applications, volume 3. Elsevier, 2002.","title":"Computing equilibria for two-person games","authors":"B Stengel","year":"2002","venue":"Handbook of game theory with economic applications","doi":null,"url":null,"internal_url":null},{"position":77,"raw":"[Wang and Wellman, 2023a] Y. Wang and M. P. Wellman. Empirical Game-Theoretic Analysis for Mean Field Games. In AAMAS, 2023.","title":"Empirical Game-Theoretic Analysis for Mean Field Games","authors":"Wang, Wellman, ] Y Wang, M P Wellman","year":"2023","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":78,"raw":"[Wang and Wellman, 2023b] Y. Wang and M. P. Wellman. Regularization for Strategy Exploration in Empirical Game-Theoretic Analysis. arXiv:2302.04928, 2023.","title":"Regularization for Strategy Exploration in Empirical Game-Theoretic Analysis","authors":"Wang, Wellman, ] Y Wang, M P Wellman","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":79,"raw":"[Wang and Wellman, 2024] Y. Wang and M. P. Wellman. Generalized response objectives for strategy exploration in empirical game-theoretic analysis. In AAMAS, 2024.","title":"Generalized response objectives for strategy exploration in empirical game-theoretic analysis","authors":"Y Wang, M P Wellman","year":"2024","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":80,"raw":"[Wang et al., 2019] Y. Wang, Z. R. Shi, L. Yu, Y. Wu, R. Singh, L. Joppa, and F. Fang. Deep reinforcement learn- ing for green security games with real-time information. In AAAI, 2019.","title":"Deep reinforcement learning for green security games with real-time information","authors":"Y Wang, Z R Shi, L Yu, Y Wu, R Singh, L Joppa, F Fang","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":81,"raw":"[Wang et al., 2021] C. Wang, Y. Yang, O. Slumbers, C. Han, T. Guo, H. Zhang, and J. Wang. A game-theoretic ap- proach for improving generalization ability of tsp solvers. arXiv preprint arXiv:2110.15105, 2021.","title":"A game-theoretic approach for improving generalization ability of tsp solvers","authors":"C Wang, Y Yang, O Slumbers, C Han, T Guo, H Zhang, J Wang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":82,"raw":"[Wang et al., 2022] Y. Wang, Q. Ma, and M. P. Well- man. Evaluating strategy exploration in empirical game- theoretic analysis. In AAMAS, 2022.","title":"Evaluating strategy exploration in empirical gametheoretic analysis","authors":"Y Wang, Q Ma, M P Wellman","year":"2022","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":83,"raw":"[Wang et al., 2024] C. Wang, Z. Yu, S. McAleer, T. Yu, and Y. Yang. Asp: Learn a universal neural solver! IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 2024.","title":"Asp: Learn a universal neural solver!","authors":"C Wang, Z Yu, S Mcaleer, T Yu, Y Yang","year":"2024","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":84,"raw":"[Wellman et al., 2024] M. P. Wellman, K. Tuyls, and A. Greenwald. Empirical game-theoretic analysis: A sur- vey. arXiv preprint arXiv:2403.04018, 2024.","title":"Empirical game-theoretic analysis: A survey","authors":"M P Wellman, K Tuyls, A Greenwald","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":85,"raw":"[Wellman, 2006] M. P. Wellman. Methods for empirical game-theoretic analysis. In AAAI, 2006.","title":"Methods for empirical game-theoretic analysis","authors":"M P Wellman","year":"2006","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":86,"raw":"[Wright et al., 2019] M. Wright, Y. Wang, and M. P. Well- man. Iterated deep reinforcement learning in games: History-aware training for improved stability. In ACM EC, 2019.","title":"Iterated deep reinforcement learning in games: History-aware training for improved stability","authors":"M Wright, Y Wang, M P Wellman","year":"2019","venue":"ACM EC","doi":null,"url":null,"internal_url":null},{"position":87,"raw":"[Xu et al., 2021] L. Xu, A. Perrault, F. Fang, H. Chen, and M. Tambe. Robust reinforcement learning under minimax regret for green security. In UAI, 2021.","title":"Robust reinforcement learning under minimax regret for green security","authors":"L Xu, A Perrault, F Fang, H Chen, M Tambe","year":"2021","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":88,"raw":"[Yang and Wang, 2020] Y. Yang and J. Wang. An overview of multi-agent reinforcement learning from game theoret- ical perspective. arXiv:2011.00583, 2020.","title":"An overview of multi-agent reinforcement learning from game theoretical perspective","authors":"Y Yang, J Wang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":89,"raw":"[Yang et al., 2020] Y. Yang, R. Tutunov, P. Sakulwongtana, and H. B. Ammar. α α -rank: Practically scaling α-rank through stochastic optimisation. In AAMAS, 2020.","title":"α α -rank: Practically scaling α-rank through stochastic optimisation","authors":"Y Yang, R Tutunov, P Sakulwongtana, H B Ammar","year":"2020","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":90,"raw":"[Yang et al., 2021] Y. Yang, J. Luo, et al. Diverse auto- curriculum is critical for successful real-world multiagent learning systems. arXiv preprint arXiv:2102.07659, 2021.","title":"Diverse autocurriculum is critical for successful real-world multiagent learning systems","authors":"Y Yang, J Luo","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":91,"raw":"[Yang et al., 2022] C. Yang, R. Wang, X. Wang, and Z. Wang. A game-theoretic perspective of gener- alization in reinforcement learning. arXiv preprint arXiv:2208.03650, 2022.","title":"A game-theoretic perspective of generalization in reinforcement learning","authors":"C Yang, R Wang, X Wang, Z Wang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":92,"raw":"[Yao et al., 2023] J. Yao, W. Liu, H. Fu, Y. Yang, S. McAleer, Q. Fu, and W. Yang. Policy space diversity for non-transitive games. arXiv:2306.16884, 2023.","title":"Policy space diversity for non-transitive games","authors":"J Yao, W Liu, H Fu, Y Yang, S Mcaleer, Q Fu, W Yang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":93,"raw":"[Yin et al., 2018] Y. Yin, Y. Vorobeychik, B. An, and N. Ha- zon. Optimal defense against election control by deleting voter groups. Artificial Intelligence, 259, 2018.","title":"Optimal defense against election control by deleting voter groups","authors":"Y Yin, Y Vorobeychik, B An, N Hazon","year":"2018","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":94,"raw":"[Zahavy et al., 2023] T. Zahavy, V. Veeriah, S. Hou, K. Waugh, M. Lai, E. Leurent, N. Tomasev, L. Schut, D. Hassabis, and S. Singh. Diversifying AI: To- wards creative chess with AlphaZero. arXiv preprint arXiv:2308.09175, 2023.","title":"Diversifying AI: Towards creative chess with AlphaZero","authors":"T Zahavy, V Veeriah, S Hou, K Waugh, M Lai, E Leurent, N Tomasev, L Schut, D Hassabis, S Singh","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":95,"raw":"[Zhang and Sandholm, 2024] B. H. Zhang and T. Sandholm. Exponential lower bounds on the double oracle algorithm in zero-sum games. In IJCAI, 2024.","title":"Exponential lower bounds on the double oracle algorithm in zero-sum games","authors":"B H Zhang, T Sandholm","year":"2024","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2024/336"},{"position":96,"raw":"[Zhang et al., 2023] B. Zhang, G. Farina, I. Anagnostides, F. Cacciamani, S. McAleer, A. Haupt, A. Celli, N. Gatti, V. Conitzer, and T. Sandholm. Computing optimal equi- libria and mechanisms via learning in zero-sum extensive- form games. In NeurIPS, 2023.","title":"Computing optimal equilibria and mechanisms via learning in zero-sum extensiveform games","authors":"B Zhang, G Farina, I Anagnostides, F Cacciamani, S Mcaleer, A Haupt, A Celli, N Gatti, V Conitzer, T Sandholm","year":"2023","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":97,"raw":"[Zhao et al., 2023] Z. Zhao, M. Wen, Y. Wen, and Y. Yang. Open-ended learning in general-sum games: The role of diversity in correlated equilibrium. 2023.","title":"Open-ended learning in general-sum games: The role of diversity in correlated equilibrium","authors":"Z Zhao, M Wen, Y Wen, Y Yang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":98,"raw":"[Zhou et al., 2022] M. Zhou, J. Chen, Y. Wen, W. Zhang, Y. Yang, Y. Yu, and J. Wang. Efficient Policy Space Re- sponse Oracles. arXiv:2202.00633, 2022.","title":"Efficient Policy Space Response Oracles","authors":"M Zhou, J Chen, Y Wen, W Zhang, Y Yang, Y Yu, J Wang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":99,"raw":"[Zhou et al., 2023] M. Zhou, Z. Wan, H. Wang, M. Wen, R. Wu, Y. Wen, Y. Yang, Y. Yu, J. Wang, and W. Zhang. Malib: A parallel framework for population-based multi- agent reinforcement learning. Journal of Machine Learn- ing Research, 24(150):1-12, 2023.","title":"Malib: A parallel framework for population-based multiagent reinforcement learning","authors":"M Zhou, Z Wan, H Wang, M Wen, R Wu, Y Wen, Y Yang, Y Yu, J Wang, W Zhang","year":"2023","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":100,"raw":"[Zinkevich et al., 2007] M. Zinkevich, M. Johanson, M. Bowling, and C. Piccione. Regret minimization in games with incomplete information. NIPS, 2007.","title":"Regret minimization in games with incomplete information","authors":"M Zinkevich, M Johanson, M Bowling, C Piccione","year":"2007","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":101,"raw":"[Zou et al., 2022] M. Zou, J. Chen, J. Luo, Z. Hu, and S. Chen. Equilibrium Approximating and Online Learn- ing for Anti-Jamming Game of Satellite Communication Power Allocation. Electronics, 11(21):3526, 2022.","title":"Equilibrium Approximating and Online Learning for Anti-Jamming Game of Satellite Communication Power Allocation","authors":"M Zou, J Chen, J Luo, Z Hu, S Chen","year":"2022","venue":"Electronics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}