{"type":"ScholarlyArticle","title":"Correlation-Based Algorithm for Team-Maxmin Equilibrium in Multiplayer Extensive-Form Games","authors":[{"name":"Youzhi Zhang","slug":"youzhi-zhang","given":"Youzhi","family":"Zhang","affiliation":"Department of Computer Science , Northwestern University , USA"},{"name":"Bo An","slug":"bo-an","given":"Bo","family":"An","affiliation":"School of Computer Science and Engineering , Nanyang Technological University , Singapore"},{"name":"V. S. Subrahmanian","slug":"v-s-subrahmanian","given":"V. S.","family":"Subrahmanian","affiliation":"Department of Computer Science , Northwestern University , USA; Buffet Institute for Global Affairs , Northwestern University , USA"}],"abstract":"Efficient algorithms computing a Nash equilibrium have been successfully applied to large zero- sum two-player extensive-form games (e.g., poker). However, in multiplayer games, computing a Nash equilibrium is generally hard, and the equilibria are not exchangeable, which makes players face the problem of selecting one of many different Nash equilibria. In this paper, we focus on an alternative solution concept in zero-sum multiplayer extensive-form games called Team-Maxmin Equilibrium (TME). It is a Nash equilibrium that maximizes each team member’s utility. As TME is unique in general, it avoids the equilibrium selection problem. However, it is still difficult (FNP- hard) to find a TME. Computing it can be formulated as a non-convex program, but existing algorithms are capable of solving this program for only very small games. In this paper, we first refine the complexity result for computing a TME by using a correlation plan to show that a TME can be found in polynomial time in a specific class of games according to our boundary for complexity. Second, we propose an efficient correlation-based algorithm to solve the non-convex program for TME in games not belonging to this class. The algorithm combines two special correlation plans based on McCormick envelopes for convex relaxation and von Stengel-Forges polytope for correlated equilibria. We show that restricting the feasible solution space to von Stengel-Forges polytope will strictly reduce the feasible solution space after convex re- laxation of nonlinear terms. Finally, experiments show that our algorithm is about four orders of magnitude faster than the prior state of the art and can solve many previously unsolvable games.","doi":"10.24963/ijcai.2022/86","url":"https://ijcai.org/proceedings/2022/86","pdf":"https://ijcai.org/proceedings/2022/0086.pdf","markdown":"https://ijcai.org/proceedings/2022/86.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/86","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":86,"pages":{"first":606,"last":612},"keywords":["Agent-based and Multi-agent Systems: Cooperative Games","Agent-based and Multi-agent Systems: Noncooperative Games"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987047","length_sec":83,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985190","length_sec":673,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Andrade et al., 2019] Tiago Andrade, Fabricio Oliveira, Sil- vio Hamacher, and Andrew Eberhard. Enhancing the normalized multiparametric disaggregation technique for mixed-integer quadratic programming. Journal of Global Optimization, 73(4):701-722, 2019.","title":"Enhancing the normalized multiparametric disaggregation technique for mixed-integer quadratic programming","authors":"Tiago Andrade, Fabricio Oliveira, Silvio Hamacher, Andrew Eberhard","year":"2019","venue":"Journal of Global Optimization","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Basilico et al., 2017] Nicola Basilico, Andrea Celli, Giuseppe De Nittis, and Nicola Gatti. Team-maxmin equilibrium: Efficiency bounds and algorithms. In AAAI, pages 356-362, 2017.","title":"Team-maxmin equilibrium: Efficiency bounds and algorithms","authors":"Nicola Basilico, Andrea Celli, Giuseppe De Nittis, Nicola Gatti","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bowling et al., 2015] Michael Bowling, Neil Burch, Michael Johanson, and Oskari Tammelin. Heads-up limit hold'em poker is solved. Science, 347(6218):145-149, 2015.","title":"Heads-up limit hold'em poker is solved","authors":"Michael Bowling, Neil Burch, Michael Johanson, Oskari Tammelin","year":"2015","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brown and Sandholm, 2017] Noam Brown and Tuomas Sandholm. Safe and nested subgame solving for imperfect-information games. In NeurIPS, 2017.","title":"Safe and nested subgame solving for imperfect-information games","authors":"Sandholm Brown, Noam Brown, Tuomas Sandholm","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Brown and Sandholm, 2018] Noam Brown and Tuomas Sandholm. Superhuman AI for heads-up no-limit poker: Libratus beats top professionals. Science, 359(6374):418- 424, 2018.","title":"Superhuman AI for heads-up no-limit poker: Libratus beats top professionals","authors":"Sandholm Brown, Noam Brown, Tuomas Sandholm","year":"2018","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Brown and Sandholm, 2019] Noam Brown and Tuomas Sandholm. Superhuman AI for multiplayer poker. Sci- ence, 365(6456):885-890, 2019.","title":"Superhuman AI for multiplayer poker","authors":"Sandholm Brown, Noam Brown, Tuomas Sandholm","year":"2019","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Celli and Gatti, 2018] Andrea Celli and Nicola Gatti. Com- putational results for extensive-form adversarial team games. In AAAI, pages 965-972, 2018.","title":"Computational results for extensive-form adversarial team games","authors":"Gatti Celli, Andrea Celli, Nicola Gatti","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen and Deng, 2005] Xi Chen and Xiaotie Deng. 3-Nash is PPAD-complete. In Electronic Colloquium on Compu- tational Complexity, volume 134, pages 2-29, 2005.","title":"Xi Chen and Xiaotie Deng. 3-Nash is PPAD-complete","authors":"Deng Chen","year":"2005","venue":"Electronic Colloquium on Computational Complexity","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Farina and Sandholm, 2020] G Farina and T Sandholm. Polynomial-time computation of optimal correlated equi- libria in two-player extensive-form games with public chance moves and beyond. In NeurIPS, 2020. [Gurobi, 2020] Gurobi. Non-convex quadratic opti- mization. https://pages.gurobi.com/rs/181-ZYS-005/ images/2020-01-14 Non%20Convex%20Quadratic% 20Optimization%20in%20Gurobi%209.0%20Webinar. pdf, 2020. Accessed: 2021-08-02.","title":"Polynomial-time computation of optimal correlated equilibria in two-player extensive-form games with public chance moves and beyond","authors":"G Farina, Sandholm","year":"2020","venue":"Gurobi. Non-convex quadratic optimization","doi":null,"url":"https://pages.gurobi.com/rs/181-ZYS-005/images/2020-01-14Non%20Convex%20Quadratic%20Optimization%20in%20Gurobi%209.0%20Webinar.pdf","internal_url":null},{"position":10,"raw":"[Jiang et al., 2013] Albert Xin Jiang, Ariel D Procaccia, Yundi Qian, Nisarg Shah, and Milind Tambe. Defender (mis)coordination in security games. In IJCAI, pages 220- 226, 2013.","title":"Defender (mis)coordination in security games","authors":"Albert Xin Jiang, Ariel D Procaccia, Yundi Qian, Nisarg Shah, Milind Tambe","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/42"},{"position":11,"raw":"[Kolodziej et al., 2013] Scott Kolodziej, Pedro M Castro, and Ignacio E Grossmann. Global optimization of bilin- ear programs with a multiparametric disaggregation tech- nique. Journal of Global Optimization, 57(4):1039-1063, 2013.","title":"Global optimization of bilinear programs with a multiparametric disaggregation technique","authors":"Scott Kolodziej, Pedro M Castro, Ignacio E Grossmann","year":"2013","venue":"Journal of Global Optimization","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[McCormick, 1976] Garth P McCormick. Computability of global solutions to factorable nonconvex programs: Part I-Convex underestimating problems. Mathematical Pro- gramming, 10(1):147-175, 1976.","title":"Computability of global solutions to factorable nonconvex programs: Part I-Convex underestimating problems","authors":"P Garth, Mccormick","year":"1976","venue":"Mathematical Programming","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Moravčík et al., 2017] Matěj Moravčík, Martin Schmid, Neil Burch, Viliam Lisý, Dustin Morrill, Nolan Bard, Trevor Davis, Kevin Waugh, Michael Johanson, and Michael Bowling. DeepStack: Expert-level artificial in- telligence in no-limit poker. Science, 356:508-513, 2017.","title":"DeepStack: Expert-level artificial intelligence in no-limit poker","authors":"Moravčík","year":"2017","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nash, 1951] John Nash. Non-cooperative games. Annals of Mathematics, pages 286-295, 1951.","title":"Non-cooperative games","authors":"John Nash","year":"1951","venue":"Annals of Mathematics","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Russell and Norvig, 2016] Stuart J Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. Malaysia; Pearson Education Limited, 2016.","title":"Artificial Intelligence: A Modern Approach","authors":"J Stuart, Peter Russell, Norvig","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Shoham and Leyton-Brown, 2008] Yoav Shoham and Kevin Leyton-Brown. Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, 2008.","title":"Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations","authors":"Yoav Shoham, Kevin Leyton-Brown","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Von Stengel and Forges, 2008] Bernhard Von Stengel and Franc ¸oise Forges. Extensive-form correlated equilibrium: Definition and computational complexity. Mathematics of Operations Research, 33(4):1002-1022, 2008.","title":"Extensive-form correlated equilibrium: Definition and computational complexity","authors":"Von Stengel, Forges","year":"2008","venue":"Mathematics of Operations Research","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[von Stengel and Koller, 1997] Bernhard von Stengel and Daphne Koller. Team-maxmin equilibria. Games and Eco- nomic Behavior, 21(1-2):309-321, 1997.","title":"Bernhard von Stengel and Daphne Koller","authors":"Von Stengel, Koller","year":"1997","venue":"Games and Economic Behavior","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[von Stengel, 1996] Bernhard von Stengel. Efficient compu- tation of behavior strategies. Games and Economic Behav- ior, 14(2):220-246, 1996.","title":"Efficient computation of behavior strategies","authors":"Von Stengel, ; Bernhard, Von Stengel","year":"1996","venue":"Games and Economic Behavior","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2019] Yufei Wang, Zheyuan Ryan Shi, Lantao Yu, Yi Wu, Rohit Singh, Lucas Joppa, and Fei Fang. Deep reinforcement learning for green security games with real- time information. In AAAI, volume 33, pages 1401-1408, 2019.","title":"Deep reinforcement learning for green security games with realtime information","authors":"Yufei Wang, Zheyuan, Ryan Shi, Lantao Yu, Yi Wu, Rohit Singh, Lucas Joppa, Fei Fang","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zha et al., 2021] Daochen Zha, Jingru Xie, Wenye Ma, Sheng Zhang, Xiangru Lian, Xia Hu, and Ji Liu. DouZero: Mastering DouDizhu with self-play deep reinforcement learning. In ICML, 2021.","title":"DouZero: Mastering DouDizhu with self-play deep reinforcement learning","authors":"Zha","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang and An, 2020a] Youzhi Zhang and Bo An. Com- puting team-maxmin equilibria in zero-sum multiplayer extensive-form games. In AAAI, volume 34, pages 2318- 2325, 2020.","title":"Computing team-maxmin equilibria in zero-sum multiplayer extensive-form games","authors":"An ; Zhang, Youzhi Zhang, Bo An","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang and An, 2020b] Youzhi Zhang and Bo An. Converg- ing to team-maxmin equilibria in zero-sum multiplayer games. In ICML, pages 11033-11043, 2020.","title":"Converging to team-maxmin equilibria in zero-sum multiplayer games","authors":"An ; Zhang, Youzhi Zhang, Bo An","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang and Sandholm, 2020] Brian Zhang and Tuomas Sandholm. Sparsified linear programming for zero-sum equilibrium finding. In ICML, pages 11256-11267, 2020.","title":"Sparsified linear programming for zero-sum equilibrium finding","authors":"Brian Zhang, Tuomas Sandholm","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2019] Youzhi Zhang, Qingyu Guo, Bo An, Long Tran-Thanh, and Nicholas R Jennings. Optimal in- terdiction of urban criminals with the aid of real-time in- formation. In AAAI, volume 33, pages 1262-1269, 2019.","title":"Optimal interdiction of urban criminals with the aid of real-time information","authors":"Zhang","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2021] Youzhi Zhang, Bo An, and Jakub Černỳ. Computing ex ante coordinated team-maxmin equilibria in zero-sum multiplayer extensive-form games. In AAAI, vol- ume 35, pages 5813-5821, 2021.","title":"Computing ex ante coordinated team-maxmin equilibria in zero-sum multiplayer extensive-form games","authors":"Zhang","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zinkevich et al., 2008] Martin Zinkevich, Michael Johan- son, Michael Bowling, and Carmelo Piccione. Regret minimization in games with incomplete information. In NeurIPS, pages 1729-1736, 2008.","title":"Regret minimization in games with incomplete information","authors":"Martin Zinkevich, Michael Johanson, Michael Bowling, Carmelo Piccione","year":"2008","venue":"NeurIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}