{"type":"ScholarlyArticle","title":"ADMN: Agent-Driven Modular Network for Dynamic Parameter Sharing in Cooperative Multi-Agent Reinforcement Learning","authors":[{"name":"Yang Yu","slug":"yang-yu","given":"Yang","family":"Yu","affiliation":"School of Artificial Intelligence , University of Chinese Academy of Sciences; CRISE , Institute of Automation , Chinese Academy of Sciences"},{"name":"Qiyue Yin","slug":"qiyue-yin","given":"Qiyue","family":"Yin","affiliation":"School of Artificial Intelligence , University of Chinese Academy of Sciences; CRISE , Institute of Automation , Chinese Academy of Sciences"},{"name":"Junge Zhang","slug":"junge-zhang","given":"Junge","family":"Zhang","affiliation":"School of Artificial Intelligence , University of Chinese Academy of Sciences; CRISE , Institute of Automation , Chinese Academy of Sciences"},{"name":"Pei Xu","slug":"pei-xu","given":"Pei","family":"Xu","affiliation":"CRISE , Institute of Automation , Chinese Academy of Sciences"},{"name":"Kaiqi Huang","slug":"kaiqi-huang","given":"Kaiqi","family":"Huang","affiliation":"School of Artificial Intelligence , University of Chinese Academy of Sciences; CRISE , Institute of Automation , Chinese Academy of Sciences; CAS , Center for Excellence in Brain Science and Intelligence Technology"}],"abstract":"Parameter sharing is a common strategy in multi-agent reinforcement learning (MARL) to make the training more efficient and scalable. However, applying parameter sharing among agents indiscriminately hinders the emergence of agents diversity and degrades the final cooperative performance. To better balance parameter sharing and agents diversity, we propose a novel Agent-Driven Modular Network (ADMN), where agents share a base network consisting of multiple specialized modules, and each agent has its own routing to connect these modules. In ADMN, modules are shared among agents to improve the training efficiency, while the combination of different modules brings rich diversity. The agent routing at different time steps is learned end-to-end to achieve a dynamic and adaptive balance. Specifically, we also propose an information-theoretical regularization between the routing of agents and their behavior to further guarantee the identifiability of different routing. We evaluated ADMN in challenging StarCraft micromanagement games and Google Research Football games, and results demonstrate the superior performance of ADMN, particularly in larger or heterogeneous cooperative tasks.","doi":"10.24963/ijcai.2024/34","url":"https://ijcai.org/proceedings/2024/34","pdf":"https://ijcai.org/proceedings/2024/0034.pdf","markdown":"https://ijcai.org/proceedings/2024/34.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/34","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":34,"pages":{"first":302,"last":310},"keywords":["Machine Learning: ML: Reinforcement learning","Agent-based and Multi-agent Systems: MAS: Multi-agent learning"],"videos":[],"references":[{"position":1,"raw":"[Bard et al., 2020] Nolan Bard, Jakob N Foerster, Sarath Chandar, Neil Burch, Marc Lanctot, H Francis Song, Emilio Parisotto, Vincent Dumoulin, Subhodeep Moitra, Edward Hughes, et al. The hanabi challenge: A new fron- tier for ai research. Artificial Intelligence, 280:103216, 2020.","title":"The hanabi challenge: A new frontier for ai research","authors":"Nolan Bard, Jakob N Foerster, Sarath Chandar, Neil Burch, Marc Lanctot, Francis Song, Emilio Parisotto, Subhodeep Vincent Dumoulin, Edward Moitra, Hughes","year":"2020","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bazzan, 2009] Ana LC Bazzan. Opportunities for multi- agent systems and multiagent reinforcement learning in traffic control. Autonomous Agents and Multi-Agent Sys- tems, 18:342-375, 2009.","title":"Opportunities for multiagent systems and multiagent reinforcement learning in traffic control","authors":"Ana Lc Bazzan","year":"2009","venue":"Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Berner et al., 2019] Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemysław De ¸biak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Chris Hesse, et al. Dota 2 with large scale deep reinforce- ment learning. arXiv preprint arXiv:1912.06680, 2019.","title":"Dota 2 with large scale deep reinforcement learning","authors":"Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemysław De ¸biak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Chris Hesse","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cao et al., 2012] Yongcan Cao, Wenwu Yu, Wei Ren, and Guanrong Chen. An overview of recent progress in the study of distributed multi-agent coordination. IEEE Trans- actions on Industrial informatics, 9(1):427-438, 2012.","title":"An overview of recent progress in the study of distributed multi-agent coordination","authors":"Cao","year":"2012","venue":"IEEE Transactions on Industrial informatics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Christianos et al., 2021] Filippos Christianos, Georgios Pa- poudakis, Muhammad A Rahman, and Stefano V Al- brecht. Scaling multi-agent reinforcement learning with selective parameter sharing. In International Conference on Machine Learning, pages 1989-1998. PMLR, 2021.","title":"Scaling multi-agent reinforcement learning with selective parameter sharing","authors":"Filippos Christianos, Georgios Papoudakis, Muhammad A Rahman, Stefano V Albrecht","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chu et al., 2019] Tianshu Chu, Jie Wang, Lara Codecà, and Zhaojian Li. Multi-agent deep reinforcement learning for large-scale traffic signal control. IEEE Transactions on In- telligent Transportation Systems, 21(3):1086-1095, 2019.","title":"Multi-agent deep reinforcement learning for large-scale traffic signal control","authors":"Tianshu Chu, Jie Wang, Lara Codecà, Zhaojian Li","year":"2019","venue":"IEEE Transactions on Intelligent Transportation Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[DeLoach and Garcia-Ojeda, 2010] Scott A DeLoach and Juan Carlos Garcia-Ojeda. O-mase: a customisable ap- proach to designing and building complex, adaptive multi- agent systems. International Journal of Agent-Oriented Software Engineering, 4(3):244-280, 2010.","title":"O-mase: a customisable approach to designing and building complex, adaptive multiagent systems","authors":"A Scott, Juan Deloach, Garcia-Ojeda Carlos","year":"2010","venue":"International Journal of Agent-Oriented Software Engineering","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Eysenbach et al., 2018] Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is all you need: Learning skills without a reward function. In Inter- national Conference on Learning Representations, 2018.","title":"Diversity is all you need: Learning skills without a reward function","authors":"Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, Sergey Levine","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Foerster et al., 2018] Jakob Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. Counterfactual multi-agent policy gradients. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 32, 2018.","title":"Counterfactual multi-agent policy gradients","authors":"Foerster","year":"2018","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hu et al., 2022] Siyi Hu, Chuanlong Xie, Xiaodan Liang, and Xiaojun Chang. Policy diagnosis via measuring role diversity in cooperative multi-agent rl. In Inter- national Conference on Machine Learning, pages 9041- 9071. PMLR, 2022.","title":"Policy diagnosis via measuring role diversity in cooperative multi-agent rl","authors":"Siyi Hu, Chuanlong Xie, Xiaodan Liang, Xiaojun Chang","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hüttenrauch et al., 2019] Maximilian Hüttenrauch, Sosic Adrian, Gerhard Neumann, et al. Deep reinforcement learning for swarm systems. Journal of Machine Learn- ing Research, 20(54):1-31, 2019.","title":"Deep reinforcement learning for swarm systems","authors":"Maximilian Hüttenrauch, Sosic Adrian, Gerhard Neumann","year":"2019","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Iqbal et al., 2021] Shariq Iqbal, Christian A Schroeder De Witt, Bei Peng, Wendelin Böhmer, Shimon Whiteson, and Fei Sha. Randomized entity-wise factorization for multi-agent reinforcement learning. In International Con- ference on Machine Learning, pages 4596-4606. PMLR, 2021.","title":"Randomized entity-wise factorization for multi-agent reinforcement learning","authors":"Shariq Iqbal, Christian A Schroeder De Witt, Bei Peng, Wendelin Böhmer, Shimon Whiteson, Fei Sha","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jiang and Lu, 2021] Jiechuan Jiang and Zongqing Lu. The emergence of individuality. In International Conference on Machine Learning, pages 4992-5001. PMLR, 2021.","title":"The emergence of individuality","authors":"Lu ; Jiang, Jiechuan Jiang, Zongqing Lu","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kim and Sung, 2023] Woojun Kim and Youngchul Sung. Parameter sharing with network pruning for scalable multi-agent deep reinforcement learning. In Proceed- ings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, pages 1942-1950, 2023.","title":"Parameter sharing with network pruning for scalable multi-agent deep reinforcement learning","authors":"Sung ; Kim, Woojun Kim, Youngchul Sung","year":"2023","venue":"Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kurach et al., 2020] Karol Kurach, Anton Raichuk, Piotr Stańczyk, Michał Zaja ¸c, Olivier Bachem, Lasse Espe- holt, Carlos Riquelme, Damien Vincent, Marcin Michal- ski, Olivier Bousquet, et al. Google research football: A novel reinforcement learning environment. In Proceed- ings of the AAAI Conference on Artificial Intelligence, vol- ume 34, pages 4501-4510, 2020.","title":"Google research football: A novel reinforcement learning environment","authors":"Karol Kurach, Anton Raichuk, Piotr Stańczyk, Michał Zaja ¸c, Olivier Bachem, Lasse Espeholt, Carlos Riquelme, Damien Vincent, Marcin Michalski, Olivier Bousquet","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lhaksmana et al., 2018] Kemas M Lhaksmana, Yohei Mu- rakami, and Toru Ishida. Role-based modeling for design- ing agent behavior in self-organizing multi-agent systems. International Journal of Software Engineering and Knowl- edge Engineering, 28(01):79-96, 2018.","title":"Role-based modeling for designing agent behavior in self-organizing multi-agent systems","authors":"Yohei Kemas M Lhaksmana, Toru Murakami, Ishida","year":"2018","venue":"International Journal of Software Engineering and Knowledge Engineering","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2021] Chenghao Li, Tonghan Wang, Chengjie Wu, Qianchuan Zhao, Jun Yang, and Chongjie Zhang. Cel- ebrating diversity in shared multi-agent reinforcement learning. Advances in Neural Information Processing Sys- tems, 34:3991-4002, 2021.","title":"Celebrating diversity in shared multi-agent reinforcement learning","authors":"Li","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lowe et al., 2017] Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environ- ments. Advances in Neural Information Processing Sys- tems, 30:6379-6390, 2017.","title":"Multi-agent actor-critic for mixed cooperative-competitive environments","authors":"Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, Igor Mordatch","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Matignon et al., 2012] Laetitia Matignon, Guillaume J Lau- rent, and Nadine Le Fort-Piat. Independent reinforcement learners in cooperative markov games: a survey regarding coordination problems. The Knowledge Engineering Re- view, 27(1):1-31, 2012.","title":"Independent reinforcement learners in cooperative markov games: a survey regarding coordination problems","authors":"Laetitia Matignon, Guillaume J Laurent, Nadine Le Fort-Piat","year":"2012","venue":"The Knowledge Engineering Review","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mnih et al., 2015] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Belle- mare, Alex Graves, Martin Riedmiller, Andreas K Fidje- land, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature, 518(7540):529-533, 2015.","title":"Human-level control through deep reinforcement learning","authors":"Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski","year":"2015","venue":"nature","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Peng et al., 2012] Chunyi Peng, Minkyong Kim, Zhe Zhang, and Hui Lei. Vdn: Virtual machine image distribu- tion network for cloud data centers. In 2012 Proceedings IEEE INFOCOM, pages 181-189. IEEE, 2012.","title":"Vdn: Virtual machine image distribution network for cloud data centers","authors":"Chunyi Peng, Minkyong Kim, Zhe Zhang, Hui Lei","year":"2012","venue":"Proceedings IEEE INFOCOM","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rashid et al., 2018] Tabish Rashid, Mikayel Samvelyan, Christian Schroeder, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning. In International Conference on Machine Learning, pages 4295-4304. PMLR, 2018.","title":"Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning","authors":"Tabish Rashid, Mikayel Samvelyan, Christian Schroeder, Gregory Farquhar, Jakob Foerster, Shimon Whiteson","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rashid et al., 2020] Tabish Rashid, Gregory Farquhar, Bei Peng, and Shimon Whiteson. Weighted qmix: Expand- ing monotonic value function factorisation for deep multi- agent reinforcement learning. Advances in Neural Infor- mation Processing Systems, 33:10199-10210, 2020.","title":"Weighted qmix: Expanding monotonic value function factorisation for deep multiagent reinforcement learning","authors":"Tabish Rashid, Gregory Farquhar, Bei Peng, Shimon Whiteson","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rosenbaum et al., 2019] Clemens Rosenbaum, Ignacio Cases, Matthew Riemer, and Tim Klinger. Routing net- works and the challenges of modular and compositional computation. arXiv preprint arXiv:1904.12774, 2019.","title":"Routing networks and the challenges of modular and compositional computation","authors":"Clemens Rosenbaum, Ignacio Cases, Matthew Riemer, Tim Klinger","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Samvelyan et al., 2019] Mikayel Samvelyan, Tabish Rashid, Christian Schroeder De Witt, Gregory Farquhar, Nantas Nardelli, Tim GJ Rudner, Chia-Man Hung, Philip HS Torr, Jakob Foerster, and Shimon Whiteson. The starcraft multi-agent challenge. arXiv preprint arXiv:1902.04043, 2019.","title":"The starcraft multi-agent challenge","authors":"Tabish Mikayel Samvelyan, Christian Rashid, Schroeder De, Gregory Witt, Nantas Farquhar, Tim Gj Nardelli, Chia-Man Rudner, Hung, Jakob Philip Hs Torr, Shimon Foerster, Whiteson","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Son et al., 2019] Kyunghwan Son, Daewoo Kim, Wan Ju Kang, David Earl Hostallero, and Yung Yi. Qtran: Learn- ing to factorize with transformation for cooperative multi- agent reinforcement learning. In International Conference on Machine Learning, pages 5887-5896. PMLR, 2019.","title":"Qtran: Learning to factorize with transformation for cooperative multiagent reinforcement learning","authors":"Son","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sutton and Barto, 2018] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.","title":"Reinforcement learning: An introduction","authors":"S Richard, Andrew G Sutton, Barto","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Tan, 1993] Ming Tan. Multi-agent reinforcement learning: Independent vs. cooperative agents. In International Con- ference on Machine Learning, pages 330-337. PMLR, 1993.","title":"Multi-agent reinforcement learning: Independent vs. cooperative agents","authors":"Ming Tan","year":"1993","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Van der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Van Hasselt et al., 2016] Hado Van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with dou- ble q-learning. In Proceedings of the AAAI conference on artificial intelligence, volume 30, 2016.","title":"Deep reinforcement learning with double q-learning","authors":"Van Hasselt","year":"2016","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2021a] Jianhao Wang, Zhizhou Ren, Terry Liu, Yang Yu, and Chongjie Zhang. Qplex: Duplex dueling multi-agent q-learning. In International Conference on Learning Representations. OpenReview, 2021.","title":"Qplex: Duplex dueling multi-agent q-learning","authors":"Wang","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2021b] Tonghan Wang, Tarun Gupta, Anuj Mahajan, Bei Peng, Shimon Whiteson, and Chongjie Zhang. Rode: learning roles to decompose multi-agent tasks. In Proceedings of the International Conference on Learning Representations. OpenReview, 2021.","title":"Rode: learning roles to decompose multi-agent tasks","authors":"Wang","year":"2021","venue":"Proceedings of the International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wooldridge et al., 2000] Michael Wooldridge, Nicholas R Jennings, and David Kinny. The gaia methodology for agent-oriented analysis and design. Autonomous Agents and multi-agent systems, 3:285-312, 2000.","title":"The gaia methodology for agent-oriented analysis and design","authors":"Michael Wooldridge, Nicholas R Jennings, David Kinny","year":"2000","venue":"Autonomous Agents and multi-agent systems","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xu et al., 2023a] Pei Xu, Junge Zhang, and Kaiqi Huang. Exploration via joint policy diversity for sparse-reward multi-agent tasks. In Edith Elkind, editor, Proceedings of the Thirty-Second International Joint Conference on Arti- ficial Intelligence, IJCAI-23, pages 326-334. International Joint Conferences on Artificial Intelligence Organization, 8 2023. Main Track.","title":"Exploration via joint policy diversity for sparse-reward multi-agent tasks","authors":"Xu","year":"2023","venue":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI-23","doi":null,"url":null,"internal_url":"/proceedings/2023/37"},{"position":35,"raw":"[Xu et al., 2023b] Zhiwei Xu, Bin Zhang, Dapeng Li, Zeren Zhang, Guangchong Zhou, Hao Chen, and Guoliang Fan. Consensus learning for cooperative multi-agent reinforce- ment learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11726-11734, 2023.","title":"Consensus learning for cooperative multi-agent reinforcement learning","authors":"Xu","year":"2023","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Yang et al., 2020] Ruihan Yang, Huazhe Xu, Yi Wu, and Xiaolong Wang. Multi-task reinforcement learning with soft modularization. Advances in Neural Information Pro- cessing Systems, 33:4767-4777, 2020.","title":"Multi-task reinforcement learning with soft modularization","authors":"Yang","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Yang et al., 2022] Mingyu Yang, Jian Zhao, Xunhan Hu, Wengang Zhou, Jiangcheng Zhu, and Houqiang Li. Ldsa: Learning dynamic subtask assignment in cooperative multi-agent reinforcement learning. Advances in Neural Information Processing Systems, 35:1698-1710, 2022.","title":"Ldsa: Learning dynamic subtask assignment in cooperative multi-agent reinforcement learning","authors":"Mingyu Yang, Jian Zhao, Xunhan Hu, Wengang Zhou, Jiangcheng Zhu, Houqiang Li","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yu et al., 2021] Chao Yu, Akash Velu, Eugene Vinitsky, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effectiveness of mappo in cooperative, multi-agent games. arXiv preprint arXiv:2103.01955, 2021.","title":"The surprising effectiveness of mappo in cooperative, multi-agent games","authors":"Chao Yu, Akash Velu, Eugene Vinitsky, Yu Wang, Alexandre Bayen, Yi Wu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhang et al., 2021a] Kaiqing Zhang, Zhuoran Yang, and Tamer Bas ¸ar. Multi-agent reinforcement learning: A se- lective overview of theories and algorithms. Handbook of Reinforcement Learning and Control, pages 321-384, 2021.","title":"Multi-agent reinforcement learning: A selective overview of theories and algorithms","authors":"Zhang","year":"2021","venue":"Handbook of Reinforcement Learning and Control","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhang et al., 2021b] Tianhao Zhang, Yueheng Li, Chen Wang, Guangming Xie, and Zongqing Lu. Fop: Factoriz- ing optimal joint policy of maximum-entropy multi-agent reinforcement learning. In International Conference on Machine Learning, pages 12491-12500. PMLR, 2021.","title":"Fop: Factorizing optimal joint policy of maximum-entropy multi-agent reinforcement learning","authors":"Zhang","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zhou et al., 2020] Meng Zhou, Ziyu Liu, Pengwei Sui, Yix- uan Li, and Yuk Ying Chung. Learning implicit credit as- signment for cooperative multi-agent reinforcement learn- ing. Advances in neural information processing systems, 33:11853-11864, 2020.","title":"Learning implicit credit assignment for cooperative multi-agent reinforcement learning","authors":"Meng Zhou, Ziyu Liu, Pengwei Sui, Yixuan Li, Yuk Ying, Chung","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}