{"type":"ScholarlyArticle","title":"Deep Hierarchical Communication Graph in Multi-Agent Reinforcement Learning","authors":[{"name":"Zeyang Liu","slug":"zeyang-liu","given":"Zeyang","family":"Liu","affiliation":"National Key Laboratory of Human-Machine Hybrid Augmented Intelligence , National Engineering Research Center for Visual Information and Applications , and Institute of Artificial Intelligence and Robotics , Xi'an Jiaotong University"},{"name":"Lipeng Wan","slug":"lipeng-wan","given":"Lipeng","family":"Wan","affiliation":"National Key Laboratory of Human-Machine Hybrid Augmented Intelligence , National Engineering Research Center for Visual Information and Applications , and Institute of Artificial Intelligence and Robotics , Xi'an Jiaotong University"},{"name":"Xue Sui","slug":"xue-sui","given":"Xue","family":"Sui","affiliation":"National Key Laboratory of Human-Machine Hybrid Augmented Intelligence , National Engineering Research Center for Visual Information and Applications , and Institute of Artificial Intelligence and Robotics , Xi'an Jiaotong University"},{"name":"Zhuoran Chen","slug":"zhuoran-chen","given":"Zhuoran","family":"Chen","affiliation":"National Key Laboratory of Human-Machine Hybrid Augmented Intelligence , National Engineering Research Center for Visual Information and Applications , and Institute of Artificial Intelligence and Robotics , Xi'an Jiaotong University"},{"name":"Kewu Sun","slug":"kewu-sun","given":"Kewu","family":"Sun","affiliation":"Intelligent Science & Technology Academy"},{"name":"Xuguang Lan","slug":"xuguang-lan","given":"Xuguang","family":"Lan","affiliation":"National Key Laboratory of Human-Machine Hybrid Augmented Intelligence , National Engineering Research Center for Visual Information and Applications , and Institute of Artificial Intelligence and Robotics , Xi'an Jiaotong University"}],"abstract":"Sharing intentions is crucial for efficient cooperation in communication-enabled multi-agent reinforcement learning. Recent work applies static or undirected graphs to determine the order of interaction. However, the static graph is not general for complex cooperative tasks, and the parallel message-passing update in the undirected graph with cycles cannot guarantee convergence. To solve this problem, we propose Deep Hierarchical Communication Graph (DHCG) to learn the dependency relationships between agents based on their messages. The relationships are formulated as directed acyclic graphs (DAGs), where the selection of the proper topology is viewed as an action and trained in an end-to-end fashion. To eliminate the cycles in the graph, we apply an acyclicity constraint as intrinsic rewards and then project the graph in the admissible solution set of DAGs. As a result, DHCG removes redundant communication edges for cost improvement and guarantees convergence. To show the effectiveness of the learned graphs, we propose policy-based and value-based DHCG. Policy-based DHCG factorizes the joint policy in an auto-regressive manner, and value-based DHCG factorizes the joint value function to individual value functions and pairwise payoff functions. Empirical results show that our method improves performance across various cooperative multi-agent tasks, including Predator-Prey, Multi-Agent Coordination Challenge, and StarCraft Multi-Agent Challenge.","doi":"10.24963/ijcai.2023/24","url":"https://ijcai.org/proceedings/2023/24","pdf":"https://ijcai.org/proceedings/2023/0024.pdf","markdown":"https://ijcai.org/proceedings/2023/24.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/24","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":24,"pages":{"first":208,"last":216},"keywords":["Agent-based and Multi-agent Systems: MAS: Agent communication","Agent-based and Multi-agent Systems: MAS: Coordination and cooperation","Agent-based and Multi-agent Systems: MAS: Multi-agent learning"],"videos":[],"references":[{"position":1,"raw":"[Bargiacchi et al., 2018] Eugenio Bargiacchi, Timothy Ver- straeten, Diederik Roijers, Ann Nowé, and Hado Hasselt. Learning to coordinate with coordination graphs in re- peated single-stage multi-agent decision problems. In In- ternational Conference on Machine Learning, pages 482- 490. PMLR, 2018.","title":"Learning to coordinate with coordination graphs in repeated single-stage multi-agent decision problems","authors":"Eugenio Bargiacchi, Timothy Verstraeten, Diederik Roijers, Ann Nowé, Hado Hasselt","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bertsekas, 2019] Dimitri P. Bertsekas. Multiagent roll- out algorithms and reinforcement learning. CoRR, abs/1910.00120, 2019.","title":"Multiagent rollout algorithms and reinforcement learning","authors":"Dimitri P Bertsekas","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Böhmer et al., 2020] Wendelin Böhmer, Vitaly Kurin, and Shimon Whiteson. Deep coordination graphs. In Interna- tional Conference on Machine Learning, pages 980-991.","title":"Deep coordination graphs","authors":"Wendelin Böhmer, Shimon Vitaly Kurin, Whiteson","year":null,"venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"PMLR, 2020.","title":null,"authors":"Pmlr","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Conte and Sichman, 2002] Rosaria Conte and Jaime Simão Sichman. Dependence graphs: Dependence within and between groups. Computational & Mathematical Organi- zation Theory, 8(2):87-112, 2002.","title":"Dependence graphs: Dependence within and between groups","authors":"Rosaria Conte, Jaime Simão Sichman","year":"2002","venue":"Computational & Mathematical Organization Theory","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Das et al., 2019] Abhishek Das, Théophile Gervet, Joshua Romoff, Dhruv Batra, Devi Parikh, Mike Rabbat, and Joelle Pineau. Tarmac: Targeted multi-agent communi- cation. In International Conference on Machine Learning, pages 1538-1546. PMLR, 2019.","title":"Tarmac: Targeted multi-agent communication","authors":"Abhishek Das, Théophile Gervet, Joshua Romoff, Dhruv Batra, Devi Parikh, Mike Rabbat, Joelle Pineau","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dimakopoulou and Van Roy, 2018] Maria Dimakopoulou and Benjamin Van Roy. Coordinated exploration in concurrent reinforcement learning. In International Conference on Machine Learning, pages 1271-1279.","title":"Maria Dimakopoulou and Benjamin Van Roy. Coordinated exploration in concurrent reinforcement learning","authors":"Van Dimakopoulou, Roy","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"PMLR, 2018.","title":null,"authors":"Pmlr","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dimakopoulou et al., 2018] Maria Dimakopoulou, Ian Os- band, and Benjamin Van Roy. Scalable coordinated ex- ploration in concurrent reinforcement learning. Advances in Neural Information Processing Systems, 31:4223-4232, 2018.","title":"Scalable coordinated exploration in concurrent reinforcement learning","authors":"Maria Dimakopoulou, Ian Osband, Benjamin Van, Roy","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fu et al., 2022] Wei Fu, Chao Yu, Zelai Xu, Jiaqi Yang, and Yi Wu. Revisiting some common practices in cooperative multi-agent reinforcement learning. In International Con- ference on Machine Learning, pages 6863-6877. PMLR, 2022.","title":"Revisiting some common practices in cooperative multi-agent reinforcement learning","authors":"Wei Fu, Chao Yu, Zelai Xu, Jiaqi Yang, Yi Wu","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guestrin et al., 2002] Carlos Guestrin, Michail Lagoudakis, and Ronald Parr. Coordinated reinforcement learning. In International Conference on Machine Learning, pages 227-234. Citeseer, 2002.","title":"Coordinated reinforcement learning","authors":"Guestrin","year":"2002","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hernandez-Leal et al., 2019] Pablo Hernandez-Leal, Bilal Kartal, and Matthew E Taylor. A survey and critique of multiagent deep reinforcement learning. In International Conference on Autonomous Agents and Multi-Agent Sys- tems, pages 750-797, 2019.","title":"A survey and critique of multiagent deep reinforcement learning","authors":"Hernandez-Leal","year":"2019","venue":"International Conference on Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jiang and Lu, 2018] Jiechuan Jiang and Zongqing Lu. Learning attentional communication for multi-agent coop- eration. Advances in Neural Information Processing Sys- tems, 31:7254-7264, 2018.","title":"Learning attentional communication for multi-agent cooperation","authors":"Lu ; Jiang, Jiechuan Jiang, Zongqing Lu","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kim et al., 2021] Woojun Kim, Jongeui Park, and Youngchul Sung. Communication in multi-agent re- inforcement learning: Intention sharing. In International Conference on Learning Representations, 2021.","title":"Communication in multi-agent reinforcement learning: Intention sharing","authors":"Kim","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kuba et al., 2022] Jakub Grudzien Kuba, Ruiqing Chen, Muning Wen, Ying Wen, Fanglei Sun, Jun Wang, and Yaodong Yang. Trust region policy optimisation in multi- agent reinforcement learning. In International Conference on Learning Representations, 2022.","title":"Trust region policy optimisation in multiagent reinforcement learning","authors":"Ruiqing Jakub Grudzien Kuba, Muning Chen, Ying Wen, Fanglei Wen, Jun Sun, Yaodong Wang, Yang","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2019] Xihan Li, Jia Zhang, Jiang Bian, Yunhai Tong, and Tie-Yan Liu. A cooperative multi-agent rein- forcement learning framework for resource balancing in complex logistics network. In International Conference on Autonomous Agents and Multi-Agent Systems, pages 980- 988, 2019.","title":"A cooperative multi-agent reinforcement learning framework for resource balancing in complex logistics network","authors":"Xihan Li, Jia Zhang, Jiang Bian, Yunhai Tong, Tie-Yan Liu","year":"2019","venue":"International Conference on Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li et al., 2021] Sheng Li, Jayesh K Gupta, Peter Morales, Ross Allen, and Mykel J Kochenderfer. Deep implicit co- ordination graphs for multi-agent reinforcement learning. In International Conference on Autonomous Agents and Multi-Agent Systems, pages 764-772, 2021.","title":"Deep implicit coordination graphs for multi-agent reinforcement learning","authors":"Sheng Li, K Jayesh, Peter Gupta, Ross Morales, Allen, Mykel, Kochenderfer","year":"2021","venue":"International Conference on Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lillicrap et al., 2016] Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. In International Conference on Learning Representations, 2016.","title":"Continuous control with deep reinforcement learning","authors":"Jonathan J Timothy P Lillicrap, Alexander Hunt, Nicolas Pritzel, Tom Heess, Yuval Erez, David Tassa, Daan Silver, Wierstra","year":"2016","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2020] Yong Liu, Weixun Wang, Yujing Hu, Jianye Hao, Xingguo Chen, and Yang Gao. Multi-agent game abstraction via graph attention neural network. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 34(05):7211-7218, 2020.","title":"Multi-agent game abstraction via graph attention neural network","authors":"Yong Liu, Weixun Wang, Yujing Hu, Jianye Hao, Xingguo Chen, Yang Gao","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Oliehoek and Amato, 2016] Frans A. Oliehoek and Christo- pher Amato. A Concise Introduction to Decentral- ized POMDPs. Springer Briefs in Intelligent Systems. Springer, 2016.","title":"A Concise Introduction to Decentralized POMDPs","authors":"Frans A Oliehoek, Christopher Amato","year":"2016","venue":"Springer Briefs in Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Palanisamy, 2020] Praveen Palanisamy. Multi-agent con- nected autonomous driving using deep reinforcement learning. In International Joint Conference on Neural Net- works, pages 1-7. IEEE, 2020.","title":"Multi-agent connected autonomous driving using deep reinforcement learning","authors":"Praveen Palanisamy","year":"2020","venue":"International Joint Conference on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pearl, 1989] Judea Pearl. Probabilistic reasoning in intel- ligent systems -networks of plausible inference. Morgan Kaufmann series in representation and reasoning. Morgan Kaufmann, 1989.","title":"Judea Pearl. Probabilistic reasoning in intelligent systems -networks of plausible inference","authors":"Pearl","year":"1989","venue":"Morgan Kaufmann series in representation and reasoning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rashid et al., 2018] Tabish Rashid, Mikayel Samvelyan, Christian Schroeder, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning. In International Conference on Machine Learning, pages 4295-4304. PMLR, 2018.","title":"Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning","authors":"Tabish Rashid, Mikayel Samvelyan, Christian Schroeder, Gregory Farquhar, Jakob Foerster, Shimon Whiteson","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rashid et al., 2020] Tabish Rashid, Gregory Farquhar, Bei Peng, and Shimon Whiteson. Weighted qmix: Expand- ing monotonic value function factorisation for deep multi- agent reinforcement learning. Advances in Neural Infor- mation Processing Systems, 33:10199-10210, 2020.","title":"Weighted qmix: Expanding monotonic value function factorisation for deep multiagent reinforcement learning","authors":"Tabish Rashid, Gregory Farquhar, Bei Peng, Shimon Whiteson","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Samvelyan et al., 2019] Mikayel Samvelyan, Tabish Rashid, Christian Schröder de Witt, Gregory Farquhar, Nantas Nardelli, Tim GJ Rudner, Chia-Man Hung, Philip HS Torr, Jakob N Foerster, and Shimon Whiteson. The starcraft multi-agent challenge. In International Conference on Autonomous Agents and Multi-Agent Systems, pages 2186-2188, 2019.","title":"The starcraft multi-agent challenge","authors":"Tabish Mikayel Samvelyan, Christian Rashid, Gregory Schröder De Witt, Nantas Farquhar, Tim Gj Nardelli, Chia-Man Rudner, Hung, Jakob N Philip Hs Torr, Shimon Foerster, Whiteson","year":"2019","venue":"International Conference on Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Schulman et al., 2016] John Schulman, Philipp Moritz, Sergey Levine, Michael I. Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation. In International Conference on Learning Representations, 2016.","title":"High-dimensional continuous control using generalized advantage estimation","authors":"John Schulman, Philipp Moritz, Sergey Levine, Michael I Jordan, Pieter Abbeel","year":"2016","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. CoRR, abs/1707.06347, 2017.","title":"Proximal policy optimization algorithms","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Shi et al., 2019] Zhenyu Shi, Runsheng Yu, Xinrun Wang, Rundong Wang, Youzhi Zhang, Hanjiang Lai, and Bo An. Learning expensive coordination: An event-based deep rl approach. In International Conference on Learning Rep- resentations, 2019.","title":"Learning expensive coordination: An event-based deep rl approach","authors":"Zhenyu Shi, Runsheng Yu, Xinrun Wang, Rundong Wang, Youzhi Zhang, Hanjiang Lai, Bo An","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Son et al., 2019] Kyunghwan Son, Daewoo Kim, Wan Ju Kang, David Earl Hostallero, and Yung Yi. Qtran: Learn- ing to factorize with transformation for cooperative multi- agent reinforcement learning. In International Conference on Machine Learning, pages 5887-5896. PMLR, 2019.","title":"Qtran: Learning to factorize with transformation for cooperative multiagent reinforcement learning","authors":"Son","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Sukhbaatar et al., 2016] Sainbayar Sukhbaatar, Rob Fergus, et al. Learning multiagent communication with backprop- agation. Advances in Neural Information Processing Sys- tems, 29, 2016.","title":"Learning multiagent communication with backpropagation","authors":"Sukhbaatar","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wainwright et al., 2002] Martin J Wainwright, Tommi Jaakkola, and Alan Willsky. Exact map estimates by (hy- per) tree agreement. Advances in Neural Information Pro- cessing Systems, 15, 2002.","title":"Exact map estimates by (hyper) tree agreement","authors":"J Martin, Tommi Wainwright, Alan Jaakkola, Willsky","year":"2002","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wainwright et al., 2004] Martin Wainwright, Tommi Jaakkola, and Alan Willsky. Tree consistency and bounds on the performance of the max-product algorithm and its generalizations. Statistics and computing, 14(2):143-166, 2004.","title":"Tree consistency and bounds on the performance of the max-product algorithm and its generalizations","authors":"Martin Wainwright, Tommi Jaakkola, Alan Willsky","year":"2004","venue":"Statistics and computing","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wang et al., 2019] Tonghan Wang, Jianhao Wang, Chongyi Zheng, and Chongjie Zhang. Learning nearly decom- posable value functions via communication minimization. In International Conference on Learning Representations, 2019.","title":"Learning nearly decomposable value functions via communication minimization","authors":"Wang","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wang et al., 2020] Jianhao Wang, Zhizhou Ren, Terry Liu, Yang Yu, and Chongjie Zhang. Qplex: Duplex dueling multi-agent q-learning. In International Conference on Learning Representations, 2020.","title":"Qplex: Duplex dueling multi-agent q-learning","authors":"Wang","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wang et al., 2022] Tonghan Wang, Liang Zeng, Weijun Dong, Qianlan Yang, Yang Yu, and Chongjie Zhang. Context-aware sparse deep coordination graphs. In Inter- national Conference on Learning Representations, 2022.","title":"Context-aware sparse deep coordination graphs","authors":"Tonghan Wang, Liang Zeng, Weijun Dong, Qianlan Yang, Yang Yu, Chongjie Zhang","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wen et al., 2022] Muning Wen, Jakub Kuba, Runji Lin, Weinan Zhang, Ying Wen, Jun Wang, and Yaodong Yang. Multi-agent reinforcement learning is a sequence model- ing problem. Advances in Neural Information Processing Systems, 35:16509-16521, 2022.","title":"Multi-agent reinforcement learning is a sequence modeling problem","authors":"Wen","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Yang et al., 2022] Qianlan Yang, Weijun Dong, Zhizhou Ren, Jianhao Wang, Tonghan Wang, and Chongjie Zhang. Self-organized polynomial-time coordination graphs. In International Conference on Machine Learning, pages 24963-24979. PMLR, 2022.","title":"Self-organized polynomial-time coordination graphs","authors":"Qianlan Yang, Weijun Dong, Zhizhou Ren, Jianhao Wang, Tonghan Wang, Chongjie Zhang","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yu et al., 2021] Yue Yu, Tian Gao, Naiyu Yin, and Qiang Ji. Dags with no curl: An efficient dag structure learning approach. In International Conference on Machine Learn- ing, pages 12156-12166. PMLR, 2021.","title":"Dags with no curl: An efficient dag structure learning approach","authors":"Yue Yu, Tian Gao, Naiyu Yin, Qiang Ji","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Yu et al., 2022] Chao Yu, Akash Velu, Eugene Vinitsky, Ji- axuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Sys- tems, 35:24611-24624, 2022.","title":"The surprising effectiveness of ppo in cooperative multi-agent games","authors":"Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, Yi Wu","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Yuan et al., 2022] Lei Yuan, Jianhao Wang, Fuxiang Zhang, Chenghe Wang, Zongzhang Zhang, Yang Yu, and Chongjie Zhang. Multi-agent incentive communication via decentralized teammate modeling. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 36(9):9466-9474, 2022.","title":"Multi-agent incentive communication via decentralized teammate modeling","authors":"Lei Yuan, Jianhao Wang, Fuxiang Zhang, Chenghe Wang, Zongzhang Zhang, Yang Yu, Chongjie Zhang","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zhang and Lesser, 2013] Chongjie Zhang and Victor Lesser. Coordinating multi-agent reinforcement learning with limited communication. In International Conference on Autonomous Agents and Multi-Agent Systems, pages 1101-1108, 2013.","title":"Chongjie Zhang and Victor Lesser. Coordinating multi-agent reinforcement learning with limited communication","authors":"Lesser Zhang","year":"2013","venue":"International Conference on Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhang et al., 2019a] Huichu Zhang, Siyuan Feng, Chang Liu, Yaoyao Ding, Yichen Zhu, Zihan Zhou, Weinan Zhang, Yong Yu, Haiming Jin, and Zhenhui Li. Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario. In The world wide web Conference, pages 3620-3624, 2019.","title":"Cityflow: A multi-agent reinforcement learning environment for large scale city traffic scenario","authors":"Zhang","year":"2019","venue":"The world wide web Conference","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhang et al., 2019b] Sai Qian Zhang, Qi Zhang, and Jieyu Lin. Efficient communication in multi-agent reinforce- ment learning via variance based control. Advances in Neural Information Processing Systems, 32:3230-3239, 2019.","title":"Efficient communication in multi-agent reinforcement learning via variance based control","authors":"Zhang","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zheng et al., 2018] Xun Zheng, Bryon Aragam, Pradeep K Ravikumar, and Eric P Xing. Dags with no tears: Continu- ous optimization for structure learning. Advances in Neu- ral Information Processing Systems, 31:9492-9503, 2018.","title":"Dags with no tears: Continuous optimization for structure learning","authors":"Xun Zheng, Bryon Aragam, Eric P Pradeep K Ravikumar, Xing","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}