{"type":"ScholarlyArticle","title":"Dynamic Belief for Decentralized Multi-Agent Cooperative Learning","authors":[{"name":"Yunpeng Zhai","slug":"yunpeng-zhai","given":"Yunpeng","family":"Zhai","affiliation":"National Key Laboratory for Multimedia Information Processing , School of Computer Science , Peking University , Beijing , China"},{"name":"Peixi Peng","slug":"peixi-peng","given":"Peixi","family":"Peng","affiliation":"National Key Laboratory for Multimedia Information Processing , School of Computer Science , Peking University , Beijing , China; Peng Cheng Laboratory , Shenzhen , China"},{"name":"Chen Su","slug":"chen-su","given":"Chen","family":"Su","affiliation":"School of Electronic and Computer Engineering , Peking University Shenzhen Graduate School , Shenzhen , China"},{"name":"Yonghong Tian","slug":"yonghong-tian","given":"Yonghong","family":"Tian","affiliation":"National Key Laboratory for Multimedia Information Processing , School of Computer Science , Peking University , Beijing , China; Peng Cheng Laboratory , Shenzhen , China; School of Electronic and Computer Engineering , Peking University Shenzhen Graduate School , Shenzhen , China"}],"abstract":"Decentralized multi-agent cooperative learning is a practical task due to the partially observed setting both in training and execution. Every agent learns to cooperate without access to the observations and policies of others. However, the decentralized training of multi-agent is of great difficulty due to non-stationarity, especially when other agents' policies are also in learning during training. To overcome this, we propose to learn a dynamic policy belief for each agent to predict the current policies of other agents and accordingly condition the policy of its own. To quickly adapt to the development of others' policies, we introduce a historical context to learn the belief inference according to a few recent action histories of other agents and a latent variational inference to model their policies by a learned distribution. We evaluate our method on the StarCraft II micro management task (SMAC) and demonstrate its superior performance in the decentralized training settings and comparable results with the state-of-the-art CTDE methods.","doi":"10.24963/ijcai.2023/39","url":"https://ijcai.org/proceedings/2023/39","pdf":"https://ijcai.org/proceedings/2023/0039.pdf","markdown":"https://ijcai.org/proceedings/2023/39.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/39","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":39,"pages":{"first":344,"last":352},"keywords":["Agent-based and Multi-agent Systems: MAS: Coordination and cooperation","Agent-based and Multi-agent Systems: MAS: Multi-agent learning"],"videos":[],"references":[{"position":1,"raw":"[Abdallah and Kaisers, 2016] Sherief Abdallah and Michael Kaisers. Addressing environment non-stationarity by re- peating q-learning updates. JMLR, 17(1):1582-1612, 2016.","title":"Addressing environment non-stationarity by repeating q-learning updates","authors":"Kaisers ; ] Abdallah, Abdallah Sherief, Michael Kaisers","year":"2016","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Banino et al., 2021] Andrea Banino, Adrià Puidomenech Badia, Jacob Walker, Tim Scholtes, Jovana Mitrovic, and Charles Blundell. Coberl: Contrastive bert for reinforce- ment learning. arXiv preprint arXiv:2107.05431, 2021.","title":"Coberl: Contrastive bert for reinforcement learning","authors":"Andrea Banino, Adrià Puidomenech Badia, Jacob Walker, Tim Scholtes, Jovana Mitrovic, Charles Blundell","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bowling and Veloso, 2002] Michael Bowling and Manuela Veloso. Multiagent learning using a variable learning rate. Artificial Intelligence, 136(2):215-250, 2002.","title":"Multiagent learning using a variable learning rate","authors":"Michael Bowling, Manuela Veloso","year":"2002","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cao et al., 2012] Yongcan Cao, Wenwu Yu, Wei Ren, and Guanrong Chen. An overview of recent progress in the study of distributed multi-agent coordination. IEEE Trans- actions on Industrial Informatics, 9(1):427-438, 2012.","title":"An overview of recent progress in the study of distributed multi-agent coordination","authors":"Cao","year":"2012","venue":"IEEE Transactions on Industrial Informatics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Carroll et al., 2019] Micah Carroll, Rohin Shah, Mark K Ho, Tom Griffiths, Sanjit Seshia, Pieter Abbeel, and Anca Dragan. On the utility of learning about humans for human-ai coordination. NeurIPS, 32:5174-5185, 2019.","title":"On the utility of learning about humans for human-ai coordination","authors":"Micah Carroll, Rohin Shah, Mark K Ho, Tom Griffiths, Sanjit Seshia, Pieter Abbeel, Anca Dragan","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chao et al., 2021] YU Chao, A VELU, E VINITSKY, et al. The surprising effectiveness of ppo in cooperative, multi- agent games. arXiv:2103.01955, 2021.","title":"The surprising effectiveness of ppo in cooperative, multiagent games","authors":"A Yu Chao, E Velu, Vinitsky","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2017] Yu Fan Chen, Miao Liu, Michael Everett, and Jonathan P How. Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning. In ICRA, pages 285-292, 2017.","title":"Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning","authors":"Miao Yu Fan Chen, Michael Liu, Jonathan P Everett, How","year":"2017","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2021] Lili Chen, Kevin Lu, Aravind Ra- jeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence model- ing. NeurIPS, 34, 2021.","title":"Decision transformer: Reinforcement learning via sequence modeling","authors":"Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, Igor Mordatch","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[de Witt et al., 2020] Christian Schroeder de Witt, Tarun Gupta, Denys Makoviichuk, Viktor Makoviychuk, Philip HS Torr, Mingfei Sun, and Shimon Whiteson. Is independent learning all you need in the starcraft multi-agent challenge? arXiv:2011.09533, 2020.","title":"Christian Schroeder de Witt","authors":"De Witt","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Duan et al., 2017] Yan Duan, Marcin Andrychowicz, Bradly C Stadie, Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, and Wojciech Zaremba. One-shot imitation learning. arXiv:1703.07326, 2017.","title":"One-shot imitation learning","authors":"Yan Duan, Marcin Andrychowicz, Bradly C Stadie, Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, Wojciech Zaremba","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Foerster et al., 2018] Jakob N Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. Counterfactual multi-agent policy gradients. In AAAI, pages 2974-2982, 2018.","title":"Counterfactual multi-agent policy gradients","authors":"Jakob N Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, Shimon Whiteson","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Fu et al., 2022] Wei Fu, Chao Yu, Zelai Xu, Jiaqi Yang, and Yi Wu. Revisiting some common practices in cooperative multi-agent reinforcement learning. In ICML, pages 6863- 6877. PMLR, 2022.","title":"Revisiting some common practices in cooperative multi-agent reinforcement learning","authors":"Wei Fu, Chao Yu, Zelai Xu, Jiaqi Yang, Yi Wu","year":"2022","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2016] He He, Jordan Boyd-Graber, Kevin Kwok, and Hal Daumé III. Opponent modeling in deep reinforce- ment learning. In ICML, pages 1804-1813. PMLR, 2016.","title":"Opponent modeling in deep reinforcement learning","authors":"He","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hernandez-Leal et al., 2019] Pablo Hernandez-Leal, Bilal Kartal, and Matthew E Taylor. Agent modeling as aux- iliary task for deep reinforcement learning. In AIIDE, vol- ume 15, pages 31-37, 2019.","title":"Agent modeling as auxiliary task for deep reinforcement learning","authors":"Hernandez-Leal","year":"2019","venue":"AIIDE","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hong et al., 2017] Zhang-Wei Hong, Shih-Yang Su, Tzu- Yun Shann, Yi-Hsiang Chang, and Chun-Yi Lee. A deep policy inference q-network for multi-agent systems. arXiv:1712.07893, 2017.","title":"A deep policy inference q-network for multi-agent systems","authors":"Zhang-Wei Hong, Shih-Yang Su, Tzu-Yun Shann, Yi-Hsiang Chang, Chun-Yi Lee","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hu et al., 2020] Siyi Hu, Fengda Zhu, Xiaojun Chang, and Xiaodan Liang. Updet: Universal multi-agent rl via policy decoupling with transformers. In ICLR, 2020.","title":"Updet: Universal multi-agent rl via policy decoupling with transformers","authors":"Siyi Hu, Fengda Zhu, Xiaojun Chang, Xiaodan Liang","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hu et al., 2021] Jian Hu, Siyang Jiang, Seth Austin Hard- ing, Haibin Wu, and Shih-wei Liao. Rethinking the im- plementation tricks and monotonicity constraint in coop- erative multi-agent reinforcement learning. arXiv e-prints, pages arXiv-2102, 2021.","title":"Rethinking the implementation tricks and monotonicity constraint in cooperative multi-agent reinforcement learning","authors":"Jian Hu, Siyang Jiang, Seth Austin Harding, Haibin Wu, Shih-Wei Liao","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Inala et al., 2020] Jeevana Priya Inala, Yichen Yang, James Paulos, Yewen Pu, Osbert Bastani, Vijay Kumar, Mar- tin Rinard, and Armando Solar-Lezama. Neurosymbolic transformers for multi-agent communication. NeurIPS, 33:13597-13608, 2020.","title":"Neurosymbolic transformers for multi-agent communication","authors":"Jeevana Priya Inala, Yichen Yang, James Paulos, Yewen Pu, Osbert Bastani, Vijay Kumar, Martin Rinard, Armando Solar-Lezama","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Janner et al., 2021] Michael Janner, Qiyang Li, and Sergey Levine. Offline reinforcement learning as one big se- quence modeling problem. NeurIPS, 34, 2021.","title":"Offline reinforcement learning as one big sequence modeling problem","authors":"Michael Janner, Qiyang Li, Sergey Levine","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv:1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kraemer and Banerjee, 2016] Landon Kraemer and Bikramjit Banerjee. Multi-agent reinforcement learning as a rehearsal for decentralized planning. Neurocomput- ing, 190:82-94, 2016.","title":"Multi-agent reinforcement learning as a rehearsal for decentralized planning","authors":"Landon Kraemer, Bikramjit Banerjee","year":"2016","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lowe et al., 2017] Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environ- ments. arXiv:1706.02275, 2017.","title":"Multi-agent actor-critic for mixed cooperative-competitive environments","authors":"Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, Igor Mordatch","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lyu and Amato, 2018] Xueguang Lyu and Christopher Am- ato. Likelihood quantile networks for coordinating multi-agent reinforcement learning. arXiv preprint arXiv:1812.06319, 2018.","title":"Xueguang Lyu and Christopher Amato. Likelihood quantile networks for coordinating multi-agent reinforcement learning","authors":"Amato Lyu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ma et al., 2021a] Xiaoteng Ma, Yiqin Yang, Chenghao Li, Yiwen Lu, Qianchuan Zhao, and Jun Yang. Modeling the interaction between agents in cooperative multi-agent re- inforcement learning. In AAMAS, pages 853-861, 2021.","title":"Modeling the interaction between agents in cooperative multi-agent reinforcement learning","authors":"Ma","year":"2021","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Ma et al., 2021b] Yuxi Ma, Meng Shen, Yuhang Zhao, Zhao Li, Xiaoyao Tong, Quanxin Zhang, and Zhi Wang. Opponent portrait for multiagent reinforcement learning in competitive environment. International Journal of Intelli- gent Systems, 36(12):7461-7474, 2021.","title":"Opponent portrait for multiagent reinforcement learning in competitive environment","authors":"Ma","year":"2021","venue":"International Journal of Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Meng et al., 2021] Linghui Meng, Muning Wen, Yaodong Yang, Chenyang Le, Xiyun Li, Weinan Zhang, Ying Wen, Haifeng Zhang, Jun Wang, and Bo Xu. Offline pre- trained multi-agent decision transformer: One big se- quence model conquers all starcraftii tasks. arXiv preprint arXiv:2112.02845, 2021.","title":"Offline pretrained multi-agent decision transformer: One big sequence model conquers all starcraftii tasks","authors":"Linghui Meng, Muning Wen, Yaodong Yang, Chenyang Le, Xiyun Li, Weinan Zhang, Ying Wen, Haifeng Zhang, Jun Wang, Bo Xu","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Moreno et al., 2021] Pol Moreno, Edward Hughes, Kevin R McKee, Bernardo Avila Pires, and Théophane Weber. Neural recursive belief states in multi-agent reinforcement learning. arXiv preprint arXiv:2102.02274, 2021.","title":"Neural recursive belief states in multi-agent reinforcement learning","authors":"Pol Moreno, Edward Hughes, Kevin R Mckee, Bernardo Avila Pires, Théophane Weber","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Oliehoek and Amato, 2016] Frans Oliehoek and Christo- pher Amato. A concise introduction to decentralized POMDPs. Springer, 2016.","title":"Frans Oliehoek and Christopher Amato. A concise introduction to decentralized POMDPs","authors":"Amato Oliehoek","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Palmer et al., 2018] Gregory Palmer, Rahul Savani, and Karl Tuyls. Negative update intervals in deep multi-agent reinforcement learning. arXiv preprint arXiv:1809.05096, 2018.","title":"Negative update intervals in deep multi-agent reinforcement learning","authors":"Gregory Palmer, Rahul Savani, Karl Tuyls","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Papoudakis and Albrecht, 2020] Georgios Papoudakis and Stefano V Albrecht. Variational autoencoders for oppo- nent modeling in multi-agent systems. arXiv:2001.10829, 2020.","title":"Variational autoencoders for opponent modeling in multi-agent systems","authors":"Georgios Papoudakis, Stefano V Albrecht","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Parisotto et al., 2020] Emilio Parisotto, Francis Song, Jack Rae, Razvan Pascanu, Caglar Gulcehre, Siddhant Jayaku- mar, Max Jaderberg, Raphael Lopez Kaufman, Aidan Clark, Seb Noury, et al. Stabilizing transformers for re- inforcement learning. In ICML, pages 7487-7498. PMLR, 2020.","title":"Stabilizing transformers for reinforcement learning","authors":"Emilio Parisotto, Francis Song, Jack Rae, Razvan Pascanu, Caglar Gulcehre, Siddhant Jayakumar, Max Jaderberg, Raphael Lopez Kaufman, Aidan Clark, Seb Noury","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Pu et al., 2016] Yunchen Pu, Zhe Gan, Ricardo Henao, Xin Yuan, Chunyuan Li, Andrew Stevens, and Lawrence Carin. Variational autoencoder for deep learning of im- ages, labels and captions. NeurIPS, 29:2352-2360, 2016.","title":"Variational autoencoder for deep learning of images, labels and captions","authors":"Yunchen Pu, Zhe Gan, Ricardo Henao, Xin Yuan, Chunyuan Li, Andrew Stevens, Lawrence Carin","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Rabinowitz et al., 2018] Neil Rabinowitz, Frank Perbet, Francis Song, Chiyuan Zhang, SM Ali Eslami, and Matthew Botvinick. Machine theory of mind. In ICML, pages 4218-4227. PMLR, 2018.","title":"Machine theory of mind","authors":"Neil Rabinowitz, Frank Perbet, Francis Song, Chiyuan Zhang, S M Ali Eslami, Matthew Botvinick","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Rashid et al., 2018] Tabish Rashid, Mikayel Samvelyan, Christian Schroeder, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning. In ICML, pages 4292-4301, 2018.","title":"QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning","authors":"Tabish Rashid, Mikayel Samvelyan, Christian Schroeder, Gregory Farquhar, Jakob Foerster, Shimon Whiteson","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Samvelyan et al., 2019] Mikayel Samvelyan, Tabish Rashid, Christian Schroeder De Witt, Gregory Farquhar, Nantas Nardelli, Tim GJ Rudner, Chia-Man Hung, Philip HS Torr, Jakob Foerster, and Shimon Whiteson. The starcraft multi-agent challenge. arXiv:1902.04043, 2019.","title":"The starcraft multi-agent challenge","authors":"Tabish Mikayel Samvelyan, Christian Rashid, Schroeder De, Gregory Witt, Nantas Farquhar, Tim Gj Nardelli, Chia-Man Rudner, Hung, Jakob Philip Hs Torr, Shimon Foerster, Whiteson","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.","title":"Proximal policy optimization algorithms","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Son et al., 2019] Kyunghwan Son, Daewoo Kim, Wan Ju Kang, David Earl Hostallero, and Yung Yi. Qtran: Learn- ing to factorize with transformation for cooperative multi- agent reinforcement learning. In ICML, pages 5887-5896.","title":"Qtran: Learning to factorize with transformation for cooperative multiagent reinforcement learning","authors":"Son","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[PMLR, 2019. Sukhbaatar et al., 2016] Sainbayar Sukhbaatar, Rob Fergus, et al. Learning multiagent communication with backprop- agation. NeurIPS, 29:2244-2252, 2016.","title":"Learning multiagent communication with backpropagation","authors":"Pmlr ; Sukhbaatar","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Sun et al., 2022] Mingfei Sun, Sam Devlin, Katja Hof- mann, and Shimon Whiteson. Monotonic improvement guarantees under non-stationarity for decentralized ppo. arXiv preprint arXiv:2202.00082, 2022.","title":"Monotonic improvement guarantees under non-stationarity for decentralized ppo","authors":"Sun","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Sunehag et al., 2017] Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Marc Lanctot, Nicolas Sonnerat, Joel Z Leibo, Karl Tuyls, et al. Value-decomposition networks for cooperative multi-agent learning. arXiv:1706.05296, 2017.","title":"Value-decomposition networks for cooperative multi-agent learning","authors":"Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojciech, Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Marc Lanctot, Nicolas Sonnerat, Joel Z Leibo, Karl Tuyls","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Tan, 1993] Ming Tan. Multi-agent reinforcement learning: Independent vs. cooperative agents. In ICML, pages 330- 337, 1993.","title":"Multi-agent reinforcement learning: Independent vs. cooperative agents","authors":"Ming Tan","year":"1993","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurIPS, 30, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Wang et al., 2020a] Jianhao Wang, Zhizhou Ren, Terry Liu, Yang Yu, and Chongjie Zhang. Qplex: Duplex dueling multi-agent q-learning. arXiv:2008.01062, 2020.","title":"Qplex: Duplex dueling multi-agent q-learning","authors":"Wang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wang et al., 2020b] Tonghan Wang, Tarun Gupta, Anuj Mahajan, Bei Peng, Shimon Whiteson, and Chongjie Zhang. Rode: Learning roles to decompose multi-agent tasks. arXiv:2010.01523, 2020.","title":"Rode: Learning roles to decompose multi-agent tasks","authors":"Wang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Wen et al., 2019] Ying Wen, Yaodong Yang, Rui Luo, Jun Wang, and Wei Pan. Probabilistic recursive reasoning for multi-agent reinforcement learning. arXiv preprint arXiv:1901.09207, 2019.","title":"Probabilistic recursive reasoning for multi-agent reinforcement learning","authors":"Ying Wen, Yaodong Yang, Rui Luo, Jun Wang, Wei Pan","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Wen et al., 2021] Ying Wen, Yaodong Yang, and Jun Wang. Modelling bounded rationality in multi-agent interactions by generalized recursive reasoning. In IJCAI, pages 414- 421, 2021.","title":"Modelling bounded rationality in multi-agent interactions by generalized recursive reasoning","authors":"Ying Wen, Yaodong Yang, Jun Wang","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/58"},{"position":47,"raw":"[Yang et al., 2019] Tianpei Yang, Jianye Hao, Zhaopeng Meng, Chongjie Zhang, Yan Zheng, and Ze Zheng. To- wards efficient detection and optimal response against so- phisticated opponents. In IJCAI, 2019.","title":"Towards efficient detection and optimal response against sophisticated opponents","authors":"Yang","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/88"},{"position":48,"raw":"[Zintgraf et al., 2021] Luisa Zintgraf, Sam Devlin, Kamil Ciosek, Shimon Whiteson, and Katja Hofmann. Deep interactive bayesian reinforcement learning via meta- learning. arXiv:2101.03864, 2021.","title":"Deep interactive bayesian reinforcement learning via metalearning","authors":"Luisa Zintgraf, Sam Devlin, Kamil Ciosek, Shimon Whiteson, Katja Hofmann","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}