{"type":"ScholarlyArticle","title":"CompetEvo: Towards Morphological Evolution from Competition","authors":[{"name":"Kangyao Huang","slug":"kangyao-huang","given":"Kangyao","family":"Huang","affiliation":"Department of Computer Science and Technology , Tsinghua University"},{"name":"Di Guo","slug":"di-guo","given":"Di","family":"Guo","affiliation":"School of Artificial Intelligence , Beijing University of Posts and Telecommunications"},{"name":"Xinyu Zhang","slug":"xinyu-zhang","given":"Xinyu","family":"Zhang","affiliation":"School of Vehicle and Mobility , Tsinghua University"},{"name":"Xiangyang Ji","slug":"xiangyang-ji","given":"Xiangyang","family":"Ji","affiliation":"Department of Automation , Tsinghua University"},{"name":"Huaping Liu","slug":"huaping-liu","given":"Huaping","family":"Liu","affiliation":"Department of Computer Science and Technology , Tsinghua University"}],"abstract":"Training an agent to adapt to specific tasks through co-optimization of morphology and control has widely attracted attention. However, whether there exists an optimal configuration and tactics for agents in a multiagent competition scenario is still an issue that is challenging to definitively conclude. In this context, we propose competitive evolution (CompetEvo), which co-evolves agents' designs and tactics in confrontation. We build arenas consisting of three animals and their evolved derivatives, placing agents with different morphologies in direct competition with each other. The results reveal that our method enables agents to evolve a more suitable design and strategy for fighting compared to fixed-morph agents, allowing them to obtain advantages in combat scenarios. Moreover, we demonstrate the amazing and impressive behaviors that emerge when confrontations are conducted under asymmetrical morphs.","doi":"10.24963/ijcai.2024/10","url":"https://ijcai.org/proceedings/2024/10","pdf":"https://ijcai.org/proceedings/2024/0010.pdf","markdown":"https://ijcai.org/proceedings/2024/10.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/10","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":10,"pages":{"first":85,"last":93},"keywords":["Agent-based and Multi-agent Systems: MAS: Agent-based simulation and emergence","Robotics: ROB: Learning in robotics","Search: S: Evolutionary computation"],"videos":[],"references":[{"position":1,"raw":"[Al-Shedivat et al., 2018] Maruan Al-Shedivat, Trapit Bansal, Yura Burda, Ilya Sutskever, Igor Mordatch, and Pieter Abbeel. Continuous adaptation via meta-learning in nonstationary and competitive environments. 6th Inter- national Conference on Learning Representations, ICLR 2018 -Conference Track Proceedings, (March):1-21, 2018.","title":"Continuous adaptation via meta-learning in nonstationary and competitive environments","authors":"Al-Shedivat","year":"2018","venue":"6th International Conference on Learning Representations, ICLR 2018 -Conference Track Proceedings","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bansal et al., 2018] Trapit Bansal, Jakub Pachocki, Szymon Sidor, Ilya Sutskever, and Igor Mordatch. Emergent com- plexity via multi-agent competition. 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings, 2018.","title":"Emergent complexity via multi-agent competition","authors":"Trapit Bansal, Jakub Pachocki, Szymon Sidor, Ilya Sutskever, Igor Mordatch","year":"2018","venue":"6th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2023] Yishuai Cai, Shaowu Yang, Minglong Li, Xinglin Chen, Yunxin Mao, Xiaodong Yi, and Wenjing Yang. Task2Morph: Differentiable Task-Inspired Frame- work for Contact-Aware Robot Design. In IEEE/RSJ In- ternational Conference on Intelligent Robots and Systems (IROS), pages 452-459, 2023.","title":"Task2Morph: Differentiable Task-Inspired Framework for Contact-Aware Robot Design","authors":"Yishuai Cai, Shaowu Yang, Minglong Li, Xinglin Chen, Yunxin Mao, Xiaodong Yi, Wenjing Yang","year":"2023","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2023a] Xinglin Chen, Da Huang, Minglong Li, Yishuai Cai, Zhuoer Wen, Zhongxuan Cai, and Wenjing Yang. Evolving Physical Instinct for Morphology and Control Co-Adaption. In IEEE/RSJ International Con- ference on Intelligent Robots and Systems (IROS), pages 6616-6623, 2023.","title":"Evolving Physical Instinct for Morphology and Control Co-Adaption","authors":"Chen","year":"2023","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2023b] Yang Chen, Yu Luo, and Fuchun Sun. Cooperative distributed model predictive control for robot in-hand manipulation. Robotic Intelligence and Automa- tion, 43(1):65-74, 2023.","title":"Cooperative distributed model predictive control for robot in-hand manipulation","authors":"Chen","year":"2023","venue":"Robotic Intelligence and Automation","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gleave et al., 2020] Adam Gleave, Michael Dennis, Cody Wild, Neel Kant, Sergey Levine, and Stuart Russell. Ad- versarial Policies: Attacking Deep Reinforcement Learn- ing. 8th International Conference on Learning Represen- tations, ICLR 2020, 2020.","title":"Adversarial Policies: Attacking Deep Reinforcement Learning","authors":"Adam Gleave, Michael Dennis, Cody Wild, Neel Kant, Sergey Levine, Stuart Russell","year":"2020","venue":"8th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gupta et al., 2021] Agrim Gupta, Silvio Savarese, Surya Ganguli, and Li Fei-Fei. Embodied intelligence via learn- ing and evolution. Nature Communications, 12(1), 2021.","title":"Embodied intelligence via learning and evolution","authors":"Gupta","year":"2021","venue":"Nature Communications","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ha, 2019] David Ha. Reinforcement learning for improving agent design. Artificial Life, 25(4):352-365, 2019.","title":"Reinforcement learning for improving agent design","authors":"David Ha","year":"2019","venue":"Artificial Life","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Haarnoja et al., 2023] Tuomas Haarnoja, Ben Moran, Guy Lever, Sandy H. Huang, Dhruva Tirumala, Markus Wulfmeier, Jan Humplik, Saran Tunyasuvunakool, Noah Y. Siegel, Roland Hafner, Michael Bloesch, Kristian Hartikainen, Arunkumar Byravan, Leonard Hasenclever, Yuval Tassa, Fereshteh Sadeghi, Nathan Batchelor, Federico Casarini, Stefano Saliceti, Charles Game, Neil Sreendra, Kushal Patel, Marlon Gwira, Andrea Huber, Nicole Hurley, Francesco Nori, Raia Hadsell, and Nicolas Heess. Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning. arXiv preprint arXiv:2304.13653, 2023.","title":"Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning","authors":"Tuomas Haarnoja, Ben Moran, Guy Lever, Sandy H Huang, Dhruva Tirumala, Markus Wulfmeier, Jan Humplik, Saran Tunyasuvunakool, Noah Y Siegel, Roland Hafner, Michael Bloesch, Kristian Hartikainen, Arunkumar Byravan, Leonard Hasenclever, Yuval Tassa, Fereshteh Sadeghi, Nathan Batchelor, Federico Casarini, Stefano Saliceti, Charles Game, Neil Sreendra, Kushal Patel, Marlon Gwira, Andrea Huber, Nicole Hurley, Francesco Nori, Raia Hadsell, Nicolas Heess","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hu et al., 2023] Jiaheng Hu, Julian Whitman, and Howie Choset. GLSO: Grammar-guided Latent Space Optimiza- tion for Sample-efficient Robot Design Automation. Pro- ceedings of Machine Learning Research, 205:1321-1331, 2023.","title":"GLSO: Grammar-guided Latent Space Optimization for Sample-efficient Robot Design Automation","authors":"Hu","year":"2023","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huang et al., 2021] Kangyao Huang, Jingyu Chen, and John Oyekan. Decentralised aerial swarm for adaptive and energy efficient transport of unknown loads. Swarm and Evolutionary Computation, 67, 2021.","title":"Decentralised aerial swarm for adaptive and energy efficient transport of unknown loads","authors":"Huang","year":"2021","venue":"Swarm and Evolutionary Computation","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang et al., 2022] Kangyao Huang, Jingyu Chen, John Oyekan, and Xinyu Zhang. Bio-inspired Multi-agent Model and Optimization Strategy for Collaborative Aerial Transport. Lecture Notes in Electrical Engineering, 801 LNEE:591-598, 2022.","title":"Bio-inspired Multi-agent Model and Optimization Strategy for Collaborative Aerial Transport","authors":"Huang","year":"2022","venue":"Lecture Notes in Electrical Engineering","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2024] Kangyao Huang, Di Guo, Xinyu Zhang, Xiangyang Ji, and Huaping Liu. Stimulate the Po- tential of Robots via Competition. arXiv preprint arXiv:2403.10487, 2024.","title":"Stimulate the Potential of Robots via Competition","authors":"Kangyao Huang, Di Guo, Xinyu Zhang, Xiangyang Ji, Huaping Liu","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2019] Siqi Liu, Guy Lever, Josh Merel, Saran Tunyasuvunakool, Nicolas Heess, and Thore Graepel. Emergent coordination through competition. 7th Inter- national Conference on Learning Representations, ICLR 2019, 2019.","title":"Emergent coordination through competition. 7th International Conference on Learning Representations","authors":"Siqi Liu, Guy Lever, Josh Merel, Saran Tunyasuvunakool, Nicolas Heess, Thore Graepel","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2022] Siqi Liu, Guy Lever, Zhe Wang, Josh Merel, S. M.Ali Eslami, Daniel Hennes, Wojciech M. Czarnecki, Yuval Tassa, Shayegan Omidshafiei, Abbas Abdolmaleki, Noah Y. Siegel, Leonard Hasenclever, Luke Marris, Saran Tunyasuvunakool, H. Francis Song, Markus Wulfmeier, Paul Muller, Tuomas Haarnoja, Brendan Tracey, Karl Tuyls, Thore Graepel, and Nicolas Heess. From motor control to team play in simulated humanoid football. Science Robotics, 7(69), 2022.","title":"From motor control to team play in simulated humanoid football","authors":"Siqi Liu, Guy Lever, Zhe Wang, Josh Merel, S M Ali Eslami, Daniel Hennes, Wojciech M Czarnecki, Yuval Tassa, Shayegan Omidshafiei, Abbas Abdolmaleki, Noah Y Siegel, Leonard Hasenclever, Luke Marris, Saran Tunyasuvunakool, H Francis Song, Markus Wulfmeier, Paul Muller, Tuomas Haarnoja, Brendan Tracey, Karl Tuyls, Thore Graepel, Nicolas Heess","year":"2022","venue":"Science Robotics","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2023] Huaping Liu, Di Guo, Fuchun Sun, and Xinyu Zhang. Morphology-based embodied intelligence: Historical retrospect and research progress. Acta Automat- ica Sinica, 49(6):11311154, 2023.","title":"Morphology-based embodied intelligence: Historical retrospect and research progress","authors":"Huaping Liu, Di Guo, Fuchun Sun, Xinyu Zhang","year":"2023","venue":"Acta Automatica Sinica","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Riedmiller et al., 2018] Martin Riedmiller, Roland Hafner, Thomas Lampe, Michael Neunert, Jonas Degrave, Tom Van De Wiele, Volodymyr Mnih, Nicolas Heess, and To- bias Springenberg. Learning by playing -Solving sparse reward tasks from scratch. 35th International Conference on Machine Learning, ICML 2018, 10:6910-6919, 2018.","title":"Learning by playing -Solving sparse reward tasks from scratch","authors":"Martin Riedmiller, Roland Hafner, Thomas Lampe, Michael Neunert, Jonas Degrave, Tom Van De Wiele, Volodymyr Mnih, Nicolas Heess","year":"2018","venue":"35th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Schaff et al., 2019] Charles Schaff, David Yunis, Ayan Chakrabarti, and Matthew R. Walter. Jointly learning to construct and control agents using deep reinforcement learning. Proceedings -IEEE International Conference on Robotics and Automation, 2019-May:9798-9805, 2019.","title":"Jointly learning to construct and control agents using deep reinforcement learning","authors":"Charles Schaff, David Yunis, Ayan Chakrabarti, Matthew R Walter","year":"2019","venue":"Proceedings -IEEE International Conference on Robotics and Automation","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Silver et al., 2016] David Silver, Aja Huang, Chris J. Mad- dison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Do- minik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mas- tering the game of Go with deep neural networks and tree search. Nature, 529(7587):484-489, 2016.","title":"Mastering the game of Go with deep neural networks and tree search","authors":"David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den, Julian Driessche, Ioannis Schrittwieser, Veda Antonoglou, Marc Panneershelvam, Sander Lanctot, Dominik Dieleman, John Grewe, Nal Nham, Ilya Kalchbrenner, Timothy Sutskever, Madeleine Lillicrap, Koray Leach, Thore Kavukcuoglu, Demis Graepel, Hassabis","year":"2016","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Silver et al., 2018] David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. A general reinforcement learning algo- rithm that masters chess, shogi, and Go through self-play. Science, 362(6419):1140-1144, 2018.","title":"A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play","authors":"David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, Demis Hassabis","year":"2018","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sims, 2023] Karl Sims. Evolving Virtual Creatures. Semi- nal Graphics Papers: Pushing the Boundaries, Volume 2, pages 699-706, 2023.","title":"Evolving Virtual Creatures. Seminal Graphics Papers: Pushing the Boundaries","authors":"Karl Sims","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vinyals et al., 2019] Oriol Vinyals, Igor Babuschkin, Woj- ciech M. Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H. Choi, Richard Powell, Timo Ewalds, Petko Georgiev, Junhyuk Oh, Dan Horgan, Manuel Kroiss, Ivo Danihelka, Aja Huang, Laurent Sifre, Trevor Cai, John P. Agapiou, Max Jaderberg, Alexan- der S. Vezhnevets, Rémi Leblond, Tobias Pohlen, Valentin Dalibard, David Budden, Yury Sulsky, James Molloy, Tom L. Paine, Caglar Gulcehre, Ziyu Wang, Tobias Pfaff, Yuhuai Wu, Roman Ring, Dani Yogatama, Dario Wünsch, Katrina McKinney, Oliver Smith, Tom Schaul, Timo- thy Lillicrap, Koray Kavukcuoglu, Demis Hassabis, Chris Apps, and David Silver. Grandmaster level in Star- Craft II using multi-agent reinforcement learning. Nature, 575(7782):350-354, 2019.","title":"Grandmaster level in Star-Craft II using multi-agent reinforcement learning","authors":"Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, Junhyuk Oh, Dan Horgan, Manuel Kroiss, Ivo Danihelka, Aja Huang, Laurent Sifre, Trevor Cai, John P Agapiou, Max Jaderberg, Alexander S Vezhnevets, Rémi Leblond, Tobias Pohlen, Valentin Dalibard, David Budden, Yury Sulsky, James Molloy, Tom L Paine, Caglar Gulcehre, Ziyu Wang, Tobias Pfaff, Yuhuai Wu, Roman Ring, Dani Yogatama, Dario Wünsch, Katrina Mckinney, Oliver Smith, Tom Schaul, Timothy Lillicrap, Koray Kavukcuoglu, Demis Hassabis, Chris Apps, David Silver","year":"2019","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2018] Tingwu Wang, Renjie Liao, Jimmy Ba, and Sanja Fidler. Nervenet: Learning structured policy with graph neural networks. 6th International Confer- ence on Learning Representations, ICLR 2018 -Confer- ence Track Proceedings, 2018.","title":"Nervenet: Learning structured policy with graph neural networks","authors":"Wang","year":"2018","venue":"6th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2019] Tingwu Wang, Yuhao Zhou, Sanja Fi- dler, and Jimmy Ba. Neural graph evolution: Towards efficient automatic robot design. 7th International Con- ference on Learning Representations, ICLR 2019, 2019.","title":"Neural graph evolution: Towards efficient automatic robot design","authors":"Wang","year":"2019","venue":"7th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2023a] Minghao Wang, Ming Cong, Yu Du, Dong Liu, and Xiaojing Tian. Multi-robot raster map fu- sion without initial relative position. Robotic Intelligence and Automation, 43(5):498-508, 2023.","title":"Multi-robot raster map fusion without initial relative position","authors":"Wang","year":"2023","venue":"Robotic Intelligence and Automation","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2023b] Yuxing Wang, Shuang Wu, Tiantian Zhang, Yongzhe Chang, Haobo Fu, Qiang Fu, and Xue- qian Wang. PreCo: Enhancing Generalization in Co- Design of Modular Soft Robots via Brain-Body Pre- Training. Proceedings of Machine Learning Research, 229:478-498, 2023.","title":"PreCo: Enhancing Generalization in Co-Design of Modular Soft Robots via Brain-Body Pre-Training","authors":"Wang","year":"2023","venue":"Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yuan et al., 2022] Ye Yuan, Yuda Song, Zhengyi Luo, Wen Sun, and Kris M. Kitani. Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design. ICLR 2022 -10th International Conference on Learning Representations, 2022.","title":"Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design","authors":"Ye Yuan, Yuda Song, Zhengyi Luo, Wen Sun, Kris M Kitani","year":"2022","venue":"ICLR 2022 -10th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Yuan et al., 2023] Lei Yuan, Ziqian Zhang, Ke Xue, Hao Yin, Feng Chen, Cong Guan, Lihe Li, Chao Qian, and Yang Yu. Robust Multi-Agent Coordination via Evolution- ary Generation of Auxiliary Adversarial Attackers. Pro- ceedings of the 37th AAAI Conference on Artificial Intelli- gence, AAAI 2023, 37:11753-11762, 2023.","title":"Robust Multi-Agent Coordination via Evolutionary Generation of Auxiliary Adversarial Attackers","authors":"Lei Yuan, Ziqian Zhang, Ke Xue, Feng Hao Yin, Cong Chen, Lihe Guan, Chao Li, Yang Qian, Yu","year":"2023","venue":"Proceedings of the 37th AAAI Conference on Artificial Intelligence, AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2022a] Xinyu Zhang, Yuanhao Huang, Kangyao Huang, Xiaoyu Wang, Dafeng Jin, Huaping Liu, and Jun Li. A Multi-modal Deformable Land-air Robot for Complex Environments. 2023 IEEE/ASME Interna- tional Conference on Advanced Intelligent Mechatronics, 2022.","title":"A Multi-modal Deformable Land-air Robot for Complex Environments","authors":"Zhang","year":"2022","venue":"IEEE/ASME International Conference on Advanced Intelligent Mechatronics","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2022b] Xinyu Zhang, Yuanhao Huang, Kangyao Huang, Ziqi Zhao, Jingwei Li, Huaping Liu, and Jun Li. Coupled Modeling and Fusion Control for a Multi-modal Deformable Land-air Robot. arXiv preprint arXiv:2211.04185, 2022.","title":"Coupled Modeling and Fusion Control for a Multi-modal Deformable Land-air Robot","authors":"Zhang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}