{"type":"ScholarlyArticle","title":"Large Language Model Based Multi-agents: A Survey of Progress and Challenges","authors":[{"name":"Taicheng Guo","slug":"taicheng-guo","given":"Taicheng","family":"Guo","affiliation":"University of Notre Dame"},{"name":"Xiuying Chen","slug":"xiuying-chen","given":"Xiuying","family":"Chen","affiliation":"King Abdullah University of Science and Technology"},{"name":"Yaqi Wang","slug":"yaqi-wang","given":"Yaqi","family":"Wang","affiliation":"Southern University of Science and Technology"},{"name":"Ruidi Chang","slug":"ruidi-chang","given":"Ruidi","family":"Chang","affiliation":"Independent Researcher"},{"name":"Shichao Pei","slug":"shichao-pei","given":"Shichao","family":"Pei","affiliation":"University of Massachusetts Boston"},{"name":"Nitesh V. Chawla","slug":"nitesh-v-chawla","given":"Nitesh V.","family":"Chawla","affiliation":"University of Notre Dame"},{"name":"Olaf Wiest","slug":"olaf-wiest","given":"Olaf","family":"Wiest","affiliation":"University of Notre Dame"},{"name":"Xiangliang Zhang","slug":"xiangliang-zhang","given":"Xiangliang","family":"Zhang","affiliation":"University of Notre Dame"}],"abstract":"Large Language Models (LLMs) have achieved remarkable success across a wide array of tasks. Due to their notable capabilities in planning and reasoning, LLMs have been utilized as autonomous agents for the automatic execution of various tasks. Recently, LLM-based agent systems have rapidly evolved from single-agent planning or decision-making to operating as multi-agent systems, enhancing their ability in complex problem-solving and world simulation. To offer an overview of this dynamic field,  we present this survey to offer an in-depth discussion on the essential aspects and challenges of LLM-based multi-agent (LLM-MA) systems. Our objective is to provide readers with an in-depth understanding of these key points: the domains and settings where LLM-MA systems operate or simulate; the profiling and communication methods of these agents; and the means by which these agents develop their skills. For those interested in delving into this field, we also summarize the commonly used datasets or benchmarks. To keep researchers updated on the latest studies,  we maintain an open-source GitHub repository (github.com/taichengguo/LLM_MultiAgents_Survey_Papers), dedicated to outlining the research of LLM-MA research.","doi":"10.24963/ijcai.2024/890","url":"https://ijcai.org/proceedings/2024/890","pdf":"https://ijcai.org/proceedings/2024/0890.pdf","markdown":"https://ijcai.org/proceedings/2024/890.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/890","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Survey Track","track":"Survey Track","number":890,"pages":{"first":8048,"last":8057},"keywords":["Agent-based and Multi-agent Systems: MAS: Other","Natural Language Processing: NLP: Language models"],"videos":[],"references":[{"position":1,"raw":"[Agashe et al., 2023] Saaket Agashe, Yue Fan, and Xin Eric Wang. Evaluating multi-agent coordination abilities in large language models. arXiv preprint arXiv:2310.03903, 2023.","title":"Evaluating multi-agent coordination abilities in large language models","authors":"Agashe","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aher et al., 2023] Gati V Aher, Rosa I Arriaga, and Adam Tauman Kalai. Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning, pages 337-371. PMLR, 2023.","title":"Using large language models to simulate multiple humans and replicate human subject studies","authors":"Rosa I Gati V Aher, Adam Arriaga, Kalai Tauman","year":"2023","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Akata et al., 2023] Elif Akata, Lion Schulz, Julian Coda-Forno, Seong Joon Oh, Matthias Bethge, and Eric Schulz. Playing repeated games with large language models. arXiv preprint arXiv:2305.16867, 2023.","title":"Playing repeated games with large language models","authors":"Elif Akata, Lion Schulz, Julian Coda-Forno, Seong Joon Oh, Matthias Bethge, Eric Schulz","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chan et al., 2023] Chi-Min Chan, Weize Chen, Yusheng Su, Jianx- uan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better llm-based evaluators through multi- agent debate. arXiv preprint arXiv:2308.07201, 2023.","title":"Chateval: Towards better llm-based evaluators through multiagent debate","authors":"Chan","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2023a] Guangyao Chen, Siwei Dong, et al. Autoa- gents: A framework for automatic agent generation. arXiv preprint arXiv:2309.17288, 2023.","title":"Autoagents: A framework for automatic agent generation","authors":"Chen","year":"2023","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2024/3"},{"position":6,"raw":"[Chen et al., 2023b] Huaben Chen, Wenkang Ji, et al. Multi-agent consensus seeking via large language models. arXiv preprint arXiv:2310.20151, 2023.","title":"Multi-agent consensus seeking via large language models","authors":"Chen","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2023c] Weize Chen, Yusheng Su, et al. Agentverse: Facilitating multi-agent collaboration and exploring emergent be- haviors in agents. arXiv preprint arXiv:2308.10848, 2023.","title":"Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents","authors":"Chen","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2023d] Yongchao Chen, Arkin, et al. Scalable multi- robot collaboration with large language models: Centralized or decentralized systems? arXiv preprint arXiv:2309.15943, 2023.","title":"Scalable multirobot collaboration with large language models: Centralized or decentralized systems","authors":"Chen","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dibia, 2023] Victor Dibia. Multi-agent llm applications -a re- view of current research, tools, and challenges, 2023.","title":"Multi-agent llm applications -a review of current research, tools, and challenges","authors":"Victor Dibia","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dong et al., 2022] Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhi- fang Sui. A survey on in-context learning. arXiv preprint arXiv:2301.00234, 2022.","title":"A survey on in-context learning","authors":"Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, Zhifang Sui","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Dong et al., 2023] Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. Self-collaboration code generation via chatgpt. arXiv preprint arXiv:2304.07590, 2023.","title":"Self-collaboration code generation via chatgpt","authors":"Yihong Dong, Xue Jiang, Zhi Jin, Ge Li","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Du et al., 2023] Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. Improving factuality and rea- soning in language models through multiagent debate. arXiv preprint arXiv:2305.14325, 2023.","title":"Improving factuality and reasoning in language models through multiagent debate","authors":"Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, Igor Mordatch","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Fan et al., 2023] Caoyun Fan, Jindou Chen, et al. Can large lan- guage models serve as rational players in game theory? a sys- tematic analysis. arXiv preprint arXiv:2312.05488, 2023.","title":"Caoyun Fan, Jindou Chen, et al. Can large language models serve as rational players in game theory? a systematic analysis","authors":"Fan","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Farmer and Axtell, 2022] J. Doyne Farmer and Robert L. Axtell. Agent-Based Modeling in Economics and Finance: Past, Present, and Future. INET Oxford Working Papers 2022-10, June 2022.","title":"Agent-Based Modeling in Economics and Finance: Past, Present, and Future","authors":"J, Doyne Farmer, Robert L Axtell","year":"2022","venue":"INET Oxford Working Papers 2022-10","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Gao et al., 2023a] Chen Gao, Xiaochong Lan, et al. S 3 : Social-network simulation system with large language model- empowered agents. arXiv preprint arXiv:2307.14984, 2023.","title":"S 3 : Social-network simulation system with large language modelempowered agents","authors":"Gao","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gao et al., 2023b] Yunfan Gao, Yun Xiong, Xinyu Gao, Kangx- iang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. Retrieval-augmented generation for large language mod- els: A survey. arXiv preprint arXiv:2312.10997, 2023.","title":"Retrieval-augmented generation for large language models: A survey","authors":"Gao","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Geva et al., 2021] Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strate- gies. Transactions of the Association for Computational Linguis- tics, 9:346-361, 2021.","title":"Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies","authors":"Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, Jonathan Berant","year":"2021","venue":"Transactions of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Hendrycks et al., 2020] Dan Hendrycks, Collin Burns, et al. Mea- suring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020.","title":"Measuring massive multitask language understanding","authors":"Dan Hendrycks, Collin Burns","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Hong et al., 2023] Sirui Hong, Xiawu Zheng, et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352, 2023.","title":"Metagpt: Meta programming for multi-agent collaborative framework","authors":"Sirui Hong, Xiawu Zheng","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Horton, 2023] John J Horton. Large language models as simulated economic agents: What can we learn from homo silicus? Tech- nical report, National Bureau of Economic Research, 2023.","title":"Large language models as simulated economic agents: What can we learn from homo silicus?","authors":"J John, Horton","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Hua et al., 2023] Wenyue Hua, Lizhou Fan, Lingyao Li, Kai Mei, Jianchao Ji, Yingqiang Ge, Libby Hemphill, and Yongfeng Zhang. War and peace (waragent): Large language model- based multi-agent simulation of world wars. arXiv preprint arXiv:2311.17227, 2023.","title":"War and peace (waragent): Large language modelbased multi-agent simulation of world wars","authors":"Hua","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Huang et al., 2023a] Dong Huang, Qingwen Bu, Jie M Zhang, Michael Luck, and Heming Cui. Agentcoder: Multi-agent-based code generation with iterative testing and optimisation. arXiv preprint arXiv:2312.13010, 2023.","title":"Agentcoder: Multi-agent-based code generation with iterative testing and optimisation","authors":"Huang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Huang et al., 2023b] Lei Huang, Weijiang Yu, et al. A sur- vey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. arXiv preprint arXiv:2311.05232, 2023.","title":"A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions","authors":"Huang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Kaiya et al., 2023] Zhao Kaiya, Michelangelo Naim, et al. Lyfe agents: Generative agents for low-cost real-time social interac- tions. arXiv preprint arXiv:2310.02172, 2023.","title":"Lyfe agents: Generative agents for low-cost real-time social interactions","authors":"Zhao Kaiya, Michelangelo Naim","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Khot et al., 2022] Tushar Khot, Harsh Trivedi, Matthew Fin- layson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sab- harwal. Decomposed prompting: A modular approach for solv- ing complex tasks. arXiv preprint arXiv:2210.02406, 2022.","title":"Decomposed prompting: A modular approach for solving complex tasks","authors":"Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, Ashish Sabharwal","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Kovač et al., 2023] Kovač, Portelas, et al. The socialai school: In- sights from developmental psychology towards artificial socio- cultural agents. arXiv preprint arXiv:2307.07871, 2023.","title":"The socialai school: Insights from developmental psychology towards artificial sociocultural agents","authors":"Kovač","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Lewis et al., 2020] Patrick Lewis, Ethan Perez, Aleksandra Pik- tus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33:9459-9474, 2020.","title":"Retrieval-augmented generation for knowledge-intensive nlp tasks","authors":"Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-Tau Yih, Tim Rocktäschel","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Lex and Schedl, 2022] Elisabeth Lex and Markus Schedl. Psychology-informed recommender systems: A human-centric perspective on recommender systems. CHIIR '22. ACM, 2022.","title":"Psychology-informed recommender systems: A human-centric perspective on recommender systems","authors":"Lex, Schedl, Elisabeth Lex, Markus Schedl","year":"2022","venue":"CHIIR '22","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Li et al., 2023a] Chao Li, Xing Su, et al. Quantifying the impact of large language models on collective opinion dynamics. arXiv preprint arXiv:2308.03313, 2023.","title":"Quantifying the impact of large language models on collective opinion dynamics","authors":"Li","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Li et al., 2023b] Guohao Li, Hammoud, et al. Camel: Commu- nicative agents for\" mind\" exploration of large scale language model society. arXiv preprint arXiv:2303.17760, 2023.","title":"Camel: Communicative agents for\" mind\" exploration of large scale language model society","authors":"Li","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Li et al., 2023c] Huao Li, Yu Quan Chong, Simon Stepputtis, Joseph Campbell, Dana Hughes, Michael Lewis, and Katia Sycara. Theory of mind for multi-agent collaboration via large language models. arXiv preprint arXiv:2310.10701, 2023.","title":"Theory of mind for multi-agent collaboration via large language models","authors":"Li","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Li et al., 2023d] Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. Api-bank: A comprehensive benchmark for tool- augmented llms. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.","title":"Api-bank: A comprehensive benchmark for toolaugmented llms","authors":"Li","year":"2023","venue":"The 2023 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Li et al., 2023e] Nian Li, Chen Gao, Yong Li, and Qingmin Liao. Large language model-empowered agents for simulating macroe- conomic activities. arXiv preprint arXiv:2310.10436, 2023.","title":"Large language model-empowered agents for simulating macroeconomic activities","authors":"Li","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Li et al., 2023f] Siyu Li, Jin Yang, et al. Are you in a masquer- ade? exploring the behavior and impact of large language model driven social bots in online social networks. arXiv preprint arXiv:2307.10337, 2023.","title":"Are you in a masquerade? exploring the behavior and impact of large language model driven social bots in online social networks","authors":"Li","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Li et al., 2023g] Yang Li, Yangyang Yu, Haohang Li, Zhi Chen, and Khaldoun Khashanah. Tradinggpt: Multi-agent system with layered memory and distinct characters for enhanced financial trading performance. arXiv preprint arXiv:2309.03736, 2023.","title":"Tradinggpt: Multi-agent system with layered memory and distinct characters for enhanced financial trading performance","authors":"Li","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Li et al., 2023h] Yuan Li, Yixuan Zhang, et al. Metaagents: Simulating interactions of human behaviors for llm-based task- oriented coordination via collaborative generative agents. arXiv preprint arXiv:2310.06500, 2023.","title":"Metaagents: Simulating interactions of human behaviors for llm-based taskoriented coordination via collaborative generative agents","authors":"Li","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Light et al., 2023a] Jonathan Light, Min Cai, et al. From text to tactic: Evaluating llms playing the game of avalon. arXiv preprint arXiv:2310.05036, 2023.","title":"From text to tactic: Evaluating llms playing the game of avalon","authors":"Light","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Light et al., 2023b] Jonathan Light, Min Cai, Sheng Shen, and Ziniu Hu. Avalonbench: Evaluating llms playing the game of avalon. In NeurIPS 2023 Foundation Models for Decision Mak- ing Workshop, 2023.","title":"Avalonbench: Evaluating llms playing the game of avalon","authors":"Light","year":"2023","venue":"NeurIPS 2023 Foundation Models for Decision Making Workshop","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Liu et al., 2023] Zijun Liu, Yanzhe Zhang, et al. Dynamic llm- agent network: An llm-agent collaboration framework with agent team optimization. arXiv preprint arXiv:2310.02170, 2023.","title":"Dynamic llmagent network: An llm-agent collaboration framework with agent team optimization","authors":"Zijun Liu, Yanzhe Zhang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Mandi et al., 2023] Zhao Mandi, Shreeya Jain, and Shuran Song. Roco: Dialectic multi-robot collaboration with large language models. arXiv preprint arXiv:2307.04738, 2023.","title":"Roco: Dialectic multi-robot collaboration with large language models","authors":"Zhao Mandi, Shreeya Jain, Shuran Song","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Mao et al., 2023] Shaoguang Mao, Yuzhe Cai, et al. Alympics: Language agents meet game theory. arXiv preprint arXiv:2311.03220, 2023.","title":"Alympics: Language agents meet game theory","authors":"Mao","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Moura, 2023] João Moura. Crewai. https://github.com/ joaomdmoura/crewAI, 2023.","title":"Crewai","authors":"João Moura","year":"2023","venue":null,"doi":null,"url":"https://github.com/joaomdmoura/crewAI","internal_url":null},{"position":43,"raw":"[Mukobi et al., 2023] Gabriel Mukobi, Hannah Erlebach, et al. Welfare diplomacy: Benchmarking language model cooperation. arXiv preprint arXiv:2310.08901, 2023.","title":"Welfare diplomacy: Benchmarking language model cooperation","authors":"Gabriel Mukobi, Hannah Erlebach","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Nascimento et al., 2023] Nathalia Nascimento, Paulo Alencar, and Donald Cowan. Self-adaptive large language model (llm)-based multiagent systems. In ACSOS-C, pages 104-109. IEEE, 2023.","title":"Self-adaptive large language model (llm)-based multiagent systems","authors":"Nathalia Nascimento, Paulo Alencar, Donald Cowan","year":"2023","venue":"ACSOS-C","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Navid et al., 2023] Navid, Majumdar, et al. Generative agent- based modeling: Unveiling social system dynamics through cou- pling mechanistic models with generative artificial intelligence. arXiv preprint arXiv:2309.11456, 2023.","title":"Generative agentbased modeling: Unveiling social system dynamics through coupling mechanistic models with generative artificial intelligence","authors":"Navid, Majumdar","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Park et al., 2022] Joon Sung Park, Lindsay Popowski, et al. Social simulacra: Creating populated prototypes for social computing systems. In UIST, pages 1-18, 2022.","title":"Social simulacra: Creating populated prototypes for social computing systems","authors":"Sung Joon, Lindsay Park, Popowski","year":"2022","venue":"UIST","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Park et al., 2023] Joon Sung Park, Joseph C O'Brien, et al. Gen- erative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442, 2023.","title":"Generative agents: Interactive simulacra of human behavior","authors":"Sung Joon, Park, C Joseph, O'brien","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Qian et al., 2023] Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. Communicative agents for software development. arXiv preprint arXiv:2307.07924, 2023.","title":"Communicative agents for software development","authors":"Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, Maosong Sun","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Ruan et al., 2023] Jingqing Ruan, Yihong Chen, Bin Zhang, Zhi- wei Xu, Tianpeng Bao, Guoqing Du, Shiwei Shi, Hangyu Mao, Ziyue Li, Xingyu Zeng, et al. Tptu: large language model- based ai agents for task planning and tool usage. arXiv preprint arXiv:2308.03427, 2023.","title":"Tptu: large language modelbased ai agents for task planning and tool usage","authors":"Ruan","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Shinn et al., 2023] Noah Shinn, Federico Cassano, et al. Reflex- ion: Language agents with verbal reinforcement learning, 2023.","title":"Reflexion: Language agents with verbal reinforcement learning","authors":"Shinn","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Sumers et al., 2023] Theodore R Sumers, Shunyu Yao, et al. Cognitive architectures for language agents. arXiv preprint arXiv:2309.02427, 2023.","title":"Cognitive architectures for language agents","authors":"Shunyu Theodore R Sumers, Yao","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Tang et al., 2023] Xiangru Tang, Anni Zou, et al. Medagents: Large language models as collaborators for zero-shot medical reasoning, 2023.","title":"Medagents: Large language models as collaborators for zero-shot medical reasoning","authors":"Tang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Wang et al., 2021] Zijie J. Wang, Dongjin Choi, Shenyu Xu, and Diyi Yang. Putting humans in the natural language processing loop: A survey, 2021.","title":"Putting humans in the natural language processing loop: A survey","authors":"J Zijie, Dongjin Wang, Shenyu Choi, Diyi Xu, Yang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Wang et al., 2023a] Kuan Wang, Yadong Lu, et al. Adapting llm agents through communication, 2023.","title":"Adapting llm agents through communication","authors":"Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Wang et al., 2023b] Lei Wang, Chen Ma, et al. A survey on large language model based autonomous agents, 2023.","title":"A survey on large language model based autonomous agents","authors":"Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":56,"raw":"[Wang et al., 2023c] Shenzhi Wang, Chang Liu, et al. Avalon's game of thoughts: Battle against deception through recursive contemplation. arXiv preprint arXiv:2310.01320, 2023.","title":"Avalon's game of thoughts: Battle against deception through recursive contemplation","authors":"Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":57,"raw":"[Weiss et al., 2024] Martin Weiss, Nasim Rahaman, Manuel Wuthrich, Yoshua Bengio, Li Erran Li, Bernhard Schölkopf, and Christopher Pal. Rethinking the buyer's inspection paradox in information markets with language agents, 2024.","title":"Rethinking the buyer's inspection paradox in information markets with language agents","authors":"Martin Weiss, Nasim Rahaman, Manuel Wuthrich, Yoshua Bengio, Li Erran Li, Bernhard Schölkopf, Christopher Pal","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":58,"raw":"[Weng, 2023] Lilian Weng. Llm powered autonomous agents. https://lilianweng.github.io/posts/2023-06-23-agent/, 2023.","title":"Llm powered autonomous agents","authors":"Lilian Weng","year":"2023","venue":null,"doi":null,"url":"https://lilianweng.github.io/posts/2023-06-23-agent/","internal_url":null},{"position":59,"raw":"[Williams et al., 2023] Ross Williams, Niyousha Hosseinichimeh, et al. Epidemic modeling with generative agents. arXiv preprint arXiv:2307.04986, 2023.","title":"Epidemic modeling with generative agents","authors":"Ross Williams, Niyousha Hosseinichimeh","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":60,"raw":"[Wu et al., 2023] Qingyun Wu, Gagan Bansal, et al. Autogen: En- abling next-gen llm applications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155, 2023.","title":"Autogen: Enabling next-gen llm applications via multi-agent conversation framework","authors":"Qingyun Wu, Gagan Bansal","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":61,"raw":"[Xi et al., 2023] Zhiheng Xi, Wenxiang Chen, et al. The rise and potential of large language model based agents: A survey, 2023.","title":"The rise and potential of large language model based agents: A survey","authors":"Zhiheng Xi, Wenxiang Chen","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":62,"raw":"[Xiao et al., 2023] Bushi Xiao, Ziyuan Yin, et al. Simulating public administration crisis: A novel generative agent-based simulation system to lower technology barriers in social science research. arXiv preprint arXiv:2311.06957, 2023.","title":"Simulating public administration crisis: A novel generative agent-based simulation system to lower technology barriers in social science research","authors":"Bushi Xiao, Ziyuan Yin","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":63,"raw":"[Xie et al., 2023] Tianbao Xie, Fan Zhou, et al. Openagents: An open platform for language agents in the wild. arXiv preprint arXiv:2310.10634, 2023.","title":"Openagents: An open platform for language agents in the wild","authors":"Tianbao Xie, Fan Zhou","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":64,"raw":"[Xiong et al., 2023] Kai Xiong, Xiao Ding, et al. Examining inter- consistency of large language models collaboration: An in-depth analysis via debate, 2023.","title":"Examining interconsistency of large language models collaboration: An in-depth analysis via debate","authors":"Kai Xiong, Xiao Ding","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":65,"raw":"[Xu et al., 2023a] Lin Xu, Zhiyuan Hu, et al. Magic: Investiga- tion of large language model powered multi-agent in cognition, adaptability, rationality and collaboration, 2023.","title":"Magic: Investigation of large language model powered multi-agent in cognition, adaptability, rationality and collaboration","authors":"Xu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":66,"raw":"[Xu et al., 2023b] Yuzhuang Xu, Shuo Wang, et al. Exploring large language models for communication games: An empirical study on werewolf. arXiv preprint arXiv:2309.04658, 2023.","title":"Exploring large language models for communication games: An empirical study on werewolf","authors":"Xu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":67,"raw":"[Xu et al., 2023c] Zelai Xu, Chao Yu, et al. Language agents with reinforcement learning for strategic play in the werewolf game. arXiv preprint arXiv:2310.18940, 2023.","title":"Language agents with reinforcement learning for strategic play in the werewolf game","authors":"Xu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":68,"raw":"[Yao et al., 2023] Shunyu Yao, Dian Yu, et al. Tree of thoughts: Deliberate problem solving with large language models, 2023.","title":"Tree of thoughts: Deliberate problem solving with large language models","authors":"Yao","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":69,"raw":"[Yu et al., 2023] Bangguo Yu, Hamidreza Kasaei, and Ming Cao. Co-navgpt: Multi-robot cooperative visual semantic navigation using large language models, 2023.","title":"Co-navgpt: Multi-robot cooperative visual semantic navigation using large language models","authors":"Yu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":70,"raw":"[Zhang et al., 2023a] An Zhang, Leheng Sheng, Yuxin Chen, Hao Li, Yang Deng, Xiang Wang, and Tat-Seng Chua. On generative agents in recommendation. arXiv preprint arXiv:2310.10108, 2023.","title":"On generative agents in recommendation","authors":"Zhang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":71,"raw":"[Zhang et al., 2023b] Ceyao Zhang, Kaijie Yang, et al. Proagent: Building proactive cooperative ai with large language models. arXiv preprint arXiv:2308.11339, 2023.","title":"Proagent: Building proactive cooperative ai with large language models","authors":"Zhang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":72,"raw":"[Zhang et al., 2023c] Hongxin Zhang, Weihua Du, et al. Build- ing cooperative embodied agents modularly with large language models. arXiv preprint arXiv:2307.02485, 2023.","title":"Building cooperative embodied agents modularly with large language models","authors":"Zhang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":73,"raw":"[Zhang et al., 2023d] Jintian Zhang, Xin Xu, and Shumin Deng. Exploring collaboration mechanisms for llm agents: A social psychology view, 2023.","title":"Exploring collaboration mechanisms for llm agents: A social psychology view","authors":"Zhang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":74,"raw":"[Zhang et al., 2023e] Junjie Zhang, Yupeng Hou, et al. Agentcf: Collaborative learning with autonomous language agents for rec- ommender systems, 2023.","title":"Agentcf: Collaborative learning with autonomous language agents for recommender systems","authors":"Zhang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":75,"raw":"[Zhao et al., 2023] Qinlin Zhao, Jindong Wang, et al. Com- peteai: Understanding the competition behaviors in large lan- guage model-based agents, 2023.","title":"Competeai: Understanding the competition behaviors in large language model-based agents","authors":"Zhao","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":76,"raw":"[Zheng et al., 2023] Zhiling Zheng, Oufan Zhang, et al. Chatgpt research group for optimizing the crystallinity of mofs and cofs. ACS Central Science, 9(11):2161-2170, 2023.","title":"Chatgpt research group for optimizing the crystallinity of mofs and cofs","authors":"Zhiling Zheng, Oufan Zhang","year":"2023","venue":"ACS Central Science","doi":null,"url":null,"internal_url":null},{"position":77,"raw":"[Zhou et al., 2023a] Wangchunshu Zhou, Yuchen Eleanor Jiang, et al. Agents: An open-source framework for autonomous lan- guage agents. arXiv preprint arXiv:2309.07870, 2023.","title":"Agents: An open-source framework for autonomous language agents","authors":"Zhou","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":78,"raw":"[Zhou et al., 2023b] Xuhui Zhou, Hao Zhu, Leena Mathur, Ruo- hong Zhang, Haofei Yu, Zhengyang Qi, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, et al. Sotopia: Interactive evaluation for social intelligence in language agents. arXiv preprint arXiv:2310.11667, 2023.","title":"Sotopia: Interactive evaluation for social intelligence in language agents","authors":"Zhou","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":79,"raw":"[Ziems et al., 2023] Caleb Ziems, Omar Shaikh, et al. Can large language models transform computational social science? Com- putational Linguistics, pages 1-53, 2023.","title":"Can large language models transform computational social science? Computational Linguistics","authors":"Caleb Ziems, Omar Shaikh","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}