{"type":"ScholarlyArticle","title":"Large Language Model as a Policy Teacher for Training Reinforcement Learning Agents","authors":[{"name":"Zihao Zhou","slug":"zihao-zhou","given":"Zihao","family":"Zhou","affiliation":"Zhejiang Lab"},{"name":"Bin Hu","slug":"bin-hu","given":"Bin","family":"Hu","affiliation":"Zhejiang Lab"},{"name":"Chenyang Zhao","slug":"chenyang-zhao","given":"Chenyang","family":"Zhao","affiliation":"Zhejiang Lab"},{"name":"Pu Zhang","slug":"pu-zhang","given":"Pu","family":"Zhang","affiliation":"Zhejiang Lab"},{"name":"Bin Liu","slug":"bin-liu","given":"Bin","family":"Liu","affiliation":"Zhejiang Lab"}],"abstract":"Recent studies have uncovered the potential of Large Language Models (LLMs) in addressing complex sequential decision-making tasks through the provision of high-level instructions. However, LLM-based agents lack specialization in tackling specific target problems, particularly in real-time dynamic environments. Additionally, deploying an LLM-based agent in practical scenarios can be both costly and time-consuming. On the other hand, reinforcement learning (RL) approaches train agents that specialize in the target task but often suffer from low sampling efficiency and high exploration costs. In this paper, we introduce a novel framework that addresses these challenges by training a smaller, specialized student RL agent using instructions from an LLM-based teacher agent. By incorporating the guidance from the teacher agent, the student agent can distill the prior knowledge of the LLM into its own model. Consequently, the student agent can be trained with significantly less data. Moreover, through further training with environment feedback, the student agent surpasses the capabilities of its teacher for completing the target task. We conducted experiments on challenging MiniGrid and Habitat environments, specifically designed for embodied AI research, to evaluate the effectiveness of our framework. The results clearly demonstrate that our approach achieves superior performance compared to strong baseline methods. Our code is available at https://github.com/ZJLAB-AMMI/LLM4Teach.","doi":"10.24963/ijcai.2024/627","url":"https://ijcai.org/proceedings/2024/627","pdf":"https://ijcai.org/proceedings/2024/0627.pdf","markdown":"https://ijcai.org/proceedings/2024/627.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/627","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":627,"pages":{"first":5671,"last":5679},"keywords":["Natural Language Processing: NLP: Language models","Uncertainty in AI: UAI: Sequential decision making","Machine Learning: ML: Reinforcement learning"],"videos":[],"references":[{"position":1,"raw":"[Agarwal et al., 2022] Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron C Courville, and Marc Belle- mare. Reincarnating reinforcement learning: Reusing prior computation to accelerate progress. Advances in Neural In- formation Processing Systems, 35:28955-28971, 2022.","title":"Reincarnating reinforcement learning: Reusing prior computation to accelerate progress","authors":"Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron C Courville, Marc Bellemare","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ahn et al., 2022] Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, et al. Do as I can, not as I say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022.","title":"Do as I can, not as I say: Grounding language in robotic affordances","authors":"Anthony Michael Ahn, Noah Brohan, Yevgen Brown, Omar Chebotar, Byron Cortes, Chelsea David, Chuyuan Finn, Keerthana Fu, Karol Gopalakrishnan, Hausman","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Biggie et al., 2023] Harel Biggie, Ajay Narasimha Mopi- devi, Dusty Woods, and Christoffer Heckman. Tell me where to go: A composable framework for context- aware embodied robot navigation. arXiv preprint arXiv:2306.09523, 2023.","title":"Tell me where to go: A composable framework for contextaware embodied robot navigation","authors":"Ajay Narasimha Harel Biggie, Dusty Mopidevi, Christoffer Woods, Heckman","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brown et al., 2020] Tom Brown, Benjamin Mann, Nick Ry- der, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learn- ers. Advances in neural information processing systems, 33:1877-1901, 2020.","title":"Language models are few-shot learners","authors":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Carta et al., 2023] Thomas Carta, Clément Romac, Thomas Wolf, Sylvain Lamprier, Olivier Sigaud, and Pierre-Yves Oudeyer. Grounding large language models in interactive environments with online reinforcement learning. arXiv preprint arXiv:2302.02662, 2023.","title":"Grounding large language models in interactive environments with online reinforcement learning","authors":"Thomas Carta, Clément Romac, Thomas Wolf, Sylvain Lamprier, Olivier Sigaud, Pierre-Yves Oudeyer","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chevalier-Boisvert et al., 2023] Maxime Chevalier- Boisvert, Bolun Dai, and al et. Minigrid & miniworld: Modular & customizable reinforcement learning environ- ments for goal-oriented tasks. CoRR, abs/2306.13831, 2023.","title":"Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks","authors":"Maxime Chevalier-Boisvert, Bolun Dai","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chiang et al., 2023] Wei-Lin Chiang, Zhuohan Li, Zi Lin, and et. al. Vicuna: An open-source chatbot impressing gpt- 4 with 90%* chatgpt quality, March 2023.","title":"Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality","authors":"Wei-Lin Chiang, Zhuohan Li, Zi Lin","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Colas et al., 2023] Cédric Colas, Laetitia Teodorescu, Pierre-Yves Oudeyer, Xingdi Yuan, and Marc-Alexandre Côté. Augmenting autotelic agents with large language models. arXiv preprint arXiv:2305.12487, 2023.","title":"Augmenting autotelic agents with large language models","authors":"Cédric Colas, Laetitia Teodorescu, Pierre-Yves Oudeyer, Xingdi Yuan, Marc-Alexandre Côté","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Da Silva et al., 2017] Felipe Leno Da Silva, Ruben Glatt, and Anna Helena Reali Costa. Simultaneously learning and advising in multiagent reinforcement learning. In Pro- ceedings of the 16th conference on autonomous agents and multiagent systems, pages 1100-1108, 2017.","title":"Simultaneously learning and advising in multiagent reinforcement learning","authors":"Da Silva","year":"2017","venue":"Proceedings of the 16th conference on autonomous agents and multiagent systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Da Silva et al., 2020] Felipe Leno Da Silva, Garrett War- nell, Anna Helena Reali Costa, and Peter Stone. Agents teaching agents: a survey on inter-agent transfer learn- ing. Autonomous Agents and Multi-Agent Systems, 34:1- 17, 2020.","title":"Felipe Leno Da Silva, Garrett Warnell, Anna Helena Reali Costa, and Peter Stone. Agents teaching agents: a survey on inter-agent transfer learning","authors":"Da Silva","year":"2020","venue":"Autonomous Agents and Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Driess et al., 2023] Danny Driess, Fei Xia, Mehdi SM Saj- jadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. Palm-e: An embodied multimodal language model. arXiv preprint arXiv:2303.03378, 2023.","title":"Palm-e: An embodied multimodal language model","authors":"Danny Driess, Fei Xia, S M Mehdi, Corey Sajjadi, Aakanksha Lynch, Brian Chowdhery, Ayzaan Ichter, Jonathan Wahid, Quan Tompson, Tianhe Vuong, Yu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Du et al., 2022] Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. Glm: Gen- eral language model pretraining with autoregressive blank infilling. In Proc. of the 60th Annual Meeting of the As- sociation for Computational Linguistics, pages 320-335, 2022.","title":"Glm: General language model pretraining with autoregressive blank infilling","authors":"Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, Jie Tang","year":"2022","venue":"Proc. of the 60th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Du et al., 2023] Yuqing Du, Olivia Watkins, Zihan Wang, Cédric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, and Jacob Andreas. Guiding pretraining in re- inforcement learning with large language models. arXiv preprint arXiv:2302.06692, 2023.","title":"Guiding pretraining in reinforcement learning with large language models","authors":"Yuqing Du, Olivia Watkins, Zihan Wang, Cédric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, Jacob Andreas","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hu et al., 2023] Bin Hu, Chenyang Zhao, Pu Zhang, Zi- hao Zhou, Yuanhang Yang, Zenglin Xu, and Bin Liu. Enabling efficient interaction between an agent and an llm: A reinforcement learning approach. arXiv preprint arXiv:2306.03604, 2023.","title":"Enabling efficient interaction between an agent and an llm: A reinforcement learning approach","authors":"Bin Hu, Chenyang Zhao, Pu Zhang, Zihao Zhou, Yuanhang Yang, Zenglin Xu, Bin Liu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hu et al., 2024] Bin Hu, Chenyang Zhao, Pu Zhang, Zihao Zhou, Yuanhang Yang, Zenglin Xu, and Bin Liu. Enabling intelligent interactions between an agent and an llm: A re- inforcement learning approach. Reinforcement Learning Conference (RLC), 2024.","title":"Enabling intelligent interactions between an agent and an llm: A reinforcement learning approach","authors":"Bin Hu, Chenyang Zhao, Pu Zhang, Zihao Zhou, Yuanhang Yang, Zenglin Xu, Bin Liu","year":"2024","venue":"Reinforcement Learning Conference (RLC)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Huang et al., 2022] Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In International Conference on Machine Learning, pages 9118-9147. PMLR, 2022.","title":"Language models as zero-shot planners: Extracting actionable knowledge for embodied agents","authors":"Wenlong Huang, Pieter Abbeel, Deepak Pathak, Igor Mordatch","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Klissarov et al., 2023] Martin Klissarov, Pierluca D'Oro, Shagun Sodhani, Roberta Raileanu, Pierre-Luc Bacon, Pascal Vincent, Amy Zhang, and Mikael Henaff. Motif: Intrinsic motivation from artificial intelligence feedback. arXiv preprint arXiv:2310.00166, 2023.","title":"Motif: Intrinsic motivation from artificial intelligence feedback","authors":"Martin Klissarov, D' Pierluca, Shagun Oro, Roberta Sodhani, Pierre-Luc Raileanu, Pascal Bacon, Amy Vincent, Mikael Zhang, Henaff","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Knox and Stone, 2009] W Bradley Knox and Peter Stone. Interactively shaping agents via human reinforcement: The tamer framework. In Proceedings of the tth International conference on Knowledge capture, pages 9-16, 2009.","title":"Interactively shaping agents via human reinforcement: The tamer framework","authors":"Knox Bradley, Peter Stone","year":"2009","venue":"Proceedings of the tth International conference on Knowledge capture","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kwon et al., 2023] Minae Kwon, Sang Michael Xie, Kale- sha Bullard, and Dorsa Sadigh. Reward design with lan- guage models. arXiv preprint arXiv:2303.00001, 2023.","title":"Reward design with language models","authors":"Minae Kwon, Sang Michael Xie, Kalesha Bullard, Dorsa Sadigh","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mao et al., 2023] Jiageng Mao, Yuxi Qian, Hang Zhao, and Yue Wang. Gpt-driver: Learning to drive with gpt. arXiv preprint arXiv:2310.01415, 2023.","title":"Gpt-driver: Learning to drive with gpt","authors":"Jiageng Mao, Yuxi Qian, Hang Zhao, Yue Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Matthews et al., 2022] Michael Matthews, Mikayel Samvelyan, Jack Parker-Holder, Edward Grefenstette, and Tim Rocktäschel. Hierarchical kickstarting for skill transfer in reinforcement learning. arXiv preprint arXiv:2207.11584, 2022.","title":"Hierarchical kickstarting for skill transfer in reinforcement learning","authors":"Michael Matthews, Mikayel Samvelyan, Jack Parker-Holder, Edward Grefenstette, Tim Rocktäschel","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Nottingham et al., 2023] Kolby Nottingham, Yasaman Razeghi, Kyungmin Kim, JB Lanier, Pierre Baldi, Roy Fox, and Sameer Singh. Selective perception: Optimizing state descriptions with reinforcement learning for language model actors. arXiv preprint arXiv:2307.11922, 2023.","title":"Selective perception: Optimizing state descriptions with reinforcement learning for language model actors","authors":"Kolby Nottingham, Yasaman Razeghi, Kyungmin Kim, Pierre Lanier, Roy Baldi, Sameer Fox, Singh","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.","title":"Language models are unsupervised multitask learners","authors":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever","year":"2019","venue":"OpenAI blog","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Radford et al., 2021] Alec Radford, Jong Wook Kim, and al et. Learning transferable visual models from natural lan- guage supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.","title":"Learning transferable visual models from natural language supervision","authors":"Radford","year":"2021","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Schaal, 1996] Stefan Schaal. Learning from demonstra- tion. Advances in neural information processing systems, 9, 1996.","title":"Learning from demonstration","authors":"Stefan Schaal, Schaal","year":"1996","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Schmitt et al., 2018] Simon Schmitt, Jonathan J Hudson, Augustin Zidek, Simon Osindero, Carl Doersch, Woj- ciech M Czarnecki, Joel Z Leibo, Heinrich Kuttler, An- drew Zisserman, Karen Simonyan, et al. Kickstarting deep reinforcement learning. arXiv preprint arXiv:1803.03835, 2018.","title":"Kickstarting deep reinforcement learning","authors":"Simon Schmitt, Jonathan J Hudson, Augustin Zidek, Simon Osindero, Carl Doersch, Wojciech M Czarnecki, Joel Z Leibo, Heinrich Kuttler, Andrew Zisserman, Karen Simonyan","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.","title":"Proximal policy optimization algorithms","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Sha et al., 2023] Hao Sha, Yao Mu, Yuxuan Jiang, Li Chen, Chenfeng Xu, Ping Luo, Shengbo Eben Li, Masayoshi Tomizuka, Wei Zhan, and Mingyu Ding. Languagempc: Large language models as decision makers for autonomous driving. arXiv preprint arXiv:2310.03026, 2023.","title":"Languagempc: Large language models as decision makers for autonomous driving","authors":"Sha Hao, Yao Mu, Yuxuan Jiang, Li Chen, Chenfeng Xu, Ping Luo, Eben Shengbo, Masayoshi Li, Wei Tomizuka, Mingyu Zhan, Ding","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Shinn et al., 2023] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R Narasimhan, and Shunyu Yao. Re- flexion: Language agents with verbal reinforcement learn- ing. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.","title":"Reflexion: Language agents with verbal reinforcement learning","authors":"Shinn","year":"2023","venue":"Thirty-seventh Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Song et al., 2023] Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M Sadler, Wei-Lun Chao, and Yu Su. Llm-planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2998-3009, 2023.","title":"Llm-planner: Few-shot grounded planning for embodied agents with large language models","authors":"Hee Chan, Jiaman Song, Clayton Wu, Brian M Washington, Wei-Lun Sadler, Yu Chao, Su","year":"2023","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Szot et al., 2021] Andrew Szot, Alexander Clegg, and al et. Habitat 2.0: Training home assistants to rearrange their habitat. Advances in Neural Information Processing Sys- tems, 34:251-266, 2021.","title":"Habitat 2.0: Training home assistants to rearrange their habitat","authors":"Andrew Szot, Alexander Clegg","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Uchendu et al., 2023] Ikechukwu Uchendu, Ted Xiao, Yao Lu, Banghua Zhu, Mengyuan Yan, Joséphine Simon, Matthew Bennice, Chuyuan Fu, Cong Ma, Jiantao Jiao, et al. Jump-start reinforcement learning. In International Conference on Machine Learning, pages 34556-34583.","title":"Jump-start reinforcement learning","authors":"Ikechukwu Uchendu, Ted Xiao, Yao Lu, Banghua Zhu, Mengyuan Yan, Joséphine Simon, Matthew Bennice, Chuyuan Fu, Cong Ma, Jiantao Jiao","year":null,"venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"PMLR, 2023.","title":null,"authors":"Pmlr","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wang et al., 2022] Xuezhi Wang, Jason Wei, and al et. Self- consistency improves chain of thought reasoning in lan- guage models. arXiv preprint arXiv:2203.11171, 2022.","title":"Selfconsistency improves chain of thought reasoning in language models","authors":"Wang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wang et al., 2023a] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open- ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023.","title":"Voyager: An openended embodied agent with large language models","authors":"Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Wang et al., 2023b] Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Ji- akai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. arXiv preprint arXiv:2308.11432, 2023.","title":"A survey on large language model based autonomous agents","authors":"Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wei et al., 2022] Jason Wei, Xuezhi Wang, Dale Schuur- mans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large lan- guage models. arXiv preprint arXiv:2201.11903, 2022.","title":"Chain of thought prompting elicits reasoning in large language models","authors":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, Denny Zhou","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Xi et al., 2023] Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. The rise and poten- tial of large language model based agents: A survey. arXiv preprint arXiv:2309.07864, 2023.","title":"The rise and potential of large language model based agents: A survey","authors":"Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Yang et al., 2023] Sherry Yang, Ofir Nachum, Yilun Du, Ja- son Wei, Pieter Abbeel, and Dale Schuurmans. Foundation models for decision making: Problems, methods, and op- portunities. arXiv preprint arXiv:2303.04129, 2023.","title":"Foundation models for decision making: Problems, methods, and opportunities","authors":"Sherry Yang, Ofir Nachum, Yilun Du, Jason Wei, Pieter Abbeel, Dale Schuurmans","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Yao et al., 2022] Shunyu Yao, Jeffrey Zhao, Dian Yu, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In NeurIPS 2022 Foundation Models for Decision Making Workshop, 2022.","title":"React: Synergizing reasoning and acting in language models","authors":"Shunyu Yao, Jeffrey Zhao, Dian Yu, Izhak Shafran, Yuan Karthik R Narasimhan, Cao","year":"2022","venue":"NeurIPS 2022 Foundation Models for Decision Making Workshop","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Yu et al., 2023] Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, Sean Kirmani, Kuang-Huei Lee, Montse Gonzalez Arenas, Hao-Tien Lewis Chiang, Tom Erez, Leonard Hasenclever, Jan Humplik, et al. Language to rewards for robotic skill synthesis. arXiv preprint arXiv:2306.08647, 2023.","title":"Language to rewards for robotic skill synthesis","authors":"Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, Sean Kirmani, Kuang-Huei Lee, Montse Gonzalez Arenas, Lewis Hao-Tien, Tom Chiang, Leonard Erez, Jan Hasenclever, Humplik","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhen et al., 2023] Yue Zhen, Sheng Bi, Lu Xing-tong, Pan Wei-qin, Shi Hai-peng, Chen Zi-rui, and Fang Yi-shu. Robot task planning based on large language model rep- resenting knowledge with directed graph structures. arXiv preprint arXiv:2306.05171, 2023.","title":"Robot task planning based on large language model representing knowledge with directed graph structures","authors":"Sheng Yue Zhen, Lu Bi, Pan Xing-Tong, Shi Wei-Qin, Chen Hai-Peng, Fang Zi-Rui, Yi-Shu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zhou et al., 2023] Zihao Zhou, Bin Hu, Pu Zhang, Chenyang Zhao, and Bin Liu. Large language model as a policy teacher for training reinforcement learning agents. arXiv preprint arXiv:2311.13373, 2023.","title":"Large language model as a policy teacher for training reinforcement learning agents","authors":"Zhou","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}