{"type":"ScholarlyArticle","title":"Large Language Models for Human-AI Co-Creation of Robotic Dance Performances","authors":[{"name":"Allegra De Filippo","slug":"allegra-de-filippo","given":"Allegra","family":"De Filippo","affiliation":"Department of Computer Science and Engineering , University of Bologna , Italy"},{"name":"Michela Milano","slug":"michela-milano","given":"Michela","family":"Milano","affiliation":"Department of Computer Science and Engineering , University of Bologna , Italy"}],"abstract":"This paper focuses on the potential of Generative Artificial Intelligence (AI), particularly Large Language Models (LLMs), in the still unexplored domain of robotic dance creation. In particular, we assess whether a LLM (GPT-3.5 turbo) can create robotic dance choreographies, and we investigate if the feedback provided by human creators can improve the quality of the output. To this end, we design three prompt engineering techniques for robotic dance creation. In the prompts, we gradually introduce human knowledge through examples and feedback in natural language in order to explore the dynamics of human-AI co-creation. The experimental analysis shows that the capabilities of the LLM can be improved through human collaboration, by producing choreographies with a major artistic impact on the evaluation audience. The findings offer valuable insights into the interplay between human creativity and AI generative models, paving the way for enhanced collaborative frameworks in creative domains.","doi":"10.24963/ijcai.2024/844","url":"https://ijcai.org/proceedings/2024/844","pdf":"https://ijcai.org/proceedings/2024/0844.pdf","markdown":"https://ijcai.org/proceedings/2024/844.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/844","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"AI, Arts & Creativity","track":"AI, Arts & Creativity","number":844,"pages":{"first":7627,"last":7635},"keywords":["Application domains: Performances, dance","Methods and resources: AI systems for collaboration and co-creation"],"videos":[],"references":[{"position":1,"raw":"[Aljanabi et al., 2023] Mohammad Aljanabi, Mo- hanad Ghazi Yaseen, Ahmed Hussein Ali, and Mostafa Abdulghafoor Mohammed. Prompt engineering: Guiding the way to effective large language models. Iraqi Journal for Computer Science and Mathematics, 4(4):151-155, 2023.","title":"Prompt engineering: Guiding the way to effective large language models","authors":"Mohammad Aljanabi, Mohanad Ghazi Yaseen, Ahmed Hussein Ali, Mostafa Abdulghafoor, Mohammed","year":"2023","venue":"Iraqi Journal for Computer Science and Mathematics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aucouturier et al., 2008] Jean-Julien Aucouturier, Kat- sushi Ikeuchi, Hirohisa Hirukawa, Shin'ichiro Nakaoka, Takaaki Shiratori, Shunsuke Kudoh, Fumio Kanehiro, Tetsuya Ogata, Hideki Kozima, Hiroshi G Okuno, et al. Cheek to chip: Dancing robots and ai's future. IEEE Intelligent Systems, 23(2):74-84, 2008.","title":"Cheek to chip: Dancing robots and ai's future","authors":"Jean-Julien Aucouturier, Katsushi Ikeuchi, Hirohisa Hirukawa, Takaaki Shin'ichiro Nakaoka, Shunsuke Shiratori, Fumio Kudoh, Tetsuya Kanehiro, Hideki Ogata, Kozima, G Hiroshi, Okuno","year":"2008","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brown et al., 2006] Steven Brown, Michael J Martinez, and Lawrence M Parsons. The neural basis of human dance. Cerebral cortex, 16(8):1157-1167, 2006.","title":"The neural basis of human dance","authors":"Steven Brown, Michael J Martinez, Lawrence M Parsons","year":"2006","venue":"Cerebral cortex","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chakrabarty et al., 2023a] Tuhin Chakrabarty, Philippe La- ban, Divyansh Agarwal, Smaranda Muresan, and Chien- Sheng Wu. Art or artifice? large language models and the false promise of creativity, 2023.","title":"Art or artifice? large language models and the false promise of creativity","authors":"Chakrabarty","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chakrabarty et al., 2023b] Tuhin Chakrabarty, Vishakh Padmakumar, Faeze Brahman, and Smaranda Muresan. Creativity support in the age of large language models: An empirical study involving emerging writers. arXiv preprint arXiv:2309.12570, 2023.","title":"Creativity support in the age of large language models: An empirical study involving emerging writers","authors":"Chakrabarty","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Charness et al., 2012] Gary Charness, Uri Gneezy, and Michael A Kuhn. Experimental methods: Between- subject and within-subject design. Journal of economic behavior & organization, 81(1):1-8, 2012.","title":"Experimental methods: Betweensubject and within-subject design","authors":"Gary Charness, Uri Gneezy, Michael A Kuhn","year":"2012","venue":"Journal of economic behavior & organization","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2023] Banghao Chen, Zhaofeng Zhang, Nico- las Langrené, and Shengxin Zhu. Unleashing the poten- tial of prompt engineering in large language models: a comprehensive review. arXiv preprint arXiv:2310.14735, 2023.","title":"Unleashing the potential of prompt engineering in large language models: a comprehensive review","authors":"Chen","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[De Filippo et al., 2019] Allegra De Filippo, Michele Lom- bardi, Michela Milano, et al. How to tame your anticipa- tory algorithm. In Proceedings of the Twenty-Eighth Inter- national Joint Conference on Artificial Intelligence, pages 1071-1077, 2019.","title":"How to tame your anticipatory algorithm","authors":"Allegra De Filippo, Michele Lombardi, Michela Milano","year":"2019","venue":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2019/150"},{"position":9,"raw":"[De Filippo et al., 2022a] Allegra De Filippo, Andrea Borghesi, Andrea Boscarino, and Michela Milano. Hada: An automated tool for hardware dimensioning of ai applications. Knowledge-Based Systems, 251:109199, 2022.","title":"Hada: An automated tool for hardware dimensioning of ai applications","authors":"De Filippo","year":"2022","venue":"Knowledge-Based Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[De Filippo et al., 2022b] Allegra De Filippo, Paola Mello, and Michela Milano. Do you like dancing robots? ai can tell you why. In PAIS 2022, pages 45-58. IOS Press, 2022.","title":"Do you like dancing robots? ai can tell you why","authors":"De Filippo","year":"2022","venue":"PAIS 2022","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[De Filippo et al., 2023] Allegra De Filippo, Luca Giu- liani, Eleonora Mancini, Andrea Borghesi, Paola Mello, Michela Milano, et al. Towards symbiotic creativity: A methodological approach to compare human and ai robotic dance creations. In Thirty-Second International Joint Con- ference on Artificial Intelligence, pages 5806-5814, 2023.","title":"Towards symbiotic creativity: A methodological approach to compare human and ai robotic dance creations","authors":"Allegra De Filippo, Luca Giuliani, Eleonora Mancini, Andrea Borghesi, Paola Mello, Michela Milano","year":"2023","venue":"Thirty-Second International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2023/644"},{"position":12,"raw":"[Deng and Lin, 2022] Jianyang Deng and Yijia Lin. The benefits and challenges of chatgpt: An overview. Frontiers in Computing and Intelligent Systems, 2(2):81-83, 2022.","title":"The benefits and challenges of chatgpt: An overview","authors":"Lin ; Deng, Jianyang Deng, Yijia Lin","year":"2022","venue":"Frontiers in Computing and Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ekin, 2023] Sabit Ekin. Prompt engineering for chatgpt: A quick guide to techniques, tips, and best practices. Au- thorea Preprints, 2023.","title":"Prompt engineering for chatgpt: A quick guide to techniques, tips, and best practices","authors":"Sabit Ekin","year":"2023","venue":"Authorea Preprints","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[El-Zanfaly et al., 2023] Dina El-Zanfaly, Yiwei Huang, and Yanwen Dong. Sand-in-the-loop: Investigating embodied co-creation for shared understandings of generative ai. In Companion Publication of the 2023 ACM Designing Inter- active Systems Conference, pages 256-260, 2023.","title":"Sand-in-the-loop: Investigating embodied co-creation for shared understandings of generative ai","authors":"Dina El-Zanfaly, Yiwei Huang, Yanwen Dong","year":"2023","venue":"Companion Publication of the 2023 ACM Designing Interactive Systems Conference","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Epstein et al., 2023] Ziv Epstein, Aaron Hertzmann, Inves- tigators of Human Creativity, Memo Akten, Hany Farid, Jessica Fjeld, Morgan R Frank, Matthew Groh, Laura Her- man, Neil Leach, et al. Art and the science of generative ai. Science, 380(6650):1110-1111, 2023.","title":"Art and the science of generative ai","authors":"Ziv Epstein, Aaron Hertzmann, Investigators Of Human, Memo Creativity, Hany Akten, Jessica Farid, Morgan R Fjeld, Matthew Frank, Laura Groh, Neil Herman, Leach","year":"2023","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Franceschelli and Musolesi, 2023] Giorgio Franceschelli and Mirco Musolesi. On the creativity of large language models. arXiv preprint arXiv:2304.00008, 2023.","title":"Giorgio Franceschelli and Mirco Musolesi. On the creativity of large language models","authors":"Musolesi Franceschelli","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Fui-Hoon Nah et al., 2023] Fiona Fui-Hoon Nah, Ruilin Zheng, Jingyuan Cai, Keng Siau, and Langtao Chen. Gen- erative ai and chatgpt: Applications, challenges, and ai- human collaboration, 2023.","title":"Generative ai and chatgpt: Applications, challenges, and aihuman collaboration","authors":"Fiona Fui-Hoon, Ruilin Nah, Jingyuan Zheng, Keng Cai, Langtao Siau, Chen","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Han et al., 2021] Xu Han, Zhengyan Zhang, Ning Ding, Yuxian Gu, Xiao Liu, Yuqi Huo, Jiezhong Qiu, Yuan Yao, Ao Zhang, Liang Zhang, et al. Pre-trained models: Past, present and future. AI Open, 2:225-250, 2021.","title":"Pre-trained models: Past, present and future","authors":"Xu Han, Zhengyan Zhang, Ning Ding, Yuxian Gu, Xiao Liu, Yuqi Huo, Jiezhong Qiu, Yuan Yao, Ao Zhang, Liang Zhang","year":"2021","venue":"AI Open","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Hong and Curran, 2019] Joo-Wha Hong and Nathaniel Ming Curran. Artificial intelligence, artists, and art: attitudes toward artwork produced by humans vs. artificial intelligence. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 15(2s):1-16, 2019.","title":"Artificial intelligence, artists, and art: attitudes toward artwork produced by humans vs. artificial intelligence","authors":"Curran ; Joo-Wha Hong, Nathaniel Ming Hong, Curran","year":"2019","venue":"ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Huang et al., 2020] Cheng-Zhi Anna Huang, Hendrik Vin- cent Koops, Ed Newton-Rex, Monica Dinculescu, and Carrie J Cai. Ai song contest: Human-ai co-creation in songwriting. arXiv preprint arXiv:2010.05388, 2020.","title":"Ai song contest: Human-ai co-creation in songwriting","authors":"Cheng-Zhi Anna Huang, Hendrik Vincent Koops, Ed Newton-Rex, Monica Dinculescu, Carrie J Cai","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Joshi and Chakrabarty, 2021] Manish Joshi and Sangeeta Chakrabarty. An extensive review of computational dance automation techniques and applications. Proceedings of the Royal Society A, 477(2251):20210071, 2021.","title":"An extensive review of computational dance automation techniques and applications","authors":"Manish Joshi, Sangeeta Chakrabarty","year":"2021","venue":"Proceedings of the Royal Society A","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lee et al., 2023] Philseok Lee, Shea Fyffe, Mina Son, Zi- hao Jia, and Ziyu Yao. A paradigm shift from \"human writing\" to \"machine generation\" in personality test de- velopment: An application of state-of-the-art natural lan- guage processing. Journal of Business and Psychology, 38(1):163-190, 2023.","title":"A paradigm shift from \"human writing\" to \"machine generation\" in personality test development: An application of state-of-the-art natural language processing","authors":"Lee","year":"2023","venue":"Journal of Business and Psychology","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu et al., 2020] Yuechang Liu, Dongbo Xie, Hankz Han- kui Zhuo, and Liqian Lai. Plan2dance: Planning based choreographing from music. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 13624-13625, 2020.","title":"Plan2dance: Planning based choreographing from music","authors":"Liu","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[McKight and Najab, 2010] Patrick E McKight and Julius Najab. Kruskal-wallis test. The corsini encyclopedia of psychology, pages 1-1, 2010.","title":"The corsini encyclopedia of psychology","authors":"E Patrick, Julius Mckight, Najab","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[McKnight and Najab, 2010] Patrick E McKnight and Julius Najab. Mann-whitney u test. The Corsini encyclopedia of psychology, pages 1-1, 2010.","title":"The Corsini encyclopedia of psychology","authors":"E Patrick, Julius Mcknight, Najab","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Mirowski et al., 2023] Piotr Mirowski, Kory W Mathew- son, Jaylen Pittman, and Richard Evans. Co-writing screenplays and theatre scripts with language models: Evaluation by industry professionals. In Proceedings of the 2023 CHI Conference on Human Factors in Comput- ing Systems, pages 1-34, 2023.","title":"Co-writing screenplays and theatre scripts with language models: Evaluation by industry professionals","authors":"Piotr Mirowski, Kory W Mathewson, Jaylen Pittman, Richard Evans","year":"2023","venue":"Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Naveed et al., 2023] Humza Naveed, Shi Khan, Asad Ullah afnd Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Nick Barnes, and Ajmal Mian. A comprehen- sive overview of large language models. arXiv preprint arXiv:2307.06435, 2023.","title":"A comprehensive overview of large language models","authors":"Humza Naveed, Shi Khan, Asad Ullah Afnd Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Nick Barnes, Ajmal Mian","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Oliveira et al., 2012] João Lobato Oliveira, Luis Paulo Reis, and Brígida Mónica Faria. Empiric evaluation of robot dancing framework based. TELKOMNIKA, 10(8):1701-8, 2012.","title":"Empiric evaluation of robot dancing framework based","authors":"João Lobato Oliveira, Luis, Paulo Reis, Brígida Mónica, Faria","year":"2012","venue":"TELKOMNIKA","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[OpenAI, 2023] OpenAI. Mastering temperature and top_p in chatgpt api. Technical report, 2023. Accessed January 2024.","title":"OpenAI. Mastering temperature and top_p in chatgpt api","authors":"Openai","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[OpenAI, 2024] OpenAI. Six strategies for getting better re- sults. Technical report, 2024. Accessed January 2024.","title":"OpenAI. Six strategies for getting better results","authors":"Openai","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Ouyang et al., 2022] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instruc- tions with human feedback. Advances in Neural Informa- tion Processing Systems, 35:27730-27744, 2022.","title":"Training language models to follow instructions with human feedback","authors":"Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Pignatiello et al., 2020] Grant A Pignatiello, Richard J Mar- tin, and Ronald L Hickman Jr. Decision fatigue: A con- ceptual analysis. Journal of health psychology, 25(1):123- 135, 2020.","title":"Decision fatigue: A conceptual analysis","authors":"Richard J Grant A Pignatiello, Ronald L Martin, Hickman","year":"2020","venue":"Journal of health psychology","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Ramos et al., 2015] Oscar E Ramos, Nicolas Mansard, Olivier Stasse, Christophe Benazeth, Sovannara Hak, and Layale Saab. Dancing humanoid robots: Systematic use of osid to compute dynamically consistent movements fol- lowing a motion capture pattern. IEEE Robotics & Au- tomation Magazine, 22(4):16-26, 2015.","title":"Dancing humanoid robots: Systematic use of osid to compute dynamically consistent movements following a motion capture pattern","authors":"Ramos","year":"2015","venue":"IEEE Robotics & Automation Magazine","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Sætra, 2023] Henrik Skaug Sætra. Generative ai: Here to stay, but for good? Technology in Society, 75:102372, 2023.","title":"Generative ai: Here to stay, but for good","authors":"Henrik Skaug Sætra, Sætra","year":"2023","venue":"Technology in Society","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Sagasti, 2019] Francisco Sagasti. Information technology and the arts: the evolution of computer choreography dur- ing the last half century. Dance Chronicle, 42(1):1-52, 2019.","title":"Francisco Sagasti. Information technology and the arts: the evolution of computer choreography during the last half century","authors":"Sagasti","year":"2019","venue":"Dance Chronicle","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Saravia, 2022] Elvis Saravia. Prompt Engineering Guide. https://github.com/dair-ai/Prompt-Engineering-Guide, 12 2022.","title":"Elvis Saravia","authors":"Saravia","year":"2022","venue":"Prompt Engineering Guide","doi":null,"url":"https://github.com/dair-ai/Prompt-Engineering-Guide,122022","internal_url":null},{"position":37,"raw":"[Scaringella et al., 2006] Nicolas Scaringella, Giorgio Zoia, and Daniel Mlynek. Automatic genre classification of mu- sic content: a survey. IEEE Signal Processing Magazine, 23(2):133-141, 2006.","title":"Automatic genre classification of music content: a survey","authors":"Nicolas Scaringella, Giorgio Zoia, Daniel Mlynek","year":"2006","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Shinozaki et al., 2007] Kuniya Shinozaki, Akitsugu Iwatani, and Ryohei Nakatsu. Concept and construction of a dance robot system. In Proceedings of the 2nd international conference on Digital interactive media in entertainment and arts, pages 161-164, 2007.","title":"Concept and construction of a dance robot system","authors":"Shinozaki","year":"2007","venue":"Proceedings of the 2nd international conference on Digital interactive media in entertainment and arts","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Shinozaki et al., 2008] Kuniya Shinozaki, Akitsugu Iwatani, and Ryohei Nakatsu. Construction and evalua- tion of a robot dance system. In Entertainment Computing Symposium, pages 83-94. Springer, 2008.","title":"Construction and evaluation of a robot dance system","authors":"Shinozaki","year":"2008","venue":"Entertainment Computing Symposium","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Stiennon et al., 2020] Nisan Stiennon, Long Ouyang, Jef- frey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008-3021, 2020.","title":"Learning to summarize with human feedback","authors":"Stiennon","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Sturm, 2012] Bob L Sturm. Two systems for automatic mu- sic genre recognition: What are they really recognizing? In Proceedings of the second international ACM work- shop on Music information retrieval with user-centered and multimodal strategies, pages 69-74, 2012.","title":"Two systems for automatic music genre recognition: What are they really recognizing?","authors":"L Bob, Sturm","year":"2012","venue":"Proceedings of the second international ACM workshop on Music information retrieval with user-centered and multimodal strategies","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Wang et al., 2023a] Chi Wang, Susan Xueqing Liu, and Ahmed H Awadallah. Cost-effective hyperparameter op- timization for large language model generation inference. arXiv preprint arXiv:2303.04673, 2023.","title":"Cost-effective hyperparameter optimization for large language model generation inference","authors":"Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Wang et al., 2023b] Xiaolei Wang, Xinyu Tang, Wayne Xin Zhao, Jingyuan Wang, and Ji-Rong Wen. Rethinking the evaluation for conversational recommendation in the era of large language models. arXiv preprint arXiv:2305.13112, 2023.","title":"Rethinking the evaluation for conversational recommendation in the era of large language models","authors":"Wang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wang, 2022] Zhigang Wang. Music choreography algo- rithm based on feature matching and fragment segmenta- tion. Computational Intelligence and Neuroscience, 2022, 2022.","title":"Music choreography algorithm based on feature matching and fragment segmentation","authors":"Zhigang Wang","year":"2022","venue":"Computational Intelligence and Neuroscience","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Wei et al., 2023] Chengwei Wei, Yun-Cheng Wang, Bin Wang, and C-C Jay Kuo. An overview on language mod- els: Recent developments and outlook. arXiv preprint arXiv:2303.05759, 2023.","title":"An overview on language models: Recent developments and outlook","authors":"Chengwei Wei, Yun-Cheng Wang, Bin Wang, C-C Jay Kuo","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[White et al., 2023] Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C Schmidt. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382, 2023.","title":"A prompt pattern catalog to enhance prompt engineering with chatgpt","authors":"Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, Douglas C Schmidt","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Wu and Hu, 2023] Yangjian Wu and Gang Hu. Explor- ing prompt engineering with gpt language models for document-level machine translation: Insights and find- ings. In Proceedings of the Eighth Conference on Machine Translation, pages 166-169, 2023.","title":"Exploring prompt engineering with gpt language models for document-level machine translation: Insights and findings","authors":"Hu Wu, Yangjian Wu, Gang Hu","year":"2023","venue":"Proceedings of the Eighth Conference on Machine Translation","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Zamfirescu-Pereira et al., 2023] JD Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang. Why johnny can't prompt: how non-ai experts try (and fail) to design llm prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1-21, 2023.","title":"Why johnny can't prompt: how non-ai experts try (and fail) to design llm prompts","authors":"Zamfirescu-Pereira","year":"2023","venue":"Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Zhou et al., 2023] Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911, 2023.","title":"Instruction-following evaluation for large language models","authors":"Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, Le Hou","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}