{"type":"ScholarlyArticle","title":"Evaluating GPT-3 Generated Explanations for Hateful Content Moderation","authors":[{"name":"Han Wang","slug":"han-wang","given":"Han","family":"Wang","affiliation":"Singapore University of Technology and Design"},{"name":"Ming Shan Hee","slug":"ming-shan-hee","given":"Ming Shan","family":"Hee","affiliation":"Singapore University of Technology and Design"},{"name":"Md Rabiul Awal","slug":"md-rabiul-awal","given":"Md Rabiul","family":"Awal","affiliation":"Mila -Quebec AI Institute han"},{"name":"Kenny Tsu Wei Choo","slug":"kenny-tsu-wei-choo","given":"Kenny Tsu Wei","family":"Choo","affiliation":"Singapore University of Technology and Design"},{"name":"Roy Ka-Wei Lee","slug":"roy-ka-wei-lee","given":"Roy Ka-Wei","family":"Lee","affiliation":"Singapore University of Technology and Design"}],"abstract":"Recent research has focused on using large language models (LLMs) to generate explanations for hate speech through fine-tuning or prompting. Despite the growing interest in this area, these generated explanations' effectiveness and potential limitations remain poorly understood. A key concern is that these explanations, generated by LLMs, may lead to erroneous judgments about the nature of flagged content by both users and content moderators. For instance, an LLM-generated explanation might inaccurately convince a content moderator that a benign piece of content is  hateful. In light of this, we propose an analytical framework for examining hate speech explanations and conducted an extensive survey on evaluating such explanations. Specifically, we prompted GPT-3 to generate explanations for both hateful and non-hateful content, and a survey was conducted with 2,400 unique respondents to evaluate the generated explanations. Our findings reveal that (1) human evaluators rated the GPT-generated explanations as high quality in terms of linguistic fluency, informativeness, persuasiveness, and logical soundness, (2) the persuasive nature of these explanations, however, varied depending on the prompting strategy employed, and (3) this persuasiveness may result in incorrect judgments about the hatefulness of the content. Our study underscores the need for caution in applying LLM-generated explanations for content moderation. Code and results are available at https://github.com/Social-AI-Studio/GPT3-HateEval.","doi":"10.24963/ijcai.2023/694","url":"https://ijcai.org/proceedings/2023/694","pdf":"https://ijcai.org/proceedings/2023/0694.pdf","markdown":"https://ijcai.org/proceedings/2023/694.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/694","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI for Good","track":"AI for Good","number":694,"pages":{"first":6255,"last":6263},"keywords":["AI for Good: Natural Language Processing","AI for Good: Data Mining","AI for Good: AI Ethics, Trust, Fairness"],"videos":[],"references":[{"position":1,"raw":"[Ashida and Komachi, 2022] Mana Ashida and Mamoru Ko- machi. Towards automatic generation of messages coun- tering online hate speech and microaggressions. In Pro- ceedings of the Sixth Workshop on Online Abuse and Harms (WOAH), Stroudsburg, PA, USA, 2022. Associa- tion for Computational Linguistics (ACL).","title":"Mana Ashida and Mamoru Komachi. Towards automatic generation of messages countering online hate speech and microaggressions","authors":"Komachi Ashida","year":"2022","venue":"Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Awal et al., 2021] Md Rabiul Awal, Rui Cao, Roy Ka-Wei Lee, and Sandra Mitrovic. Angrybert: Joint learning tar- get and emotion for hate speech detection. arXiv preprint arXiv:2103.11800, 2021.","title":"Angrybert: Joint learning target and emotion for hate speech detection","authors":"Md Rabiul Awal, Rui Cao, Roy, Ka-Wei Lee, Sandra Mitrovic","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Awal et al., 2023] Md Rabiul Awal, Roy Ka-Wei Lee, Es- haan Tanwar, Tanmay Garg, and Tanmoy Chakraborty. Model-agnostic meta-learning for multilingual hate speech detection. IEEE Transactions on Computational Social Systems, 2023.","title":"Model-agnostic meta-learning for multilingual hate speech detection","authors":"Md Rabiul, Awal, Roy, Ka-Wei Lee, Eshaan Tanwar, Tanmay Garg, Tanmoy Chakraborty","year":"2023","venue":"IEEE Transactions on Computational Social Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Badjatiya et al., 2017] Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva Varma. Deep learning for hate speech detection in tweets. In Proceedings of the 26th international conference on World Wide Web companion, April 2017.","title":"Deep learning for hate speech detection in tweets","authors":"Badjatiya","year":"2017","venue":"Proceedings of the 26th international conference on World Wide Web companion","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Brown et al., 2020] Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, and Neelakantan et al. Language models are Few-Shot learners. Advances in neural information processing sys- tems, May 2020.","title":"Language models are Few-Shot learners","authors":"Benjamin Tom B Brown, Nick Mann, Melanie Ryder, Jared Subbiah, Prafulla Kaplan, Neelakantan Dhariwal","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Buchanan et al., 2021] Ben Buchanan, Andrew Lohn, Micah Musser, and Katerina Sedova. Truth, lies, and au- tomation. Center for Security and Emerging Technology, 1(1), 2021.","title":"Truth, lies, and automation","authors":"Ben Buchanan, Andrew Lohn, Micah Musser, Katerina Sedova","year":"2021","venue":"Center for Security and Emerging Technology","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Camburu et al., 2018] Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blun- som. e-snli: Natural language inference with natural language explanations. Advances in Neural Information Processing Systems, 31, 2018.","title":"e-snli: Natural language inference with natural language explanations","authors":"Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, Phil Blunsom","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Cao and Lee, 2020] Rui Cao and Roy Ka-Wei Lee. Hate- gan: Adversarial generative-based data augmentation for hate speech detection. In Proceedings of the 28th Inter- national Conference on Computational Linguistics, pages 6327-6338, 2020.","title":"Hategan: Adversarial generative-based data augmentation for hate speech detection","authors":"Rui Cao, Roy, Ka-Wei Lee","year":"2020","venue":"Proceedings of the 28th International Conference on Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cao et al., 2020] Rui Cao, Roy Ka-Wei Lee, and Tuan-Anh Hoang. DeepHate: Hate speech detection via multi- faceted text representations. In 12th ACM conference on web science, March 2020.","title":"DeepHate: Hate speech detection via multifaceted text representations","authors":"Rui Cao, Roy, Ka-Wei Lee, Tuan-Anh Hoang","year":"2020","venue":"12th ACM conference on web science","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Chiu et al., 2021] Ke-Li Chiu, Annie Collins, and Ro- han Alexander. Detecting hate speech with GPT-3. arXiv:2103.12407, March 2021.","title":"Detecting hate speech with GPT-3","authors":"Ke-Li Chiu, Annie Collins, Rohan Alexander","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Davidson et al., 2017] Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. Automated hate speech detection and the problem of offensive language. In Pro- ceedings of the international AAAI conference on web and social media, March 2017.","title":"Automated hate speech detection and the problem of offensive language","authors":"Thomas Davidson, Dana Warmsley, Michael Macy, Ingmar Weber","year":"2017","venue":"Proceedings of the international AAAI conference on web and social media","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Djuric et al., 2015] Nemanja Djuric, Jing Zhou, Robin Mor- ris, Mihajlo Grbovic, Vladan Radosavljevic, and Narayan Bhamidipati. Hate speech detection with comment embed- dings. In Proceedings of the 24th International Conference on World Wide Web, New York, NY, USA, May 2015. As- sociation for Computing Machinery (ACM).","title":"Hate speech detection with comment embeddings","authors":"Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, Narayan Bhamidipati","year":"2015","venue":"Proceedings of the 24th International Conference on World Wide Web","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Dou et al., 2022] Yao Dou, Maxwell Forbes, Rik Koncel- Kedziorski, Noah Smith, and Yejin Choi. Is GPT-3 text indistinguishable from human text? scarecrow: A frame- work for scrutinizing machine text. In Proceedings of the 60th Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), Stroudsburg, PA, USA, 2022. Association for Computational Linguis- tics (ACL).","title":"Is GPT-3 text indistinguishable from human text? scarecrow: A framework for scrutinizing machine text","authors":"Yao Dou, Maxwell Forbes, Rik Koncel-Kedziorski, Noah Smith, Yejin Choi","year":"2022","venue":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[ElSherief et al., 2021] Mai ElSherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Mun- mun De Choudhury, and Diyi Yang. Latent hatred: A benchmark for understanding implicit hate speech. arXiv preprint arXiv:2109.05322, September 2021.","title":"Latent hatred: A benchmark for understanding implicit hate speech","authors":"Elsherief","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Fortuna and Nunes, 2018] Paula Fortuna and Sérgio Nunes. A survey on automatic detection of hate speech in text. ACM Comput. Surv., 51(4):1-30, July 2018.","title":"Paula Fortuna and Sérgio Nunes. A survey on automatic detection of hate speech in text","authors":"Fortuna, Nunes","year":"2018","venue":"ACM Comput. Surv","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gillespie, 2018] Tarleton Gillespie. Custodians of the Inter- net: Platforms, content moderation, and the hidden deci- sions that shape social media. Yale University Press, 2018.","title":"Tarleton Gillespie. Custodians of the Internet: Platforms, content moderation, and the hidden decisions that shape social media","authors":"Gillespie","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Haimson et al., 2021] Oliver L Haimson, Daniel Del- monaco, Peipei Nie, and Andrea Wegner. Dispropor- tionate removals and differing content moderation expe- riences for conservative, transgender, and black social me- dia users: Marginalization and moderation gray areas. Proc. ACM Hum. Comput. Interact., 5(CSCW2):1-35, Oc- tober 2021.","title":"Disproportionate removals and differing content moderation experiences for conservative, transgender, and black social media users: Marginalization and moderation gray areas","authors":"Daniel Oliver L Haimson, Peipei Delmonaco, Andrea Nie, Wegner","year":"2021","venue":"Proc. ACM Hum. Comput. Interact","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Huang et al., 2022] Fan Huang, Haewoon Kwak, and Jisun An. Chain of explanation: New prompting method to generate higher quality natural language explanation for implicit hate speech. arXiv preprint arXiv:2209.04889, September 2022.","title":"Chain of explanation: New prompting method to generate higher quality natural language explanation for implicit hate speech","authors":"Fan Huang, Haewoon Kwak, Jisun An","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kojima et al., 2022] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are Zero-Shot reasoners. arXiv preprint arXiv:2205.11916, May 2022.","title":"Large language models are Zero-Shot reasoners","authors":"Takeshi Kojima, Shane Shixiang, Machel Gu, Yutaka Reid, Yusuke Matsuo, Iwasawa","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lampe and Resnick, 2004] Cliff Lampe and Paul Resnick. Slash (dot) and burn: distributed moderation in a large on- line conversation space. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 543-550. 2004.","title":"Cliff Lampe and Paul Resnick. Slash (dot) and burn: distributed moderation in a large online conversation space","authors":"Resnick Lampe","year":"2004","venue":"Proceedings of the SIGCHI conference on Human factors in computing systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lin et al., 2021] Ken-Yu Lin, Roy Ka-Wei Lee, Wei Gao, and Wen-Chih Peng. Early prediction of hate speech prop- agation. In 2021 International Conference on Data Mining Workshops (ICDMW), pages 967-974. Institute of Electri- cal and Electronics Engineers (IEEE), 2021.","title":"Early prediction of hate speech propagation","authors":"Ken-Yu Lin, Roy, Ka-Wei Lee, Wei Gao, Wen-Chih Peng","year":"2021","venue":"2021 International Conference on Data Mining Workshops (ICDMW)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Markov et al., 2022] Todor Markov, Chong Zhang, Sand- hini Agarwal, Tyna Eloundou, Teddy Lee, Steven Adler, Angela Jiang, and Lilian Weng. A holistic approach to un- desired content detection in the real world. arXiv preprint arXiv:2208.03274, August 2022.","title":"A holistic approach to undesired content detection in the real world","authors":"Todor Markov, Chong Zhang, Sandhini Agarwal, Tyna Eloundou, Teddy Lee, Steven Adler, Angela Jiang, Lilian Weng","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Mathew et al., 2021] Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. HateXplain: A benchmark dataset for explainable hate speech detection. Proc. Conf. AAAI Artif. Intell., 35(17):14867-14875, May 2021.","title":"HateXplain: A benchmark dataset for explainable hate speech detection","authors":"Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, Animesh Mukherjee","year":"2021","venue":"Proc. Conf. AAAI Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Meng et al., 2022] Qing Meng, Tharun Suresh, Roy Ka- Wei Lee, and Tanmoy Chakraborty. Predicting hate in- tensity of twitter conversation threads. arXiv preprint arXiv:2206.08406, 2022.","title":"Predicting hate intensity of twitter conversation threads","authors":"Qing Meng, Tharun Suresh, Roy, Ka-Wei Lee, Tanmoy Chakraborty","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Park et al., 2018] Ji Ho Park, Jamin Shin, and Pascale Fung. Reducing gender bias in abusive language detection. arXiv preprint arXiv:1808.07231, August 2018.","title":"Reducing gender bias in abusive language detection","authors":"Ji Ho, Park, Jamin Shin, Pascale Fung","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Petroni et al., 2019] Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. Language models as knowl- edge bases? arXiv preprint arXiv:1909.01066, September 2019.","title":"Language models as knowledge bases","authors":"Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, Sebastian Riedel","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Poletto et al., 2019] Fabio Poletto, Valerio Basile, Cristina Bosco, Viviana Patti, and Maria Stranisci. Annotating hate speech: Three schemes at comparison. In CEUR WORK- SHOP PROCEEDINGS, volume 2481, pages 1-8. CEUR- WS, 2019.","title":"Annotating hate speech: Three schemes at comparison","authors":"Fabio Poletto, Valerio Basile, Cristina Bosco, Viviana Patti, Maria Stranisci","year":"2019","venue":"CEUR WORK-SHOP PROCEEDINGS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI blog, 1(8), 2019.","title":"Language models are unsupervised multitask learners","authors":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever","year":"2019","venue":"OpenAI blog","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Sanh et al., 2021] Victor Sanh, Albert Webson, Colin Raf- fel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, and Chaffin et al. Multitask prompted training enables zero- shot task generalization. arXiv preprint arXiv:2110.08207, October 2021.","title":"Multitask prompted training enables zeroshot task generalization","authors":"Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Chaffin","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Sap et al., 2019] Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A Smith. The risk of racial bias in hate speech detection. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1668-1678. 2019.","title":"The risk of racial bias in hate speech detection","authors":"Sap","year":"2019","venue":"Proceedings of the 57th annual meeting of the association for computational linguistics","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Sap et al., 2020] Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A Smith, and Yejin Choi. Social bias frames: Reasoning about social and power implications of language. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Strouds- burg, PA, USA, 2020. Association for Computational Lin- guistics (ACL).","title":"Social bias frames: Reasoning about social and power implications of language","authors":"Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A Smith, Yejin Choi","year":"2020","venue":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Saunders et al., 2022] William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. Self-critiquing models for assisting human eval- uators. arXiv preprint arXiv:2206.05802, June 2022.","title":"Self-critiquing models for assisting human evaluators","authors":"William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, Jan Leike","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Schmidt and Wiegand, 2017] Anna Schmidt and Michael Wiegand. A survey on hate speech detection using natural language processing. In Proceedings of the Fifth Interna- tional Workshop on Natural Language Processing for So- cial Media, Stroudsburg, PA, USA, 2017. Association for Computational Linguistics (ACL).","title":"A survey on hate speech detection using natural language processing","authors":"Anna Schmidt, Michael Wiegand","year":"2017","venue":"Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Steiger et al., 2021] Timir J Steiger, Miriah, Sukrit Bharucha, Martin J Venkatagiri, and Matthew Riedl. The psychological well-being of content moderators: the emotional labor of commercial moderation and avenues for improving support. In Proceedings of the 2021 CHI conference on human factors in computing systems, pages 1-14. 2021.","title":"The psychological well-being of content moderators: the emotional labor of commercial moderation and avenues for improving support","authors":"J Timir, Steiger, Sukrit Miriah, Martin J Bharucha, Matthew Venkatagiri, Riedl","year":"2021","venue":"Proceedings of the 2021 CHI conference on human factors in computing systems","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Suzor et al., 2019] Nicolas P Suzor, Andrew West, and Jil- lian Quodling. What do we mean when we talk about transparency? toward meaningful transparency in com- mercial content moderation. International Journal of Communication Systems (IJCS), 13, 2019.","title":"What do we mean when we talk about transparency? toward meaningful transparency in commercial content moderation","authors":"Nicolas P Suzor, Andrew West, Jillian Quodling","year":"2019","venue":"International Journal of Communication Systems (IJCS)","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Tekiroglu et al., 2020] Serra Sinem Tekiroglu, Yi-Ling Chung, and Marco Guerini. Generating counter narra- tives against online hate speech: Data and strategies. arXiv preprint arXiv:2004.04216, April 2020.","title":"Generating counter narratives against online hate speech: Data and strategies","authors":"Serra Sinem Tekiroglu, Yi-Ling Chung, Marco Guerini","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Vaccaro et al., 2020] Kristen Vaccaro, Christian Sandvig, and Karrie Karahalios. At the end of the day facebook does what ItWants\" how users experience contesting algo- rithmic content moderation. Proceedings of the ACM on human-computer interaction, 4(CSCW2):1-22, 2020.","title":"At the end of the day facebook does what ItWants\" how users experience contesting algorithmic content moderation","authors":"Kristen Vaccaro, Christian Sandvig, Karrie Karahalios","year":"2020","venue":"Proceedings of the ACM on human-computer interaction","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wang, 2022] Yingshan Wang, Yau-Shian. Toxicity de- tection with generative prompt-based inference. arXiv preprint arXiv:2205.12390, 2022.","title":"Toxicity detection with generative prompt-based inference","authors":"Yingshan Wang, Yau-Shian","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Watanabe et al., 2018] Hajime Watanabe, Mondher Bouaz- izi, and Tomoaki Ohtsuki. Hate speech on twitter: A prag- matic approach to collect hateful and offensive expres- sions and perform hate speech detection. IEEE Access, 6:13825-13835, 2018.","title":"Hate speech on twitter: A pragmatic approach to collect hateful and offensive expressions and perform hate speech detection","authors":"Hajime Watanabe, Mondher Bouazizi, Tomoaki Ohtsuki","year":"2018","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Wei et al., 2022] Jason Wei, Xuezhi Wang, Dale Schuur- mans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large lan- guage models. arXiv preprint arXiv:2201.11903, 2022.","title":"Chain of thought prompting elicits reasoning in large language models","authors":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, Denny Zhou","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Wiegreffe et al., 2022] Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, and Yejin Choi. Re- framing human-AI collaboration for generating free-text explanations. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Stroudsburg, PA, USA, 2022. Association for Computational Linguistics (ACL).","title":"Reframing human-AI collaboration for generating free-text explanations","authors":"Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, Yejin Choi","year":"2022","venue":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhang et al., 2022] Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493, October 2022.","title":"Automatic chain of thought prompting in large language models","authors":"Zhang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhao et al., 2021] Tony Z Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Im- proving few-shot performance of language models. In In- ternational Conference on Machine Learning, February 2021.","title":"Calibrate before use: Improving few-shot performance of language models","authors":"Tony Z Zhao, Eric Wallace, Shi Feng, Dan Klein, Sameer Singh","year":"2021","venue":"ternational Conference on Machine Learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}