{"type":"ScholarlyArticle","title":"Decoding the Underlying Meaning of Multimodal Hateful Memes","authors":[{"name":"Ming Shan Hee","slug":"ming-shan-hee","given":"Ming Shan","family":"Hee","affiliation":"Singapore University of Technology and Design"},{"name":"Wen-Haw Chong","slug":"wen-haw-chong","given":"Wen-Haw","family":"Chong","affiliation":"Singapore Management University"},{"name":"Roy Ka-Wei Lee","slug":"roy-ka-wei-lee","given":"Roy Ka-Wei","family":"Lee","affiliation":"Singapore University of Technology and Design"}],"abstract":"Recent studies have proposed models that yielded promising performance for the hateful meme classification task. Nevertheless, these proposed models do not generate interpretable explanations that uncover the underlying meaning and support the classification output. A major reason for the lack of explainable hateful meme methods is the absence of a hateful meme dataset that contains ground truth explanations for benchmarking or training. Intuitively, having such explanations can educate and assist content moderators in interpreting and removing flagged hateful memes. This paper address this research gap by introducing Hateful meme with Reasons Dataset (HatReD), which is a new multimodal hateful meme dataset annotated with the underlying hateful contextual reasons. We also define a new conditional generation task that aims to automatically generate underlying reasons to explain hateful memes and establish the baseline performance of state-of-the-art pre-trained language models on this task.  We further demonstrate the usefulness of HatReD by analyzing the challenges of the new conditional generation task in explaining memes in seen and unseen domains. The dataset and benchmark models are made available here: https://github.com/Social-AI-Studio/HatRed","doi":"10.24963/ijcai.2023/665","url":"https://ijcai.org/proceedings/2023/665","pdf":"https://ijcai.org/proceedings/2023/0665.pdf","markdown":"https://ijcai.org/proceedings/2023/665.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/665","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI for Good","track":"AI for Good","number":665,"pages":{"first":5995,"last":6003},"keywords":["AI for Good: Knowledge Representation and Reasoning","AI for Good: Natural Language Processing","AI for Good: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Anderson et al., 2018] Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for im- age captioning and visual question answering. In CVPR, 2018.","title":"Bottom-up and top-down attention for image captioning and visual question answering","authors":"Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, Lei Zhang","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2022] Rui Cao, Roy Ka-Wei Lee, Wen-Haw Chong, and Jing Jiang. Prompting for multimodal hateful meme classification. In Proceedings of the 2022 Confer- ence on Empirical Methods in Natural Language Process- ing, pages 321-332, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguis- tics.","title":"Prompting for multimodal hateful meme classification","authors":"Rui Cao, Roy, Ka-Wei Lee, Wen-Haw Chong, Jing Jiang","year":"2022","venue":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cho et al., 2021] Jaemin Cho, Jie Lei, Hao Tan, and Mohit Bansal. Unifying vision-and-language tasks via text gen- eration. In ICML, 2021.","title":"Unifying vision-and-language tasks via text generation","authors":"Jaemin Cho, Jie Lei, Hao Tan, Mohit Bansal","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[ElSherief et al., 2021] Mai ElSherief, Caleb Ziems, David Muchlinski, Vaishnavi Anupindi, Jordyn Seybolt, Mun- mun De Choudhury, and Diyi Yang. Latent hatred: A benchmark for understanding implicit hate speech. In Pro- ceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 345-363, Online and Punta Cana, Dominican Republic, November 2021. Asso- ciation for Computational Linguistics.","title":"Latent hatred: A benchmark for understanding implicit hate speech","authors":"Elsherief","year":"2021","venue":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fersini et al., 2022] Elisabetta Fersini, Francesca Gasparini, Giulia Rizzi, Aurora Saibene, Berta Chulvi, Paolo Rosso, Alyssa Lees, and Jeffrey Sorensen. Semeval-2022 task 5: Multimedia automatic misogyny identification. In Pro- ceedings of the 16th International Workshop on Seman- tic Evaluation (SemEval-2022). Association for Computa- tional Linguistics, 2022.","title":"Semeval-2022 task 5: Multimedia automatic misogyny identification","authors":"Elisabetta Fersini, Francesca Gasparini, Giulia Rizzi, Aurora Saibene, Berta Chulvi, Paolo Rosso, Alyssa Lees, Jeffrey Sorensen","year":"2022","venue":"Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gasparini et al., 2021] Francesca Gasparini, Giulia Rizzi, Aurora Saibene, and Elisabetta Fersini. Benchmark dataset of memes with text transcriptions for automatic detection of multi-modal misogynistic content. arXiv preprint arXiv:2106.08409, 2021.","title":"Benchmark dataset of memes with text transcriptions for automatic detection of multi-modal misogynistic content","authors":"Francesca Gasparini, Giulia Rizzi, Aurora Saibene, Elisabetta Fersini","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hee et al., 2022] Ming Shan Hee, Roy Ka-Wei Lee, and Wen-Haw Chong. On explaining multimodal hateful meme detection models. In Proceedings of the ACM Web Conference 2022, WWW '22, page 3651-3655, New York, NY, USA, 2022. Association for Computing Ma- chinery.","title":"On explaining multimodal hateful meme detection models","authors":"Ming Shan, Hee, Roy, Ka-Wei Lee, Wen-Haw Chong","year":"2022","venue":"Proceedings of the ACM Web Conference 2022","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kärkkäinen and Joo, 2019] Kimmo Kärkkäinen and Jungseock Joo. Fairface: Face attribute dataset for balanced race, gender, and age. arXiv preprint arXiv:1908.04913, 2019.","title":"Fairface: Face attribute dataset for balanced race, gender, and age","authors":"Joo Kärkkäinen, Kimmo Kärkkäinen, Jungseock Joo","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kiela et al., 2020] Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. The hateful memes challenge: Detecting hate speech in multimodal memes. Advances in Neural Information Processing Systems, 33:2611-2624, 2020.","title":"The hateful memes challenge: Detecting hate speech in multimodal memes","authors":"Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, Davide Testuggine","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kiela et al., 2021] Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Casey A. Fitzpatrick, Peter Bull, Greg Lipstein, Tony Nelli, Ron Zhu, Niklas Muennighoff, Riza Velioglu, Jewgeni Rose, Phillip Lippe, Nithin Holla, Shantanu Chandra, Santhosh Rajamanickam, Georgios Antoniou, Ekaterina Shutova, Helen Yannakoudakis, Vlad Sandulescu, Umut Ozertem, Patrick Pantel, Lucia Specia, and Devi Parikh. The hate- ful memes challenge: Competition report. In Hugo Jair Escalante and Katja Hofmann, editors, Proceedings of the NeurIPS 2020 Competition and Demonstration Track, vol- ume 133 of Proceedings of Machine Learning Research, pages 344-360. PMLR, 06-12 Dec 2021.","title":"The hateful memes challenge: Competition report","authors":"Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Casey A Fitzpatrick, Peter Bull, Greg Lipstein, Tony Nelli, Ron Zhu, Niklas Muennighoff, Riza Velioglu, Jewgeni Rose, Phillip Lippe, Nithin Holla, Shantanu Chandra, Santhosh Rajamanickam, Georgios Antoniou, Ekaterina Shutova, Helen Yannakoudakis, Vlad Sandulescu, Umut Ozertem, Patrick Pantel, Lucia Specia, Devi Parikh","year":"2021","venue":"Proceedings of the NeurIPS 2020 Competition and Demonstration Track","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lee et al., 2021] Roy Ka-Wei Lee, Rui Cao, Ziqing Fan, Jing Jiang, and Wen-Haw Chong. Disentangling hate in online memes. In Proceedings of the 29th ACM Interna- tional Conference on Multimedia, pages 5138-5147, 2021.","title":"Disentangling hate in online memes","authors":"Ka-Wei Roy, Rui Lee, Ziqing Cao, Jing Fan, Wen-Haw Jiang, Chong","year":"2021","venue":"Proceedings of the 29th ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2019] Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557, 2019.","title":"Visualbert: A simple and performant baseline for vision and language","authors":"Li","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin, 2004] Chin-Yew Lin. ROUGE: A package for auto- matic evaluation of summaries. In Text Summarization Branches Out, pages 74-81, Barcelona, Spain, July 2004. Association for Computational Linguistics.","title":"ROUGE: A package for automatic evaluation of summaries","authors":"Chin-Yew Lin","year":"2004","venue":"Text Summarization Branches Out","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lippe et al., 2020] Phillip Lippe, Nithin Holla, Shantanu Chandra, Santhosh Rajamanickam, Georgios Antoniou, Ekaterina Shutova, and Helen Yannakoudakis. A mul- timodal framework for the detection of hateful memes. arXiv preprint arXiv:2012.12871, 2020.","title":"A multimodal framework for the detection of hateful memes","authors":"Phillip Lippe, Nithin Holla, Shantanu Chandra, Santhosh Rajamanickam, Georgios Antoniou, Ekaterina Shutova, Helen Yannakoudakis","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.","title":"Roberta: A robustly optimized bert pretraining approach","authors":"Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mathias et al., 2021] Lambert Mathias, Shaoliang Nie, Aida Mostafazadeh Davani, Douwe Kiela, Vinodkumar Prabhakaran, Bertie Vidgen, and Zeerak Waseem. Find- ings of the WOAH 5 shared task on fine grained hateful memes detection. In Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021), pages 201-206, Online, August 2021. Association for Computational Lin- guistics. [Mokady et al., 2021]","title":"Findings of the WOAH 5 shared task on fine grained hateful memes detection","authors":"Lambert Mathias, Shaoliang Nie, Aida Mostafazadeh Davani, Douwe Kiela, Bertie Vinodkumar Prabhakaran, Zeerak Vidgen, Waseem","year":"2021","venue":"Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"Ron Mokady, Amir Hertz, and Amit H. Bermano. Clipcap: CLIP prefix for image cap- tioning. CoRR, 2021.","title":"Clipcap: CLIP prefix for image captioning","authors":"Ron Mokady, Amir Hertz, Amit H Bermano","year":"2021","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Muennighoff, 2020] Niklas Muennighoff. Vilio: State-of- the-art visio-linguistic models applied to hateful memes. CoRR, 2020.","title":"Niklas Muennighoff. Vilio: State-ofthe-art visio-linguistic models applied to hateful memes","authors":"Muennighoff","year":"2020","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Papineni et al., 2002] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for au- tomatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- tational Linguistics, pages 311-318, Philadelphia, Penn- sylvania, USA, July 2002. Association for Computational Linguistics.","title":"Bleu: a method for automatic evaluation of machine translation","authors":"Kishore Papineni, Salim Roukos, Todd Ward, Wei-Jing Zhu","year":"2002","venue":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pramanick et al., 2021a] Shraman Pramanick, Dimi- tar Dimitrov, Rituparna Mukherjee, Shivam Sharma, Md. Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. Detecting harmful memes and their targets. In Findings of the Association for Computational Linguistics: ACL/IJCNLP, pages 2783-2796, 2021.","title":"Detecting harmful memes and their targets","authors":"Pramanick","year":"2021","venue":"Findings of the Association for Computational Linguistics: ACL/IJCNLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pramanick et al., 2021b] Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md. Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. MOMENTA: A multi- modal framework for detecting harmful memes and their targets. In Findings of the Association for Computational Linguistics: EMNLP, pages 4439-4455, 2021.","title":"MOMENTA: A multimodal framework for detecting harmful memes and their targets","authors":"Pramanick","year":"2021","venue":"Findings of the Association for Computational Linguistics: EMNLP","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.","title":"Language models are unsupervised multitask learners","authors":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever","year":"2019","venue":"OpenAI blog","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Raffel et al., 2020] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67, 2020.","title":"Exploring the limits of transfer learning with a unified text-to-text transformer","authors":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu","year":"2020","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Roberts et al., 2020] Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parameters of a language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), pages 5418-5426, Online, November 2020. Association for Computational Linguis- tics. Rothe et al., 2020] Sascha Rothe, Shashi Narayan, and Ali- aksei Severyn. Leveraging pre-trained checkpoints for se- quence generation tasks. Transactions of the Association for Computational Linguistics, 8:264-280, 2020.","title":"How much knowledge can you pack into the parameters of a language model?","authors":"Adam Roberts, Colin Raffel, Noam Shazeer","year":"2020","venue":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sap et al., 2020] Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith, and Yejin Choi. Social bias frames: Reasoning about social and power implications of language. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5477-5490, Online, July 2020. Association for Compu- tational Linguistics.","title":"Social bias frames: Reasoning about social and power implications of language","authors":"Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A Smith, Yejin Choi","year":"2020","venue":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sharma et al., 2022a] Shivam Sharma, Md Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. Disarm: Detect- ing the victims targeted by harmful memes. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1572-1588, 2022.","title":"Disarm: Detecting the victims targeted by harmful memes","authors":"Sharma","year":"2022","venue":"Findings of the Association for Computational Linguistics: NAACL 2022","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sharma et al., 2022b] Shivam Sharma, Firoj Alam, Md Akhtar, Dimitar Dimitrov, Giovanni Da San Martino, Hamed Firooz, Alon Halevy, Fabrizio Silvestri, Preslav Nakov, Tanmoy Chakraborty, et al. Detecting and un- derstanding harmful memes: A survey. arXiv preprint arXiv:2205.04274, 2022.","title":"Detecting and understanding harmful memes: A survey","authors":"Sharma","year":"2022","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2022/781"},{"position":28,"raw":"[Sharma et al., 2023] Shivam Sharma, Atharva Kulkarni, Tharun Suresh, Himanshi Mathur, Preslav Nakov, Md Akhtar, Tanmoy Chakraborty, et al. Characterizing the entities in harmful memes: Who is the hero, the villain, the victim? arXiv preprint arXiv:2301.11219, 2023.","title":"Characterizing the entities in harmful memes: Who is the hero, the villain, the victim","authors":"Shivam Sharma, Atharva Kulkarni, Tharun Suresh, Himanshi Mathur, Preslav Nakov, Md Akhtar, Tanmoy Chakraborty","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Sundararajan et al., 2017] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep net- works. In International conference on machine learning, pages 3319-3328. PMLR, 2017.","title":"Axiomatic attribution for deep networks","authors":"Mukund Sundararajan, Ankur Taly, Qiqi Yan","year":"2017","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Suryawanshi et al., 2020] Shardul Suryawanshi, Bharathi Raja Chakravarthi, Mihael Arcan, and Paul Buitelaar. Multimodal meme dataset (multioff) for identi- fying offensive content in image and text. In Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying, pages 32-41, 2020.","title":"Multimodal meme dataset (multioff) for identifying offensive content in image and text","authors":"Shardul Suryawanshi, Raja Bharathi, Mihael Chakravarthi, Paul Arcan, Buitelaar","year":"2020","venue":"Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wu et al., 2019] Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick. Detec- tron2. https://github.com/facebookresearch/detectron2, 2019. (accessed February 28, 2023).","title":"Detectron2","authors":"Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, Ross Girshick","year":"2019","venue":null,"doi":null,"url":"https://github.com/facebookresearch/detectron2","internal_url":null},{"position":32,"raw":"[Yang et al., 2022] Chuanpeng Yang, Fuqing Zhu, Guihua Liu, Jizhong Han, and Songlin Hu. Multimodal hate speech detection via cross-domain knowledge transfer. In Proceedings of the 30th ACM International Conference on Multimedia, MM '22, page 4505-4514, New York, NY, USA, 2022. Association for Computing Machinery.","title":"Multimodal hate speech detection via cross-domain knowledge transfer","authors":"Yang","year":"2022","venue":"Proceedings of the 30th ACM International Conference on Multimedia, MM '22","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2019] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019.","title":"Bertscore: Evaluating text generation with bert","authors":"Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, Yoav Artzi","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}