{"type":"ScholarlyArticle","title":"What is Beneath Misogyny: Misogynous Memes Classification and Explanation","authors":[{"name":"Kushal Kanwar","slug":"kushal-kanwar","given":"Kushal","family":"Kanwar","affiliation":"Jaypee University of Information Technology , Solan , India"},{"name":"Dushyant Singh Chauhan","slug":"dushyant-singh-chauhan","given":"Dushyant","family":"Singh Chauhan","affiliation":"CortexTor Labs , India"},{"name":"Gopendra Vikram Singh","slug":"gopendra-vikram-singh","given":"Gopendra Vikram","family":"Singh","affiliation":"Amity Center for Artificial Intelligence"},{"name":"Asif Ekbal","slug":"asif-ekbal","given":"Asif","family":"Ekbal","affiliation":"Department of Computer Science , Indian Institute of Technology Patna , India; School of AI and Data Science , IIT Jodhpur , India"}],"abstract":"Memes are popular in the modern world and are distributed primarily for entertainment. However, harmful ideologies such as misogyny can be propagated through innocent-looking memes. The detection and understanding of why a meme is misogynous is a research challenge due to its multimodal nature (image and text) and its nuanced manifestations across different societal contexts. We introduce a novel multimodal approach, namely, MM-Misogyny to detect, categorize, and explain misogynistic content in memes. MM-Misogyny processes text and image modalities separately and unifies them into a multimodal context through a cross-attention mechanism. The resulting multimodal context is then easily processed for labeling, categorization, and explanation via a classifier and Large Language Model (LLM). The evaluation of the proposed model is performed on a newly curated dataset (What’s Beneath Misogynous Stereotyping (WBMS)) created by collecting misogynous memes from cyberspace and categorizing them into four categories, namely, Kitchen, Leadership, Working, and Shopping. The model not only detects and classifies misogyny, but also provides a granular understanding of how misogyny operates in operates in domains of life. The results demonstrate the superiority of our approach compared to existing methods. The code and dataset are available at https://github.com/Misogyny.","doi":"10.24963/ijcai.2025/1083","url":"https://ijcai.org/proceedings/2025/1083","pdf":"https://ijcai.org/proceedings/2025/1083.pdf","markdown":"https://ijcai.org/proceedings/2025/1083.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/1083","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"AI and Social Good","track":"AI and Social Good","number":1083,"pages":{"first":9746,"last":9753},"keywords":["Multidisciplinary Topics and Applications: General","Machine Learning: General","AI Ethics, Trust, Fairness: General"],"videos":[],"references":[{"position":1,"raw":"[AI, 2023] Mistral AI. Mistral 7b: A fast and efficient trans- former model. https://mistral.ai/news/announcing-mistral- 7b/, 2023. [AI, 2024] Meta AI. Llama 3 model card. https://ai.meta.com/resources/models-and-libraries/llama- downloads/, 2024.","title":"Mistral AI. Mistral 7b: A fast and efficient transformer model","authors":"Ai","year":"2023","venue":null,"doi":null,"url":"https://ai.meta.com/resources/models-and-libraries/llama-downloads/,2024","internal_url":null},{"position":2,"raw":"[Anzovino et al., 2018] Maria Anzovino, Elisabetta Fersini, and Paolo Rosso. Automatic identification and classifica- tion of misogynistic language on twitter. In Natural Lan- guage Processing and Information Systems: 23rd Interna- tional Conference on Applications of Natural Language to Information Systems, NLDB 2018, Paris, France, June 13- 15, 2018, Proceedings 23, pages 57-64. Springer, 2018.","title":"Automatic identification and classification of misogynistic language on twitter","authors":"Maria Anzovino, Elisabetta Fersini, Paolo Rosso","year":"2018","venue":"Natural Language Processing and Information Systems: 23rd International Conference on Applications of Natural Language to Information Systems, NLDB 2018","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Attanasio et al., 2022] Giuseppe Attanasio, Debora Nozza, Eliana Pastor, Dirk Hovy, et al. Benchmarking post-hoc interpretability approaches for transformer-based misog- yny detection. In Proceedings of NLP Power! The First Workshop on Efficient Benchmarking in NLP. Association for Computational Linguistics, 2022.","title":"Benchmarking post-hoc interpretability approaches for transformer-based misogyny detection","authors":"Giuseppe Attanasio, Debora Nozza, Eliana Pastor, Dirk Hovy","year":"2022","venue":"Proceedings of NLP Power! The First Workshop on Efficient Benchmarking in NLP. Association for Computational Linguistics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bakarov, 2018] Amir Bakarov. Vector space models for au- tomatic misogyny identification. In Proceedings of Sixth Evaluation Campaign of Natural Language. Processing and Speech Tools for Italian. Final Workshop (EVALITA 2018), pages 211-213, 2018.","title":"Amir Bakarov. Vector space models for automatic misogyny identification","authors":"Bakarov","year":"2018","venue":"Proceedings of Sixth Evaluation Campaign of Natural Language. Processing and Speech Tools for Italian. Final Workshop (EVALITA 2018)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Calderon-Suarez et al., 2023] Ricardo Calderon-Suarez, Rosa M Ortega-Mendoza, Manuel Montes-Y-Gómez, Carina Toxqui-Quitl, and Marco A Marquez-Vera. En- hancing the detection of misogynistic content in social media by transferring knowledge from song phrases. IEEE Access, 11:13179-13190, 2023.","title":"Enhancing the detection of misogynistic content in social media by transferring knowledge from song phrases","authors":"Ricardo Calderon-Suarez, Rosa M Ortega-Mendoza, Manuel Montes-Y-Gómez, Carina Toxqui-Quitl, Marco A Marquez-Vera","year":"2023","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Faysse et al., 2023] Manuel Faysse, Gautier Viaud, Céline Hudelot, and Pierre Colombo. Revisiting instruction fine- tuned model evaluation to guide industrial applications. In Proceedings of the 2023 Conference on Empirical Meth- ods in Natural Language Processing, pages 9033-9048, 2023.","title":"Revisiting instruction finetuned model evaluation to guide industrial applications","authors":"Gautier Manuel Faysse, Céline Viaud, Pierre Hudelot, Colombo","year":"2023","venue":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fersini et al., 2022] Elisabetta Fersini, Francesca Gasparini, Giulia Rizzi, Aurora Saibene, Berta Chulvi, Paolo Rosso, Alyssa Lees, and Jeffrey Sorensen. Semeval-2022 task 5: Multimedia automatic misogyny identification. In Pro- ceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 533-549, 2022.","title":"Semeval-2022 task 5: Multimedia automatic misogyny identification","authors":"Elisabetta Fersini, Francesca Gasparini, Giulia Rizzi, Aurora Saibene, Berta Chulvi, Paolo Rosso, Alyssa Lees, Jeffrey Sorensen","year":"2022","venue":"Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Flesch, 2007] Rudolf Flesch. Flesch-kincaid readability test. Retrieved October, 26(3):2007, 2007.","title":"Flesch-kincaid readability test","authors":"Rudolf Flesch","year":"2007","venue":"Retrieved","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[García-Díaz et al., 2021] José Antonio García-Díaz, Mar Cánovas-García, Ricardo Colomo-Palacios, and Rafael Valencia-García. Detecting misogyny in spanish tweets. an approach based on linguistics features and word embed- dings. Future Generation Computer Systems, 114:506- 518, 2021.","title":"José Antonio García-Díaz, Mar Cánovas-García, Ricardo Colomo-Palacios, and Rafael Valencia-García. Detecting misogyny in spanish tweets. an approach based on linguistics features and word embeddings","authors":"García-Díaz","year":"2021","venue":"Future Generation Computer Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gu et al., 2022] Yimeng Gu, Ignacio Castro, and Gareth Tyson. Mmvae at semeval-2022 task 5: A multi-modal multi-task vae on misogynous meme detection. In Pro- ceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 700-710, 2022.","title":"Mmvae at semeval-2022 task 5: A multi-modal multi-task vae on misogynous meme detection","authors":"Yimeng Gu, Ignacio Castro, Gareth Tyson","year":"2022","venue":"Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hwang and Shwartz, 2023] EunJeong Hwang and Vered Shwartz. Memecap: A dataset for captioning and inter- preting memes. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1433-1445, 2023.","title":"Memecap: A dataset for captioning and interpreting memes","authors":"Shwartz Hwang, Eunjeong Hwang, Vered Shwartz","year":"2023","venue":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Joshi et al., 2024] Saurav Joshi, Filip Ilievski, and Luca Luceri. Contextualizing internet memes across social me- dia platforms. In Companion Proceedings of the ACM on Web Conference 2024, pages 1831-1840, 2024.","title":"Contextualizing internet memes across social media platforms","authors":"Saurav Joshi, Filip Ilievski, Luca Luceri","year":"2024","venue":"Companion Proceedings of the ACM on Web Conference 2024","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Muti et al., 2022a] Arianna Muti, Francesco Fernicola, and Alberto Barrón-Cedeño. Misogyny and aggressiveness tend to come together and together we address them. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 4142-4148, 2022.","title":"Misogyny and aggressiveness tend to come together and together we address them","authors":"Muti","year":"2022","venue":"Proceedings of the Thirteenth Language Resources and Evaluation Conference","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Muti et al., 2022b] Arianna Muti, Katerina Korre, and Al- berto Barrón-Cedeño. Unibo at semeval-2022 task 5: A multimodal bi-transformer approach to the binary and fine- grained identification of misogyny in memes. In Proceed- ings of the 16th International Workshop on Semantic Eval- uation (SemEval-2022), pages 663-672, 2022.","title":"Unibo at semeval-2022 task 5: A multimodal bi-transformer approach to the binary and finegrained identification of misogyny in memes","authors":"Muti","year":"2022","venue":"Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pamungkas et al., 2018] Endang Wahyu Pamungkas, Alessandra Teresa Cignarella, Valerio Basile, Viviana Patti, et al. Automatic identification of misogyny in en- glish and italian tweets at evalita 2018 with a multilingual hate lexicon. In CEUR Workshop Proceedings, volume 2263, pages 1-6. CEUR-WS, 2018.","title":"Automatic identification of misogyny in english and italian tweets at evalita 2018 with a multilingual hate lexicon","authors":"Endang Wahyu Pamungkas, Alessandra Teresa Cignarella, Valerio Basile, Viviana Patti","year":"2018","venue":"CEUR Workshop Proceedings","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language super- vision. International Conference on Machine Learning (ICML), 2021.","title":"Learning transferable visual models from natural language supervision","authors":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Rizzi et al., 2023] Giulia Rizzi, Francesca Gasparini, Au- rora Saibene, Paolo Rosso, and Elisabetta Fersini. Rec- ognizing misogynous memes: Biased models and tricky archetypes. Information Processing & Management, 60(5):103474, 2023.","title":"Recognizing misogynous memes: Biased models and tricky archetypes","authors":"Giulia Rizzi, Francesca Gasparini, Aurora Saibene, Paolo Rosso, Elisabetta Fersini","year":"2023","venue":"Information Processing & Management","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Samghabadi et al., 2020] Niloofar Safi Samghabadi, Parth Patwa, Srinivas Pykl, Prerana Mukherjee, Amitava Das, and Thamar Solorio. Aggression and misogyny detection using bert: A multi-task approach. In Proceedings of the second workshop on trolling, aggression and cyberbully- ing, pages 126-131, 2020.","title":"Aggression and misogyny detection using bert: A multi-task approach","authors":"Niloofar Safi Samghabadi, Parth Patwa, Srinivas Pykl, Prerana Mukherjee, Amitava Das, Thamar Solorio","year":"2020","venue":"Proceedings of the second workshop on trolling, aggression and cyberbullying","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Singh et al., 2023] Gopendra Singh, Soumitra Ghosh, Atul Verma, Chetna Painkra, and Asif Ekbal. Standardizing dis- tress analysis: Emotion-driven distress identification and cause extraction (dice) in multimodal online posts. In Pro- ceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4517-4532, 2023.","title":"Standardizing distress analysis: Emotion-driven distress identification and cause extraction (dice) in multimodal online posts","authors":"Singh","year":"2023","venue":"Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Singh et al., 2024] Gopendra Vikram Singh, Atul Verma, Asif Ekbal, et al. Multiseao-mix: A multimodal multi- task framework for sentiment, emotion, support, and of- fensive analysis in code-mixed setting. IEEE Transactions on Computational Social Systems, 2024.","title":"Multiseao-mix: A multimodal multitask framework for sentiment, emotion, support, and offensive analysis in code-mixed setting","authors":"Singh","year":"2024","venue":"IEEE Transactions on Computational Social Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Teh and Uwasomba, 2024] Phoey Lee Teh and Chuk- wudi Festus Uwasomba. Impact of large language models on scholarly publication titles and abstracts: A compara- tive analysis. Journal of Social Computing, 5(2):105-121, 2024. [Teknium, 2024] Teknium. Openher- mes 2.5 - fine-tuned mistral model. https://huggingface.co/teknium/OpenHermes-2.5-Mistral, 2024.","title":"Phoey Lee Teh and Chukwudi Festus Uwasomba. Impact of large language models on scholarly publication titles and abstracts: A comparative analysis","authors":"Uwasomba Teh","year":"2024","venue":"Journal of Social Computing","doi":null,"url":"https://huggingface.co/teknium/OpenHermes-2.5-Mistral,2024","internal_url":null},{"position":22,"raw":"[Touvron et al., 2023] Hugo Touvron, Thibaut Lavril, Gau- tier Izacard, Xavier Martinet, Marie-Anne Lachaux, Tim- othée Lacroix, Baptiste Rozière, Naman Goyal, Eric Ham- bro, Faisal Azhar, et al. Llama: Open and efficient founda- tion language models. arXiv preprint arXiv:2302.13971, 2023.","title":"Hugo Touvron, Thibaut Lavril, Gautier Izacard","authors":"Touvron","year":"2023","venue":"Llama: Open and efficient foundation language models","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang and Wang, 2022] Jing Zhang and Yujin Wang. Srcb at semeval-2022 task 5: Pretraining based image to text late sequential fusion system for multimodal misogynous meme identification. In Proceedings of the 16th Interna- tional Workshop on Semantic Evaluation (SemEval-2022), pages 585-596, 2022.","title":"Srcb at semeval-2022 task 5: Pretraining based image to text late sequential fusion system for multimodal misogynous meme identification","authors":"Jing Zhang, Yujin Wang","year":"2022","venue":"Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}