{"type":"ScholarlyArticle","title":"Multimodal Emotion Recognition with Large Language Models","authors":[{"name":"Hongrui Zhang","slug":"hongrui-zhang","given":"Hongrui","family":"Zhang","affiliation":""},{"name":"Daiqing Wu","slug":"daiqing-wu","given":"Daiqing","family":"Wu","affiliation":""},{"name":"Yangyang Li","slug":"yangyang-li","given":"Yangyang","family":"Li","affiliation":""},{"name":"Kuien Liu","slug":"kuien-liu","given":"Kuien","family":"Liu","affiliation":""},{"name":"Yuhui Wang","slug":"yuhui-wang","given":"Yuhui","family":"Wang","affiliation":""},{"name":"Yu Zhou","slug":"yu-zhou","given":"Yu","family":"Zhou","affiliation":""},{"name":"Sicheng Zhao","slug":"sicheng-zhao","given":"Sicheng","family":"Zhao","affiliation":""}],"abstract":"Multimodal Emotion Recognition (MER) focuses on identifying and interpreting emotions from modality-compound inputs. Closely mirroring human cognitive processes in real-world environments, MER has drawn substantial attention from both academia and industry. Recently, a paradigm shift has been unveiled in MER, from leveraging small-scale, task-specific models to Large Language Models (LLMs). We refer to the latter as the MER-with-LLMs paradigm, which offers unprecedented generality, spurring numerous empirical attempts, even alongside speculation about their potential to achieve general emotional intelligence. However, with these new opportunities come new challenges, including the scarcity of emotionally annotated data, the affective gap both within and across modalities, and the opacity of affective interpretation. To systematically review existing research and guide future exploration, this paper categorizes prior works according to their focus on addressing these challenges into three directions: Affective Data Augmentation, Multimodal Affective Representation, and Multimodal Affective Reasoning. By thoroughly tracing the development, emerging trends, and remaining issues within each direction, this paper aims to provide a clear academic map of the MER-with-LLMs paradigm and foster its structured advancement.","doi":"10.24963/ijcai.2026/897","url":"https://ijcai.org/proceedings/2026/897","pdf":"https://ijcai.org/proceedings/2026/0897.pdf","markdown":"https://ijcai.org/proceedings/2026/897.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/897","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Survey Track","track":"Survey Track","number":897,"pages":{"first":8088,"last":8096},"keywords":["Computer Vision: Multimodal learning","Natural Language Processing: Language generation","Computer Vision: Interpretability and transparency","Natural Language Processing: Language models","Computer Vision: Video analysis and understanding"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}