Hybrid Relational Graphs with Sentiment-laden Semantic Alignment for Multimodal Emotion Recognition in Conversation

Ji, Hongru; Li, Xianghua; Li, Mingxin; Zhao, Meng; Gao, Chao

doi:10.24963/ijcai.2025/331

Hybrid Relational Graphs with Sentiment-laden Semantic Alignment for Multimodal Emotion Recognition in Conversation

Hongru Ji, Xianghua Li, Mingxin Li, Meng Zhao, Chao Gao

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 2973-2981.

https://doi.org/10.24963/ijcai.2025/331

Keywords

Data Mining: DM: Mining graphs
Natural Language Processing: NLP: Sentiment analysis, stylistic analysis, and argument mining
Data Mining: DM: Applications

Abstract

Multimodal Emotion Recognition in Conversation (MERC) focuses on detecting the emotions expressed by speakers in each utterance. Recent research has increasingly leveraged graph-based models to capture interactive relationships in conversations, enhancing the ability to extract emotional cues. However, existing methods primarily focus on explicit utterance-level relationships, neglecting both the implicit connections within individual modality and the differences in implicit relationships across modalities. Moreover, these methods often overlook the role of sentimental features in conversation history in cross-modal semantic alignment. To address these issues, we propose a novel model that employs modality-adaptive hybrid relational graphs to enrich the dialogue graph by inferring implicit relationships between nodes within each modality. Furthermore, we introduce historical sentiment through a progressive strategy that utilizes contrastive learning to refine cross-modal semantic alignment. Experimental results demonstrate the superior performance of our approach over state-of-the-art methods on the IEMOCAP and MELD datasets. Our code is available at https://github.com/cgao-comp/HRG-SSA.

Hongru Ji, Xianghua Li, Mingxin Li, Meng Zhao, and Chao Gao. 2025. Hybrid Relational Graphs with Sentiment-laden Semantic Alignment for Multimodal Emotion Recognition in Conversation. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25. IJCAI, 2973–2981. https://doi.org/10.24963/ijcai.2025/331

@inproceedings{ijcai2025p331,
  title     = {Hybrid Relational Graphs with Sentiment-laden Semantic Alignment for Multimodal Emotion Recognition in Conversation},
  author    = {Ji, Hongru and Li, Xianghua and Li, Mingxin and Zhao, Meng and Gao, Chao},
  booktitle = {Proceedings of the Thirty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-25}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {James Kwok},
  pages     = {2973--2981},
  year      = {2025},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2025/331},
  url       = {https://doi.org/10.24963/ijcai.2025/331},
}

TY  - CPAPER
TI  - Hybrid Relational Graphs with Sentiment-laden Semantic Alignment for Multimodal Emotion Recognition in Conversation
AU  - Ji, Hongru
AU  - Li, Xianghua
AU  - Li, Mingxin
AU  - Zhao, Meng
AU  - Gao, Chao
T2  - Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2025
SP  - 2973
EP  - 2981
DO  - 10.24963/ijcai.2025/331
UR  - https://doi.org/10.24963/ijcai.2025/331
ER  -