{"type":"ScholarlyArticle","title":"DFMU: Distribution-based Framework for Modeling Aleatoric Uncertainty in Multimodal Sentiment Analysis","authors":[{"name":"Chen Tang","slug":"chen-tang","given":"Chen","family":"Tang","affiliation":""},{"name":"Tingrui Shen","slug":"tingrui-shen","given":"Tingrui","family":"Shen","affiliation":""},{"name":"Xinrong Gong","slug":"xinrong-gong","given":"Xinrong","family":"Gong","affiliation":""},{"name":"Chong Zhao","slug":"chong-zhao","given":"Chong","family":"Zhao","affiliation":""},{"name":"Tong Zhang","slug":"tong-zhang","given":"Tong","family":"Zhang","affiliation":""}],"abstract":"In Multimodal Sentiment Analysis (MSA), data noise arising from various sources can lead to uncertainty in Aleatoric Uncertainty (AU), significantly impacting model performance. Current efforts to address AU have insufficiently explored its sources. They primarily focus on modeling noise rather than implementing targeted modeling based on its origin. Consequently, these approaches struggle to effectively mitigate the influence of AU, resulting in sustained limitations in model performance. Our research identifies that the AU primarily stems from two problems: subjective bias in the annotation process and the complex set relationships of sentiment features. To specifically address them, we propose DFMU, a Distribution-based Framework for Modeling Aleatoric Uncertainty, which incorporates an uncertainty modeling block capable of encoding uncertainty distributions and adaptively adjusting optimization objectives. Furthermore, we introduce distribution-based contrastive learning with sentiment words replacement to better capture the complex relationships among features. Extensive experiments on three public MSA datasets, i.e., MOSI, MOSEI, and SIMS, demonstrate that the proposed model maintains robust performance even under high noise conditions and achieves state-of-the-art results on these popular datasets.","doi":"10.24963/ijcai.2025/917","url":"https://ijcai.org/proceedings/2025/917","pdf":"https://ijcai.org/proceedings/2025/0917.pdf","markdown":"https://ijcai.org/proceedings/2025/917.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/917","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Natural Language Processing","track":"Main Track","number":917,"pages":{"first":8250,"last":8258},"keywords":["Machine Learning: ML: Multi-modal learning","AI Ethics, Trust, Fairness: ETF: Trustworthy AI","Natural Language Processing: NLP: Sentiment analysis, stylistic analysis, and argument mining","Uncertainty in AI: UAI: Uncertainty representations"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}