{"type":"ScholarlyArticle","title":"Trusted Multi-view Learning with Label Noise","authors":[{"name":"Cai Xu","slug":"cai-xu","given":"Cai","family":"Xu","affiliation":"School of Computer Science and Technology , Xidian University"},{"name":"Yilin Zhang","slug":"yilin-zhang","given":"Yilin","family":"Zhang","affiliation":"School of Computer Science and Technology , Xidian University"},{"name":"Ziyu Guan","slug":"ziyu-guan","given":"Ziyu","family":"Guan","affiliation":"School of Computer Science and Technology , Xidian University"},{"name":"Wei Zhao","slug":"wei-zhao","given":"Wei","family":"Zhao","affiliation":"School of Computer Science and Technology , Xidian University"}],"abstract":"Multi-view learning methods often focus on improving decision accuracy while neglecting the decision uncertainty, which significantly restricts their applications in safety-critical applications. To address this issue, researchers propose trusted multi-view methods that learn the class distribution for each instance, enabling the estimation of classification probabilities and uncertainty. However, these methods heavily rely on high-quality ground-truth labels. This motivates us to delve into a new generalized trusted multi-view learning problem: how to develop a reliable multi-view learning model under the guidance of noisy labels? We propose a trusted multi-view noise refining method to solve this problem. We first construct view-opinions using evidential deep neural networks, which consist of belief mass vectors and uncertainty estimates. Subsequently, we design view-specific noise correlation matrices that transform the original opinions into noisy opinions aligned with the noisy labels. Considering label noises originating from low-quality data features and easily-confused classes, we ensure that the diagonal elements of these matrices are inversely proportional to the uncertainty, while incorporating class relations into the off-diagonal elements. Finally, we aggregate the noisy opinions and employ a generalized maximum likelihood loss on the aggregated opinion for model training, guided by the noisy labels. We empirically compare TMNR with state-of-the-art trusted multi-view learning and label noise learning baselines on 5 publicly available datasets. Experiment results show that TMNR outperforms baseline methods on accuracy, reliability and robustness. The code and appendix are released at https://github.com/YilinZhang107/TMNR.","doi":"10.24963/ijcai.2024/582","url":"https://ijcai.org/proceedings/2024/582","pdf":"https://ijcai.org/proceedings/2024/0582.pdf","markdown":"https://ijcai.org/proceedings/2024/582.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/582","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":582,"pages":{"first":5263,"last":5271},"keywords":["Machine Learning: ML: Trustworthy machine learning","Machine Learning: ML: Multi-modal learning","Machine Learning: ML: Multi-view learning","Machine Learning: ML: Classification"],"videos":[],"references":[{"position":1,"raw":"[Berthon et al., 2021] Antonin Berthon, Bo Han, Gang Niu, Tongliang Liu, and Masashi Sugiyama. Confidence scores make instance-dependent label-noise learning possible. In Proceedings of the 38th International Conference on Ma- chine Learning, pages 825-836, 2021.","title":"Confidence scores make instance-dependent label-noise learning possible","authors":"Antonin Berthon, Bo Han, Gang Niu, Tongliang Liu, Masashi Sugiyama","year":"2021","venue":"Proceedings of the 38th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2024] Changrui Chen, Jungong Han, and Kurt Debattista. Virtual category learning: A semi-supervised learning method for dense prediction with extremely lim- ited labels. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.","title":"Virtual category learning: A semi-supervised learning method for dense prediction with extremely limited labels","authors":"Chen","year":"2024","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cheng et al., 2020] Hao Cheng, Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, and Yang Liu. Learning with instance-dependent label noise: A sample sieve approach. In International Conference on Learning Representations, 2020.","title":"Learning with instance-dependent label noise: A sample sieve approach","authors":"Hao Cheng, Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, Yang Liu","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cheng et al., 2022] De Cheng, Tongliang Liu, Yixiong Ning, Nannan Wang, Bo Han, Gang Niu, Xinbo Gao, and Masashi Sugiyama. Instance-dependent label-noise learning with manifold-regularized transition matrix esti- mation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16630- 16639, 2022.","title":"Instance-dependent label-noise learning with manifold-regularized transition matrix estimation","authors":"De Cheng, Tongliang Liu, Yixiong Ning, Nannan Wang, Bo Han, Gang Niu, Xinbo Gao, Masashi Sugiyama","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Deng et al., 2023] Danruo Deng, Guangyong Chen, Yang Yu, Furui Liu, and Pheng-Ann Heng. Uncertainty esti- mation by fisher information-based evidential deep learn- ing. In Proceedings of the 40th International Conference on Machine Learning, pages 7596-7616, 2023.","title":"Uncertainty estimation by fisher information-based evidential deep learning","authors":"Deng","year":"2023","venue":"Proceedings of the 40th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gal and Ghahramani, 2016] Yarin Gal and Zoubin Ghahra- mani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of The 33rd International Conference on Machine Learning, page 1050-1059, 2016.","title":"Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning","authors":"Ghahramani Gal","year":"2016","venue":"Proceedings of The 33rd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gan et al., 2021] Jiangzhang Gan, Ziwen Peng, Xiaofeng Zhu, Rongyao Hu, Junbo Ma, and Guorong Wu. Brain functional connectivity analysis based on multi-graph fu- sion. Medical image analysis, 71:102057, 2021.","title":"Brain functional connectivity analysis based on multi-graph fusion","authors":"Gan","year":"2021","venue":"Medical image analysis","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Han et al., 2020] Zongbo Han, Changqing Zhang, Huazhu Fu, and Joey Tianyi Zhou. Trusted multi-view classifica- tion. In International Conference on Learning Represen- tations, 2020.","title":"Trusted multi-view classification","authors":"Han","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Han et al., 2022] Zongbo Han, Changqing Zhang, Huazhu Fu, and Joey Tianyi Zhou. Trusted multi-view classifica- tion with dynamic evidential fusion. IEEE transactions on pattern analysis and machine intelligence, 45(2):2551- 2566, 2022.","title":"Trusted multi-view classification with dynamic evidential fusion","authors":"Han","year":"2022","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hendrycks et al., 2018] Dan Hendrycks, Mantas Mazeika, Duncan Wilson, and Kevin Gimpel. Using trusted data to train deep networks on labels corrupted by severe noise. Advances in neural information processing systems, 31, 2018.","title":"Using trusted data to train deep networks on labels corrupted by severe noise","authors":"Hendrycks","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hu et al., 2024] Shizhe Hu, Chengkun Zhang, Guoliang Zou, Zhengzheng Lou, and Yangdong Ye. Deep multiview clustering by pseudo-label guided contrastive learning and dual correlation learning. IEEE Transactions on Neural Networks and Learning Systems, 2024.","title":"Deep multiview clustering by pseudo-label guided contrastive learning and dual correlation learning","authors":"Hu","year":"2024","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Huang et al., 2023] Shudong Huang, Yixi Liu, Ivor W Tsang, Zenglin Xu, and Jiancheng Lv. Multi-view sub- space clustering by joint measuring of consistency and di- versity. IEEE Transactions on Knowledge and Data Engi- neering, 35(8):8270-8281, 2023.","title":"Multi-view subspace clustering by joint measuring of consistency and diversity","authors":"Huang","year":"2023","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jillani et al., 2020] Rashad Jillani, Syed Fawad Hussain, and Hari Kalva. Multi-view clustering for fast intra mode decision in hevc. In 2020 IEEE International Conference on Consumer Electronics (ICCE), 2020.","title":"Multi-view clustering for fast intra mode decision in hevc","authors":"Rashad Jillani, Hari Syed Fawad Hussain, Kalva","year":"2020","venue":"IEEE International Conference on Consumer Electronics (ICCE)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jøsang, 2016] Audun Jøsang. Subjective logic: A formalism for reasoning under uncertainty. Springer, 2016.","title":"Subjective logic: A formalism for reasoning under uncertainty","authors":"Audun Jøsang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liang et al., 2021] Xinyan Liang, Qian Guo, Yuhua Qian, Weiping Ding, and Qingfu Zhang. Evolutionary deep fusion method and its application in chemical structure recognition. IEEE Transactions on Evolutionary Compu- tation, 25(5):883-893, 2021.","title":"Evolutionary deep fusion method and its application in chemical structure recognition","authors":"Xinyan Liang, Qian Guo, Yuhua Qian, Weiping Ding, Qingfu Zhang","year":"2021","venue":"IEEE Transactions on Evolutionary Computation","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lin et al., 2022] Yijie Lin, Yuanbiao Gou, Xiaotian Liu, Jinfeng Bai, Jiancheng Lv, and Xi Peng. Dual contrastive prediction for incomplete multi-view representation learn- ing. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4447-4461, 2022.","title":"Dual contrastive prediction for incomplete multi-view representation learning","authors":"Yijie Lin, Yuanbiao Gou, Xiaotian Liu, Jinfeng Bai, Jiancheng Lv, Xi Peng","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lin et al., 2023] Zhenghong Lin, Yanchao Tan, Yunfei Zhan, Weiming Liu, Fan Wang, Chaochao Chen, Shiping Wang, and Carl Yang. Contrastive intra-and inter-modality generation for enhancing incomplete multimedia recom- mendation. In Proceedings of the 31st ACM International Conference on Multimedia, pages 6234-6242, 2023.","title":"Contrastive intra-and inter-modality generation for enhancing incomplete multimedia recommendation","authors":"Lin","year":"2023","venue":"Proceedings of the 31st ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2022] Wei Liu, Xiaodong Yue, Yufei Chen, and Thierry Denoeux. Trusted multi-view deep learning with opinion aggregation. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 36, pages 7585- 7593, 2022.","title":"Trusted multi-view deep learning with opinion aggregation","authors":"Wei Liu, Xiaodong Yue, Yufei Chen, Thierry Denoeux","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2023] Chengliang Liu, Jie Wen, Zhihao Wu, Xi- aoling Luo, Chao Huang, and Yong Xu. Information recovery-driven deep incomplete multiview clustering net- work. IEEE Transactions on Neural Networks and Learn- ing Systems, 2023.","title":"Information recovery-driven deep incomplete multiview clustering network","authors":"Chengliang Liu, Jie Wen, Zhihao Wu, Xiaoling Luo, Chao Huang, Yong Xu","year":"2023","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2024] Haoran Liu, Ying Ma, Ming Yan, Yingke Chen, Dezhong Peng, and Xu Wang. Dida: Disambiguated domain alignment for cross-domain retrieval with partial labels. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 3612-3620, 2024.","title":"Dida: Disambiguated domain alignment for cross-domain retrieval with partial labels","authors":"Haoran Liu, Ying Ma, Ming Yan, Yingke Chen, Dezhong Peng, Xu Wang","year":"2024","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Min et al., 2023] Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, and Dan Roth. Re- cent advances in natural language processing via large pre- trained language models: A survey. ACM Computing Sur- veys, 56(2):1-40, 2023.","title":"Recent advances in natural language processing via large pretrained language models: A survey","authors":"Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran, Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, Dan Roth","year":"2023","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Nikzad-Khasmakhi et al., 2021] Narjes Nikzad- Khasmakhi, Mohammad Ali Balafar, M Reza Feizi- Derakhshi, and Cina Motamed. Berters: Multimodal representation learning for expert recommendation system with transformers and graph embeddings. Chaos, Solitons & Fractals, 151:111260, 2021.","title":"Berters: Multimodal representation learning for expert recommendation system with transformers and graph embeddings","authors":"Nikzad-Khasmakhi","year":"2021","venue":"Chaos, Solitons & Fractals","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Patrini et al., 2017] Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recognition, pages 2233-2241, 2017.","title":"Making deep neural networks robust to label noise: A loss correction approach","authors":"Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, Lizhen Qu","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Qin et al., 2022] Yang Qin, Dezhong Peng, Xi Peng, Xu Wang, and Peng Hu. Deep evidential learning with noisy correspondence for cross-modal retrieval. In Pro- ceedings of the 30th ACM International Conference on Multimedia, pages 4948-4956, 2022.","title":"Deep evidential learning with noisy correspondence for cross-modal retrieval","authors":"Yang Qin, Dezhong Peng, Xi Peng, Xu Wang, Peng Hu","year":"2022","venue":"Proceedings of the 30th ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sensoy et al., 2018] Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncertainty. Advances in neural information processing systems, 31, 2018.","title":"Evidential deep learning to quantify classification uncertainty","authors":"Murat Sensoy, Lance Kaplan, Melih Kandemir","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2015] Weiran Wang, Raman Arora, Karen Livescu, and Jeff Bilmes. On deep multi-view repre- sentation learning. In Proceedings of the 32nd Inter- national Conference on Machine Learning, pages 1083- 1092, 2015.","title":"On deep multi-view representation learning","authors":"Wang","year":"2015","venue":"Proceedings of the 32nd International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2022] Xu Wang, Peng Hu, Pei Liu, and Dezhong Peng. Deep semisupervised class-and correlation-collapsed cross-view learning. IEEE Transac- tions on Cybernetics, 52(3):1588-1601, 2022.","title":"Deep semisupervised class-and correlation-collapsed cross-view learning","authors":"Xu Wang, Peng Hu, Pei Liu, Dezhong Peng","year":"2022","venue":"IEEE Transactions on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wen et al., 2022] Jie Wen, Zheng Zhang, Lunke Fei, Bob Zhang, Yong Xu, Zhao Zhang, and Jinxing Li. A survey on incomplete multiview clustering. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 53(2):1136- 1149, 2022.","title":"A survey on incomplete multiview clustering","authors":"Jie Wen, Zheng Zhang, Lunke Fei, Bob Zhang, Yong Xu, Zhao Zhang, Jinxing Li","year":"2022","venue":"IEEE Transactions on Systems, Man, and Cybernetics: Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xie et al., 2023] Mengyao Xie, Zongbo Han, Changqing Zhang, Yichen Bai, and Qinghua Hu. Exploring and ex- ploiting uncertainty for incomplete multi-view classifica- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19873- 19882, 2023.","title":"Exploring and exploiting uncertainty for incomplete multi-view classification","authors":"Xie","year":"2023","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xu et al., 2022] Lei Xu, Hui Wu, Chunming He, Jun Wang, Changqing Zhang, Feiping Nie, and Lei Chen. Multi- modal sequence learning for alzheimer's disease progres- sion prediction with incomplete variable-length longitudi- nal data. Medical Image Analysis, 82:102643, 2022.","title":"Multimodal sequence learning for alzheimer's disease progression prediction with incomplete variable-length longitudinal data","authors":"Lei Xu, Hui Wu, Chunming He, Jun Wang, Changqing Zhang, Feiping Nie, Lei Chen","year":"2022","venue":"Medical Image Analysis","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xu et al., 2023] Jie Xu, Yazhou Ren, Xiaoshuang Shi, Heng Tao Shen, and Xiaofeng Zhu. Untie: Clustering analysis with disentanglement in multi-view information fusion. Information Fusion, 100:101937, 2023.","title":"Untie: Clustering analysis with disentanglement in multi-view information fusion","authors":"Jie Xu, Yazhou Ren, Xiaoshuang Shi, Heng Tao Shen, Xiaofeng Zhu","year":"2023","venue":"Information Fusion","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xu et al., 2024] Cai Xu, Jiajun Si, Ziyu Guan, Wei Zhao, Yue Wu, and Xiyue Gao. Reliable conflictive multi-view learning. In Proceedings of the AAAI Conference on Arti- ficial Intelligence, volume 38, pages 16129-16137, 2024.","title":"Reliable conflictive multi-view learning","authors":"Cai Xu, Jiajun Si, Ziyu Guan, Wei Zhao, Yue Wu, Xiyue Gao","year":"2024","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhou et al., 2023] Hai Zhou, Zhe Xue, Ying Liu, Boang Li, Junping Du, Meiyu Liang, and Yuankai Qi. Calm: An enhanced encoding and confidence evaluating framework for trustworthy multi-view learning. In Proceedings of the 31st ACM International Conference on Multimedia, pages 3108-3116, 2023.","title":"Calm: An enhanced encoding and confidence evaluating framework for trustworthy multi-view learning","authors":"Hai Zhou, Zhe Xue, Ying Liu, Boang Li, Junping Du, Meiyu Liang, Yuankai Qi","year":"2023","venue":"Proceedings of the 31st ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}