{"type":"ScholarlyArticle","title":"Simple Contrastive Multi-View Clustering with Data-Level Fusion","authors":[{"name":"Caixuan Luo","slug":"caixuan-luo","given":"Caixuan","family":"Luo","affiliation":"School of Computer Science and Engineering , Guangxi Normal University , Guilin 541004 , China; Guangxi Key Lab of Multi-source Information Mining & Security , Guilin 541004 , China"},{"name":"Jie Xu","slug":"jie-xu","given":"Jie","family":"Xu","affiliation":"School of Computer Science and Engineering , University of Electronic Science and Technology of China , Chengdu 611731 , China"},{"name":"Yazhou Ren","slug":"yazhou-ren","given":"Yazhou","family":"Ren","affiliation":"School of Computer Science and Engineering , University of Electronic Science and Technology of China , Chengdu 611731 , China"},{"name":"Junbo Ma","slug":"junbo-ma","given":"Junbo","family":"Ma","affiliation":"School of Communication Engineering , Hangzhou Dianzi University , Hangzhou 310018 , China"},{"name":"Xiaofeng Zhu","slug":"xiaofeng-zhu","given":"Xiaofeng","family":"Zhu","affiliation":"School of Computer Science and Engineering , Guangxi Normal University , Guilin 541004 , China; School of Computer Science and Engineering , University of Electronic Science and Technology of China , Chengdu 611731 , China"}],"abstract":"Previous deep multi-view clustering methods usually design un-shared encoders to explore the cluster information among multi-view data, but they are difficult to customize the encoders for individual views and easily increase information loss. To address these issues, we propose a simple yet effective contrastive multi-view clustering framework. Specifically, different from using feature-level fusion in previous methods, we first propose a data-level fusion method to fuse multi-view information, which produces a fused data to replace all views and thus avoids customizing networks for different views. Then, we simulate the data noise and unavailability in multiple views to design two kinds of data augmentation for the fused data, making a shared encoder with simple contrastive learning to learn robust features and achieve the interaction across views. As a result, our method is a general framework and we base on it to conduct feature clustering and end-to-end clustering. Extensive experiments demonstrate that our method can explore the discriminative information in multi-view data and achieve superior clustering performance.","doi":"10.24963/ijcai.2024/519","url":"https://ijcai.org/proceedings/2024/519","pdf":"https://ijcai.org/proceedings/2024/0519.pdf","markdown":"https://ijcai.org/proceedings/2024/519.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/519","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":519,"pages":{"first":4697,"last":4705},"keywords":["Machine Learning: ML: Self-supervised Learning","Machine Learning: ML: Multi-modal learning","Machine Learning: ML: Clustering","Machine Learning: ML: Multi-view learning"],"videos":[],"references":[{"position":1,"raw":"[Abavisani and Patel, 2018] Mahdi Abavisani and Vishal M Patel. Deep multimodal subspace clustering networks. IEEE J-STSP, 12(6):1601-1614, 2018.","title":"Mahdi Abavisani and Vishal M Patel. Deep multimodal subspace clustering networks","authors":"Patel Abavisani","year":"2018","venue":"IEEE J-STSP","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andrew et al., 2013] Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. Deep canonical correlation analysis. In ICML, pages 1247-1255, 2013.","title":"Deep canonical correlation analysis","authors":"Galen Andrew, Raman Arora, Jeff Bilmes, Karen Livescu","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bickel and Scheffer, 2004] Steffen Bickel and Tobias Scheffer. Multi-view clustering. In ICDM, pages 19-26, 2004.","title":"Multi-view clustering","authors":"Scheffer Bickel, Steffen Bickel, Tobias Scheffer","year":"2004","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai et al., 2012] Xiao Cai, Hua Wang, Heng Huang, and Chris Ding. Joint stage recognition and anatomical an- notation of drosophila gene expression patterns. Bioinfor- matics, 28(12):i16-i24, 2012.","title":"Joint stage recognition and anatomical annotation of drosophila gene expression patterns","authors":"Xiao Cai, Hua Wang, Heng Huang, Chris Ding","year":"2012","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2022] Man-Sheng Chen, Jia-Qi Lin, Xiang- Long Li, Bao-Yu Liu, Chang-Dong Wang, Dong Huang, and Jian-Huang Lai. Representation learning in multi- view clustering: A literature review. Data Science and Engineering, 7(3):225-241, 2022.","title":"Representation learning in multiview clustering: A literature review","authors":"Chen","year":"2022","venue":"Data Science and Engineering","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2023] Jie Chen, Hua Mao, Wai Lok Woo, and Xi Peng. Deep multiview clustering by contrasting cluster assignments. In ICCV, pages 16752-16761, 2023.","title":"Deep multiview clustering by contrasting cluster assignments","authors":"Jie Chen, Hua Mao, Wai Lok Woo, Xi Peng","year":"2023","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Everingham et al., 2010] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zis- serman. The pascal visual object classes (voc) challenge. IJCV, 88:303-338, 2010.","title":"The pascal visual object classes (voc) challenge","authors":"Mark Everingham, Luc Van Gool, K I Christopher, John Williams, Andrew Winn, Zisserman","year":"2010","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fang et al., 2023] Uno Fang, Man Li, Jianxin Li, Longxi- ang Gao, Tao Jia, and Yanchun Zhang. A comprehensive survey on multi-view clustering. TKDE, 35(12):12350- 12368, 2023.","title":"A comprehensive survey on multi-view clustering","authors":"Uno Fang, Man Li, Jianxin Li, Longxiang Gao, Tao Jia, Yanchun Zhang","year":"2023","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2019] Zhenyu Huang, Joey Tianyi Zhou, Xi Peng, Changqing Zhang, Hongyuan Zhu, and Jiancheng Lv. Multi-view spectral clustering network. In IJCAI, pages 2563-2569, 2019.","title":"Multi-view spectral clustering network","authors":"Zhenyu Huang, Joey Tianyi Zhou, Xi Peng, Changqing Zhang, Hongyuan Zhu, Jiancheng Lv","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/356"},{"position":10,"raw":"[Hussain et al., 2010] Syed Fawad Hussain, Gilles Bisson, and Clément Grimal. An improved co-similarity mea- sure for document clustering. In 2010 Ninth International Conference on Machine Learning and Applications, pages 190-197, 2010.","title":"An improved co-similarity measure for document clustering","authors":"Gilles Syed Fawad Hussain, Clément Bisson, Grimal","year":"2010","venue":"2010 Ninth International Conference on Machine Learning and Applications","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jin et al., 2023] Jiaqi Jin, Siwei Wang, Zhibin Dong, Xin- wang Liu, and En Zhu. Deep incomplete multi-view clus- tering with cross-view partial sample and prototype align- ment. In CVPR, pages 11600-11609, 2023.","title":"Deep incomplete multi-view clustering with cross-view partial sample and prototype alignment","authors":"Jiaqi Jin, Siwei Wang, Zhibin Dong, Xinwang Liu, En Zhu","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kingma and Ba, 2014] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lin et al., 2012] Yan-Ching Lin, Min-Chun Hu, Wen- Huang Cheng, Yung-Huan Hsieh, and Hong-Ming Chen. Human action recognition and retrieval using sole depth information. In ACM MM, pages 1053-1056, 2012.","title":"Human action recognition and retrieval using sole depth information","authors":"Yan-Ching Lin, Min-Chun Hu, Wen-Huang Cheng, Yung-Huan Hsieh, Hong-Ming Chen","year":"2012","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2021] Yijie Lin, Yuanbiao Gou, Zitao Liu, Boyun Li, Jiancheng Lv, and Xi Peng. COMPLETER: Incom- plete multi-view clustering via contrastive prediction. In CVPR, pages 11174-11183, 2021.","title":"COMPLETER: Incomplete multi-view clustering via contrastive prediction","authors":"Yijie Lin, Yuanbiao Gou, Zitao Liu, Boyun Li, Jiancheng Lv, Xi Peng","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2022] Fangfei Lin, Bing Bai, Kun Bai, Yazhou Ren, Peng Zhao, and Zenglin Xu. Contrastive multi-view hyperbolic hierarchical clustering. In IJCAI, pages 3250- 3256, 2022.","title":"Contrastive multi-view hyperbolic hierarchical clustering","authors":"Lin","year":"2022","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2022/451"},{"position":16,"raw":"[Liu et al., 2023] Jiyuan Liu, Xinwang Liu, Yuexiang Yang, Qing Liao, and Yuanqing Xia. Contrastive multi-view ker- nel learning. TPAMI, pages 1-15, 2023.","title":"Contrastive multi-view kernel learning","authors":"Jiyuan Liu, Xinwang Liu, Yuexiang Yang, Qing Liao, Yuanqing Xia","year":"2023","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[MacQueen, 1967] James MacQueen. Some methods for classification and analysis of multivariate observations. In Proceedings of the Berkeley Symposium on Mathematical Statistics and Probability, pages 281-297, 1967.","title":"Some methods for classification and analysis of multivariate observations","authors":"James Macqueen","year":"1967","venue":"Proceedings of the Berkeley Symposium on Mathematical Statistics and Probability","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nair and Hinton, 2010] Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann ma- chines. In ICML, pages 807-814, 2010.","title":"Rectified linear units improve restricted boltzmann machines","authors":"Vinod Nair, Geoffrey E Hinton","year":"2010","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nene et al., 1996] Sameer A Nene, Shree K Nayar, Hi- roshi Murase, et al. Columbia object image library (coil-100). http://www1.cs.columbia.edu/CAVE/software/ softlib/coil-100.php, 1996.","title":"Columbia object image library (coil-100","authors":"Sameer A Nene, Hiroshi Shree K Nayar, Murase","year":"1996","venue":null,"doi":null,"url":"http://www1.cs.columbia.edu/CAVE/software/softlib/coil-100.php","internal_url":null},{"position":20,"raw":"[Oord et al., 2018] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03748, 2018.","title":"Representation learning with contrastive predictive coding","authors":"Aaron Van Den Oord, Yazhe Li, Oriol Vinyals","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Peng et al., 2019] Xi Peng, Zhenyu Huang, Jiancheng Lv, Hongyuan Zhu, and Joey Tianyi Zhou. COMIC: Multi- view clustering without parameter selection. In ICML, pages 5092-5101, 2019.","title":"COMIC: Multiview clustering without parameter selection","authors":"Xi Peng, Zhenyu Huang, Jiancheng Lv, Hongyuan Zhu, Joey Tianyi Zhou","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ren et al., 2022] Yazhou Ren, Jingyu Pu, Zhimeng Yang, Jie Xu, Guofeng Li, Xiaorong Pu, Philip S Yu, and Li- fang He. Deep clustering: A comprehensive survey. arXiv preprint arXiv:2210.04142, 2022.","title":"Deep clustering: A comprehensive survey","authors":"Jingyu Yazhou Ren, Zhimeng Pu, Jie Yang, Guofeng Xu, Xiaorong Li, Philip S Pu, Lifang Yu, He","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rodriguez and Laio, 2014] Alex Rodriguez and Alessandro Laio. Clustering by fast search and find of density peaks. science, 344(6191):1492-1496, 2014.","title":"Clustering by fast search and find of density peaks","authors":"Alex Rodriguez, Alessandro Laio","year":"2014","venue":"science","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rosenblatt and others, 1962] Frank Rosenblatt et al. Prin- ciples of neurodynamics: Perceptrons and the theory of brain mechanisms. Spartan books Washington, DC, 55, 1962.","title":"Principles of neurodynamics: Perceptrons and the theory of brain mechanisms","authors":"Frank Rosenblatt","year":"1962","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Shorten and Khoshgoftaar, 2019] Connor Shorten and Taghi M Khoshgoftaar. A survey on image data augmen- tation for deep learning. Journal of big data, 6(1):1-48, 2019.","title":"A survey on image data augmentation for deep learning","authors":"Connor Shorten, M Taghi, Khoshgoftaar","year":"2019","venue":"Journal of big data","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sun et al., 2007] Ting-Kai Sun, Song-Can Chen, Zhong Jin, and Jing-Yu Yang. Kernelized discriminative canonical correlation analysis. In 2007 International Conference on Wavelet Analysis and Pattern Recognition, volume 3, pages 1283-1287, 2007.","title":"Kernelized discriminative canonical correlation analysis","authors":"Ting-Kai Sun, Song-Can Chen, Zhong Jin, Jing-Yu Yang","year":"2007","venue":"2007 International Conference on Wavelet Analysis and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tang and Liu, 2022a] Huayi Tang and Yong Liu. Deep safe incomplete multi-view clustering: Theorem and al- gorithm. In ICML, pages 21090-21110, 2022.","title":"Deep safe incomplete multi-view clustering: Theorem and algorithm","authors":"Liu Tang, Huayi Tang, Yong Liu","year":"2022","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Tang and Liu, 2022b] Huayi Tang and Yong Liu. Deep safe multi-view clustering: Reducing the risk of clustering per- formance degradation caused by view increase. In CVPR, pages 202-211, 2022.","title":"Deep safe multi-view clustering: Reducing the risk of clustering performance degradation caused by view increase","authors":"Liu Tang, Huayi Tang, Yong Liu","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Tian et al., 2020] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. In ECCV, pages 776-794, 2020.","title":"Contrastive multiview coding","authors":"Tian","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Trosten et al., 2021] Daniel J. Trosten, Sigurd Løkse, Robert Jenssen, and Michael Kampffmeyer. Reconsider- ing representation alignment for multi-view clustering. In CVPR, pages 1255-1265, 2021.","title":"Reconsidering representation alignment for multi-view clustering","authors":"J Daniel, Sigurd Trosten, Robert Løkse, Michael Jenssen, Kampffmeyer","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Trosten et al., 2023] Daniel J Trosten, Sigurd Løkse, Robert Jenssen, and Michael C Kampffmeyer. On the effects of self-supervision and contrastive alignment in deep multi- view clustering. In CVPR, pages 23976-23985, 2023.","title":"On the effects of self-supervision and contrastive alignment in deep multiview clustering","authors":"Sigurd Daniel J Trosten, Robert Løkse, Michael C Jenssen, Kampffmeyer","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Wang et al., 2015] Weiran Wang, Raman Arora, Karen Livescu, and Jeff Bilmes. On deep multi-view represen- tation learning. In ICML, pages 1083-1092, 2015.","title":"On deep multi-view representation learning","authors":"Wang","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wen et al., 2022] Jie Wen, Zheng Zhang, Lunke Fei, Bob Zhang, Yong Xu, Zhao Zhang, and Jinxing Li. A survey on incomplete multiview clustering. TSMCS, 53(2):1136- 1149, 2022.","title":"A survey on incomplete multiview clustering","authors":"Jie Wen, Zheng Zhang, Lunke Fei, Bob Zhang, Yong Xu, Zhao Zhang, Jinxing Li","year":"2022","venue":"TSMCS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Xiao et al., 2017] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for bench- marking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.","title":"Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms","authors":"Han Xiao, Kashif Rasul, Roland Vollgraf","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Xu et al., 2013] Chang Xu, Dacheng Tao, and Chao Xu. A survey on multi-view learning. arXiv preprint arXiv:1304.5634, 2013.","title":"A survey on multi-view learning","authors":"Chang Xu, Dacheng Tao, Chao Xu","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Xu et al., 2022] Jie Xu, Huayi Tang, Yazhou Ren, Liang Peng, Xiaofeng Zhu, and Lifang He. Multi-level feature learning for contrastive multi-view clustering. In CVPR, pages 16051-16060, 2022.","title":"Multi-level feature learning for contrastive multi-view clustering","authors":"Jie Xu, Huayi Tang, Yazhou Ren, Liang Peng, Xiaofeng Zhu, Lifang He","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Xu et al., 2023] Jie Xu, Shuo Chen, Yazhou Ren, Xi- aoshuang Shi, Hengtao Shen, Gang Niu, and Xiaofeng Zhu. Self-weighted contrastive learning among multi- ple views for mitigating representation degeneration. In NeurIPS, volume 36, pages 1119-1131, 2023.","title":"Self-weighted contrastive learning among multiple views for mitigating representation degeneration","authors":"Jie Xu, Shuo Chen, Yazhou Ren, Xiaoshuang Shi, Hengtao Shen, Gang Niu, Xiaofeng Zhu","year":"2023","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Yan et al., 2023] Weiqing Yan, Yuanyang Zhang, Chenlei Lv, Chang Tang, Guanghui Yue, Liang Liao, and Weisi Lin. GCFAgg: Global and cross-view feature aggregation for multi-view clustering. In CVPR, pages 19863-19872, 2023.","title":"GCFAgg: Global and cross-view feature aggregation for multi-view clustering","authors":"Weiqing Yan, Yuanyang Zhang, Chenlei Lv, Chang Tang, Guanghui Yue, Liang Liao, Weisi Lin","year":"2023","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Yang et al., 2023] Xihong Yang, Jin Jiaqi, Siwei Wang, Ke Liang, Yue Liu, Yi Wen, Suyuan Liu, Sihang Zhou, Xinwang Liu, and En Zhu. Dealmvc: Dual contrastive calibration for multi-view clustering. In ACM MM, pages 337-346, 2023.","title":"Dealmvc: Dual contrastive calibration for multi-view clustering","authors":"Xihong Yang, Jin Jiaqi, Siwei Wang, Ke Liang, Yue Liu, Yi Wen, Suyuan Liu, Sihang Zhou, Xinwang Liu, En Zhu","year":"2023","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Yin et al., 2020] Ming Yin, Weitian Huang, and Junbin Gao. Shared generative latent representation learning for multi-view clustering. In AAAI, pages 6688-6695, 2020.","title":"Shared generative latent representation learning for multi-view clustering","authors":"Yin","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Zhang and He, 2023] Zheng Zhang and Wen-Jue He. Ten- sorized topological graph learning for generalized in- complete multi-view clustering. Information Fusion, 100:101914, 2023.","title":"Tensorized topological graph learning for generalized incomplete multi-view clustering","authors":"Zheng Zhang, Wen-Jue He","year":"2023","venue":"Information Fusion","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhang et al., 2024] Zheng Zhang, Xu Yuan, Lei Zhu, Jingkuan Song, and Liqiang Nie. Badcm: Invisible back- door attack against cross-modal learning. TIP, 33:2558- 2571, 2024.","title":"Badcm: Invisible backdoor attack against cross-modal learning","authors":"Zheng Zhang, Xu Yuan, Lei Zhu, Jingkuan Song, Liqiang Nie","year":"2024","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhou and Shen, 2020] Runwu Zhou and Yi-Dong Shen. End-to-end adversarial-attention network for multi-modal clustering. In CVPR, pages 14619-14628, 2020.","title":"End-to-end adversarial-attention network for multi-modal clustering","authors":"Shen Zhou, Runwu Zhou, Yi-Dong Shen","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}