{"type":"ScholarlyArticle","title":"Robust Contrastive Multi-view Kernel Clustering","authors":[{"name":"Peng Su","slug":"peng-su","given":"Peng","family":"Su","affiliation":"College of Computer Science , Sichuan University , Chengdu 610065 , China; Engineering Research Center of Machine Learning and Industry Intelligence , Ministry of Education , Chengdu 610065 , China"},{"name":"Yixi Liu","slug":"yixi-liu","given":"Yixi","family":"Liu","affiliation":"College of Computer Science , Sichuan University , Chengdu 610065 , China; Engineering Research Center of Machine Learning and Industry Intelligence , Ministry of Education , Chengdu 610065 , China"},{"name":"Shujian Li","slug":"shujian-li","given":"Shujian","family":"Li","affiliation":"College of Computer Science , Sichuan University , Chengdu 610065 , China; Engineering Research Center of Machine Learning and Industry Intelligence , Ministry of Education , Chengdu 610065 , China"},{"name":"Shudong Huang","slug":"shudong-huang","given":"Shudong","family":"Huang","affiliation":"College of Computer Science , Sichuan University , Chengdu 610065 , China; Engineering Research Center of Machine Learning and Industry Intelligence , Ministry of Education , Chengdu 610065 , China"},{"name":"Jiancheng Lv","slug":"jiancheng-lv","given":"Jiancheng","family":"Lv","affiliation":"College of Computer Science , Sichuan University , Chengdu 610065 , China; Engineering Research Center of Machine Learning and Industry Intelligence , Ministry of Education , Chengdu 610065 , China"}],"abstract":"Multi-view kernel clustering (MKC) aims to fully reveal the consistency and complementarity of multiple views in a potential Hilbert space, thereby enhancing clustering performance. The clustering results of most MKC methods are highly sensitive to the quality of the constructed kernels, as traditional methods independently compute kernel matrices for each view without fully considering complementary information across views. In previous contrastive multi-view kernel learning, the goal was to bring cross-view instances of the same sample closer during the kernel construction process while pushing apart instances across samples to achieve a comprehensive integration of cross-view information. However, its inherent drawback is the potential inappropriate amplification of distances between different instances of the same clusters (i.e., false negative pairs) during the training process, leading to a reduction in inter-class discriminability. To address this challenge, we propose a Robust Contrastive multi-view kernel Learning approach (R-CMK) against false negative pairs. It partitions negative pairs into different intervals based on distance or similarity, and for false negative pairs, reverses their optimization gradient. This effectively avoids further amplification of distances for false negative pairs while simultaneously pushing true negative pairs farther apart. We conducted comprehensive experiments on various MKC methods to validate the effectiveness of the proposed method. The code is available at https://github.com/Duo-laimi/rcmk_main.","doi":"10.24963/ijcai.2024/546","url":"https://ijcai.org/proceedings/2024/546","pdf":"https://ijcai.org/proceedings/2024/0546.pdf","markdown":"https://ijcai.org/proceedings/2024/546.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/546","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":546,"pages":{"first":4938,"last":4945},"keywords":["Machine Learning: ML: Multi-view learning","Machine Learning: ML: Kernel methods","Machine Learning: ML: Clustering"],"videos":[],"references":[{"position":1,"raw":"[Bisson and Grimal, 2012] Gilles Bisson and Clément Gri- mal. Co-clustering of multi-view datasets: A parallelizable approach. In 2012 IEEE 12th International Conference on Data Mining, pages 828-833, 2012.","title":"Co-clustering of multi-view datasets: A parallelizable approach","authors":"Grimal Bisson, Gilles Bisson, Clément Grimal","year":"2012","venue":"IEEE 12th International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2020] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Inter- national Conference on Machine Learning, volume 119, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cristianini et al., 2001] Nello Cristianini, John Shawe- Taylor, and Jaz Kandola. Spectral kernel methods for clus- tering. Advances in Neural Information Processing Sys- tems, 14, 2001.","title":"Spectral kernel methods for clustering","authors":"Nello Cristianini, John Shawe-Taylor, Jaz Kandola","year":"2001","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dhillon et al., 2004] Inderjit S Dhillon, Yuqiang Guan, and Brian Kulis. Kernel k-means: spectral clustering and nor- malized cuts. In Proceedings of the tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 551-556, 2004.","title":"Kernel k-means: spectral clustering and normalized cuts","authors":"Yuqiang Inderjit S Dhillon, Brian Guan, Kulis","year":"2004","venue":"Proceedings of the tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gong et al., 2012] Maoguo Gong, Yan Liang, Jiao Shi, Wenping Ma, and Jingjing Ma. Fuzzy c-means cluster- ing with local information and kernel metric for image segmentation. IEEE Transactions on Image Processing, 22(2):573-584, 2012.","title":"Fuzzy c-means clustering with local information and kernel metric for image segmentation","authors":"Gong","year":"2012","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Greene and Cunningham, 2006] Derek Greene and Pádraig Cunningham. Practical solutions to the problem of di- agonal dominance in kernel document clustering. In Proc. 23rd International Conference on Machine learning (ICML'06), pages 377-384, 2006.","title":"Practical solutions to the problem of diagonal dominance in kernel document clustering","authors":"Cunningham Greene, Derek Greene, Pádraig Cunningham","year":"2006","venue":"Proc. 23rd International Conference on Machine learning (ICML'06)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[He et al., 2020] Kaiming He, Haoqi Fan, Yuxin Wu, Sain- ing Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9726-9735, 2020.","title":"Momentum contrast for unsupervised visual representation learning","authors":"Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, Ross Girshick","year":"2020","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2012] Hsin-Chien Huang, Yung-Yu Chuang, and Chu-Song Chen. Multiple kernel fuzzy cluster- ing. IEEE Transactions on Fuzzy Systems, 20(1):120-134, 2012.","title":"Multiple kernel fuzzy clustering","authors":"Hsin-Chien Huang, Yung-Yu Chuang, Chu-Song Chen","year":"2012","venue":"IEEE Transactions on Fuzzy Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2019] Shudong Huang, Zhao Kang, Ivor W Tsang, and Zenglin Xu. Auto-weighted multi-view clus- tering via kernelized graph learning. Pattern Recognition, 88:174-184, 2019.","title":"Auto-weighted multi-view clustering via kernelized graph learning","authors":"Shudong Huang, Zhao Kang, Ivor W Tsang, Zenglin Xu","year":"2019","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kang et al., 2018] Zhao Kang, Xiao Lu, Jinfeng Yi, and Zenglin Xu. Self-weighted multiple kernel learning for graph-based clustering and semi-supervised classification. In Proceedings of the 27th International Joint Conference on Artificial Intelligence, pages 2312-2318, 2018.","title":"Self-weighted multiple kernel learning for graph-based clustering and semi-supervised classification","authors":"Zhao Kang, Xiao Lu, Jinfeng Yi, Zenglin Xu","year":"2018","venue":"Proceedings of the 27th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2018/320"},{"position":11,"raw":"[Kingma and Ba, 2017] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2016] Xinwang Liu, Yong Dou, Jianping Yin, Lei Wang, and En Zhu. Multiple kernel k-means clustering with matrix-induced regularization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 30, pages 1888-1894, 2016.","title":"Multiple kernel k-means clustering with matrix-induced regularization","authors":"Xinwang Liu, Yong Dou, Jianping Yin, Lei Wang, En Zhu","year":"2016","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2017] Xinwang Liu, Sihang Zhou, Yueqing Wang, Miaomiao Li, Yong Dou, En Zhu, and Jianping Yin. Optimal neighborhood kernel clustering with multiple ker- nels. In Proceedings of the AAAI Conference on Artificial Intelligence, AAAI Conference on Artificial Intelligence, pages 2266-2272, 2017.","title":"Optimal neighborhood kernel clustering with multiple kernels","authors":"Liu","year":"2017","venue":"Proceedings of the AAAI Conference on Artificial Intelligence, AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2020a] Jing Liu, Fuyuan Cao, Xiao-Zhi Gao, Liqin Yu, and Jiye Liang. A cluster-weighted kernel k- means method for multi-view clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 4860-4867, 2020.","title":"A cluster-weighted kernel kmeans method for multi-view clustering","authors":"Liu","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2020b] Jiyuan Liu, Xinwang Liu, Jian Xiong, Qing Liao, Sihang Zhou, Siwei Wang, and Yuexiang Yang. Optimal neighborhood multiple kernel clustering with adaptive local kernels. IEEE Transactions on Knowl- edge and Data Engineering, 34(6):2872-2885, 2020.","title":"Optimal neighborhood multiple kernel clustering with adaptive local kernels","authors":"Liu","year":"2020","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2021] Jiyuan Liu, Xinwang Liu, Siwei Wang, Si- hang Zhou, and Yuexiang Yang. Hierarchical multiple ker- nel clustering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 8671-8679, 2021.","title":"Hierarchical multiple kernel clustering","authors":"Jiyuan Liu, Xinwang Liu, Siwei Wang, Sihang Zhou, Yuexiang Yang","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2023] Jiyuan Liu, Xinwang Liu, Yuexiang Yang, Qing Liao, and Yuanqing Xia. Contrastive multi-view ker- nel learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(8):9552-9566, 2023.","title":"Contrastive multi-view kernel learning","authors":"Jiyuan Liu, Xinwang Liu, Yuexiang Yang, Qing Liao, Yuanqing Xia","year":"2023","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu, 2023] Xinwang Liu. Simplemkkm: Simple multiple kernel k-means. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):5174-5186, 2023.","title":"Simplemkkm: Simple multiple kernel k-means","authors":"Xinwang Liu, Liu","year":"2023","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nie et al., 2017] Feiping Nie, Guohao Cai, and Xuelong Li. Multi-view clustering and semi-supervised classifi- cation with adaptive neighbours. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, page 2408-2414, 2017.","title":"Multi-view clustering and semi-supervised classification with adaptive neighbours","authors":"Nie","year":"2017","venue":"Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ren et al., 2020] Zhenwen Ren, Simon X Yang, Quansen Sun, and Tao Wang. Consensus affinity graph learning for multiple kernel clustering. IEEE Transactions on Cyber- netics, 51(6):3273-3284, 2020.","title":"Consensus affinity graph learning for multiple kernel clustering","authors":"Ren","year":"2020","venue":"IEEE Transactions on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tang et al., 2023] Yiming Tang, Zhifu Pan, Xianghui Hu, Witold Pedrycz, and Renhao Chen. Knowledge-induced multiple kernel fuzzy clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.","title":"Knowledge-induced multiple kernel fuzzy clustering","authors":"Yiming Tang, Zhifu Pan, Xianghui Hu, Witold Pedrycz, Renhao Chen","year":"2023","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2019] Siwei Wang, Xinwang Liu, En Zhu, Chang Tang, Jiyuan Liu, Jingtao Hu, Jingyuan Xia, and Jianping Yin. Multi-view clustering via late fusion align- ment maximization. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 3778-3784, 2019.","title":"Multi-view clustering via late fusion alignment maximization","authors":"Siwei Wang, Xinwang Liu, En Zhu, Chang Tang, Jiyuan Liu, Jingtao Hu, Jingyuan Xia, Jianping Yin","year":"2019","venue":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19","doi":null,"url":null,"internal_url":"/proceedings/2019/524"},{"position":23,"raw":"[Xiao et al., 2015] Shijie Xiao, Mingkui Tan, Dong Xu, and Zhao Yang Dong. Robust kernel low-rank representation. IEEE Transactions on Neural Networks and Learning Sys- tems, 27(11):2268-2281, 2015.","title":"Robust kernel low-rank representation","authors":"Shijie Xiao, Mingkui Tan, Dong Xu, Zhao Yang, Dong","year":"2015","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xie et al., 2017] Xingyu Xie, Xianglin Guo, Guangcan Liu, and Jun Wang. Implicit block diagonal low-rank rep- resentation. IEEE Transactions on Image Processing, 27(1):477-489, 2017.","title":"Implicit block diagonal low-rank representation","authors":"Xingyu Xie, Xianglin Guo, Guangcan Liu, Jun Wang","year":"2017","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yang et al., 2023] Mouxing Yang, Yunfan Li, Peng Hu, Jin- feng Bai, Jiancheng Lv, and Xi Peng. Robust multi- view clustering with incomplete information. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 45(1):1055-1069, 2023.","title":"Robust multiview clustering with incomplete information","authors":"Yang","year":"2023","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}