{"type":"ScholarlyArticle","title":"Nonconvex Multiview Subspace Clustering Framework with Efficient Method Designs and Theoretical Analysis","authors":[{"name":"Zhi Wang","slug":"zhi-wang","given":"Zhi","family":"Wang","affiliation":"College of Computer and Information Science , Southwest University , Chongqing , China"},{"name":"Zhuo Liu","slug":"zhuo-liu","given":"Zhuo","family":"Liu","affiliation":"College of Computer and Information Science , Southwest University , Chongqing , China"},{"name":"Dong Hu","slug":"dong-hu","given":"Dong","family":"Hu","affiliation":"College of Computer and Information Science , Southwest University , Chongqing , China"},{"name":"Tao Jia","slug":"tao-jia","given":"Tao","family":"Jia","affiliation":"College of Computer and Information Science , Southwest University , Chongqing , China"}],"abstract":"Multi-view subspace clustering (MvSC) is one of the most effective methods for understanding and processing high-dimensional data. However, existing MvSC methods still have two shortcomings: (1) they adopt the nuclear norm as the low-rank constraint, which makes it impossible to fully exploit the mutually complementary subspace information, and (2) they do not handle disjoint and confounding points carefully, which may degrade the purity and distinctiveness of cross-view fusion. To address these issues, in this paper we propose a novel MvSC model with nonconvex ℓq regularization. Specially, our proposed model can not only effectively capture the intrinsic global low-rank structure, but also accurately cluster disjoint and confounding data samples into corresponding subspaces. Then, an efficient algorithm is developed with convergence guarantee. Furthermore, we prove that the sequence generated by our proposed algorithm converges to the desirable Karush-Kuhn-Tucker (KKT) critical point. Extensive experiments on various datasets verify the superiority of our proposed model. MATLAB code is available at https://github.com/wangzhi-swu/NLRSC-MvSC.","doi":"10.24963/ijcai.2024/571","url":"https://ijcai.org/proceedings/2024/571","pdf":"https://ijcai.org/proceedings/2024/0571.pdf","markdown":"https://ijcai.org/proceedings/2024/571.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/571","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":571,"pages":{"first":5162,"last":5170},"keywords":["Machine Learning: ML: Clustering","Machine Learning: ML: Optimization","Machine Learning: ML: Multi-view learning","Machine Learning: ML: Matrix/tensor methods"],"videos":[],"references":[{"position":1,"raw":"[Basri and Jacobs, 2003] Ronen Basri and David W Jacobs. Lambertian reflectance and linear subspaces. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 2003.","title":"Lambertian reflectance and linear subspaces","authors":"Jacobs Basri, Ronen Basri, David W Jacobs","year":"2003","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Brbić and Kopriva, 2018] Maria Brbić and Ivica Kopriva. Multi-view low-rank sparse subspace clustering. Pattern Recognition, 2018.","title":"Multi-view low-rank sparse subspace clustering","authors":"Kopriva Brbić, Maria Brbić, Ivica Kopriva","year":"2018","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2011] Deng Cai, Xiaofei He, Jiawei Han, and Thomas S Huang. Graph regularized nonnegative matrix factorization for data representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011.","title":"Graph regularized nonnegative matrix factorization for data representation","authors":"Deng Cai, Xiaofei He, Jiawei Han, Thomas S Huang","year":"2011","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cao et al., 2013] Wenfei Cao, Jian Sun, and Zongben Xu. Fast image deconvolution using closed-form thresholding formulas of l q (q = 1/2, 2/3) regularization. Journal of Visual Communication and Image Representation, 2013.","title":"Fast image deconvolution using closed-form thresholding formulas of l q (q = 1/2, 2/3) regularization","authors":"Cao","year":"2013","venue":"Journal of Visual Communication and Image Representation","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cao et al., 2015] Xiaochun Cao, Changqing Zhang, Huazhu Fu, Si Liu, and Hua Zhang. Diversity-induced multi-view subspace clustering. In CVPR, 2015.","title":"Diversity-induced multi-view subspace clustering","authors":"Xiaochun Cao, Changqing Zhang, Huazhu Fu, Si Liu, Hua Zhang","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chartrand and Yin, 2008] Rick Chartrand and Wotao Yin. Iteratively reweighted algorithms for compressive sensing. In ICASSP, 2008.","title":"Iteratively reweighted algorithms for compressive sensing","authors":"Yin Chartrand, Rick Chartrand, Wotao Yin","year":"2008","venue":"ICASSP","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chartrand, 2007] Rick Chartrand. Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Processing Letters, 2007.","title":"Exact reconstruction of sparse signals via nonconvex minimization","authors":"Rick Chartrand","year":"2007","venue":"IEEE Signal Processing Letters","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2022] Jie Chen, Shengxiang Yang, Hua Mao, and Conor Fahy. Multiview subspace clustering using low- rank representation. IEEE Transactions on Cybernetics, 2022.","title":"Multiview subspace clustering using lowrank representation","authors":"Jie Chen, Shengxiang Yang, Hua Mao, Conor Fahy","year":"2022","venue":"IEEE Transactions on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Elhamifar and Vidal, 2013] Ehsan Elhamifar and René Vi- dal. Sparse subspace clustering: Algorithm, theory, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013.","title":"Sparse subspace clustering: Algorithm, theory, and applications","authors":"Ehsan Elhamifar, René Vidal","year":"2013","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fang et al., 2023] Uno Fang, Man Li, Jianxin Li, Longxi- ang Gao, Tao Jia, and Yanchun Zhang. A comprehensive survey on multi-view clustering. IEEE Transactions on Knowledge and Data Engineering, 2023.","title":"A comprehensive survey on multi-view clustering","authors":"Uno Fang, Man Li, Jianxin Li, Longxiang Gao, Tao Jia, Yanchun Zhang","year":"2023","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gao et al., 2015] Hongchang Gao, Feiping Nie, Xuelong Li, and Heng Huang. Multi-view subspace clustering. In ICCV, 2015.","title":"Multi-view subspace clustering","authors":"Hongchang Gao, Feiping Nie, Xuelong Li, Heng Huang","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ho et al., 2003] Jeffrey Ho, Ming-Husang Yang, Jongwoo Lim, Kuang-Chih Lee, and David Kriegman. Clustering appearances of objects under varying illumination condi- tions. In CVPR, 2003.","title":"Clustering appearances of objects under varying illumination conditions","authors":"Jeffrey Ho, Ming-Husang Yang, Jongwoo Lim, Kuang-Chih Lee, David Kriegman","year":"2003","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kang et al., 2020] Zhao Kang, Wangtao Zhou, Zhitong Zhao, Junming Shao, Meng Han, and Zenglin Xu. Large- scale multi-view subspace clustering in linear time. In AAAI, 2020. [Kuybeda et al., 2013] Oleg Kuybeda, Gabriel A Frank, Al- berto Bartesaghi, Mario Borgnia, Sriram Subramaniam, and Guillermo Sapiro. A collaborative framework for 3d alignment and classification of heterogeneous subvolumes in cryo-electron tomography. Journal of Structural Biol- ogy, 2013.","title":"A collaborative framework for 3d alignment and classification of heterogeneous subvolumes in cryo-electron tomography","authors":"Zhao Kang, Wangtao Zhou, Zhitong Zhao, Junming Shao, Meng Han, Zenglin Xu, ; Kuybeda","year":"2013","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2019a] Ruihuang Li, Changqing Zhang, Qinghua Hu, Pengfei Zhu, and Zheng Wang. Flexible multi-view representation learning for subspace clustering. In IJCAI, 2019.","title":"Flexible multi-view representation learning for subspace clustering","authors":"Li","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/404"},{"position":15,"raw":"[Li et al., 2019b] Xuelong Li, Quanmao Lu, Yongsheng Dong, and Dacheng Tao. Robust subspace clustering by cauchy loss function. IEEE Transactions on Neural Net- works and Learning Systems, 2019.","title":"Robust subspace clustering by cauchy loss function","authors":"Li","year":"2019","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2010] Guangcan Liu, Zhouchen Lin, and Yong Yu. Robust subspace segmentation by low-rank represen- tation. In ICML, 2010.","title":"Robust subspace segmentation by low-rank representation","authors":"Liu","year":"2010","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2012] Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, and Yi Ma. Robust recovery of sub- space structures by low-rank representation. IEEE Trans- actions on Pattern Analysis and Machine Intelligence, 2012.","title":"Robust recovery of subspace structures by low-rank representation","authors":"Liu","year":"2012","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2022] Suyuan Liu, Siwei Wang, Pei Zhang, Kai Xu, Xinwang Liu, Changwang Zhang, and Feng Gao. Ef- ficient one-pass multi-view subspace clustering with con- sensus anchors. In AAAI, 2022.","title":"Efficient one-pass multi-view subspace clustering with consensus anchors","authors":"Suyuan Liu, Siwei Wang, Pei Zhang, Kai Xu, Xinwang Liu, Changwang Zhang, Feng Gao","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2023a] Baoyu Liu, Ling Huang, Changdong Wang, Jianhuang Lai, and Philip S Yu. Multiview cluster- ing via proximity learning in latent representation space. IEEE Transactions on Neural Networks and Learning Sys- tems, 2023.","title":"Multiview clustering via proximity learning in latent representation space","authors":"Liu","year":"2023","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2023b] Tianyu Liu, Dong Hu, Zhi Wang, Jian- ping Gou, and Wu Chen. Hyperspectral image denoising using nonconvex fraction function. IEEE Geoscience and Remote Sensing Letters, 2023.","title":"Hyperspectral image denoising using nonconvex fraction function","authors":"Liu","year":"2023","venue":"IEEE Geoscience and Remote Sensing Letters","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2023c] Zhuo Liu, Dong Hu, Zhi Wang, Jianping Gou, and Tao Jia. Latlrr for subspace clustering via reweighted frobenius norm minimization. Expert Systems with Applications, 2023.","title":"Latlrr for subspace clustering via reweighted frobenius norm minimization","authors":"Liu","year":"2023","venue":"Expert Systems with Applications","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Luo et al., 2018] Shirui Luo, Changqing Zhang, Wei Zhang, and Xiaochun Cao. Consistent and specific multi- view subspace clustering. In AAAI, 2018.","title":"Consistent and specific multiview subspace clustering","authors":"Luo","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Ma et al., 2008] Yi Ma, Allen Y Yang, Harm Derksen, and Robert Fossum. Estimation of subspace arrangements with applications in modeling and segmenting mixed data. SIAM Review, 2008.","title":"Estimation of subspace arrangements with applications in modeling and segmenting mixed data","authors":"Yi Ma, Allen Y Yang, Harm Derksen, Robert Fossum","year":"2008","venue":"SIAM Review","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Marjanovic and Solo, 2012] Goran Marjanovic and Victor Solo. On l q optimization and matrix completion. IEEE Transactions on Signal Processing, 2012.","title":"On l q optimization and matrix completion","authors":"Solo Marjanovic, Goran Marjanovic, Victor Solo","year":"2012","venue":"IEEE Transactions on Signal Processing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Muja and Lowe, 2014] Marius Muja and David G Lowe. Scalable nearest neighbor algorithms for high dimensional data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014.","title":"Scalable nearest neighbor algorithms for high dimensional data","authors":"Lowe Muja, Marius Muja, David G Lowe","year":"2014","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Ng et al., 2001] Andrew Ng, Michael Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. In NeurIPS, 2001.","title":"On spectral clustering: Analysis and an algorithm","authors":"Andrew Ng, Michael Jordan, Yair Weiss","year":"2001","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Qu et al., 2022] Qin Qu, Zhi Wang, and Wu Chen. Robust subspace clustering based on latent low-rank representa- tion with weighted schatten-p norm minimization. In PRI- CAI, 2022.","title":"Robust subspace clustering based on latent low-rank representation with weighted schatten-p norm minimization","authors":"Qin Qu, Zhi Wang, Wu Chen","year":"2022","venue":"PRI-CAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Shan et al., 2023] Yiwen Shan, Dong Hu, Zhi Wang, and Tao Jia. Multi-channel nuclear norm minus frobenius norm minimization for color image denoising. Signal Pro- cessing, 2023.","title":"Multi-channel nuclear norm minus frobenius norm minimization for color image denoising","authors":"Yiwen Shan, Dong Hu, Zhi Wang, Tao Jia","year":"2023","venue":"Signal Processing","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Slepčev and Thorpe, 2019] Dejan Slepčev and Matthew Thorpe. Analysis of p-laplacian regularization in semisu- pervised learning. SIAM Journal on Mathematical Analy- sis, 2019.","title":"Dejan Slepčev and Matthew Thorpe. Analysis of p-laplacian regularization in semisupervised learning","authors":"Thorpe Slepčev","year":"2019","venue":"SIAM Journal on Mathematical Analysis","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Tan et al., 2023] Yuze Tan, Yixi Liu, Hongjie Wu, Jiancheng Lv, and Shudong Huang. Metric multi-view graph clustering. In AAAI, 2023.","title":"Metric multi-view graph clustering","authors":"Tan","year":"2023","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Tang et al., 2014] Kewei Tang, Risheng Liu, Zhixun Su, and Jie Zhang. Structure-constrained low-rank represen- tation. IEEE Transactions on Neural Networks and Learn- ing Systems, 2014.","title":"Structure-constrained low-rank representation","authors":"Kewei Tang, Risheng Liu, Zhixun Su, Jie Zhang","year":"2014","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Tang et al., 2023] Kewei Tang, Kaiqiang Xu, Wei Jiang, Zhixun Su, Xiyan Sun, and Xiaonan Luo. Selecting the best part from multiple laplacian autoencoders for multi- view subspace clustering. IEEE Transactions on Knowl- edge and Data Engineering, 2023.","title":"Selecting the best part from multiple laplacian autoencoders for multiview subspace clustering","authors":"Kewei Tang, Kaiqiang Xu, Wei Jiang, Zhixun Su, Xiyan Sun, Xiaonan Luo","year":"2023","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[van der Maaten and Hinton, 2008] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Vidal et al., 2008] René Vidal, Roberto Tron, and Richard Hartley. Multiframe motion segmentation with missing data using powerfactorization and gpca. International Journal of Computer Vision, 2008.","title":"Multiframe motion segmentation with missing data using powerfactorization and gpca","authors":"Vidal","year":"2008","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Vidal, 2011] René Vidal. Subspace clustering. IEEE Signal Processing Magazine, 2011.","title":"Subspace clustering","authors":"René Vidal, Vidal","year":"2011","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wang et al., 2019] Zhi Wang, Wendong Wang, Jianjun Wang, and Siqi Chen. Fast and efficient algorithm for matrix completion via closed-form 2/3-thresholding op- erator. Neurocomputing, 2019.","title":"Fast and efficient algorithm for matrix completion via closed-form 2/3-thresholding operator","authors":"Zhi Wang, Wendong Wang, Jianjun Wang, Siqi Chen","year":"2019","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Wang et al., 2020] Zhi Wang, Chao Gao, Xiaohu Luo, Ming Tang, Jianjun Wang, and Wu Chen. Accelerated in- exact matrix completion algorithm via closed-form q- thresholding (q = 1/2, 2/3) operator. International Jour- nal of Machine Learning and Cybernetics, 2020.","title":"Accelerated inexact matrix completion algorithm via closed-form qthresholding (q = 1/2, 2/3) operator","authors":"Zhi Wang, Chao Gao, Xiaohu Luo, Ming Tang, Jianjun Wang, Wu Chen","year":"2020","venue":"International Journal of Machine Learning and Cybernetics","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wang et al., 2021] Zhi Wang, Dong Hu, Xiaohu Luo, Wen- dong Wang, Jianjun Wang, and Wu Chen. Performance guarantees of transformed schatten-1 regularization for ex- act low-rank matrix recovery. International Journal of Ma- chine Learning and Cybernetics, 2021.","title":"Performance guarantees of transformed schatten-1 regularization for exact low-rank matrix recovery","authors":"Zhi Wang, Dong Hu, Xiaohu Luo, Wendong Wang, Jianjun Wang, Wu Chen","year":"2021","venue":"International Journal of Machine Learning and Cybernetics","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Wang et al., 2022] Zhi Wang, Yu Liu, Xin Luo, Jianjun Wang, Chao Gao, Dezhong Peng, and Wu Chen. Large- scale affine matrix rank minimization with a novel noncon- vex regularizer. IEEE Transactions on Neural Networks and Learning Systems, 2022.","title":"Largescale affine matrix rank minimization with a novel nonconvex regularizer","authors":"Zhi Wang, Yu Liu, Xin Luo, Jianjun Wang, Chao Gao, Dezhong Peng, Wu Chen","year":"2022","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Xu et al., 2012] Zongben Xu, Xiangyu Chang, Fengmin Xu, and Hai Zhang. l 1/2 regularization: A thresholding representation theory and a fast solver. IEEE Transactions on Neural Networks and Learning Systems, 2012.","title":"l 1/2 regularization: A thresholding representation theory and a fast solver","authors":"Xu","year":"2012","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Yang et al., 2008] Allen Y Yang, John Wright, Yi Ma, and S. Shankar Sastry. Unsupervised segmentation of natural images via lossy data compression. Computer Vision and Image Understanding, 2008.","title":"Unsupervised segmentation of natural images via lossy data compression","authors":"John Allen Y Yang, Yi Wright, S Shankar Ma, Sastry","year":"2008","venue":"Computer Vision and Image Understanding","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Zhang et al., 2017] Changqing Zhang, Qinghua Hu, Huazhu Fu, Pengfei Zhu, and Xiaochun Cao. Latent multi-view subspace clustering. In CVPR, 2017.","title":"Latent multi-view subspace clustering","authors":"Changqing Zhang, Qinghua Hu, Huazhu Fu, Pengfei Zhu, Xiaochun Cao","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Zhang, 2017] Xian-Da Zhang. Matrix Analysis and Appli- cations. Cambridge University Press, 2017.","title":"Matrix Analysis and Applications","authors":"Xian-Da Zhang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}