{"type":"ScholarlyArticle","title":"From Ensemble Clustering to Multi-View Clustering","authors":[{"name":"Zhiqiang Tao","slug":"zhiqiang-tao","given":"Zhiqiang","family":"Tao","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA 02115 , USA"},{"name":"Hongfu Liu","slug":"hongfu-liu","given":"Hongfu","family":"Liu","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA 02115 , USA"},{"name":"Sheng Li","slug":"sheng-li","given":"Sheng","family":"Li","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA 02115 , USA"},{"name":"Zhengming Ding","slug":"zhengming-ding","given":"Zhengming","family":"Ding","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA 02115 , USA"},{"name":"Yun Fu","slug":"yun-fu","given":"Yun","family":"Fu","affiliation":"Department of Electrical and Computer Engineering , Northeastern University , Boston , MA 02115 , USA; College of Computer and Information Science , Northeastern University , Boston , MA 02115 , USA"}],"abstract":"Multi-View Clustering (MVC) aims to find the cluster structure shared by multiple views of a particular dataset. Existing MVC methods mainly integrate the raw data from different views, while ignoring the high-level information. Thus, their performance may degrade due to the conflict between heterogeneous features and the noises existing in each individual view. To overcome this problem, we propose a novel Multi-View Ensemble Clustering (MVEC) framework to solve MVC in an Ensemble Clustering (EC) way, which generates Basic Partitions (BPs) for each view individually and seeks for a consensus partition among all the BPs. By this means, we naturally leverage the complementary information of multi-view data in the same partition space. Instead of directly fusing BPs, we employ the low-rank and sparse decomposition to explicitly consider the connection between different views and detect the noises in each view. Moreover, the spectral ensemble clustering task is also involved by our framework with a carefully designed constraint, making MVEC a unified optimization framework to achieve the final consensus partition. Experimental results on six real-world datasets show the efficacy of our approach compared with both MVC and EC methods.","doi":"10.24963/ijcai.2017/396","url":"https://ijcai.org/proceedings/2017/396","pdf":"https://ijcai.org/proceedings/2017/0396.pdf","markdown":"https://ijcai.org/proceedings/2017/396.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/396","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":396,"pages":{"first":2843,"last":2849},"keywords":["Machine Learning: Unsupervised Learning","Machine Learning: Multi-instance/Multi-label/Multi-view learning"],"videos":[],"references":[{"position":1,"raw":"[Bickel and Scheffer, 2004] Steffen Bickel and Tobias Scheffer. Multi-view clustering. In ICDM, 2004.","title":"Multi-view clustering","authors":"Steffen Bickel, Tobias Scheffer","year":"2004","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bruno and Marchand-Maillet, 2009] Eric Bruno and Stphane Marchand-Maillet. Multiview clustering: a late fusion approach using latent models. In SIGIR, 2009.","title":"Multiview clustering: a late fusion approach using latent models","authors":"Marchand-Maillet ; Bruno, Eric Bruno, Stphane Marchand-Maillet","year":"2009","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2010] Jian-Feng Cai, Emmanuel J. Candès, and Zuowei Shen. A singular value thresholding algorithm for matrix com- pletion. SIAM Journal on Optimization, 20(4):1956-1982, 2010.","title":"A singular value thresholding algorithm for matrix completion","authors":"Jian-Feng Cai, Emmanuel J Candès, Zuowei Shen","year":"2010","venue":"SIAM Journal on Optimization","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cao et al., 2015] Xiaochun Cao, Changqing Zhang, Huazhu Fu, Si Liu, and Hua Zhang. Diversity-induced multi-view subspace clustering. In CVPR, 2015.","title":"Diversity-induced multi-view subspace clustering","authors":"Xiaochun Cao, Changqing Zhang, Huazhu Fu, Si Liu, Hua Zhang","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chaudhuri et al., 2009] Kamalika Chaudhuri, Sham M. Kakade, Karen Livescu, and Karthik Sridharan. Multi-view clustering via canonical correlation analysis. In ICML, 2009.","title":"Multi-view clustering via canonical correlation analysis","authors":"Kamalika Chaudhuri, M Sham, Karen Kakade, Karthik Livescu, Sridharan","year":"2009","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dhillon et al., 2004] Inderjit S. Dhillon, Yuqiang Guan, and Brian Kulis. Kernel k-means: Spectral clustering and normalized cuts. In SIGKDD, 2004.","title":"Kernel k-means: Spectral clustering and normalized cuts","authors":"S Inderjit, Yuqiang Dhillon, Brian Guan, Kulis","year":"2004","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ding and Fu, 2014] Zhengming Ding and Yun Fu. Low-rank com- mon subspace for multi-view learning. In ICDM, 2014.","title":"Low-rank common subspace for multi-view learning","authors":"Zhengming Ding, Yun Fu","year":"2014","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Duchi et al., 2008] John C. Duchi, Shai Shalev-Shwartz, Yoram Singer, and Tushar Chandra. Efficient projections onto the l 1 - ball for learning in high dimensions. In ICML, 2008.","title":"Efficient projections onto the l 1ball for learning in high dimensions","authors":"John C Duchi, Shai Shalev-Shwartz, Yoram Singer, Tushar Chandra","year":"2008","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fred and Jain, 2005] Ana L. N. Fred and Anil K. Jain. Combining multiple clusterings using evidence accumulation. IEEE Trans. Pattern Anal. Mach. Intell., 27(6):835-850, 2005.","title":"Combining multiple clusterings using evidence accumulation","authors":"L N Ana, Anil K Fred, Jain","year":"2005","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Greene and Cunningham, 2009] Derek Greene and Pádraig Cun- ningham. A matrix factorization approach for integrating multi- ple data views. In ECML PKDD, 2009.","title":"A matrix factorization approach for integrating multiple data views","authors":"Derek Greene, Pádraig Cunningham","year":"2009","venue":"ECML PKDD","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kumar and III, 2011] Abhishek Kumar and Hal Daum III. A co- training approach for multi-view spectral clustering. In ICML, 2011.","title":"Abhishek Kumar and Hal Daum III. A cotraining approach for multi-view spectral clustering","authors":"Iii Kumar","year":"2011","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kumar et al., 2011] Abhishek Kumar, Piyush Rai, and Hal Daume. Co-regularized multi-view spectral clustering. In NIPS, 2011.","title":"Co-regularized multi-view spectral clustering","authors":"Abhishek Kumar, Piyush Rai, Hal Daume","year":"2011","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2015a] Sheng Li, Ming Shao, and Yun Fu. Multi-view low-rank analysis for outlier detection. In SDM, 2015.","title":"Multi-view low-rank analysis for outlier detection","authors":"Li","year":"2015","venue":"SDM","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2015b] Yeqing Li, Feiping Nie, Heng Huang, and Jun- zhou Huang. Large-scale multi-view spectral clustering via bi- partite graph. In AAAI, 2015.","title":"Large-scale multi-view spectral clustering via bipartite graph","authors":"Li","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2016] Sheng Li, Yaliang Li, and Yun Fu. Multi-view time series classification: A discriminative bilinear projection ap- proach. In CIKM, 2016.","title":"Multi-view time series classification: A discriminative bilinear projection approach","authors":"Sheng Li, Yaliang Li, Yun Fu","year":"2016","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lin et al., 2011] Zhouchen Lin, Risheng Liu, and Zhixun Su. Lin- earized alternating direction method with adaptive penalty for low-rank representation. In NIPS, 2011.","title":"Linearized alternating direction method with adaptive penalty for low-rank representation","authors":"Lin","year":"2011","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2013a] Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, and Yi Ma. Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell., 35(1):171-184, 2013.","title":"Robust recovery of subspace structures by low-rank representation","authors":"Liu","year":"2013","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2013b] Jialu Liu, Chi Wang, Jing Gao, and Jiawei Han. Multi-view clustering via joint nonnegative matrix factorization. In SDM, 2013.","title":"Multi-view clustering via joint nonnegative matrix factorization","authors":"Liu","year":"2013","venue":"SDM","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2015] Hongfu Liu, Tongliang Liu, Junjie Wu, Dacheng Tao, and Yun Fu. Spectral ensemble clustering. In SIGKDD, 2015.","title":"Spectral ensemble clustering","authors":"Hongfu Liu, Tongliang Liu, Junjie Wu, Dacheng Tao, Yun Fu","year":"2015","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2016] Hongfu Liu, Ming Shao, Sheng Li, and Yun Fu. Infinite ensemble for image clustering. In SIGKDD, 2016.","title":"Infinite ensemble for image clustering","authors":"Hongfu Liu, Ming Shao, Sheng Li, Yun Fu","year":"2016","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2017] Hongfu Liu, Junjie Wu, Tongliang Liu, Dacheng Tao, and Yun Fu. Spectral ensemble clustering via weighted k- means: Theoretical and practical evidence. IEEE Trans. Knowl. Data Eng., 29(5):1129-1143, 2017.","title":"Spectral ensemble clustering via weighted kmeans: Theoretical and practical evidence","authors":"Hongfu Liu, Junjie Wu, Tongliang Liu, Dacheng Tao, Yun Fu","year":"2017","venue":"IEEE Trans. Knowl. Data Eng","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Ng et al., 2001] Andrew Y. Ng, Michael I. Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. In NIPS, 2001.","title":"On spectral clustering: Analysis and an algorithm","authors":"Andrew Y Ng, Michael I Jordan, Yair Weiss","year":"2001","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Strehl and Ghosh, 2003] Alexander Strehl and Joydeep Ghosh. Cluster ensembles -a knowledge reuse framework for combin- ing multiple partitions. Journal of Machine Learning Research, 3:583-617, 2003.","title":"Cluster ensembles -a knowledge reuse framework for combining multiple partitions","authors":"Ghosh Strehl, Alexander Strehl, Joydeep Ghosh","year":"2003","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Talwalkar et al., 2013] Ameet Talwalkar, Lester Mackey, Yadong Mu, Shih-Fu Chang, and Michael I. Jordan. Distributed low-rank subspace segmentation. In ICCV, 2013.","title":"Distributed low-rank subspace segmentation","authors":"Ameet Talwalkar, Lester Mackey, Yadong Mu, Shih-Fu Chang, Michael I Jordan","year":"2013","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Tao et al., 2016] Zhiqiang Tao, Hongfu Liu, Sheng Li, and Yun Fu. Robust spectral ensemble clustering. In CIKM, 2016.","title":"Robust spectral ensemble clustering","authors":"Zhiqiang Tao, Hongfu Liu, Sheng Li, Yun Fu","year":"2016","venue":"CIKM","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Tao et al., 2017] Zhiqiang Tao, Hongfu Liu, and Yun Fu. Simulta- neous clustering and ensemble. In AAAI, 2017.","title":"Simultaneous clustering and ensemble","authors":"Zhiqiang Tao, Hongfu Liu, Yun Fu","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2016] Yang Wang, Wenjie Zhang, Lin Wu, Xuemin Lin, Meng Fang, and Shirui Pan. Iterative views agreement: An iterative low-rank based structured optimization method to multi- view spectral clustering. In IJCAI, 2016.","title":"Iterative views agreement: An iterative low-rank based structured optimization method to multiview spectral clustering","authors":"Yang Wang, Wenjie Zhang, Lin Wu, Xuemin Lin, Meng Fang, Shirui Pan","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2017] Shuyang Wang, Zhengming Ding, and Yun Fu. Coupled marginalized auto-encoders for cross-domain multi- view learning. In IJCAI, 2017.","title":"Coupled marginalized auto-encoders for cross-domain multiview learning","authors":"Shuyang Wang, Zhengming Ding, Yun Fu","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wu et al., 2009] Junjie Wu, Hui Xiong, and Jian Chen. Adapting the right measures for k-means clustering. In SIGKDD, 2009.","title":"Adapting the right measures for k-means clustering","authors":"Junjie Wu, Hui Xiong, Jian Chen","year":"2009","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wu et al., 2015] Junjie Wu, Hongfu Liu, Hui Xiong, Jie Cao, and Jian Chen. K-means-based consensus clustering: A unified view. IEEE Trans. Knowl. Data Eng., 27(1):155-169, 2015.","title":"K-means-based consensus clustering: A unified view","authors":"Junjie Wu, Hongfu Liu, Hui Xiong, Jie Cao, Jian Chen","year":"2015","venue":"IEEE Trans. Knowl. Data Eng","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xia et al., 2014] Rongkai Xia, Yan Pan, Lei Du, and Jian Yin. Ro- bust multi-view spectral clustering via low-rank and sparse de- composition. In AAAI, 2014.","title":"Robust multi-view spectral clustering via low-rank and sparse decomposition","authors":"Rongkai Xia, Yan Pan, Lei Du, Jian Yin","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xiao et al., 2015] Shijie Xiao, Wen Li, Dong Xu, and Dacheng Tao. Falrr: A fast low rank representation solver. In CVPR, 2015.","title":"Falrr: A fast low rank representation solver","authors":"Shijie Xiao, Wen Li, Dong Xu, Dacheng Tao","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Ye et al., 2012] Guangnan Ye, Dong Liu, I-Hong Jhuo, and Shih- Fu Chang. Robust late fusion with rank minimization. In CVPR, 2012.","title":"Robust late fusion with rank minimization","authors":"Guangnan Ye, Dong Liu, I-Hong Jhuo, Shih-Fu Chang","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zha et al., 2002] Hongyuan Zha, Xiaofeng He, Chris Ding, Ming Gu, and Horst D. Simon. Spectral relaxation for k-means cluster- ing. In NIPS, 2002.","title":"Spectral relaxation for k-means clustering","authors":"Hongyuan Zha, Xiaofeng He, Chris Ding, Ming Gu, Horst D Simon","year":"2002","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhang et al., 2009] Yifan Zhang, Changsheng Xu, Hanqing Lu, and Yueh-Min Huang. Character identification in feature-length films using global face-name matching. IEEE Trans. Multimedia, 11(7):1276-1288, 2009.","title":"Character identification in feature-length films using global face-name matching","authors":"Yifan Zhang, Changsheng Xu, Hanqing Lu, Yueh-Min Huang","year":"2009","venue":"IEEE Trans. Multimedia","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhang et al., 2014] Xin Zhang, Fuchun Sun, Guangcan Liu, and Yi Ma. Fast low-rank subspace segmentation. IEEE Trans. Knowl. Data Eng., 26(5):1293-1297, 2014.","title":"Fast low-rank subspace segmentation","authors":"Xin Zhang, Fuchun Sun, Guangcan Liu, Yi Ma","year":"2014","venue":"IEEE Trans. Knowl. Data Eng","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhao and Fu, 2015] Handong Zhao and Yun Fu. Dual-regularized multi-view outlier detection. In IJCAI, 2015.","title":"Dual-regularized multi-view outlier detection","authors":"Handong Zhao, Yun Fu","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/572"},{"position":38,"raw":"[Zhao et al., 2016] Handong Zhao, Hongfu Liu, and Yun Fu. In- complete multi-modal visual data grouping. In IJCAI, 2016.","title":"Incomplete multi-modal visual data grouping","authors":"Handong Zhao, Hongfu Liu, Yun Fu","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/341"},{"position":39,"raw":"[Zhao et al., 2017] Handong Zhao, Zhengming Ding, and Yun Fu. Multi-view clustering via deep matrix factorization. In AAAI, 2017.","title":"Multi-view clustering via deep matrix factorization","authors":"Handong Zhao, Zhengming Ding, Yun Fu","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zhuang et al., 2016] Liansheng Zhuang, Jingjing Wang, Zhouchen Lin, Allen Y. Yang, Yi Ma, and Nenghai Yu. Locality-preserving low-rank representation for graph construction from nonlinear manifolds. Neurocomputing, 175:715-722, 2016.","title":"Locality-preserving low-rank representation for graph construction from nonlinear manifolds","authors":"Liansheng Zhuang, Jingjing Wang, Zhouchen Lin, Allen Y Yang, Yi Ma, Nenghai Yu","year":"2016","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}