{"type":"ScholarlyArticle","title":"Pseudo Supervised Matrix Factorization in Discriminative Subspace","authors":[{"name":"Jiaqi Ma","slug":"jiaqi-ma","given":"Jiaqi","family":"Ma","affiliation":"School of Computer Science , Wuhan University"},{"name":"Yipeng Zhang","slug":"yipeng-zhang","given":"Yipeng","family":"Zhang","affiliation":"School of Computer Science , Wuhan University"},{"name":"Lefei Zhang","slug":"lefei-zhang","given":"Lefei","family":"Zhang","affiliation":"School of Computer Science , Wuhan University"},{"name":"Bo Du","slug":"bo-du","given":"Bo","family":"Du","affiliation":"School of Computer Science , Wuhan University"},{"name":"Dapeng Tao","slug":"dapeng-tao","given":"Dapeng","family":"Tao","affiliation":"School of Information Science and Engineering , Yunnan University"}],"abstract":"Non-negative Matrix Factorization (NMF) and spectral clustering have been proved to be efficient and effective for data clustering tasks and have been applied to various real-world scenes. However, there are still some drawbacks in traditional methods: (1) most existing algorithms only consider high-dimensional data directly while neglect the intrinsic data structure in the low-dimensional subspace; (2) the pseudo-information got in the optimization process is not relevant to most spectral clustering and manifold regularization methods. In this paper, a novel unsupervised matrix factorization method, Pseudo Supervised Matrix Factorization (PSMF), is proposed for data clustering. The main contributions are threefold: (1) to cluster in the discriminant subspace, Linear Discriminant Analysis (LDA) combines with NMF to become a unified framework; (2) we propose a pseudo supervised manifold regularization term which utilizes the pseudo-information to instruct the regularization term in order to find subspace that discriminates different classes; (3) an efficient optimization algorithm is designed to solve the proposed problem with proved convergence. Extensive experiments on multiple benchmark datasets illustrate that the proposed model outperforms other state-of-the-art clustering algorithms.","doi":"10.24963/ijcai.2019/633","url":"https://ijcai.org/proceedings/2019/633","pdf":"https://ijcai.org/proceedings/2019/0633.pdf","markdown":"https://ijcai.org/proceedings/2019/633.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/633","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning Applications","track":"Main track","number":633,"pages":{"first":4554,"last":4560},"keywords":["Machine Learning: Clustering","Machine Learning: Dimensionality Reduction and Manifold Learning","Machine Learning Applications: Applications of Unsupervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Cai et al., 2005] Deng Cai, Xiaofei He, and Jiawei Han. Document clustering using locality preserving indexing. IEEE TKDE, 17(12):1624-1637, 2005.","title":"Document clustering using locality preserving indexing","authors":"Deng Cai, Xiaofei He, Jiawei Han","year":"2005","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cai et al., 2011a] Deng Cai, Xiaofei He, and Jiawei Han. Locally consistent concept factorization for document clustering. IEEE TKDE, 23(6):902-913, 2011.","title":"Locally consistent concept factorization for document clustering","authors":"Cai","year":"2011","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2011b] Deng Cai, Xiaofei He, Jiawei Han, and Thomas S Huang. Graph regularized nonnegative ma- trix factorization for data representation. IEEE TPAMI, 33(8):1548-1560, 2011.","title":"Graph regularized nonnegative matrix factorization for data representation","authors":"Cai","year":"2011","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cai, 2011] Deng Cai. Litekmeans: the fastest matlab imple- mentation of kmeans. Available at: http:// www.zjucadcg. cn/ dengcai/ Data/ Clustering.html, 2011.","title":"Litekmeans: the fastest matlab implementation of kmeans","authors":"Deng Cai","year":"2011","venue":null,"doi":null,"url":"http://www.zjucadcg.cn/dengcai/Data/Clustering.html","internal_url":null},{"position":5,"raw":"[Chung and Graham, 1997] Fan RK Chung and Fan Chung Graham. Spectral graph theory. American Mathematical Soc., 1997.","title":"Spectral graph theory","authors":"Fan Rk, Chung, Fan Chung, Graham","year":"1997","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ding et al., 2010] Chris H. Q. Ding, Tao Li, and Michael I. Jordan. Convex and semi-nonnegative matrix factoriza- tions. IEEE TPAMI, 32(1):45-55, 2010.","title":"Convex and semi-nonnegative matrix factorizations","authors":"H Q Chris, Tao Ding, Michael I Li, Jordan","year":"2010","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Estévez et al., 2009] Pablo A. Estévez, M. Tesmer, Clau- dio A. Perez, and Jacek M. Zurada. Normalized mutual information feature selection. IEEE TNN, 20(2):189-201, 2009.","title":"Normalized mutual information feature selection","authors":"Pablo A Estévez, M Tesmer, Claudio A Perez, Jacek M Zurada","year":"2009","venue":"IEEE TNN","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[He et al., 2005] Xiaofei He, Shuicheng Yan, Yuxiao Hu, Partha Niyogi, and HongJiang Zhang. Face recognition using laplacianfaces. IEEE TPAMI, 27(3):328-340, 2005.","title":"Face recognition using laplacianfaces","authors":"Xiaofei He, Shuicheng Yan, Yuxiao Hu, Partha Niyogi, Hongjiang Zhang","year":"2005","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2013] Jin Huang, Feiping Nie, Heng Huang, and Chris H. Q. Ding. Robust manifold nonnegative ma- trix factorization. ACM TKDD, 8(3):11:1-11:21, 2013.","title":"Robust manifold nonnegative matrix factorization","authors":"Jin Huang, Feiping Nie, Heng Huang, Chris H Q Ding","year":"2013","venue":"ACM TKDD","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jiang et al., 2004] Daxin Jiang, Chun Tang, and Aidong Zhang. Cluster analysis for gene expression data: A sur- vey. IEEE TKDE, 16(11):1370-1386, 2004.","title":"Cluster analysis for gene expression data: A survey","authors":"Daxin Jiang, Chun Tang, Aidong Zhang","year":"2004","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ke and Kanade, 2005] Qifa Ke and Takeo Kanade. Robust l1 norm factorization in the presence of outliers and miss- ing data by alternative convex programming. In Proc. CVPR, pages 739-746, 2005.","title":"Robust l1 norm factorization in the presence of outliers and missing data by alternative convex programming","authors":"Kanade ; ] Qifa Ke, Takeo Ke, Kanade","year":"2005","venue":"Proc. CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lee and Seung, 1999] Daniel D Lee and H Sebastian Seung. Learning the parts of objects by non-negative matrix fac- torization. Nature, 401(6755):788, 1999.","title":"Learning the parts of objects by non-negative matrix factorization","authors":"D Daniel, H Lee, Seung Sebastian","year":"1999","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2017] Weiwei Liu, Xiao-Bo Shen, and Ivor W. Tsang. Sparse embedded k-means clustering. In Proc. NIPS, pages 3321-3329, 2017.","title":"Sparse embedded k-means clustering","authors":"Weiwei Liu, Xiao-Bo Shen, Ivor W Tsang","year":"2017","venue":"Proc. NIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2018] Yang Liu, Quanxue Gao, Zhaohua Yang, and Shujian Wang. Learning with adaptive neighbors for image clustering. In Proc. IJCAI, pages 2483-2489, 2018.","title":"Learning with adaptive neighbors for image clustering","authors":"Yang Liu, Quanxue Gao, Zhaohua Yang, Shujian Wang","year":"2018","venue":"Proc. IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/344"},{"position":15,"raw":"[Lu et al., 2017] Yuwu Lu, Zhihui Lai, Xu Yong, Xuelong Li, David Zhang, and Chun Yuan. Nonnegative discrimi- nant matrix factorization. IEEE TCSVT, 27(7):1392-1405, 2017.","title":"Nonnegative discriminant matrix factorization","authors":"Lu","year":"2017","venue":"IEEE TCSVT","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Ma et al., 2018] Sihan Ma, Lefei Zhang, Wenbin Hu, Yipeng Zhang, Jia Wu, Xuelong Li, et al. Self- representative manifold concept factorization with adap- tive neighbors for clustering. In Proc. IJCAI, pages 2539- 2545, 2018.","title":"Selfrepresentative manifold concept factorization with adap-tive neighbors for clustering","authors":"Sihan Ma, Lefei Zhang, Wenbin Hu, Yipeng Zhang, Jia Wu, Xuelong Li","year":"2018","venue":"Proc. IJCAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ng et al., 2002] Andrew Y Ng, Michael I Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. In Proc. NIPS, pages 849-856, 2002.","title":"On spectral clustering: Analysis and an algorithm","authors":"Y Andrew, Michael I Ng, Yair Jordan, Weiss","year":"2002","venue":"Proc. NIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nie et al., 2014] Feiping Nie, Xiaoqian Wang, and Heng Huang. Clustering and projected clustering with adaptive neighbors. In Proc. ACM SIGKDD, pages 977-986, 2014.","title":"Clustering and projected clustering with adaptive neighbors","authors":"Nie","year":"2014","venue":"Proc. ACM SIGKDD","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nie et al., 2015] Feiping Nie, Hua Wang, Heng Huang, and Chris Ding. Joint schatten p-norm and ℓ p -norm robust matrix completion for missing value recovery. KAIS, 42(3):525-544, 2015.","title":"Joint schatten p-norm and ℓ p -norm robust matrix completion for missing value recovery","authors":"Nie","year":"2015","venue":"KAIS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shen et al., 2017] Xiao-Bo Shen, Weiwei Liu, Ivor W. Tsang, Fumin Shen, and Quan-Sen Sun. Compressed k- means for large-scale clustering. In Proc. AAAI, pages 2527-2533, 2017.","title":"Compressed kmeans for large-scale clustering","authors":"Xiao-Bo Shen, Weiwei Liu, Ivor W Tsang, Fumin Shen, Quan-Sen Sun","year":"2017","venue":"Proc. AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shi and Malik, 2000] Jianbo Shi and Jitendra Malik. Nor- malized cuts and image segmentation. IEEE TPAMI, 22(8):888-905, 2000.","title":"Normalized cuts and image segmentation","authors":"Jianbo Shi, Jitendra Malik","year":"2000","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2017] Boyue Wang, Yongli Hu, Junbin Gao, Yanfeng Sun, Haoran Chen, Muhammad Ali, and Baocai Yin. Locality preserving projections for grassmann mani- fold. In Proc. IJCAI, pages 2893-2900, 2017.","title":"Locality preserving projections for grassmann manifold","authors":"Wang","year":"2017","venue":"Proc. IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/403"},{"position":23,"raw":"[Wang et al., 2018] Jing Wang, Feng Tian, Weiwei Liu, Xiao Wang, Wenjie Zhang, and Kenji Yamanishi. Ranking pre- serving nonnegative matrix factorization. In Proc. IJCAI, pages 2776-2782, 2018.","title":"Ranking preserving nonnegative matrix factorization","authors":"Jing Wang, Feng Tian, Weiwei Liu, Xiao Wang, Wenjie Zhang, Kenji Yamanishi","year":"2018","venue":"Proc. IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/385"},{"position":24,"raw":"[Wechsler et al., 2012] Harry Wechsler, Jonathon P Phillips, Vicki Bruce, Francoise Fogelman Soulie, and Thomas S Huang. Face recognition: From theory to applications, volume 163. 2012.","title":"Face recognition: From theory to applications","authors":"Harry Wechsler, Jonathon P Phillips, Vicki Bruce, Francoise Fogelman Soulie, Thomas S Huang","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yang et al., 2011] Yi Yang, Heng Tao Shen, Zhigang Ma, Zi Huang, and Xiaofang Zhou. L2,1-norm regularized dis- criminative feature selection for unsupervised learning. In Proc. IJCAI, pages 1589-1594, 2011.","title":"L2,1-norm regularized discriminative feature selection for unsupervised learning","authors":"Yi Yang, Heng Tao Shen, Zhigang Ma, Zi Huang, Xiaofang Zhou","year":"2011","venue":"Proc. IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2011/267"},{"position":26,"raw":"[Zhang and Zha, 2004] Zhenyue Zhang and Hongyuan Zha. Principal manifolds and nonlinear dimensionality reduc- tion via tangent space alignment. Journal of Shanghai University, 8(4):406-424, 2004.","title":"Principal manifolds and nonlinear dimensionality reduction via tangent space alignment","authors":"Zha Zhang, Zhenyue Zhang, Hongyuan Zha","year":"2004","venue":"Journal of Shanghai University","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2009] Tianhao Zhang, Dacheng Tao, Xuelong Li, and Yang Jie. Patch alignment for dimensionality re- duction. IEEE TKDE, 21(9):1299-1313, 2009.","title":"Patch alignment for dimensionality reduction","authors":"Tianhao Zhang, Dacheng Tao, Xuelong Li, Yang Jie","year":"2009","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2017a] Lefei Zhang, Qian Zhang, Bo Du, Dacheng Tao, and Jane You. Robust manifold matrix fac- torization for joint clustering and feature extraction. In Proc. AAAI, pages 1662-1668, 2017.","title":"Robust manifold matrix factorization for joint clustering and feature extraction","authors":"Zhang","year":"2017","venue":"Proc. AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2017b] Lefei Zhang, Qian Zhang, Bo Du, Jane You, and Dacheng Tao. Adaptive manifold regularized matrix factorization for data clustering. In Proc. IJCAI, pages 3399-3405, 2017.","title":"Adaptive manifold regularized matrix factorization for data clustering","authors":"Zhang","year":"2017","venue":"Proc. IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/475"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}