{"type":"ScholarlyArticle","title":"Geometric Understanding for Unsupervised Subspace Learning","authors":[{"name":"Shihui Ying","slug":"shihui-ying","given":"Shihui","family":"Ying","affiliation":"Department of Mathematics , School of Science , Shanghai University , Shanghai , China"},{"name":"Lipeng Cai","slug":"lipeng-cai","given":"Lipeng","family":"Cai","affiliation":"Department of Mathematics , School of Science , Shanghai University , Shanghai , China"},{"name":"Changzhou He","slug":"changzhou-he","given":"Changzhou","family":"He","affiliation":"Qualcomm (Shanghai) Co. Ltd. , China"},{"name":"Yaxin Peng","slug":"yaxin-peng","given":"Yaxin","family":"Peng","affiliation":"Department of Mathematics , School of Science , Shanghai University , Shanghai , China"}],"abstract":"In this paper, we address the unsupervised subspace learning from a geometric viewpoint. First, we formulate the subspace learning as an inverse problem on Grassmannian manifold by considering all subspaces as points on it. Then, to make the model computable, we parameterize the Grassmannian manifold by using an orbit of rotation group action on all standard subspaces, which are spanned by the orthonormal basis. Further, to improve the robustness, we introduce a low-rank regularizer which makes the dimension of subspace as low as possible. Thus, the subspace learning problem is transferred to a minimization problem with variables of rotation and dimension. Then, we adopt the alternately iterative strategy to optimize the variables, where a structure-preserving method, based on the geodesic structure of the rotation group, is designed to update the rotation.  Finally, we compare the proposed approach with six state-of-the-art methods on three different kinds of real datasets. The experimental results validate that our proposed method outperforms all compared methods.","doi":"10.24963/ijcai.2019/579","url":"https://ijcai.org/proceedings/2019/579","pdf":"https://ijcai.org/proceedings/2019/0579.pdf","markdown":"https://ijcai.org/proceedings/2019/579.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/579","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":579,"pages":{"first":4171,"last":4177},"keywords":["Machine Learning: Dimensionality Reduction and Manifold Learning","Machine Learning: Data Mining"],"videos":[],"references":[{"position":1,"raw":"[Belhumeur et al., 1997] Peter N. Belhumeur, Joao P. Hes- panha, and David J. Kriegman. Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell., 19(7):711-720, 1997.","title":"Eigenfaces vs. fisherfaces: recognition using class specific linear projection","authors":"N Peter, Joao P Belhumeur, David J Hespanha, Kriegman","year":"1997","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Belkin and Niyogi, 2003] Mikhail Belkin and Partha Niyo- gi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15(6):1373- 1396, 2003.","title":"Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation","authors":"Niyogi Belkin","year":"2003","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cai et al., 2007] Deng Cai, Xiaofei He, Kun Zhou, Jiawei Han, and Hujun Bao. Locality sensitive discriminant anal- ysis. In IJCAI, pages 708-713, 2007.","title":"Locality sensitive discriminant analysis","authors":"Deng Cai, Xiaofei He, Kun Zhou, Jiawei Han, Hujun Bao","year":"2007","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2007/114"},{"position":4,"raw":"[Candès et al., 2011] Emmanuel J. Candès, Xiaodong Li, Y- i Ma, and John Wright. Robust principal component anal- ysis? Journal of the ACM, 58(3):1-37, 2011.","title":"Robust principal component analysis?","authors":"Emmanuel J Candès, Xiaodong Li, Yi Ma, John Wright","year":"2011","venue":"Journal of the ACM","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chakraborty et al., 2017] Rudrasis Chakraborty, Søren Hauberg, and Baba C. Vemuri. Intrinsic Grassmann averages for online linear and robust subspace learning. In CVPR, pages 6196-6204, 2017.","title":"Intrinsic Grassmann averages for online linear and robust subspace learning","authors":"Chakraborty","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ding and Fu, 2018] Zhengming Ding and Yun Fu. Robust multiview data analysis through collective low-rank sub- space. IEEE Trans. Neural Netw. Lear. Syst., 29(5):1986- 1997, 2018.","title":"Robust multiview data analysis through collective low-rank subspace","authors":"Zhengming Ding, Yun Fu","year":"2018","venue":"IEEE Trans. Neural Netw. Lear. Syst","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Elhamifar and Vidal, 2013] Ehsan Elhamifar and René Vi- dal. Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans. Pattern Anal. Mach. Intell., 35(11):2765-2781, 2013.","title":"Sparse subspace clustering: algorithm, theory, and applications","authors":"Ehsan Elhamifar, René Vidal","year":"2013","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fang et al., 2018] Xiaozhao Fang, Shaohua Teng, Zhihui Lai, Zhaoshui He, Shengli Xie, and Wai Keung Wong. Robust latent subspace learning for image classification. IEEE Trans. Neural Netw. Lear. Syst., 29(6):2502-2515, 2018.","title":"Robust latent subspace learning for image classification","authors":"Xiaozhao Fang, Shaohua Teng, Zhihui Lai, Zhaoshui He, Shengli Xie, Wai Keung Wong","year":"2018","venue":"IEEE Trans. Neural Netw. Lear. Syst","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hamm and Lee, 2008] Jihun Hamm and Daniel D. Lee. Grassmann discriminant analysis: a unifying view on subspace-based learning. In ICML, pages 376-383, 2008.","title":"Grassmann discriminant analysis: a unifying view on subspace-based learning","authors":"Jihun Hamm, Daniel D Lee","year":"2008","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hauberg et al., 2014] Søren Hauberg, Aasa Feragen, and Michael J. Black. Grassmann averages for scalable robust PCA. In CVPR, pages 3810-3817, 2014.","title":"Grassmann averages for scalable robust PCA","authors":"Hauberg","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hauberg et al., 2016] Søren Hauberg, Aasa Feragen, Raffi Enficiaud, and Michael J. Black. Scalable robust princi- pal component analysis using Grassmann averages. IEEE Trans. Pattern Anal. Mach. Intell., 38(11):2298-2311, 2016.","title":"Scalable robust principal component analysis using Grassmann averages","authors":"Hauberg","year":"2016","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2005a] Xiaofei He, Deng Cai, Shuicheng Yan, and Hong-jiang Zhang. Neighborhood preserving embed- ding. In ICCV, pages 1208-1213, 2005.","title":"Neighborhood preserving embedding","authors":"He","year":"2005","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2005b] Xiaofei He, Shuicheng Yan, Yuxiao Hu, Partha Niyogi, and Hong-jiang Zhang. Face recognition using Laplacianfaces. IEEE Trans. Pattern Anal. Machine Intell., 27(3):328-340, 2005.","title":"Face recognition using Laplacianfaces","authors":"He","year":"2005","venue":"IEEE Trans. Pattern Anal. Machine Intell","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Helgason, 1978] Sigurdur Helgason. Differential geometry, Lie groups, and symmetric spaces. Academic Press, 1978.","title":"Differential geometry, Lie groups, and symmetric spaces","authors":"Sigurdur Helgason","year":"1978","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Huang et al., 2014] Zhiwu Huang, Ruiping Wang, Shiguang Shan, and Xilin Chen. Learning Euclidean- to-Riemannian metric for point-to-set classification. In CVPR, pages 1677-1684, 2014.","title":"Learning Euclideanto-Riemannian metric for point-to-set classification","authors":"Zhiwu Huang, Ruiping Wang, Shiguang Shan, Xilin Chen","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lecun et al., 1998] Yann Lecun, Léon Bottou, Yoshua Ben- gio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li and Fu, 2016] Sheng Li and Yun Fu. Learning robust and discriminative subspace with low-rank constraints. IEEE Trans. Neural Netw. Lear. Syst., 27(11):2160-2173, 2016.","title":"Learning robust and discriminative subspace with low-rank constraints","authors":"Sheng Li, Yun Fu","year":"2016","venue":"IEEE Trans. Neural Netw. Lear. Syst","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2008] Xi Li, Weiming Hu, Zhongfei Zhang, Xiao- qin Zhang, Mingliang Zhu, and Jian Cheng. Visual track- ing via incremental Log-Euclidean Riemannian subspace learning. In CVPR, pages 1-8, 2008.","title":"Visual tracking via incremental Log-Euclidean Riemannian subspace learning","authors":"Xi Li, Weiming Hu, Zhongfei Zhang, Xiaoqin Zhang, Mingliang Zhu, Jian Cheng","year":"2008","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2015] Zechao Li, Jing Liu, Jinhui Tang, and Han- qing Lu. Robust structured subspace learning for data representation. IEEE Trans. Pattern Anal. Mach. Intell., 37(10):2085-2098, 2015.","title":"Robust structured subspace learning for data representation","authors":"Zechao Li, Jing Liu, Jinhui Tang, Hanqing Lu","year":"2015","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu and Yan, 2011] Guangcan Liu and Shuicheng Yan. La- tent low-rank representation for subspace segmentation and feature extraction. In ICCV, pages 1615-1622, 2011.","title":"Latent low-rank representation for subspace segmentation and feature extraction","authors":"Yan ; Liu, Guangcan Liu, Shuicheng Yan","year":"2011","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2010] Guangcan Liu, Zhouchen Lin, and Yong Yu. Robust subspace segmentation by low-rank represen- tation. In ICML, pages 663-670, 2010.","title":"Robust subspace segmentation by low-rank representation","authors":"Liu","year":"2010","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2013] Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, and Yi Ma. Robust recovery of sub- space structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell., 35(1):171-184, 2013.","title":"Robust recovery of subspace structures by low-rank representation","authors":"Liu","year":"2013","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Maaten and Hinton, 2008] Laurens Van Der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of Machine Learning Research, 9:2579-2605, 2008.","title":"Visualizing data using t-SNE","authors":"Hinton Maaten, Laurens Van, Der Maaten, Geoffrey Hinton","year":"2008","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Nene et al., 1996] Sameer A. Nene, Shree K. Nayar, and Hi- roshi Murase. Columbia object image library (coil 100). Tech. Rep. CUCS-006-96, Columbia University, New York, 1996.","title":"Columbia object image library","authors":"A Sameer, Nene, K Shree, Hiroshi Nayar, Murase","year":"1996","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Qiao et al., 2013] Hong Qiao, Peng Zhang, Di Wang, and Bo Zhang. An explicit nonlinear mapping for manifold learning. IEEE Trans. Cybern., 43(1):51-63, 2013.","title":"An explicit nonlinear mapping for manifold learning","authors":"Qiao","year":"2013","venue":"IEEE Trans. Cybern","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Saul and Roweis, 2003] Lawrence K. Saul and Sam T. Roweis. Think globally, fit locally: unsupervised learn- ing of low dimensional manifolds. Journal of Machine Learning Research, 4:119-155, 2003.","title":"Think globally, fit locally: unsupervised learning of low dimensional manifolds","authors":"Lawrence K Saul, Sam T Roweis","year":"2003","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tenenbaum et al., 2000] Joshua B. Tenenbaum, Vin de Sil- va, and John C. Langford. A global geometric frame- work for nonlinear dimensionality reduction. Science, 290(5500):2319-2323, 2000.","title":"A global geometric framework for nonlinear dimensionality reduction","authors":"Joshua B Tenenbaum, Vin De Silva, John C Langford","year":"2000","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Turk and Pentland, 1991] Matthew Turk and Alex Pentland. Eigenfaces for recognition. Journal of Cognitive Neuro- science, 3(1):71-86, 1991.","title":"Matthew Turk and Alex Pentland. Eigenfaces for recognition","authors":"Pentland Turk","year":"1991","venue":"Journal of Cognitive Neuroscience","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Vaswani et al., 2018] Namrata Vaswani, Thierry Bouwman- s, Sajid Javed, and Praneeth Narayanamurthy. Robust sub- space learning: robust PCA, robust subspace tracking, and robust subspace recovery. IEEE Signal Process. Mag., 35(4):32-55, 2018.","title":"Robust subspace learning: robust PCA, robust subspace tracking, and robust subspace recovery","authors":"Namrata Vaswani, Thierry Bouwmans, Sajid Javed, Praneeth Narayanamurthy","year":"2018","venue":"IEEE Signal Process. Mag","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Warner, 2013] Frank W. Warner. Foundations of differen- tiable manifolds and lie groups. Springer Science & Busi- ness Media, 2013.","title":"Foundations of differentiable manifolds and lie groups","authors":"Frank W Warner","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wright et al., 2009] John Wright, Allen Y. Yang, Arvind Ganesh, S. Shankar Sastry, and Yi Ma. Robust face recog- nition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell., 31(2):210-227, 2009.","title":"Robust face recognition via sparse representation","authors":"John Wright, Allen Y Yang, Arvind Ganesh, S Shankar Sastry, Yi Ma","year":"2009","venue":"IEEE Trans. Pattern Anal. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhu et al., 2018] Xiaofeng Zhu, Shichao Zhang, Rongyao Hu, Yonghua Zhu, and Jingkuan Song. Local and global structure preservation for robust unsupervised spectral fea- ture selection. IEEE Trans. Know. Data Eng., 30(3):517- 529, 2018.","title":"Local and global structure preservation for robust unsupervised spectral feature selection","authors":"Xiaofeng Zhu, Shichao Zhang, Rongyao Hu, Yonghua Zhu, Jingkuan Song","year":"2018","venue":"IEEE Trans. Know. Data Eng","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}