{"type":"ScholarlyArticle","title":"Semi-supervised Orthogonal Graph Embedding with Recursive Projections","authors":[{"name":"Hanyang Liu","slug":"hanyang-liu","given":"Hanyang","family":"Liu","affiliation":"Northwestern Polytechnical University , Xi'an 710072 , P. R. China"},{"name":"Junwei Han","slug":"junwei-han","given":"Junwei","family":"Han","affiliation":"Northwestern Polytechnical University , Xi'an 710072 , P. R. China"},{"name":"Feiping Nie","slug":"feiping-nie","given":"Feiping","family":"Nie","affiliation":"Northwestern Polytechnical University , Xi'an 710072 , P. R. China; University of Texas at Arlington , USA"}],"abstract":"Many graph based semi-supervised dimensionality reduction algorithms utilize the projection matrix to linearly map the data matrix from the original feature space to a lower dimensional representation. But the dimensionality after reduction is inevitably restricted to the number of classes, and the learned non-orthogonal projection matrix usually fails to preserve distances well and balance the weight on different projection direction. This paper proposes a novel dimensionality reduction method, called the semi-supervised orthogonal graph embedding with recursive projections (SOGE). We integrate the manifold smoothness and label fitness as well as the penalization of the linear mapping mismatch, and learn the orthogonal projection on the Stiefel manifold that empirically demonstrates better performance. Moreover, we recursively update the projection matrix in its orthocomplemented space to continuously learn more projection vectors, so as to better control the dimension of reduction. Comprehensive experiment on several benchmarks demonstrates the significant improvement over the existing methods.","doi":"10.24963/ijcai.2017/321","url":"https://ijcai.org/proceedings/2017/321","pdf":"https://ijcai.org/proceedings/2017/0321.pdf","markdown":"https://ijcai.org/proceedings/2017/321.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/321","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":321,"pages":{"first":2308,"last":2314},"keywords":["Machine Learning: Semi-Supervised Learning","Machine Learning: Data Mining","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Abernethy et al., 2008] Jacob Abernethy, Olivier Chapelle, and Carlos Castillo. Web spam identification through con- tent and hyperlinks. In Proceedings of the 4th interna- tional workshop on Adversarial information retrieval on the web, pages 41-44. ACM, 2008.","title":"Web spam identification through content and hyperlinks","authors":"Abernethy","year":"2008","venue":"Proceedings of the 4th international workshop on Adversarial information retrieval on the web","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Belhumeur et al., 1997] Peter N. Belhumeur, Joro P. Hes- panha, and David J. Kriegman. Eigenfaces vs. fisherfaces: recognition using class specific linear projection. TPAMI, 19(7):711-720, 1997.","title":"Eigenfaces vs. fisherfaces: recognition using class specific linear projection","authors":"N Peter, Joro P Belhumeur, David J Hespanha, Kriegman","year":"1997","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Belkin and Niyogi, 2003] Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionality re- duction and data representation. Neural Computation, 15(6):1373-1396, 2003.","title":"Laplacian eigenmaps for dimensionality reduction and data representation","authors":"Niyogi Belkin, Mikhail Belkin, Partha Niyogi","year":"2003","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Belkin et al., 2006] Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled exam- ples. JMLR, 7(1):2399-2434, 2006.","title":"Manifold regularization: A geometric framework for learning from labeled and unlabeled examples","authors":"Mikhail Belkin, Partha Niyogi, Vikas Sindhwani","year":"2006","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cai et al., 2006] Deng Cai, Xiaofei He, Jiawei Han, and Hong Jiang Zhang. Orthogonal laplacianfaces for face recognition. TIP, 15(11):3608-14, 2006.","title":"Orthogonal laplacianfaces for face recognition","authors":"Deng Cai, Xiaofei He, Jiawei Han, Hong Jiang Zhang","year":"2006","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cai et al., 2007] Deng Cai, Xiaofei He, and Jiawei Han. Semi-supervised discriminant analysis. In ICCV, pages 1- 7, 2007.","title":"Semi-supervised discriminant analysis","authors":"Deng Cai, Xiaofei He, Jiawei Han","year":"2007","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Enhbat, 1996] R. Enhbat. An algorithm for maximizing a convex function over a simple set. Journal of Global Op- timization, 8(4):379-391, 1996.","title":"An algorithm for maximizing a convex function over a simple set","authors":"R Enhbat","year":"1996","venue":"Journal of Global Optimization","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huang et al., 2012] Yi Huang, Dong Xu, and Feiping Nie. Semi-supervised dimension reduction using trace ratio cri- terion. IEEE Transactions on neural networks and learn- ing systems, 23(3):519-526, 2012.","title":"Semi-supervised dimension reduction using trace ratio criterion","authors":"Yi Huang, Dong Xu, Feiping Nie","year":"2012","venue":"IEEE Transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kim et al., 2016] Kwang In Kim, James Tompkin, Hanspeter Pfister, and Christian Theobalt. Semi- supervised learning with explicit relationship regulariza- tion. In CVPR, pages 2188-2196, 2016.","title":"Semisupervised learning with explicit relationship regularization","authors":"In Kwang, James Kim, Hanspeter Tompkin, Christian Pfister, Theobalt","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kokiopoulou and Saad, 2005] Effrosini Kokiopoulou and Yousef Saad. Orthogonal neighborhood preserving pro- jections. In Data Mining, Fifth IEEE International Con- ference on, pages 8-pp. IEEE, 2005.","title":"Effrosini Kokiopoulou and Yousef Saad. Orthogonal neighborhood preserving projections","authors":"Saad Kokiopoulou","year":"2005","venue":"Data Mining, Fifth IEEE International Conference on","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2006] Haifeng Li, Tao Jiang, and Keshu Zhang. Efficient and robust feature extraction by maximum mar- gin criterion. TNN, 17(1):157-65, 2006.","title":"Efficient and robust feature extraction by maximum margin criterion","authors":"Haifeng Li, Tao Jiang, Keshu Zhang","year":"2006","venue":"TNN","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2008] Wei Liu, Dacheng Tao, and Jianzhuang Liu. Transductive component analysis. In ICML, pages 433-442, 2008.","title":"Transductive component analysis","authors":"Wei Liu, Dacheng Tao, Jianzhuang Liu","year":"2008","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2017] Hanyang Liu, Junwei Han, Feiping Nie, and Xuelong Li. Balanced clustering with least square re- gression. In AAAI, 2017.","title":"Balanced clustering with least square regression","authors":"Liu","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nie et al., 2010] Feiping Nie, Dong Xu, Wai Hung Tsang, and Changshui Zhang. Flexible manifold embedding: A framework for semi-supervised and unsupervised dimen- sion reduction. TIP, 19(7):1921-1932, 2010.","title":"Flexible manifold embedding: A framework for semi-supervised and unsupervised dimension reduction","authors":"Nie","year":"2010","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nie et al., 2014] Feiping. Nie, Jiajun Yuan, and Heng Huang. Optimal mean robust principal component anal- ysis. In ICML, pages 1062-1070, 2014.","title":"Optimal mean robust principal component analysis","authors":"Nie","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nie et al., 2016] Feiping Nie, Xiaoqian Wang, Michael I Jordan, and Heng Huang. The constrained laplacian rank algorithm for graph-based clustering. In AAAI, 2016.","title":"The constrained laplacian rank algorithm for graph-based clustering","authors":"Nie","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nie et al., 2017] Feiping Nie, Rui Zhang, and Xuelong Li. A generalized power iteration method for solving quadratic problem on the stiefel manifold. Science China Information Science (SCIS), 60, 2017.","title":"A generalized power iteration method for solving quadratic problem on the stiefel manifold","authors":"Nie","year":"2017","venue":"Science China Information Science","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Rockafellar, 1970] R. Tyrrell Rockafellar. Convex analysis. Princeton Mathematical Series, 29(17):5C101, 1970.","title":"Convex analysis","authors":"R Tyrrell Rockafellar","year":"1970","venue":"Princeton Mathematical Series","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Roweis and Saul, 2000] Sam. T. Roweis and Lawrence. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323-6, 2000.","title":"Nonlinear dimensionality reduction by locally linear embedding","authors":"T Sam, Lawrence K Roweis, Saul","year":"2000","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Turk and Pentland, 2011] Matthew A. Turk and Alex P. Pentland. Face recognition using eigenfaces. Proc. CVPR, volume 84(9):586-591, 2011.","title":"Face recognition using eigenfaces","authors":"Matthew A Turk, Alex P Pentland","year":"2011","venue":"Proc. CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2014] Hua Wang, Feiping Nie, and Heng Huang. Robust distance metric learning via simultaneous l1-norm minimization and maximization. In ICML, pages 1836-1844, 2014.","title":"Robust distance metric learning via simultaneous l1-norm minimization and maximization","authors":"Hua Wang, Feiping Nie, Heng Huang","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yan et al., 2007] Shuicheng Yan, Dong Xu, Benyu Zhang, Hong-Jiang Zhang, Qiang Yang, and Stephen Lin. Graph embedding and extensions: A general framework for di- mensionality reduction. TPAMI, 29(1), 2007.","title":"Graph embedding and extensions: A general framework for dimensionality reduction","authors":"Shuicheng Yan, Dong Xu, Benyu Zhang, Hong-Jiang Zhang, Qiang Yang, Stephen Lin","year":"2007","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yan et al., 2016] Yan Yan, Zhongwen Xu, Ivor W Tsang, Guodong Long, and Yi Yang. Robust semi-supervised learning through label aggregation. In AAAI, 2016.","title":"Robust semi-supervised learning through label aggregation","authors":"Yan Yan, Zhongwen Xu, Ivor W Tsang, Guodong Long, Yi Yang","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2014] Yan-Ming Zhang, Kaizhu Huang, Xin- wen Hou, and Cheng-Lin Liu. Learning locality preserv- ing graph from data. IEEE transactions on cybernetics, 44(11):2088-2098, 2014.","title":"Learning locality preserving graph from data","authors":"Yan-Ming Zhang, Kaizhu Huang, Xinwen Hou, Cheng-Lin Liu","year":"2014","venue":"IEEE transactions on cybernetics","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhou et al., 2004] Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal, Jason Weston, and Bernhard Sch Olkopf. Learning with local and global consistency. Advances in NIPS, 16(4):321-328, 2004.","title":"Learning with local and global consistency","authors":"Zhou","year":"2004","venue":"Advances in NIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhu et al., 2003] Xiaojin Zhu, Zoubin Ghahramani, and John D. Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In ICML, pages 912-919, 2003.","title":"Semi-supervised learning using gaussian fields and harmonic functions","authors":"Xiaojin Zhu, Zoubin Ghahramani, John D Lafferty","year":"2003","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}