{"type":"ScholarlyArticle","title":"Flexible Orthogonal Neighborhood Preserving Embedding","authors":[{"name":"Tianji Pang","slug":"tianji-pang","given":"Tianji","family":"Pang","affiliation":"Northwestern Polytechnical University , Xian 710072 , P. R. China."},{"name":"Feiping Nie","slug":"feiping-nie","given":"Feiping","family":"Nie","affiliation":"Northwestern Polytechnical University , Xian 710072 , P. R. China.; University of Texas at Arlington , USA"},{"name":"Junwei Han","slug":"junwei-han","given":"Junwei","family":"Han","affiliation":"Northwestern Polytechnical University , Xian 710072 , P. R. China."}],"abstract":"In this paper, we propose a novel linear subspace learning algorithm called Flexible Orthogonal Neighborhood Preserving Embedding (FONPE), which is a linear approximation of Locally Linear Embedding (LLE) algorithm. Our novel objective function integrates two terms related to manifold smoothness and a flexible penalty defined on the projection fitness. Different from Neighborhood Preserving Embedding (NPE), we relax the hard constraint by modeling the mismatch between the approximate linear embedding and the original nonlinear embedding instead of enforcing them to be equal, which makes it better cope with the data sampled from a nonlinear manifold. Besides, instead of enforcing an orthogonality between the projected points, we enforce the mapping to be orthogonal. By using this method, FONPE tends to preserve distances and thus the overall geometry can be preserved. Unlike LLE, as FONPE has an explicit linear mapping between the input and the reduced spaces, it can handle novel testing data straightforwardly. Moreover, when the projection matrix in our model becomes an identity matrix, our model can be transformed to denoising LLE (DLLE). Compared with the standard LLE, we demonstrate that DLLE can handle data with noise better. Comprehensive experiments on several benchmark databases demonstrate the effectiveness of our algorithm.","doi":"10.24963/ijcai.2017/361","url":"https://ijcai.org/proceedings/2017/361","pdf":"https://ijcai.org/proceedings/2017/0361.pdf","markdown":"https://ijcai.org/proceedings/2017/361.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/361","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":361,"pages":{"first":2592,"last":2598},"keywords":["Machine Learning: Data Mining","Machine Learning: Machine Learning","Machine Learning: Unsupervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Belhumeur et al., 1997] Peter N. Belhumeur, João P Hes- panha, and David J. Kriegman. Eigenfaces vs. fisher- faces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine In- telligence, 19(7):711-720, 1997.","title":"Eigenfaces vs. fisherfaces: Recognition using class specific linear projection","authors":"N Peter, João P Belhumeur, David J Hespanha, Kriegman","year":"1997","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Belkin and Niyogi, 2003] Mikhail Belkin and Partha Niyo- gi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15(6):1373- 1396, 2003.","title":"Mikhail Belkin and Partha Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation","authors":"Niyogi Belkin","year":"2003","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bengio et al., 2004] Yoshua Bengio, Jean-Franc ¸ois Paiement, Pascal Vincent, Olivier Delalleau, Nicolas Le Roux, and Marie Ouimet. Out-of-sample extensions for lle, isomap, mds, eigenmaps, and spectral clustering. Advances in Neural Information Processing Systems, 16:177-184, 2004.","title":"Out-of-sample extensions for lle, isomap, mds, eigenmaps, and spectral clustering","authors":"Yoshua Bengio, Jean-Franc ¸ois Paiement, Pascal Vincent, Olivier Delalleau, Nicolas Le Roux, Marie Ouimet","year":"2004","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Elbagoury and Ibrahim, 2016] Ahmed Elbagoury and Rani- a Ibrahim. Ebek: Exemplar-based kernel preserving em- bedding. In The International Joint Conference on Artifi- cial Intelligence IJCAI, 2016.","title":"Ebek: Exemplar-based kernel preserving embedding","authors":"Ahmed Elbagoury, Rania Ibrahim","year":"2016","venue":"The International Joint Conference on Artificial Intelligence IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/207"},{"position":5,"raw":"[Feiping Nie and Li, 2017] Wei Zhu Feiping Nie and Xue- long Li. Unsupervised large graph embedding. In AAAI Conference on Artificial Intelligence., 2017.","title":"Unsupervised large graph embedding","authors":"Wei Zhu, Feiping Nie, Xuelong Li","year":"2017","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Georghiades et al., 2001] Athinodoros S. Georghiades, Pe- ter N. Belhumeur, and David J. Kriegman. From few to many: Illumination cone models for face recognition un- der variable lighting and pose. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 23(6):643-660, 2001.","title":"From few to many: Illumination cone models for face recognition under variable lighting and pose","authors":"Athinodoros S Georghiades, Peter N Belhumeur, David J Kriegman","year":"2001","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Graham and Allinson, 1995] Daniel B. Graham and Nigel Allinson. Characterizing virtual eigensignatures for gener- al purpose face recognition. Journal of Nursing Manage- ment, 3(2):87-91, 1995.","title":"Characterizing virtual eigensignatures for general purpose face recognition","authors":"Daniel B Graham, Nigel Allinson","year":"1995","venue":"Journal of Nursing Management","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[H, 2007] William H. Numerical recipes 3rd edition: The art of scientific computing. Cambridge university press, 2007.","title":"Numerical recipes 3rd edition: The art of scientific computing","authors":"H William","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He and Niyogi, 2004] Xiaofei He and Partha Niyogi. Lo- cality preserving projections. In Advances in Neural In- formation Processing Systems, pages 153-160. MIT Press, 2004.","title":"Locality preserving projections","authors":"Xiaofei He, Partha Niyogi","year":"2004","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2005] Xiaofei He, Deng Cai, Shuicheng Yan, and Hong-Jiang Zhang. Neighborhood preserving embedding. In Tenth IEEE International Conference on Computer Vi- sion (ICCV), volume 1, pages 1208-1213. IEEE, 2005.","title":"Neighborhood preserving embedding","authors":"Xiaofei He, Deng Cai, Shuicheng Yan, Hong-Jiang Zhang","year":"2005","venue":"Tenth IEEE International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huang et al., 2015] Zhiwu Huang, Ruiping Wang, Shiguang Shan, and Xilin Chen. Projection metric learning on grassmann manifold with application to video based face recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015.","title":"Projection metric learning on grassmann manifold with application to video based face recognition","authors":"Zhiwu Huang, Ruiping Wang, Shiguang Shan, Xilin Chen","year":"2015","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kokiopoulou and Saad, 2007] Effrosyni Kokiopoulou and Yousef Saad. Orthogonal neighborhood preserving projec- tions: A projection-based dimensionality reduction tech- nique. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 29(12):2143-2156, 2007.","title":"Effrosyni Kokiopoulou and Yousef Saad. Orthogonal neighborhood preserving projections: A projection-based dimensionality reduction technique","authors":"Saad Kokiopoulou","year":"2007","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li and Tao, 2012] Jun Li and Dacheng Tao. On preserving original variables in bayesian pca with application to im- age analysis. IEEE Transactions on Image Processing, 21(12):4830-4843, 2012.","title":"On preserving original variables in bayesian pca with application to image analysis","authors":"Tao ; Li, Jun Li, Dacheng Tao","year":"2012","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2002] Qingshan Liu, Rui Huang, Hanqing Lu, and Songde Ma. Face recognition using kernel-based fish- er discriminant analysis. In IEEE International Confer- ence on Automatic Face and Gesture Recognition., pages 197-201. IEEE, 2002.","title":"Face recognition using kernel-based fisher discriminant analysis","authors":"Qingshan Liu, Rui Huang, Hanqing Lu, Songde Ma","year":"2002","venue":"IEEE International Conference on Automatic Face and Gesture Recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nie et al., 2010] Feiping Nie, Dong Xu, Wai Hung Tsang, and Changshui Zhang. Flexible manifold embedding: A framework for semi-supervised and unsupervised dimen- sion reduction. IEEE Transactions on Image Processing, 19(7):1921-1932, 2010.","title":"Flexible manifold embedding: A framework for semi-supervised and unsupervised dimension reduction","authors":"Nie","year":"2010","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Roweis and Saul, 2000] Sam T Roweis and Lawrence K Saul. Nonlinear dimensionality reduction by locally lin- ear embedding. Science, 290(5500):2323-2326, 2000.","title":"Nonlinear dimensionality reduction by locally linear embedding","authors":"T Sam, Lawrence K Roweis, Saul","year":"2000","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Samaria and Harter, 1994] F. S. Samaria and A. C. Harter. Parameterisation of a stochastic model for human face i- dentification. In Proceedings of the Second IEEE Work- shop on Applications of Computer Vision., pages 138 - 142. IEEE, 1994.","title":"Parameterisation of a stochastic model for human face identification","authors":"F S Samaria, A C Harter","year":"1994","venue":"Proceedings of the Second IEEE Workshop on Applications of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Schölkopf et al., 1998] Bernhard Schölkopf, Alexander S- mola, and Klaus-Robert Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neural Compu- tation, 10(5):1299-1319, 1998.","title":"Nonlinear component analysis as a kernel eigenvalue problem","authors":"Bernhard Schölkopf, Alexander Smola, Klaus-Robert Müller","year":"1998","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Sim et al., 2002] Terence Sim, Simon Baker, and Maan B- sat. The cmu pose, illumination, and expression (pie) database. In IEEE International Conference on Automat- ic Face and Gesture Recognition., pages 46 -51. IEEE, 2002.","title":"The cmu pose, illumination, and expression (pie) database","authors":"Terence Sim, Simon Baker, Maan Bsat","year":"2002","venue":"IEEE International Conference on Automatic Face and Gesture Recognition","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Tenenbaum et al., 2000] Joshua B Tenenbaum, Vin De Sil- va, and John C Langford. A global geometric frame- work for nonlinear dimensionality reduction. Science, 290(5500):2319-2323, 2000.","title":"A global geometric framework for nonlinear dimensionality reduction","authors":"Joshua B Tenenbaum, Vin De Silva, John C Langford","year":"2000","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Turk and Pentland, 1991] Matthew A Turk and Alex P Pent- land. Face recognition using eigenfaces. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 586-591. IEEE, 1991.","title":"Face recognition using eigenfaces","authors":"A Matthew, Alex P Turk, Pentland","year":"1991","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2015a] Hua Wang, Feiping Nie, and Heng Huang. Learning robust locality preserving projection vi- a p-order minimization. In AAAI Conference on Artificial Intelligence., pages 3059-3065, 2015.","title":"Learning robust locality preserving projection via p-order minimization","authors":"Wang","year":"2015","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2015b] Rong Wang, Feiping Nie, Xiaojun Yang, Feifei Gao, and Minli Yao. Robust 2dpca with non-greedy-norm maximization for image analysis. IEEE transactions on cybernetics, 45(5):1108-1112, 2015.","title":"Robust 2dpca with non-greedy-norm maximization for image analysis","authors":"Wang","year":"2015","venue":"IEEE transactions on cybernetics","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Xu et al., 2016] Jinglin Xu, Junwei Han, and Feiping Nie. Discriminatively embedded k-means for multi-view clus- tering. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.","title":"Discriminatively embedded k-means for multi-view clustering","authors":"Jinglin Xu, Junwei Han, Feiping Nie","year":"2016","venue":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}