{"type":"ScholarlyArticle","title":"Learning Strictly Orthogonal p-Order Nonnegative Laplacian Embedding via Smoothed Iterative Reweighted Method","authors":[{"name":"Haoxuan Yang","slug":"haoxuan-yang","given":"Haoxuan","family":"Yang","affiliation":"Department of Computer Science , Colorado School of Mines , Golden , CO 80401 , U.S.A."},{"name":"Kai Liu","slug":"kai-liu","given":"Kai","family":"Liu","affiliation":"Department of Computer Science , Colorado School of Mines , Golden , CO 80401 , U.S.A."},{"name":"Hua Wang","slug":"hua-wang","given":"Hua","family":"Wang","affiliation":"Department of Computer Science , Colorado School of Mines , Golden , CO 80401 , U.S.A."},{"name":"Feiping Nie","slug":"feiping-nie","given":"Feiping","family":"Nie","affiliation":"School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL) , Northwestern Polytechnical University , Xi'an 710072 , Shaanxi , P. R. China"}],"abstract":"Laplacian Embedding (LE) is a powerful method to reveal the intrinsic geometry of high-dimensional data by using graphs. Imposing the orthogonal and nonnegative constraints onto the LE objective has proved to be effective to avoid degenerate and negative solutions, which, though, are challenging to achieve simultaneously because they are nonlinear and nonconvex. In addition, recent studies have shown that using the p-th order of the L2-norm distances in LE can find the best solution for clustering and promote the robustness of the embedding model against outliers, although this makes the optimization objective nonsmooth and difficult to efficiently solve in general. In this work, we study LE that uses the p-th order of the L2-norm distances and satisfies both orthogonal and nonnegative constraints.  We introduce a novel smoothed iterative reweighted method to tackle this challenging optimization problem and rigorously analyze its convergence.  We demonstrate the effectiveness and potential of our proposed method by extensive empirical studies on both synthetic and real data sets.","doi":"10.24963/ijcai.2019/561","url":"https://ijcai.org/proceedings/2019/561","pdf":"https://ijcai.org/proceedings/2019/0561.pdf","markdown":"https://ijcai.org/proceedings/2019/561.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/561","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":561,"pages":{"first":4040,"last":4046},"keywords":["Machine Learning: Dimensionality Reduction and Manifold Learning","Machine Learning: Unsupervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Bertsekas, 1996] Dimitri P Bertsekas. Constrained opti- mization and Lagrange multiplier methods. Academic press, 1996.","title":"Constrained optimization and Lagrange multiplier methods","authors":"Dimitri P Bertsekas","year":"1996","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boyd and Vandenberghe, 2004] Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge Univer- sity Press, 2004.","title":"Convex optimization","authors":"Vandenberghe Boyd","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Boyd et al., 2011] Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. Distributed opti- mization and statistical learning via the alternating direc- tion method of multipliers. Foundations and Trends R in Machine Learning, 3(1):1-122, 2011.","title":"Distributed optimization and statistical learning via the alternating direction method of multipliers","authors":"Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein","year":"2011","venue":"Foundations and Trends R in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Candes et al., 2008] Emmanuel J Candes, Michael B Wakin, and Stephen P Boyd. Enhancing sparsity by reweighted l-1 minimization. Journal of Fourier analysis and applications, 14(5-6):877-905, 2008.","title":"Enhancing sparsity by reweighted l-1 minimization","authors":"J Emmanuel, Candes, Stephen P Michael B Wakin, Boyd","year":"2008","venue":"Journal of Fourier analysis and applications","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chan et al., 1994] Pak K Chan, Martine DF Schlag, and Ja- son Y Zien. Spectral k-way ratio-cut partitioning and clus- tering. IEEE Transactions on computer-aided design of integrated circuits and systems, 13(9):1088-1096, 1994.","title":"Spectral k-way ratio-cut partitioning and clustering","authors":"Martine Df Pak K Chan, Jason Y Schlag, Zien","year":"1994","venue":"IEEE Transactions on computer-aided design of integrated circuits and systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ding and He, 2004] Chris Ding and Xiaofeng He. K-means clustering via principal component analysis. In Proceed- ings of the twenty-first international conference on Ma- chine learning, page 29. ACM, 2004.","title":"K-means clustering via principal component analysis","authors":"Chris Ding, Xiaofeng He","year":"2004","venue":"Proceedings of the twenty-first international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ding et al., 2005] Chris Ding, Xiaofeng He, and Horst D Si- mon. On the equivalence of nonnegative matrix factoriza- tion and spectral clustering. In Proceedings of the 2005 SIAM International Conference on Data Mining, pages 606-610. SIAM, 2005.","title":"On the equivalence of nonnegative matrix factorization and spectral clustering","authors":"Chris Ding, Xiaofeng He, Horst D Simon","year":"2005","venue":"Proceedings of the 2005 SIAM International Conference on Data Mining","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ding et al., 2006] Chris Ding, Tao Li, Wei Peng, and Hae- sun Park. Orthogonal nonnegative matrix t-factorizations for clustering. In Proceedings of the 12th ACM SIGKD- D international conference on Knowledge discovery and data mining, pages 126-135. ACM, 2006.","title":"Orthogonal nonnegative matrix t-factorizations for clustering","authors":"Chris Ding, Tao Li, Wei Peng, Haesun Park","year":"2006","venue":"Proceedings of the 12th ACM SIGKD-D international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hagen and Kahng, 1992] Lars Hagen and Andrew B Kah- ng. New spectral methods for ratio cut partitioning and clustering. IEEE transactions on computer-aided design of integrated circuits and systems, 11(9):1074-1085, 1992.","title":"New spectral methods for ratio cut partitioning and clustering","authors":"Kahng Hagen, Lars Hagen, Andrew B Kahng","year":"1992","venue":"IEEE transactions on computer-aided design of integrated circuits and systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hall, 1970] Kenneth M Hall. An r-dimensional quadratic placement algorithm. Management science, 17(3):219- 229, 1970.","title":"An r-dimensional quadratic placement algorithm","authors":"M Kenneth, Hall","year":"1970","venue":"Management science","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hartigan and Wong, 1979] John A Hartigan and Manchek A Wong. Algorithm as 136: A k-means clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics), 28(1):100-108, 1979.","title":"Algorithm as 136: A k-means clustering algorithm","authors":"A John, Manchek A Hartigan, Wong","year":"1979","venue":"Journal of the Royal Statistical Society. Series C (Applied Statistics)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lee and Seung, 2001] Daniel D Lee and H Sebastian Se- ung. Algorithms for non-negative matrix factorization. In Advances in neural information processing systems, pages 556-562, 2001.","title":"Algorithms for non-negative matrix factorization","authors":"D Daniel, H Lee, Seung Sebastian","year":"2001","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2017] Yun Liu, Yiming Guo, Hua Wang, Feiping Nie, and Heng Huang. Semi-supervised classifications via elastic and robust embedding. In Thirty-First AAAI Con- ference on Artificial Intelligence (AAAI 2017), 2017.","title":"Semi-supervised classifications via elastic and robust embedding","authors":"Yun Liu, Yiming Guo, Hua Wang, Feiping Nie, Heng Huang","year":"2017","venue":"Thirty-First AAAI Conference on Artificial Intelligence (AAAI 2017)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Luo et al., 2009] Dijun Luo, Chris Ding, Heng Huang, and Tao Li. Non-negative laplacian embedding. In Data Min- ing, 2009. ICDM'09. Ninth IEEE International Confer- ence on, pages 337-346. IEEE, 2009.","title":"Non-negative laplacian embedding","authors":"Luo","year":"2009","venue":"ICDM'09. Ninth IEEE International Conference on","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nie et al., 2010] Feiping Nie, Heng Huang, Xiao Cai, and Chris H Ding. Efficient and robust feature selection via joint ℓ 2,1 -norms minimization. In Advances in neural in- formation processing systems, pages 1813-1821, 2010.","title":"Efficient and robust feature selection via joint ℓ 2,1 -norms minimization","authors":"Nie","year":"2010","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nie et al., 2013] Feiping Nie, Hua Wang, Heng Huang, and Chris HQ Ding. Early active learning via robust represen- tation and structured sparsity. In IJCAI, pages 1572-1578, 2013.","title":"Early active learning via robust representation and structured sparsity","authors":"Nie","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nie et al., 2016] Feiping Nie, Hua Wang, Cheng Deng, Xin- bo Gao, Xuelong Li, Heng Huang, et al. New ℓ 1 -norm re- laxations and optimizations for graph clustering. In AAAI, pages 1962-1968, 2016.","title":"New ℓ 1 -norm relaxations and optimizations for graph clustering","authors":"Nie","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shi and Malik, 2000] Jianbo Shi and Jitendra Malik. Nor- malized cuts and image segmentation. IEEE Transactions on pattern analysis and machine intelligence, 22(8):888- 905, 2000.","title":"Normalized cuts and image segmentation","authors":"Jianbo Shi, Jitendra Malik","year":"2000","venue":"IEEE Transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2010] Hua Wang, Chris Ding, and Heng Huang. Directed graph learning via high-order co-linkage analysis. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 451-466. Springer, 2010.","title":"Directed graph learning via high-order co-linkage analysis","authors":"Hua Wang, Chris Ding, Heng Huang","year":"2010","venue":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2011] Hua Wang, Feiping Nie, and Heng Huang. Learning instance specific distance for multi- instance classification. In Twenty-Fifth AAAI Conference on Artificial Intelligence, 2011.","title":"Learning instance specific distance for multiinstance classification","authors":"Hua Wang, Feiping Nie, Heng Huang","year":"2011","venue":"Twenty-Fifth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2012] Hua Wang, Feiping Nie, and Heng Huang. Robust and discriminative distance for multi- instance learning. In Computer Vision and Pattern Recog- nition (CVPR), 2012 IEEE Conference on, pages 2919- 2924. IEEE, 2012.","title":"Robust and discriminative distance for multiinstance learning","authors":"Hua Wang, Feiping Nie, Heng Huang","year":"2012","venue":"Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2013a] Hua Wang, Feiping Nie, Weidong Cai, and Heng Huang. Semi-supervised robust dictionary learning via efficient ℓ 2,0 + -norms minimization. In Pro- ceedings of the IEEE International Conference on Com- puter Vision, pages 1145-1152, 2013.","title":"Semi-supervised robust dictionary learning via efficient ℓ 2,0 + -norms minimization","authors":"Wang","year":"2013","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2013b] Hua Wang, Feiping Nie, and Heng Huang. Multi-view clustering and feature learning via structured sparsity. In International conference on ma- chine learning, pages 352-360, 2013.","title":"Multi-view clustering and feature learning via structured sparsity","authors":"Wang","year":"2013","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2013c] Hua Wang, Feiping Nie, and Heng Huang. Robust and discriminative self-taught learning. In International conference on machine learning, pages 298- 306, 2013.","title":"Robust and discriminative self-taught learning","authors":"Wang","year":"2013","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2014] Hua Wang, Feiping Nie, and Heng Huang. Robust distance metric learning via simultaneous ℓ 1 -norm minimization and maximization. In Internation- al Conference on Machine Learning, pages 1836-1844, 2014.","title":"Robust distance metric learning via simultaneous ℓ 1 -norm minimization and maximization","authors":"Hua Wang, Feiping Nie, Heng Huang","year":"2014","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2015] Hua Wang, Feiping Nie, and Heng Huang. Learning robust locality preserving projection via p-order minimization. In AAAI, pages 3059-3065, 2015.","title":"Learning robust locality preserving projection via p-order minimization","authors":"Hua Wang, Feiping Nie, Heng Huang","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}