{"type":"ScholarlyArticle","title":"Learning Robust Distance Metric with Side Information via Ratio Minimization of Orthogonally Constrained L21-Norm Distances","authors":[{"name":"Kai Liu","slug":"kai-liu","given":"Kai","family":"Liu","affiliation":"Department of Computer Science , Colorado School of Mines , Golden , CO 80401 , U.S.A."},{"name":"Lodewijk Brand","slug":"lodewijk-brand","given":"Lodewijk","family":"Brand","affiliation":"Department of Computer Science , Colorado School of Mines , Golden , CO 80401 , U.S.A."},{"name":"Hua Wang","slug":"hua-wang","given":"Hua","family":"Wang","affiliation":"Department of Computer Science , Colorado School of Mines , Golden , CO 80401 , U.S.A."},{"name":"Feiping Nie","slug":"feiping-nie","given":"Feiping","family":"Nie","affiliation":"School of Computer Science and Center for OPTical IMagery Analysis and Learning (OPTIMAL) , Northwestern Polytechnical University , Xi'an 710072 , Shaanxi , P. R. China"}],"abstract":"Metric Learning, which aims at learning a distance metric for a given data set, plays an important role in measuring the distance or similarity between data objects. Due to its broad usefulness, it has attracted a lot of interest in machine learning and related areas in the past few decades. This paper proposes to learn the distance metric from the side information in the forms of must-links and cannot-links. Given the pairwise constraints, our goal is to learn a Mahalanobis distance that minimizes the ratio of the distances of the data pairs in the must-links to those in the cannot-links. Different from many existing papers that use the traditional squared L2-norm distance, we develop a robust model that is less sensitive to data noise or outliers by using the not-squared L2-norm distance.  In our objective, the orthonormal constraint is enforced to avoid degenerate solutions. To solve our objective, we have derived an efficient iterative solution algorithm. We have conducted extensive experiments, which demonstrated the superiority of our method over state-of-the-art.","doi":"10.24963/ijcai.2019/417","url":"https://ijcai.org/proceedings/2019/417","pdf":"https://ijcai.org/proceedings/2019/0417.pdf","markdown":"https://ijcai.org/proceedings/2019/417.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/417","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":417,"pages":{"first":3008,"last":3014},"keywords":["Computer Vision: Statistical Methods and Machine Learning","Machine Learning: Semi-Supervised Learning","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Bar-Hillel et al., 2003] Aharon Bar-Hillel, Tomer Hertz, Noam Shental, and Daphna Weinshall. Learning distance functions using equivalence relations. In Proceedings of the 20th International Conference on Machine Learning (ICML-03), pages 11-18, 2003.","title":"Learning distance functions using equivalence relations","authors":"Bar-Hillel","year":"2003","venue":"Proceedings of the 20th International Conference on Machine Learning (ICML-03)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bellet et al., 2013] Aurélien Bellet, Amaury Habrard, and Marc Sebban. A survey on metric learning for feature vec- tors and structured data. arXiv preprint arXiv:1306.6709, 2013.","title":"A survey on metric learning for feature vectors and structured data","authors":"Aurélien Bellet, Amaury Habrard, Marc Sebban","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Boyd et al., 2011] Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed op- timization and statistical learning via the alternating direc- tion method of multipliers. Foundations and Trends R in Machine learning, 3(1):1-122, 2011.","title":"Distributed optimization and statistical learning via the alternating direction method of multipliers","authors":"Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein","year":"2011","venue":"Foundations and Trends R in Machine learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brand et al., 2019] Lodewijk Brand, Xue Yang, Kai Liu, Saad Elbeleidy, Hua Wang, and Hao Zhang. Learning ro- bust multi-label sample specific distances for identifying HIV-1 drug resistance. In International Conference on Re- search in Computational Molecular Biology, pages 51-67. Springer, 2019.","title":"Learning robust multi-label sample specific distances for identifying HIV-1 drug resistance","authors":"Lodewijk Brand, Xue Yang, Kai Liu, Saad Elbeleidy, Hua Wang, Hao Zhang","year":"2019","venue":"International Conference on Research in Computational Molecular Biology","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Davis et al., 2007] Jason V Davis, Brian Kulis, Prateek Jain, Suvrit Sra, and Inderjit S Dhillon. Information-theoretic metric learning. In Proceedings of the 24th international conference on Machine learning, pages 209-216. ACM, 2007.","title":"Information-theoretic metric learning","authors":"Jason V Davis, Brian Kulis, Prateek Jain, Suvrit Sra, Inderjit S Dhillon","year":"2007","venue":"Proceedings of the 24th international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ding et al., 2006] C. Ding, D. Zhou, X. He, and H. Zha. R1-pca: rotational invariant l 1-norm principal component analysis for robust subspace factorization. In ICML, 2006.","title":"R1-pca: rotational invariant l 1-norm principal component analysis for robust subspace factorization","authors":"C Ding, D Zhou, X He, H Zha","year":"2006","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fisher, 1936] Ronald A Fisher. The use of multiple mea- surements in taxonomic problems. Annals of eugenics, 7(2):179-188, 1936.","title":"The use of multiple measurements in taxonomic problems","authors":"Ronald A Fisher","year":"1936","venue":"Annals of eugenics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Guo and Ying, 2014] Zheng-Chu Guo and Yiming Ying. Guaranteed classification via regularized similarity learn- ing. Neural Computation, 26(3):497-522, 2014.","title":"Guaranteed classification via regularized similarity learning","authors":"Zheng-Chu Guo, Yiming Ying","year":"2014","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hoi et al., 2006] Steven CH Hoi, Wei Liu, Michael R Lyu, and Wei-Ying Ma. Learning distance metrics with contex- tual constraints for image retrieval. In IEEE CVPR, vol- ume 2, pages 2072-2078, 2006.","title":"Learning distance metrics with contextual constraints for image retrieval","authors":"C H Steven, Wei Hoi, Liu, Wei-Ying Michael R Lyu, Ma","year":"2006","venue":"IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Huang et al., 2012] Kaizhu Huang, Rong Jin, Zenglin Xu, and Cheng-Lin Liu. Robust metric learning by smooth optimization. arXiv preprint arXiv:1203.3461, 2012.","title":"Robust metric learning by smooth optimization","authors":"Kaizhu Huang, Rong Jin, Zenglin Xu, Cheng-Lin Liu","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Koestinger et al., 2012] Martin Koestinger, Martin Hirzer, Paul Wohlhart, Peter M Roth, and Horst Bischof. Large scale metric learning from equivalence constraints. In IEEE CVPR, pages 2288-2295, 2012.","title":"Large scale metric learning from equivalence constraints","authors":"Martin Koestinger, Martin Hirzer, Paul Wohlhart, Peter M Roth, Horst Bischof","year":"2012","venue":"IEEE CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kwak, 2008] N. Kwak. Principal component analysis based on l1-norm maximization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30:1672-1680, 2008.","title":"Principal component analysis based on l1-norm maximization","authors":"N Kwak","year":"2008","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li and Tang, 2015] Zechao Li and Jinhui Tang. Weakly su- pervised deep metric learning for community-contributed image retrieval. IEEE Transactions on Multimedia, 17(11):1989-1999, 2015.","title":"Weakly supervised deep metric learning for community-contributed image retrieval","authors":"Li, Tang ; ] Zechao Li, Jinhui Tang","year":"2015","venue":"IEEE Transactions on Multimedia","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu and Wang, 2015] Kai Liu and Hua Wang. Robust multi- relational clustering via ℓ 1 -norm symmetric nonnegative matrix factorization. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguis- tics and the 7th International Joint Conference on Natural Language Processing, volume 2, pages 397-401, 2015.","title":"Robust multirelational clustering via ℓ 1 -norm symmetric nonnegative matrix factorization","authors":"Kai Liu, Hua Wang","year":"2015","venue":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu and Wang, 2018] Kai Liu and Hua Wang. High-order co-clustering via strictly orthogonal and symmetric ℓ 1 - norm nonnegative matrix tri-factorization. In IJCAI, pages 2454-2460, 2018.","title":"High-order co-clustering via strictly orthogonal and symmetric ℓ 1norm nonnegative matrix tri-factorization","authors":"Kai Liu, Hua Wang","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2017] Yun Liu, Yiming Guo, Hua Wang, Feiping Nie, and Heng Huang. Semi-supervised classifications via elastic and robust embedding. In Thirty-First AAAI Con- ference on Artificial Intelligence, 2017.","title":"Semi-supervised classifications via elastic and robust embedding","authors":"Yun Liu, Yiming Guo, Hua Wang, Feiping Nie, Heng Huang","year":"2017","venue":"Thirty-First AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2018] Kai Liu, Hua Wang, Feiping Nie, and Hao Zhang. Learning multi-instance enriched image represen- tations via non-greedy ratio maximization of the ℓ 1 -norm distances. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pages 7727-7735, 2018.","title":"Learning multi-instance enriched image representations via non-greedy ratio maximization of the ℓ 1 -norm distances","authors":"Kai Liu, Hua Wang, Feiping Nie, Hao Zhang","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Schönemann, 1966] Peter H Schönemann. A generalized solution of the orthogonal procrustes problem. Psychome- trika, 31(1):1-10, 1966.","title":"A generalized solution of the orthogonal procrustes problem","authors":"Peter H Schönemann","year":"1966","venue":"Psychometrika","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2011] Hua Wang, Feiping Nie, and Heng Huang. Learning instance specific distance for multi- instance classification. In Twenty-Fifth AAAI Conference on Artificial Intelligence, 2011.","title":"Learning instance specific distance for multiinstance classification","authors":"Hua Wang, Feiping Nie, Heng Huang","year":"2011","venue":"Twenty-Fifth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wang et al., 2012] Hua Wang, Feiping Nie, and Heng Huang. Robust and discriminative distance for multi- instance learning. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 2919-2924, 2012.","title":"Robust and discriminative distance for multiinstance learning","authors":"Hua Wang, Feiping Nie, Heng Huang","year":"2012","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2013a] Hua Wang, Feiping Nie, and Heng Huang. Multi-view clustering and feature learning via structured sparsity. In ICML (3), pages 352-360, 2013.","title":"Multi-view clustering and feature learning via structured sparsity","authors":"Wang","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2013b] Qianying Wang, Pong C Yuen, and Guocan Feng. Semi-supervised metric learning via topol- ogy preserving multiple semi-supervised assumptions. Pattern Recognition, 46(9):2576-2587, 2013.","title":"Semi-supervised metric learning via topology preserving multiple semi-supervised assumptions","authors":"Wang","year":"2013","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wang et al., 2014a] Hua Wang, Feiping Nie, and Heng Huang. Globally and locally consistent unsupervised pro- jection. In Twenty-Eighth AAAI Conference on Artificial Intelligence, 2014.","title":"Globally and locally consistent unsupervised projection","authors":"Wang","year":"2014","venue":"Twenty-Eighth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2014b] Hua Wang, Feiping Nie, and Heng Huang. Robust distance metric learning via simultane- ous ℓ 1 -norm minimization and maximization. In Inter- national Conference on Machine Learning, pages 1836- 1844, 2014.","title":"Robust distance metric learning via simultaneous ℓ 1 -norm minimization and maximization","authors":"Wang","year":"2014","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Weinberger and Saul, 2009] Kilian Q Weinberger and Lawrence K Saul. Distance metric learning for large mar- gin nearest neighbor classification. Journal of Machine Learning Research, 10(Feb):207-244, 2009.","title":"Distance metric learning for large margin nearest neighbor classification","authors":"Q Kilian, Lawrence K Weinberger, Saul","year":"2009","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Weinberger et al., 2006] Kilian Q Weinberger, John Blitzer, and Lawrence K Saul. Distance metric learning for large margin nearest neighbor classification. In NIPS, pages 1473-1480, 2006.","title":"Distance metric learning for large margin nearest neighbor classification","authors":"John Kilian Q Weinberger, Lawrence K Blitzer, Saul","year":"2006","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wright et al., 2009] John Wright, Arvind Ganesh, Shankar Rao, Yigang Peng, and Yi Ma. Robust principal compo- nent analysis: Exact recovery of corrupted. Advances in Neural Information Processing Systems, page 116, 2009.","title":"Robust principal component analysis: Exact recovery of corrupted","authors":"John Wright, Arvind Ganesh, Shankar Rao, Yigang Peng, Yi Ma","year":"2009","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Xiang et al., 2008] Shiming Xiang, Feiping Nie, and Changshui Zhang. Learning a mahalanobis distance metric for data clustering and classification. Pattern Recognition, 41(12):3600-3612, 2008.","title":"Learning a mahalanobis distance metric for data clustering and classification","authors":"Shiming Xiang, Feiping Nie, Changshui Zhang","year":"2008","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Xing et al., 2003] Eric P Xing, Michael I Jordan, Stuart J Russell, and Andrew Y Ng. Distance metric learning with application to clustering with side-information. In NIPS, pages 521-528, 2003.","title":"Distance metric learning with application to clustering with side-information","authors":"Eric P Xing, Stuart J Michael I Jordan, Andrew Y Russell, Ng","year":"2003","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Xu et al., 2003] Wei Xu, Xin Liu, and Yihong Gong. Doc- ument clustering based on non-negative matrix factoriza- tion. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in infor- maion retrieval, pages 267-273. ACM, 2003.","title":"Document clustering based on non-negative matrix factorization","authors":"Wei Xu, Xin Liu, Yihong Gong","year":"2003","venue":"Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}