{"type":"ScholarlyArticle","title":"Beyond Similar and Dissimilar Relations : A Kernel Regression Formulation for Metric Learning","authors":[{"name":"Pengfei Zhu","slug":"pengfei-zhu","given":"Pengfei","family":"Zhu","affiliation":"School of Computer Science and Technology , Tianjin University , Tianjin 300350 , China"},{"name":"Ren Qi","slug":"ren-qi","given":"Ren","family":"Qi","affiliation":"School of Computer Science and Technology , Tianjin University , Tianjin 300350 , China"},{"name":"Qinghua Hu","slug":"qinghua-hu","given":"Qinghua","family":"Hu","affiliation":"School of Computer Science and Technology , Tianjin University , Tianjin 300350 , China"},{"name":"Qilong Wang","slug":"qilong-wang","given":"Qilong","family":"Wang","affiliation":"School of Computer Science and Technology , Tianjin University , Tianjin 300350 , China"},{"name":"Changqing Zhang","slug":"changqing-zhang","given":"Changqing","family":"Zhang","affiliation":"School of Computer Science and Technology , Tianjin University , Tianjin 300350 , China"},{"name":"Liu Yang","slug":"liu-yang","given":"Liu","family":"Yang","affiliation":"School of Computer Science and Technology , Tianjin University , Tianjin 300350 , China"}],"abstract":"Most existing metric learning methods focus on learning a similarity or distance measure relying on similar and dissimilar relations between sample pairs. However, pairs of samples cannot be simply identified as similar or dissimilar in many real-world applications, e.g., multi-label learning, label distribution learning or tasks with continuous decision values. To this end, in this paper we propose a novel relation alignment metric learning (RAML)  formulation to handle the metric learning problem in those scenarios. Since the relation of two samples can be measured by the difference degree of the decision values, motivated by the consistency of the sample relations in the feature space and decision space, our proposed RAML utilizes the sample relations in the decision space to guide the metric learning in the feature space. Specifically, our RAML method formulates metric learning as a kernel regression problem, which can be efficiently optimized by the standard regression solvers. We carry out several experiments on the single-label classification, multi-label classification, and label distribution learning tasks, to demonstrate that our method achieves favorable performance against the state-of-the-art methods.","doi":"10.24963/ijcai.2018/450","url":"https://ijcai.org/proceedings/2018/450","pdf":"https://ijcai.org/proceedings/2018/0450.pdf","markdown":"https://ijcai.org/proceedings/2018/450.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/450","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":450,"pages":{"first":3242,"last":3248},"keywords":["Machine Learning: Multi-instance;Multi-label;Multi-view learning","Machine Learning: Dimensionality Reduction and Manifold Learning","Machine Learning: Classification"],"videos":[],"references":[{"position":1,"raw":"[Bhadra et al., 2017] Sahely Bhadra, Samuel Kaski, and Juho Rousu. Multi-view kernel completion. Machine Learning, 106(5):713-739, 2017.","title":"Multi-view kernel completion","authors":"Bhadra","year":"2017","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bhatia et al., 2015] Kush Bhatia, Himanshu Jain, Purushot- tam Kar, Manik Varma, and Prateek Jain. Sparse local em- beddings for extreme multi-label classification. In NIPS, pages 730-738, 2015.","title":"Sparse local embeddings for extreme multi-label classification","authors":"Bhatia","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Boutell et al., 2004] Matthew R Boutell, Jiebo Luo, Xipeng Shen, and Christopher M Brown. Learning multi-label scene classification. Pattern recognition, 37(9):1757- 1771, 2004.","title":"Learning multi-label scene classification","authors":"Jiebo Matthew R Boutell, Xipeng Luo, Christopher M Shen, Brown","year":"2004","venue":"Pattern recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cha, 2007] Sung-Hyuk Cha. Comprehensive survey on distance/similarity measures between probability density functions. City, 1(2):1, 2007.","title":"Comprehensive survey on distance/similarity measures between probability density functions","authors":"Sung-Hyuk Cha","year":"2007","venue":"City","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cortes et al., 2012] Corinna Cortes, Mehryar Mohri, and Afshin Rostamizadeh. Algorithms for learning kernels based on centered alignment. Journal of Machine Learn- ing Research, 13(Mar):795-828, 2012.","title":"Algorithms for learning kernels based on centered alignment","authors":"Corinna Cortes, Mehryar Mohri, Afshin Rostamizadeh","year":"2012","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Davis et al., 2007] Jason V Davis, Brian Kulis, Prateek Jain, Suvrit Sra, and Inderjit S Dhillon. Information-theoretic metric learning. In ICML, pages 209-216, 2007.","title":"Information-theoretic metric learning","authors":"Jason V Davis, Brian Kulis, Prateek Jain, Suvrit Sra, Inderjit S Dhillon","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Drucker et al., 1997] Harris Drucker, Christopher J. C. Burges, Linda Kaufman, Alex J. Smola, and Vladimir Vapnik. Support vector regression machines. NIPS, 28(7):779-784, 1997.","title":"Support vector regression machines","authors":"Harris Drucker, J C Christopher, Linda Burges, Alex J Kaufman, Vladimir Smola, Vapnik","year":"1997","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Geng, 2016] Xin Geng. Label distribution learning. IEEE TKDE, 28(7):1734-1748, 2016.","title":"Label distribution learning","authors":"Xin Geng","year":"2016","venue":"IEEE TKDE","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gu et al., 2016] Yuwen Gu, Hui Zou, et al. High- dimensional generalizations of asymmetric least squares regression and their applications. The Annals of Statistics, 44(6):2661-2694, 2016.","title":"Highdimensional generalizations of asymmetric least squares regression and their applications","authors":"Yuwen Gu, Hui Zou","year":"2016","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Guillaumin et al., 2009] Matthieu Guillaumin, Jakob Ver- beek, and Cordelia Schmid. Is that you? metric learning approaches for face identification. In ICCV, pages 498- 505. IEEE, 2009.","title":"Is that you? metric learning approaches for face identification","authors":"Matthieu Guillaumin, Jakob Verbeek, Cordelia Schmid","year":"2009","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hoi et al., 2006] Steven CH Hoi, Wei Liu, Michael R Lyu, and Wei-Ying Ma. Learning distance metrics with contex- tual constraints for image retrieval. In CVPR, volume 2, pages 2072-2078, 2006.","title":"Learning distance metrics with contextual constraints for image retrieval","authors":"C H Steven, Wei Hoi, Liu, Wei-Ying Michael R Lyu, Ma","year":"2006","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Law et al., 2013] Marc T Law, Nicolas Thome, and Matthieu Cord. Quadruplet-wise image similarity learn- ing. In ICCV, pages 249-256, 2013.","title":"Quadruplet-wise image similarity learning","authors":"Nicolas Marc T Law, Matthieu Thome, Cord","year":"2013","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2012] Zhen Li, Liangliang Cao, Shiyu Chang, John R Smith, and Thomas S Huang. Beyond mahalanobis distance: Learning second-order discriminant function for people verification. In CVPRW, pages 45-50. IEEE, 2012.","title":"Beyond mahalanobis distance: Learning second-order discriminant function for people verification","authors":"Zhen Li, Liangliang Cao, Shiyu Chang, John R Smith, Thomas S Huang","year":"2012","venue":"CVPRW","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liao et al., 2015] Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Z Li. Person re-identification by local maximal occurrence representation and metric learning. In CVPR, pages 2197-2206, 2015.","title":"Person re-identification by local maximal occurrence representation and metric learning","authors":"Liao","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[McFee and Lanckriet, 2011] Brian McFee and Gert Lanck- riet. Learning multi-modal similarity. Journal of machine learning research, 12(Feb):491-523, 2011.","title":"Learning multi-modal similarity","authors":"Brian Mcfee, Gert Lanckriet","year":"2011","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mensink et al., 2012] Thomas Mensink, Jakob Verbeek, Florent Perronnin, and Gabriela Csurka. Metric learning for large scale image classification: Generalizing to new classes at near-zero cost. ECCV, pages 488-501, 2012.","title":"Metric learning for large scale image classification: Generalizing to new classes at near-zero cost","authors":"Thomas Mensink, Jakob Verbeek, Florent Perronnin, Gabriela Csurka","year":"2012","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mu et al., 2010] Yadong Mu, Jialie Shen, and Shuicheng Yan. Weakly-supervised hashing in kernel space. In CVPR, pages 3344-3351. IEEE, 2010.","title":"Weakly-supervised hashing in kernel space","authors":"Yadong Mu, Jialie Shen, Shuicheng Yan","year":"2010","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Oh Song et al., 2016] Hyun Oh Song, Yu Xiang, Stefanie Jegelka, and Silvio Savarese. Deep metric learning via lift- ed structured feature embedding. In CVPR, pages 4004- 4012, 2016.","title":"Deep metric learning via lifted structured feature embedding","authors":"Hyun Oh Song, Yu Xiang, Stefanie Jegelka, Silvio Savarese","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shaw et al., 2011] Blake Shaw, Bert Huang, and Tony Je- bara. Learning a distance metric from a network. In NIPS, pages 1899-1907, 2011.","title":"Learning a distance metric from a network","authors":"Blake Shaw, Bert Huang, Tony Jebara","year":"2011","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Trohidis et al., 2008] Konstantinos Trohidis, Grigorios T- soumakas, George Kalliris, and Ioannis P Vlahavas. Multi- label classification of music into emotions. In ISMIR, vol- ume 8, pages 325-330, 2008.","title":"Multilabel classification of music into emotions","authors":"Konstantinos Trohidis, Grigorios Tsoumakas, George Kalliris, Ioannis P Vlahavas","year":"2008","venue":"ISMIR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2015] Faqiang Wang, Wangmeng Zuo, Lei Zhang, Deyu Meng, and David Zhang. A kernel clas- sification framework for metric learning. IEEE TNNLS, 26(9):1950-1962, 2015.","title":"A kernel classification framework for metric learning","authors":"Wang","year":"2015","venue":"IEEE TNNLS","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2016] Kaiye Wang, Ran He, Liang Wang, Wei Wang, and Tieniu Tan. Joint feature selection and sub- space learning for cross-modal retrieval. IEEE TPAMI, 38(10):2010-2023, 2016.","title":"Joint feature selection and subspace learning for cross-modal retrieval","authors":"Wang","year":"2016","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Weinberger and Saul, 2009] Kilian Q Weinberger and Lawrence K Saul. Distance metric learning for large mar- gin nearest neighbor classification. Journal of Machine Learning Research, 10(Feb):207-244, 2009.","title":"Distance metric learning for large margin nearest neighbor classification","authors":"Q Kilian, Lawrence K Weinberger, Saul","year":"2009","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2006] Liu Yang, Rong Jin, Rahul Sukthankar, and Yi Liu. An efficient algorithm for local distance metric learning. In AAAI, volume 2, pages 543-548, 2006.","title":"An efficient algorithm for local distance metric learning","authors":"Liu Yang, Rong Jin, Rahul Sukthankar, Yi Liu","year":"2006","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Ying and Li, 2012] Yiming Ying and Peng Li. Distance metric learning with eigenvalue optimization. Journal of Machine Learning Research, 13(Jan):1-26, 2012.","title":"Distance metric learning with eigenvalue optimization","authors":"Yiming Ying, Peng Li","year":"2012","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zadeh et al., 2016] Pourya Zadeh, Reshad Hosseini, and Suvrit Sra. Geometric mean metric learning. In ICML, pages 2464-2471, 2016.","title":"Geometric mean metric learning","authors":"Zadeh","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang and Wu, 2015] Min-Ling Zhang and Lei Wu. Lift: Multi-label learning with label-specific features. IEEE T- PAMI, 37(1):107-120, 2015.","title":"Lift: Multi-label learning with label-specific features","authors":"Wu ; ] Min-Ling Zhang, Lei Zhang, Wu","year":"2015","venue":"IEEE T-PAMI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang and Zhou, 2007] Min-Ling Zhang and Zhi-Hua Zhou. Ml-knn: A lazy learning approach to multi-label learning. Pattern recognition, 40(7):2038-2048, 2007.","title":"Ml-knn: A lazy learning approach to multi-label learning","authors":"Zhou ; ] Min-Ling Zhang, Zhi-Hua Zhang, Zhou","year":"2007","venue":"Pattern recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}