{"type":"ScholarlyArticle","title":"Learning Mahalanobis Distance Metric: Considering Instance Disturbance Helps","authors":[{"name":"Han-Jia Ye","slug":"han-jia-ye","given":"Han-Jia","family":"Ye","affiliation":"National Key Laboratory for Novel Software Technology Collaborative Innovation Center of Novel Software Technology and Industrialization; Nanjing University , Nanjing , 210023 , China"},{"name":"De-Chuan Zhan","slug":"de-chuan-zhan","given":"De-Chuan","family":"Zhan","affiliation":"National Key Laboratory for Novel Software Technology Collaborative Innovation Center of Novel Software Technology and Industrialization; Nanjing University , Nanjing , 210023 , China"},{"name":"Xue-Min Si","slug":"xue-min-si","given":"Xue-Min","family":"Si","affiliation":"National Key Laboratory for Novel Software Technology Collaborative Innovation Center of Novel Software Technology and Industrialization; Nanjing University , Nanjing , 210023 , China"},{"name":"Yuan Jiang","slug":"yuan-jiang","given":"Yuan","family":"Jiang","affiliation":"National Key Laboratory for Novel Software Technology Collaborative Innovation Center of Novel Software Technology and Industrialization; Nanjing University , Nanjing , 210023 , China"}],"abstract":"Mahalanobis distance metric takes feature weights and correlation into account in the distance computation, which can improve the performance of many similarity/dissimilarity based methods, such as kNN. Most existing distance metric learning methods obtain metric based on the raw features and side information but neglect the reliability of them. Noises or disturbances on instances will make changes on their relationships, so as to affect the learned metric.In this paper, we claim that considering disturbance of instances may help the distance metric learning approach get a robust metric, and propose the Distance metRIc learning Facilitated by disTurbances (DRIFT) approach. In DRIFT, the noise or the disturbance of each instance is learned. Therefore, the distance between each pair of (noisy) instances can be better estimated, which facilitates side information utilization and metric learning.Experiments on prediction and visualization clearly indicate the effectiveness of the proposed approach.","doi":"10.24963/ijcai.2017/463","url":"https://ijcai.org/proceedings/2017/463","pdf":"https://ijcai.org/proceedings/2017/0463.pdf","markdown":"https://ijcai.org/proceedings/2017/463.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/463","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":463,"pages":{"first":3315,"last":3321},"keywords":["Machine Learning: Feature Selection/Construction","Machine Learning: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Beck and Teboulle, 2009] A. Beck and M. Teboulle. A fast itera- tive shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1):183-202, 2009.","title":"A fast iterative shrinkage-thresholding algorithm for linear inverse problems","authors":"A Beck, M Teboulle","year":"2009","venue":"SIAM Journal on Imaging Sciences","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bellet et al., 2015] A. Bellet, A. Habrard, and M. Sebban. Metric Learning. Synthesis Lectures on Artificial Intelligence and Ma- chine Learning. Morgan & Claypool Publishers, 2015.","title":"Metric Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning","authors":"A Bellet, A Habrard, M Sebban","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bian and Tao, 2011] W. Bian and D. Tao. Learning a distance met- ric by empirical loss minimization. In IJCAI, pages 1186-1191, Barcelona, Spain, 2011.","title":"Learning a distance metric by empirical loss minimization","authors":"W Bian, D Tao","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2011/202"},{"position":4,"raw":"[Boyd and Vandenberghe, 2004] S. Boyd and L. Vandenberghe. Convex optimization. Cambridge university press, 2004.","title":"Convex optimization","authors":"S Boyd, L Vandenberghe","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cao et al., 2016] Q. Cao, Z.-C. Guo, and Y. Ying. Generalization bounds for metric and similarity learning. MLJ, 102(1):115-132, 2016.","title":"Generalization bounds for metric and similarity learning","authors":"Q Cao, Z.-C Guo, Y Ying","year":"2016","venue":"MLJ","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2014] N. Chen, J. Zhu, J. Chen, and B. Zhang. Dropout training for support vector machines. In AAAI, pages 1752-1759, Quebec, Canada, 2014.","title":"Dropout training for support vector machines","authors":"N Chen, J Zhu, J Chen, B Zhang","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2015] M. Chen, K. Q Weinberger, Z. E. Xu, and F. Sha. Marginalizing stacked linear denoising autoencoders. JMLR, 16:3849-3875, 2015.","title":"Marginalizing stacked linear denoising autoencoders","authors":"M Chen, K Weinberger, Z E Xu, F Sha","year":"2015","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Davis et al., 2007] J. V. Davis, B. Kulis, P. Jain, S. Sra, and I. S. Dhillon. Information-theoretic metric learning. In ICML, pages 209-216, Corvalis, OR., 2007.","title":"Information-theoretic metric learning","authors":"J V Davis, B Kulis, P Jain, S Sra, I S Dhillon","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huang et al., 2010] K. Huang, R. Jin, Z. Xu, and C.-L. Liu. Robust metric learning by smooth optimization. In UAI, pages 244-251, Catalina Island, CA., 2010.","title":"Robust metric learning by smooth optimization","authors":"K Huang, R Jin, Z Xu, C.-L Liu","year":"2010","venue":"UAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kulis, 2012] B. Kulis. Metric learning: A survey. Foundations and Trends R ⃝ in Machine Learning, 5(4):287-364, 2012.","title":"Metric learning: A survey","authors":"B Kulis","year":"2012","venue":"Foundations and Trends R ⃝ in Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Law et al., 2016a] M. T Law, N. Thome, and M. Cord. Learning a distance metric from relative comparisons between quadruplets of images. International Journal of Computer Vision, pages 1-30, 2016.","title":"Learning a distance metric from relative comparisons between quadruplets of images","authors":"Law","year":"2016","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Law et al., 2016b] M. T Law, Y. Yu, M. Cord, and E. P Xing. Closed-form training of mahalanobis distance for supervised clustering. In CVPR, pages 3909-3917, Las Vegas, NV., 2016.","title":"Closed-form training of mahalanobis distance for supervised clustering","authors":"Law","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2014] N. Li, R. Jin, and Z.-H. Zhou. Top rank optimiza- tion in linear time. In NIPS, pages 1502-1510. Cambridge, MA.: MIT Press, 2014.","title":"Top rank optimization in linear time","authors":"N Li, R Jin, Z.-H Zhou","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2016] Y. Li, M. Yang, Z. Xu, and Z. Zhang. Learning with marginalized corrupted features and labels together. In AAAI, pages 1251-1257, Phoenix, AZ., 2016.","title":"Learning with marginalized corrupted features and labels together","authors":"Y Li, M Yang, Z Xu, Z Zhang","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lim et al., 2013] D. Lim, G. Lanckriet, and B. McFee. Robust structural metric learning. In ICML, pages 615-623, Atlanta, GA., 2013.","title":"Robust structural metric learning","authors":"D Lim, G Lanckriet, B Mcfee","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Luo et al., 2016] Y. Luo, Y. Wen, and D. Tao. On combining side information and unlabeled data for heterogeneous multi-task met- ric learning. In IJCAI, pages 1809-1815, New York, NY., 2016.","title":"On combining side information and unlabeled data for heterogeneous multi-task metric learning","authors":"Y Luo, Y Wen, D Tao","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/259"},{"position":17,"raw":"[Mao et al., 2016] Q. Mao, L. Wang, and I. W Tsang. A unified probabilistic framework for robust manifold learning and embed- ding. MLJ, pages 1-24, 2016.","title":"A unified probabilistic framework for robust manifold learning and embedding","authors":"Q Mao, L Wang, I W Tsang","year":"2016","venue":"MLJ","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[McFee and Lanckriet, 2010] B. McFee and G. R Lanckriet. Metric learning to rank. In ICML, pages 775-782, Haifa, Israel, 2010.","title":"Metric learning to rank","authors":"B Mcfee, G R Lanckriet","year":"2010","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nesterov, 2004] Y. Nesterov. Introductory lectures on convex opti- mization, volume 87. Springer Science & Business Media, 2004.","title":"Introductory lectures on convex optimization","authors":"Y Nesterov","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Perrot and Habrard, 2015] M. Perrot and A. Habrard. Regressive virtual metric learning. In NIPS, pages 1810-1818. Cambridge, MA.: MIT Press, 2015.","title":"Regressive virtual metric learning","authors":"M Perrot, A Habrard","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Qian et al., 2014] Q. Qian, J. Hu, R. Jin, J. Pei, and S. Zhu. Dis- tance metric learning using dropout: a structured regularization approach. In ACM SIGKDD, pages 323-332, New York, NY., 2014.","title":"Distance metric learning using dropout: a structured regularization approach","authors":"Qian","year":"2014","venue":"ACM SIGKDD","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Qian et al., 2015a] Q. Qian, R. Jin, J. Yi, L. Zhang, and S. Zhu. Efficient distance metric learning by adaptive sampling and mini- batch stochastic gradient descent (sgd). MLJ, 99(3):353-372, 2015.","title":"Efficient distance metric learning by adaptive sampling and minibatch stochastic gradient descent (sgd)","authors":"Qian","year":"2015","venue":"MLJ","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Qian et al., 2015b] Q. Qian, R. Jin, S. Zhu, and Y. Lin. Fine- grained visual categorization via multi-stage metric learning. In CVPR, pages 3716-3724, Boston, MA., 2015.","title":"Finegrained visual categorization via multi-stage metric learning","authors":"Qian","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Van Der Maaten et al., 2013] L. Van Der Maaten, M. Chen, S. Tyree, and K. Q Weinberger. Learning with marginalized cor- rupted features. In ICML, pages 410-418, Atlanta, GA., 2013.","title":"Learning with marginalized corrupted features","authors":"Van Der Maaten","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Verma and Branson, 2015] N. Verma and K. Branson. Sample complexity of learning mahalanobis distance metrics. In NIPS, pages 2584-2592. Cambridge, MA.: MIT Press, 2015.","title":"Sample complexity of learning mahalanobis distance metrics","authors":"N Verma, K Branson","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wager et al., 2013] S. Wager, S. Wang, and P. S Liang. Dropout training as adaptive regularization. In NIPS, pages 351-359. Cambridge, MA.: MIT Press, 2013.","title":"Dropout training as adaptive regularization","authors":"S Wager, S Wang, P S Liang","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2012] J. Wang, A. Woznica, and A. Kalousis. Learn- ing neighborhoods for metric learning. In ECML/PKDD, pages 223-236, Bristol, UK., 2012.","title":"Learning neighborhoods for metric learning","authors":"J Wang, A Woznica, A Kalousis","year":"2012","venue":"ECML/PKDD","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wangni and Chen, 2016] J. Wangni and N. Chen. Nonlinear fea- ture extraction with max-margin data shifting. In AAAI, pages 2208-2214, Phoenix, AZ., 2016.","title":"Nonlinear feature extraction with max-margin data shifting","authors":"J Wangni, N Chen","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Weinberger and Saul, 2009] K. Q Weinberger and L. K. Saul. Dis- tance metric learning for large margin nearest neighbor classifica- tion. JMLR, 10:207-244, 2009.","title":"Distance metric learning for large margin nearest neighbor classification","authors":"K Weinberger, L K Saul","year":"2009","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Weinberger et al., 2006] K. Q Weinberger, J. Blitzer, and L. K. Saul. Distance metric learning for large margin nearest neighbor classification. In NIPS, pages 1473-1480. MIT Press, Cambridge, MA.: MIT Press, 2006.","title":"Distance metric learning for large margin nearest neighbor classification","authors":"K Weinberger, J Blitzer, L K Saul","year":"2006","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xiang et al., 2008] S. Xiang, F. Nie, and C. Zhang. Learning a mahalanobis distance metric for data clustering and classification. Pattern Recognition, 41(12):3600-3612, 2008.","title":"Learning a mahalanobis distance metric for data clustering and classification","authors":"S Xiang, F Nie, C Zhang","year":"2008","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Xing et al., 2003] E. P. Xing, A. Y. Ng, M. I. Jordan, and S. Russell. Distance metric learning with application to clustering with side- information. In NIPS, pages 505-512. Cambridge, MA.: MIT Press, 2003.","title":"Distance metric learning with application to clustering with sideinformation","authors":"E P Xing, A Y Ng, M I Jordan, S Russell","year":"2003","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Ye et al., 2016a] H.-J. Ye, D.-C. Zhan, and Y. Jiang. Instance spe- cific metric subspace learning: A bayesian approach. In AAAI, pages 2272-2278, Phoenix, AZ., 2016.","title":"Instance specific metric subspace learning: A bayesian approach","authors":"Ye","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Ye et al., 2016b] H.-J. Ye, D.-C. Zhan, X.-M. Si, and Y. Jiang. Learning feature aware metric. In ACML, pages 286-301, Hamil- ton, New Zealand, 2016.","title":"Learning feature aware metric","authors":"Ye","year":"2016","venue":"ACML","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zadeh et al., 2016] P. H. Zadeh, R. Hosseini, and S. Sra. Geomet- ric mean metric learning. In ICML, pages 2464-2471, New York, NY., 2016.","title":"Geometric mean metric learning","authors":"P H Zadeh, R Hosseini, S Sra","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhan et al., 2009] D.-C. Zhan, M. Li, Y.-F. Li, and Z.-H. Zhou. Learning instance specific distances using metric propagation. In ICML, pages 1225-1232, Montreal, Canada, 2009.","title":"Learning instance specific distances using metric propagation","authors":"D.-C Zhan, M Li, Y.-F Li, Z.-H Zhou","year":"2009","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhang et al., 2003] J. Zhang, R. Jin, Y. Yang, and A. G Hauptmann. Modified logistic regression: An approximation to svm and its applications in large-scale text categorization. In ICML, pages 888-895, Washington, D.C., 2003.","title":"Modified logistic regression: An approximation to svm and its applications in large-scale text categorization","authors":"J Zhang, R Jin, Y Yang, A G Hauptmann","year":"2003","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhang et al., 2007] W. Zhang, X. Xue, Z. Sun, Y.-F. Guo, and H. Lu. Optimal dimensionality of metric space for classification. In ICML, pages 1135-1142, Corvallis, OR., 2007.","title":"Optimal dimensionality of metric space for classification","authors":"W Zhang, X Xue, Z Sun, Y.-F Guo, H Lu","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}