{"type":"ScholarlyArticle","title":"Zero-shot Metric Learning","authors":[{"name":"Xinyi Xu","slug":"xinyi-xu","given":"Xinyi","family":"Xu","affiliation":"School of Electronic Engineering , Xidian University , Xian 710071 , China"},{"name":"Huanhuan Cao","slug":"huanhuan-cao","given":"Huanhuan","family":"Cao","affiliation":"School of Electronic Engineering , Xidian University , Xian 710071 , China"},{"name":"Yanhua Yang","slug":"yanhua-yang","given":"Yanhua","family":"Yang","affiliation":"School of Electronic Engineering , Xidian University , Xian 710071 , China"},{"name":"Erkun Yang","slug":"erkun-yang","given":"Erkun","family":"Yang","affiliation":"School of Electronic Engineering , Xidian University , Xian 710071 , China"},{"name":"Cheng Deng","slug":"cheng-deng","given":"Cheng","family":"Deng","affiliation":"School of Electronic Engineering , Xidian University , Xian 710071 , China"}],"abstract":"In this work, we tackle the zero-shot metric learning problem and propose a novel method abbreviated as ZSML, with the purpose to learn a distance metric that measures the similarity of unseen categories (even unseen datasets). ZSML achieves strong transferability by capturing multi-nonlinear yet continuous relation among data. It is motivated by two facts: 1) relations can be essentially described from various perspectives; and 2) traditional binary supervision is insufficient to represent continuous visual similarity. Specifically, we first reformulate a collection of specific-shaped convolutional kernels to combine data pairs and generate multiple relation vectors. Furthermore, we design a new cross-update regression loss to discover continuous similarity. Extensive experiments including intra-dataset transfer and inter-dataset transfer on four benchmark datasets demonstrate that ZSML can achieve state-of-the-art performance.","doi":"10.24963/ijcai.2019/555","url":"https://ijcai.org/proceedings/2019/555","pdf":"https://ijcai.org/proceedings/2019/0555.pdf","markdown":"https://ijcai.org/proceedings/2019/555.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/555","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":555,"pages":{"first":3996,"last":4002},"keywords":["Machine Learning: Multi-instance;Multi-label;Multi-view learning","Machine Learning: Dimensionality Reduction and Manifold Learning"],"videos":[],"references":[{"position":1,"raw":"[Cai et al., 2008] Deng Cai, Xiaofei He, Xiaoyun Wu, and Jiawei Han. Non-negative matrix factorization on mani- fold. In ICDM, pages 63-72, 2008.","title":"Non-negative matrix factorization on manifold","authors":"Deng Cai, Xiaofei He, Xiaoyun Wu, Jiawei Han","year":"2008","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Davis et al., 2007] Jason V Davis, Brian Kulis, Prateek Jain, Suvrit Sra, and Inderjit S Dhillon. Information-theoretic metric learning. In ICML, pages 209-216, 2007.","title":"Information-theoretic metric learning","authors":"Jason V Davis, Brian Kulis, Prateek Jain, Suvrit Sra, Inderjit S Dhillon","year":"2007","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Donahue et al., 2013] Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. Decaf: a deep convolutional activation fea- ture for generic visual recognition. 50(1):I-647, 2013.","title":"Decaf: a deep convolutional activation feature for generic visual recognition","authors":"Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, Trevor Darrell","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Farhadi et al., 2010] Ali Farhadi, Ian Endres, and Derek Hoiem. Attribute-centric recognition for cross-category generalization. In CVPR, pages 2352-2359. IEEE, 2010.","title":"Attribute-centric recognition for cross-category generalization","authors":"Ali Farhadi, Ian Endres, Derek Hoiem","year":"2010","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Griffin and Perona, 2008] Gregory Griffin and Pietro Per- ona. Learning and using taxonomies for fast visual cat- egorization. In CVPR, pages 1-8. IEEE, 2008.","title":"Gregory Griffin and Pietro Perona. Learning and using taxonomies for fast visual categorization","authors":"Perona Griffin","year":"2008","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Guillaumin et al., 2009] Matthieu Guillaumin, Jakob Ver- beek, and Cordelia Schmid. Is that you? metric learning approaches for face identification. In ICCV, pages 498- 505, 2009.","title":"Is that you? metric learning approaches for face identification","authors":"Matthieu Guillaumin, Jakob Verbeek, Cordelia Schmid","year":"2009","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hadsell et al., 2006] Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning an invariant mapping. In CVPR, volume 2, pages 1735-1742, 2006.","title":"Dimensionality reduction by learning an invariant mapping","authors":"Raia Hadsell, Sumit Chopra, Yann Lecun","year":"2006","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hoi et al., 2006] Steven CH Hoi, Wei Liu, Michael R Lyu, and Wei-Ying Ma. Learning distance metrics with contex- tual constraints for image retrieval. In CVPR, volume 2, pages 2072-2078, 2006.","title":"Learning distance metrics with contextual constraints for image retrieval","authors":"C H Steven, Wei Hoi, Liu, Wei-Ying Michael R Lyu, Ma","year":"2006","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Huang et al., 2015] Sheng Huang, Mohamed Elhoseiny, Ahmed Elgammal, and Dan Yang. Learning hypergraph- regularized attribute predictors. In CVPR, pages 409-417, 2015.","title":"Learning hypergraphregularized attribute predictors","authors":"Sheng Huang, Mohamed Elhoseiny, Ahmed Elgammal, Dan Yang","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huang et al., 2016] Chen Huang, Chen Change Loy, and Xiaoou Tang. Local similarity-aware deep feature embed- ding. In NIPS, pages 1262-1270, 2016.","title":"Local similarity-aware deep feature embedding","authors":"Chen Huang, Chen Change Loy, Xiaoou Tang","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jia et al., 2014] Yangqing Jia, Evan Shelhamer, Jeff Don- ahue, Sergey Karayev, Jonathan Long, Ross Girshick, Ser- gio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature embedding. In ACM, pages 675-678, 2014.","title":"Caffe: Convolutional architecture for fast feature embedding","authors":"Jia","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lampert et al., 2014] Christoph H Lampert, Hannes Nick- isch, and Stefan Harmeling. Attribute-based classifica- tion for zero-shot visual object categorization. IEEE Transactions on Pattern Analysis and Machine Intelli- gence (TPAMI), 36(3):453-465, 2014.","title":"Attribute-based classification for zero-shot visual object categorization","authors":"Hannes Christoph H Lampert, Stefan Nickisch, Harmeling","year":"2014","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Law et al., 2016] Marc T Law, Yaoliang Yu, Matthieu Cord, and Eric P Xing. Closed-form training of mahalanobis dis- tance for supervised clustering. In CVPR, pages 3909- 3917, 2016.","title":"Closed-form training of mahalanobis distance for supervised clustering","authors":"Yaoliang Marc T Law, Matthieu Yu, Eric P Cord, Xing","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lecun et al., 1998] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. 86(11):2278-2324, Nov 1998.","title":"Gradient-based learning applied to document recognition","authors":"Y Lecun, L Bottou, Y Bengio, P Haffner","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Marszalek and Schmid, 2007] Marcin Marszalek and Cordelia Schmid. Semantic hierarchies for visual object recognition. In CVPR, pages 1-7. IEEE, 2007.","title":"Semantic hierarchies for visual object recognition","authors":"Marcin Marszalek, Cordelia Schmid","year":"2007","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mensink et al., 2013] Thomas Mensink, Jakob Verbeek, Florent Perronnin, and Gabriela Csurka. Distance-based image classification: Generalizing to new classes at near- zero cost. TPAMI, 35(11):2624-2637, 2013.","title":"Distance-based image classification: Generalizing to new classes at nearzero cost","authors":"Thomas Mensink, Jakob Verbeek, Florent Perronnin, Gabriela Csurka","year":"2013","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Nene et al., 1996] Sameer A Nene, Shree K Nayar, Hiroshi Murase, et al. Columbia object image library (coil-20). 1996.","title":"Columbia object image library","authors":"Sameer A Nene, Hiroshi Shree K Nayar, Murase","year":"1996","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Oh Song et al., 2016] Hyun Oh Song, Yu Xiang, Stefanie Jegelka, and Silvio Savarese. Deep metric learning via lifted structured feature embedding. In CVPR, pages 4004-4012, 2016.","title":"Deep metric learning via lifted structured feature embedding","authors":"Hyun Oh Song, Yu Xiang, Stefanie Jegelka, Silvio Savarese","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Roweis et al., 2004] Sam Roweis, Geoffrey Hinton, and Ruslan Salakhutdinov. Neighbourhood component anal- ysis. NIPS, 17:513-520, 2004.","title":"Neighbourhood component analysis","authors":"Sam Roweis, Geoffrey Hinton, Ruslan Salakhutdinov","year":"2004","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Toshev and Szegedy, 2014] Alexander Toshev and Chris- tian Szegedy. Deeppose: Human pose estimation via deep neural networks. In CVPR, pages 1653-1660, 2014.","title":"Deeppose: Human pose estimation via deep neural networks","authors":"Alexander Toshev, Christian Szegedy","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wah et al., 2011a] C. Wah, S. Branson, P. Welinder, P. Per- ona, and S. Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical report, 2011.","title":"The Caltech-UCSD Birds-200-2011 Dataset","authors":"Wah","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wah et al., 2011b] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech- ucsd birds-200-2011 dataset. 2011.","title":"The caltechucsd birds-200-2011 dataset","authors":"Wah","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Weinberger and Saul, 2009] Kilian Q Weinberger and Lawrence K Saul. Distance metric learning for large mar- gin nearest neighbor classification. Journal of Machine Learning Research, 10(Feb):207-244, 2009.","title":"Distance metric learning for large margin nearest neighbor classification","authors":"Q Kilian, Lawrence K Weinberger, Saul","year":"2009","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xing et al., 2003] Eric P Xing, Michael I Jordan, Stuart J Russell, and Andrew Y Ng. Distance metric learning with application to clustering with side-information. In NIPS, pages 521-528, 2003.","title":"Distance metric learning with application to clustering with side-information","authors":"Eric P Xing, Stuart J Michael I Jordan, Andrew Y Russell, Ng","year":"2003","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xiong et al., 2012] Caiming Xiong, David Johnson, Ran Xu, and Jason J Corso. Random forests for metric learning with implicit pairwise position dependence. In SIGKDD, pages 958-966, 2012.","title":"Random forests for metric learning with implicit pairwise position dependence","authors":"Xiong","year":"2012","venue":"SIGKDD","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Ye et al., 2016] Han-Jia Ye, De-Chuan Zhan, Xue-Min Si, Yuan Jiang, and Zhi-Hua Zhou. What makes objects sim- ilar: A unified multi-metric learning approach. In NIPS, pages 1235-1243, 2016.","title":"What makes objects similar: A unified multi-metric learning approach","authors":"Han-Jia Ye, Xue-Min De-Chuan Zhan, Yuan Si, Zhi-Hua Jiang, Zhou","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zadeh et al., 2016] Pourya Zadeh, Reshad Hosseini, and Suvrit Sra. Geometric mean metric learning. In ICML, pages 2464-2471, 2016.","title":"Geometric mean metric learning","authors":"Zadeh","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhou et al., 2004] Denny Zhou, Olivier Bousquet, Thomas N Lal, Jason Weston, and Bernhard Schölkopf. Learning with local and global consistency. In NIPs, pages 321-328, 2004.","title":"Learning with local and global consistency","authors":"Denny Zhou, Olivier Bousquet, Jason Thomas N Lal, Bernhard Weston, Schölkopf","year":"2004","venue":"NIPs","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}