{"type":"ScholarlyArticle","title":"Implicit Non-linear Similarity Scoring for Recognizing Unseen Classes","authors":[{"name":"Yuchen Guo","slug":"yuchen-guo","given":"Yuchen","family":"Guo","affiliation":"† School of Software , Tsinghua University , Beijing 100084 , China"},{"name":"Guiguang Ding","slug":"guiguang-ding","given":"Guiguang","family":"Ding","affiliation":"† School of Software , Tsinghua University , Beijing 100084 , China"},{"name":"Jungong Han","slug":"jungong-han","given":"Jungong","family":"Han","affiliation":"‡ School of Computing & Communications , Lancaster University , UK"},{"name":"Sicheng Zhao","slug":"sicheng-zhao","given":"Sicheng","family":"Zhao","affiliation":"† School of Software , Tsinghua University , Beijing 100084 , China"},{"name":"Bin Wang","slug":"bin-wang","given":"Bin","family":"Wang","affiliation":"† School of Software , Tsinghua University , Beijing 100084 , China"}],"abstract":"Recognizing unseen classes is an important task for real-world applications, due to: 1) it is common that some classes in reality have no labeled image exemplar for training; and 2) novel classes emerge rapidly. Recently, to address this task many zero-shot learning (ZSL) approaches have been proposed where explicit linear scores, like inner product score, are employed to measure the similarity between a class and an image. We argue that explicit linear scoring (ELS) seems too weak to capture complicated image-class correspondence. We propose a simple yet effective framework, called Implicit Non-linear Similarity Scoring (ICINESS). In particular, we train a scoring network which uses image and class features as input, fuses them by hidden layers, and outputs the similarity. Based on the universal approximation theorem, it can approximate the true similarity function between images and classes if a proper structure is used in an implicit non-linear way, which is more flexible and powerful. With ICINESS framework, we implement ZSL algorithms by shallow and deep networks, which yield consistently superior results.","doi":"10.24963/ijcai.2018/680","url":"https://ijcai.org/proceedings/2018/680","pdf":"https://ijcai.org/proceedings/2018/0680.pdf","markdown":"https://ijcai.org/proceedings/2018/680.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/680","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Robotics","track":"Main track","number":680,"pages":{"first":4898,"last":4904},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Robotics: Robotics and Vision","Machine Learning Applications: Networks"],"videos":[],"references":[{"position":1,"raw":"[Akata et al., 2015] Zeynep Akata, Scott E. Reed, Daniel Walter, Honglak Lee, and Bernt Schiele. Evaluation of output embeddings for fine-grained image classification. In CVPR, 2015.","title":"Evaluation of output embeddings for fine-grained image classification","authors":"Zeynep Akata, Scott E Reed, Daniel Walter, Honglak Lee, Bernt Schiele","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Akata et al., 2016] Zeynep Akata, Florent Perronnin, Zaïd Harchaoui, and Cordelia Schmid. Label-embedding for image classification. IEEE Trans. Pattern Anal. Mach. In- tell., 2016.","title":"Label-embedding for image classification","authors":"Zeynep Akata, Florent Perronnin, Zaïd Harchaoui, Cordelia Schmid","year":"2016","venue":"IEEE Trans. Pattern Anal. Mach","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bishop and others, 2006] Christopher M Bishop et al. Pat- tern recognition and machine learning, volume 1. Springer, New York, 2006.","title":"Pattern recognition and machine learning","authors":"M Christopher, Bishop","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Changpinyo et al., 2016] Soravit Changpinyo, Wei-Lun Chao, Boqing Gong, and Fei Sha. Synthesized classifiers for zero-shot learning. In CVPR, 2016.","title":"Synthesized classifiers for zero-shot learning","authors":"Changpinyo","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chao et al., 2016] Wei-Lun Chao, Soravit Changpinyo, Bo- qing Gong, and Fei Sha. An empirical study and analysis of generalized zero-shot learning for object recognition in the wild. In ECCV, pages 52-68, 2016.","title":"An empirical study and analysis of generalized zero-shot learning for object recognition in the wild","authors":"Wei-Lun Chao, Soravit Changpinyo, Boqing Gong, Fei Sha","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cybenko, 1989] George Cybenko. Approximation by su- perpositions of a sigmoidal function. MCSS, 1989.","title":"Approximation by superpositions of a sigmoidal function","authors":"George Cybenko","year":"1989","venue":"MCSS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Donahue et al., 2014] Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. In ICML, 2014.","title":"Decaf: A deep convolutional activation feature for generic visual recognition","authors":"Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, Trevor Darrell","year":"2014","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Farhadi et al., 2009] Ali Farhadi, Ian Endres, Derek Hoiem, and David A. Forsyth. Describing objects by their at- tributes. In CVPR, 2009.","title":"Describing objects by their attributes","authors":"Ali Farhadi, Ian Endres, Derek Hoiem, David A Forsyth","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Frome et al., 2013] Andrea Frome, Gregory S. Corrado, Jonathon Shlens, Samy Bengio, Jeffrey Dean, Mar- c'Aurelio Ranzato, and Tomas Mikolov. Devise: A deep visual-semantic embedding model. In NIPS, 2013.","title":"Devise: A deep visual-semantic embedding model","authors":"Andrea Frome, Gregory S Corrado, Jonathon Shlens, Samy Bengio, Jeffrey Dean, Marc'aurelio Ranzato, Tomas Mikolov","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fu et al., 2015] Zhenyong Fu, Tao Xiang, Elyor Kodirov, and Shaogang Gong. Zero-shot object recognition by se- mantic manifold distance. In CVPR, 2015.","title":"Zero-shot object recognition by semantic manifold distance","authors":"Fu","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guo et al., 2016] Yuchen Guo, Guiguang Ding, Xiaoming Jin, and Jianmin Wang. Transductive zero-shot recognition via shared model space learning. In AAAI, 2016.","title":"Transductive zero-shot recognition via shared model space learning","authors":"Yuchen Guo, Guiguang Ding, Xiaoming Jin, Jianmin Wang","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Guo et al., 2017] Yuchen Guo, Guiguang Ding, Jungong Han, and Yue Gao. Synthesizing samples for zero-shot learning. In IJCAI, pages 1774-1780, 2017.","title":"Synthesizing samples for zero-shot learning","authors":"Yuchen Guo, Guiguang Ding, Jungong Han, Yue Gao","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/246"},{"position":13,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hornik, 1991] Kurt Hornik. Approximation capabilities of multilayer feedforward networks. Neural Networks, 1991.","title":"Approximation capabilities of multilayer feedforward networks","authors":"Kurt Hornik","year":"1991","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kodirov et al., 2015] Elyor Kodirov, Tao Xiang, Zhenyong Fu, and Shaogang Gong. Unsupervised domain adaptation for zero-shot learning. In ICCV, 2015.","title":"Unsupervised domain adaptation for zero-shot learning","authors":"Kodirov","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kodirov et al., 2017] Elyor Kodirov, Tao Xiang, and Shao- gang Gong. Semantic autoencoder for zero-shot learning. In CVPR, 2017. [Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.","title":"Elyor Kodirov, Tao Xiang, and Shaogang Gong. Semantic autoencoder for zero-shot learning","authors":"Kodirov","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lampert et al., 2014] Christoph H. Lampert, Hannes Nick- isch, and Stefan Harmeling. Attribute-based classification for zero-shot visual object categorization. IEEE TPAMI, 2014.","title":"Attribute-based classification for zero-shot visual object categorization","authors":"Christoph H Lampert, Hannes Nickisch, Stefan Harmeling","year":"2014","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Norouzi et al., 2013] Mohammad Norouzi, Tomas Mikolov, Samy Bengio, Yoram Singer, Jonathon Shlens, Andrea Frome, Greg Corrado, and Jeffrey Dean. Zero-shot learn- ing by convex combination of semantic embeddings. CoR- R, abs/1312.5650, 2013.","title":"Zero-shot learning by convex combination of semantic embeddings","authors":"Mohammad Norouzi, Tomas Mikolov, Samy Bengio, Yoram Singer, Jonathon Shlens, Andrea Frome, Greg Corrado, Jeffrey Dean","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Patterson and Hays, 2012] Genevieve Patterson and James Hays. SUN attribute database: Discovering, annotating, and recognizing scene attributes. In CVPR, 2012.","title":"SUN attribute database: Discovering, annotating, and recognizing scene attributes","authors":"Genevieve Patterson, James Hays","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Romera-Paredes and Torr, 2015] Bernardino Romera- Paredes and Philip H. S. Torr. An embarrassingly simple approach to zero-shot learning. In ICML, 2015.","title":"An embarrassingly simple approach to zero-shot learning","authors":"Bernardino Romera, - Paredes, Philip H S Torr","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Russakovsky et al., 2015] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z.g Huang, A. Karpathy, A. Khosla, M. S. Bernstein, A. C. Berg, and Fei-Fei Li. Imagenet large scale visual recognition challenge. IJCV, 2015.","title":"Imagenet large scale visual recognition challenge","authors":"O Russakovsky, J Deng, H Su, J Krause, S Satheesh, S Ma, Z Huang, A Karpathy, A Khosla, M S Bernstein, A C Berg, Fei-Fei Li","year":"2015","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sengupta et al., 2016] Nandini Sengupta, Md. Sahidullah, and Goutam Saha. Lung sound classification using cepstral-based statistical features. Comp. in Bio. and Med., 2016.","title":"Lung sound classification using cepstral-based statistical features","authors":"Nandini Sengupta, Md Sahidullah, Goutam Saha","year":"2016","venue":"Comp. in Bio. and Med","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Socher et al., 2013] Richard Socher, Milind Ganjoo, Christopher D. Manning, and Andrew Y. Ng. Zero-shot learning through cross-modal transfer. In NIPS, 2013.","title":"Zero-shot learning through cross-modal transfer","authors":"Richard Socher, Milind Ganjoo, Christopher D Manning, Andrew Y Ng","year":"2013","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tang et al., 2017] Sheng Tang, Yu Li, Lixi Deng, and Yong- dong Zhang. Object localization based on proposal fusion. IEEE TMM, 2017.","title":"Object localization based on proposal fusion","authors":"Sheng Tang, Yu Li, Lixi Deng, Yongdong Zhang","year":"2017","venue":"IEEE TMM","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wah et al., 2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech- UCSD Birds-200-2011 Dataset. Technical Report CNS- TR-2011-001, California Institute of Technology, 2011.","title":"The Caltech-UCSD Birds-200-2011 Dataset","authors":"Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, Serge Belongie","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xian et al., 2016] Yongqin Xian, Zeynep Akata, Gaurav Sharma, Quynh N. Nguyen, Matthias Hein, and Bern- t Schiele. Latent embeddings for zero-shot classification. In CVPR, 2016.","title":"Latent embeddings for zero-shot classification","authors":"Xian","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xian et al., 2017] Yongqin Xian, Christoph H. Lampert, Bernt Schiele, and Zeynep Akata. Zero-shot learning - A comprehensive evaluation of the good, the bad and the ugly. CoRR, abs/1707.00600, 2017.","title":"Zero-shot learning -A comprehensive evaluation of the good, the bad and the ugly","authors":"Xian","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang and Saligrama, 2015] Ziming Zhang and Venkatesh Saligrama. Zero-shot learning via semantic similarity em- bedding. In ICCV, 2015.","title":"Zero-shot learning via semantic similarity embedding","authors":"Ziming Zhang, Venkatesh Saligrama","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang and Saligrama, 2016] Ziming Zhang and Venkatesh Saligrama. Zero-shot learning via joint latent similarity embedding. In CVPR, 2016.","title":"Zero-shot learning via joint latent similarity embedding","authors":"Ziming Zhang, Venkatesh Saligrama","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2017] Rui Zhang, Sheng Tang, Yongdong Zhang, Jintao Li, and Shuicheng Yan. Scale-adaptive con- volutions for scene parsing. In ICCV, 2017.","title":"Scale-adaptive convolutions for scene parsing","authors":"Rui Zhang, Sheng Tang, Yongdong Zhang, Jintao Li, Shuicheng Yan","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}