{"type":"ScholarlyArticle","title":"Learning Image-Specific Attributes by Hyperbolic Neighborhood Graph Propagation","authors":[{"name":"Xiaofeng Xu","slug":"xiaofeng-xu","given":"Xiaofeng","family":"Xu","affiliation":"School of Computer Science and Engineering , Nanjing University of Science and Technology; Faculty of Engineering and Information Technology , University of Technology Sydney"},{"name":"Ivor W. Tsang","slug":"ivor-w-tsang","given":"Ivor W.","family":"Tsang","affiliation":"Faculty of Engineering and Information Technology , University of Technology Sydney"},{"name":"Xiaofeng Cao","slug":"xiaofeng-cao","given":"Xiaofeng","family":"Cao","affiliation":"Faculty of Engineering and Information Technology , University of Technology Sydney"},{"name":"Ruiheng Zhang","slug":"ruiheng-zhang","given":"Ruiheng","family":"Zhang","affiliation":"Faculty of Engineering and Information Technology , University of Technology Sydney; School of Mechatronical Engineering , Beijing Institute of Technology"},{"name":"Chuancai Liu","slug":"chuancai-liu","given":"Chuancai","family":"Liu","affiliation":"School of Computer Science and Engineering , Nanjing University of Science and Technology; Collaborative Innovation Center of IoT Technology and Intelligent Systems , Minjiang University"}],"abstract":"As a kind of semantic representation of visual object descriptions, attributes are widely used in various computer vision tasks. In most of existing attribute-based research, class-specific attributes (CSA), which are class-level annotations, are usually adopted due to its low annotation cost for each class instead of each individual image. However, class-specific attributes are usually noisy because of annotation errors and diversity of individual images. Therefore, it is desirable to obtain image-specific attributes (ISA), which are image-level annotations, from the original class-specific attributes. In this paper, we propose to learn image-specific attributes by graph-based attribute propagation. Considering the intrinsic property of hyperbolic geometry that its distance expands exponentially, hyperbolic neighborhood graph (HNG) is constructed to characterize the relationship between samples. Based on HNG, we define neighborhood consistency for each sample to identify inconsistent samples. Subsequently, inconsistent samples are refined based on their neighbors in HNG. Extensive experiments on five benchmark datasets demonstrate the significant superiority of the learned image-specific attributes over the original class-specific attributes in the zero-shot object classification task.","doi":"10.24963/ijcai.2019/554","url":"https://ijcai.org/proceedings/2019/554","pdf":"https://ijcai.org/proceedings/2019/0554.pdf","markdown":"https://ijcai.org/proceedings/2019/554.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/554","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":554,"pages":{"first":3989,"last":3995},"keywords":["Machine Learning: Classification","Machine Learning: Semi-Supervised Learning","Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation"],"videos":[],"references":[{"position":1,"raw":"[Akata et al., 2016] Zeynep Akata, Florent Perronnin, Zaid Harchaoui, and Cordelia Schmid. Label-embedding for image classification. IEEE TPAMI, 38(7):1425-1438, 2016.","title":"Label-embedding for image classification","authors":"Zeynep Akata, Florent Perronnin, Zaid Harchaoui, Cordelia Schmid","year":"2016","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2018] Long Chen, Hanwang Zhang, Jun Xiao, Wei Liu, and Shih-Fu Chang. Zero-shot visual recog- nition using semantics-preserving adversarial embedding networks. In CVPR, 2018.","title":"Zero-shot visual recognition using semantics-preserving adversarial embedding networks","authors":"Long Chen, Hanwang Zhang, Jun Xiao, Wei Liu, Shih-Fu Chang","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Farhadi et al., 2009] Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. Describing objects by their attributes. In CVPR, 2009.","title":"Describing objects by their attributes","authors":"Ali Farhadi, Ian Endres, Derek Hoiem, David Forsyth","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ganea et al., 2018] Octavian-Eugen Ganea, Gary Bécigneul, and Thomas Hofmann. Hyperbolic en- tailment cones for learning hierarchical embeddings. In ICML, 2018.","title":"Hyperbolic entailment cones for learning hierarchical embeddings","authors":"Ganea","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Gaur and Manjunath, 2017] Utkarsh Gaur and BS Manju- nath. Weakly supervised manifold learning for dense se- mantic object correspondence. In ICCV, 2017.","title":"Weakly supervised manifold learning for dense semantic object correspondence","authors":"Manjunath Gaur","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jang et al., 2018] Jinhyeok Jang, Hyunjoong Cho, Jaehong Kim, Jaeyeon Lee, and Seungjoon Yang. Facial at- tribute recognition by recurrent learning with visual fix- ation. IEEE Trans. Cybern., 2018.","title":"Facial attribute recognition by recurrent learning with visual fixation","authors":"Jang","year":"2018","venue":"IEEE Trans. Cybern","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kakaletsis et al., 2018] Efstratios Kakaletsis, Olga Zoidi, Ioannis Tsingalis, Anastasios Tefas, Nikos Nikolaidis, and Ioannis Pitas. Label propagation on facial images using similarity and dissimilarity labelling constraints. In ICIP, 2018.","title":"Label propagation on facial images using similarity and dissimilarity labelling constraints","authors":"Efstratios Kakaletsis, Olga Zoidi, Ioannis Tsingalis, Anastasios Tefas, Nikos Nikolaidis, Ioannis Pitas","year":"2018","venue":"ICIP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kodirov et al., 2017] Elyor Kodirov, Tao Xiang, and Shao- gang Gong. Semantic autoencoder for zero-shot learning. In CVPR, 2017.","title":"Elyor Kodirov, Tao Xiang, and Shaogang Gong. Semantic autoencoder for zero-shot learning","authors":"Kodirov","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kumar Verma et al., 2018] Vinay Kumar Verma, Gundeep Arora, Ashish Mishra, and Piyush Rai. Generalized zero- shot learning via synthesized examples. In CVPR, 2018.","title":"Generalized zeroshot learning via synthesized examples","authors":"Kumar Vinay, Gundeep Verma, Ashish Arora, Piyush Mishra, Rai","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lampert et al., 2014] Christoph H Lampert, Hannes Nick- isch, and Stefan Harmeling. Attribute-based classification for zero-shot visual object categorization. IEEE TPAMI, 36(3):453-465, 2014.","title":"Attribute-based classification for zero-shot visual object categorization","authors":"Hannes Christoph H Lampert, Stefan Nickisch, Harmeling","year":"2014","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lankford, 1969] Philip M Lankford. Regionalization: the- ory and alternative algorithms. Geographical Analysis, 1(2):196-212, 1969.","title":"Regionalization: theory and alternative algorithms","authors":"M Philip, Lankford","year":"1969","venue":"Geographical Analysis","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[LeCun et al., 2015] Yann LeCun, Yoshua Bengio, and Ge- offrey Hinton. Deep learning. Nature, 521(7553):436, 2015.","title":"Deep learning","authors":"Yann Lecun, Yoshua Bengio, Geoffrey Hinton","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2018] Yang Liu, Quanxue Gao, Jin Li, Jungong Han, and Ling Shao. Zero shot learning via low-rank em- bedded semantic autoencoder. In IJCAI, 2018.","title":"Zero shot learning via low-rank embedded semantic autoencoder","authors":"Yang Liu, Quanxue Gao, Jin Li, Jungong Han, Ling Shao","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/345"},{"position":15,"raw":"[Mahajan et al., 2018] Dhruv Mahajan, Ross Girshick, Vig- nesh Ramanathan, Kaiming He, et al. Exploring the limits of weakly supervised pretraining. In ECCV, 2018.","title":"Exploring the limits of weakly supervised pretraining","authors":"Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nickel and Kiela, 2017] Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical rep- resentations. In NIPS, 2017.","title":"Maximillian Nickel and Douwe Kiela. Poincaré embeddings for learning hierarchical representations","authors":"Kiela Nickel","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nitta and Kuroe, 2018] Tohru Nitta and Yasuaki Kuroe. Hyperbolic gradient operator and hyperbolic back- propagation learning algorithms. IEEE TNNLS, 29(5):1689-1702, 2018.","title":"Hyperbolic gradient operator and hyperbolic backpropagation learning algorithms","authors":"Kuroe ; ] Tohru Nitta, Yasuaki Nitta, Kuroe","year":"2018","venue":"IEEE TNNLS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Parikh and Grauman, 2011] Devi Parikh and Kristen Grau- man. Relative attributes. In ICCV, 2011.","title":"Relative attributes","authors":"Devi Parikh, Kristen Grauman","year":"2011","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Patterson and Hays, 2012] Genevieve Patterson and James Hays. Sun attribute database: Discovering, annotating, and recognizing scene attributes. In CVPR, 2012.","title":"Sun attribute database: Discovering, annotating, and recognizing scene attributes","authors":"Genevieve Patterson, James Hays","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Roy et al., 2018] Debaditya Roy, Sri Rama Murty Ko- dukula, et al. Unsupervised universal attribute modelling for action recognition. IEEE TMM, 2018.","title":"Debaditya Roy, Sri Rama Murty Kodukula, et al. Unsupervised universal attribute modelling for action recognition","authors":"Roy","year":"2018","venue":"IEEE TMM","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shepard, 1968] Donald Shepard. A two-dimensional inter- polation function for irregularly-spaced data. In ACM Na- tional Conference, 1968.","title":"A two-dimensional interpolation function for irregularly-spaced data","authors":"Donald Shepard","year":"1968","venue":"ACM National Conference","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tay et al., 2018] Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. Hyperbolic representation learning for fast and ef- ficient neural question answering. In ACM WSDM, 2018.","title":"Hyperbolic representation learning for fast and efficient neural question answering","authors":"Yi Tay, Anh Luu, Siu Tuan, Hui Cheung","year":"2018","venue":"ACM WSDM","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Toussaint, 1980] Godfried T Toussaint. The relative neigh- bourhood graph of a finite planar set. Pattern Recognition, 12(4):261-268, 1980.","title":"The relative neighbourhood graph of a finite planar set","authors":"T Godfried, Toussaint","year":"1980","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Welinder et al., 2011] Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah, et al. The caltech-ucsd birds-200-2011 dataset. Technical Report CNS-TR-2011- 001, California Institute of Technology, 2011.","title":"The caltech-ucsd birds-200-2011 dataset","authors":"Peter Welinder, Steve Branson, Takeshi Mita, Catherine Wah","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xian et al., 2018a] Yongqin Xian, Christoph H Lampert, Bernt Schiele, and Zeynep Akata. Zero-shot learning-a comprehensive evaluation of the good, the bad and the ugly. IEEE TPAMI, 2018.","title":"Zero-shot learning-a comprehensive evaluation of the good, the bad and the ugly","authors":"Xian","year":"2018","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xian et al., 2018b] Yongqin Xian, Tobias Lorenz, Bernt Schiele, and Zeynep Akata. Feature generating networks for zero-shot learning. In CVPR, 2018.","title":"Feature generating networks for zero-shot learning","authors":"Xian","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Xu et al., 2017] Xing Xu, Fumin Shen, Yang Yang, Dongxi- ang Zhang, Heng Tao Shen, and Jingkuan Song. Matrix tri- factorization with manifold regularizations for zero-shot learning. In CVPR, 2017.","title":"Matrix trifactorization with manifold regularizations for zero-shot learning","authors":"Xing Xu, Fumin Shen, Yang Yang, Dongxiang Zhang, Heng Tao Shen, Jingkuan Song","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2018a] Zhao Zhang, Lei Jia, Mingbo Zhao, Guangcan Liu, Meng Wang, and Shuicheng Yan. Kernel- induced label propagation by mapping for semi-supervised classification. IEEE Transactions on Big Data, 2018.","title":"Kernelinduced label propagation by mapping for semi-supervised classification","authors":"Zhang","year":"2018","venue":"IEEE Transactions on Big Data","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2018b] Zhao Zhang, Fanzhang Li, Lei Jia, Jie Qin, Li Zhang, and Shuicheng Yan. Robust adaptive em- bedded label propagation with weight learning for induc- tive classification. IEEE TNNLS, 29(8):3388-3403, 2018.","title":"Robust adaptive embedded label propagation with weight learning for inductive classification","authors":"Zhang","year":"2018","venue":"IEEE TNNLS","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhou, 2017] Zhi-Hua Zhou. A brief introduction to weakly supervised learning. National Science Review, 5(1):44-53, 2017.","title":"A brief introduction to weakly supervised learning","authors":"Zhi-Hua Zhou","year":"2017","venue":"National Science Review","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zoidi et al., 2018] Olga Zoidi, Anastasios Tefas, Nikos Nikolaidis, and Ioannis Pitas. Positive and negative label propagations. IEEE Trans. Circuits Syst. Video Technol., 28(2):342-355, 2018.","title":"Positive and negative label propagations","authors":"Olga Zoidi, Anastasios Tefas, Nikos Nikolaidis, Ioannis Pitas","year":"2018","venue":"IEEE Trans. Circuits Syst. Video Technol","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}