{"type":"ScholarlyArticle","title":"Incomplete Attribute Learning with auxiliary labels","authors":[{"name":"Kongming Liang","slug":"kongming-liang","given":"Kongming","family":"Liang","affiliation":"Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) , Institute of Computing Technology , CAS , Beijing 100190 , China; School of Computer Science , Carleton University , Ottawa , Canada; University of Chinese Academy of Sciences , Beijing 100049 , China"},{"name":"Yuhong Guo","slug":"yuhong-guo","given":"Yuhong","family":"Guo","affiliation":"School of Computer Science , Carleton University , Ottawa , Canada"},{"name":"Hong Chang","slug":"hong-chang","given":"Hong","family":"Chang","affiliation":"Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) , Institute of Computing Technology , CAS , Beijing 100190 , China"},{"name":"Xilin Chen","slug":"xilin-chen","given":"Xilin","family":"Chen","affiliation":"Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS) , Institute of Computing Technology , CAS , Beijing 100190 , China; University of Chinese Academy of Sciences , Beijing 100049 , China"}],"abstract":"Visual attribute learning is a fundamental and challenging problem for  image understanding. Considering the huge semantic space of attributes, it is economically impossible to annotate all their presence or absence for a natural image via crowd-sourcing. In this paper, we tackle the incompleteness nature of visual attributes by introducing auxiliary labels into a novel transductive learning framework. By jointly predicting the attributes from the input images and modeling the relationship of attributes and auxiliary labels, the missing attributes can be recovered effectively. In addition, the proposed model can be solved efficiently in an alternative way by optimizing quadratic programming problems and updating parameters in closed-form solutions. Moreover, we propose and investigate different methods for acquiring auxiliary labels. We conduct experiments on three widely used attribute prediction datasets. The experimental results show that our proposed method can achieve the state-of-the-art performance with access to partially observed attribute annotations.","doi":"10.24963/ijcai.2017/313","url":"https://ijcai.org/proceedings/2017/313","pdf":"https://ijcai.org/proceedings/2017/0313.pdf","markdown":"https://ijcai.org/proceedings/2017/313.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/313","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":313,"pages":{"first":2252,"last":2258},"keywords":["Machine Learning: Semi-Supervised Learning","Robotics and Vision: Vision and Perception"],"videos":[],"references":[{"position":1,"raw":"[Bird et al., 2009] Steven Bird, Ewan Klein, and Edward Loper. Natural language processing with Python: ana- lyzing text with the natural language toolkit. \" O'Reilly Media, Inc\", 2009.","title":"Natural language processing with Python: analyzing text with the natural language toolkit","authors":"Steven Bird, Ewan Klein, Edward Loper","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2013] Minmin Chen, Alice X Zheng, and Kil- ian Q Weinberger. Fast image tagging. In Proc. of ICML, pages 1274-1282, 2013.","title":"Fast image tagging","authors":"Minmin Chen, Alice X Zheng, Kilian Q Weinberger","year":"2013","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2014] Lin Chen, Qiang Zhang, and Baoxin Li. Predicting multiple attributes via relative multi-task learn- ing. In Proc. of CVPR, pages 1027-1034. IEEE, 2014.","title":"Predicting multiple attributes via relative multi-task learning","authors":"Lin Chen, Qiang Zhang, Baoxin Li","year":"2014","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Davis and Goadrich, 2006] Jesse Davis and Mark Goadrich. The relationship between precision-recall and roc curves. In Proc. of ICML, pages 233-240. ACM, 2006.","title":"The relationship between precision-recall and roc curves","authors":"Goadrich Davis, Jesse Davis, Mark Goadrich","year":"2006","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Donahue et al., 2014] Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, and Trevor Darrell. Decaf: A deep convolutional activation feature for generic visual recognition. In Proc. of ICML, pages 647-655, 2014.","title":"Decaf: A deep convolutional activation feature for generic visual recognition","authors":"Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, Trevor Darrell","year":"2014","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fang et al., 2015] Hao Fang, Saurabh Gupta, Forrest Ian- dola, Rupesh K Srivastava, Li Deng, Piotr Dollár, et al. From captions to visual concepts and back. In Proc. of CVPR, pages 1473-1482, 2015.","title":"From captions to visual concepts and back","authors":"Saurabh Hao Fang, Forrest Gupta, Iandola, K Rupesh, Li Srivastava, Piotr Deng, Dollár","year":"2015","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Farhadi et al., 2009] Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. Describing objects by their attributes. In Proc. of CVPR, pages 1778-1785. IEEE, 2009.","title":"Describing objects by their attributes","authors":"Ali Farhadi, Ian Endres, Derek Hoiem, David Forsyth","year":"2009","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fellbaum, 1998] Christiane Fellbaum. WordNet. Wiley On- line Library, 1998.","title":"WordNet","authors":null,"year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Frome et al., 2013] Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Tomas Mikolov, et al. Devise: A deep visual-semantic embedding model. In Proc. of NIPS, pages 2121-2129, 2013.","title":"Devise: A deep visual-semantic embedding model","authors":"Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Tomas Mikolov","year":"2013","venue":"Proc. of NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Huang et al., 2015] Sheng Huang, Mohamed Elhoseiny, Ahmed Elgammal, and Dan Yang. Learning hypergraph- regularized attribute predictors. In Proc. of CVPR, pages 409-417, 2015.","title":"Learning hypergraphregularized attribute predictors","authors":"Sheng Huang, Mohamed Elhoseiny, Ahmed Elgammal, Dan Yang","year":"2015","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hwang and Sigal, 2014] Sung Ju Hwang and Leonid Sigal. A unified semantic embedding: Relating taxonomies and attributes. In Proc. of NIPS, pages 271-279, 2014.","title":"A unified semantic embedding: Relating taxonomies and attributes","authors":"Ju Sung, Leonid Hwang, Sigal","year":"2014","venue":"Proc. of NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jayaraman and Grauman, 2014] Dinesh Jayaraman and Kristen Grauman. Zero-shot recognition with unreliable attributes. In Proc. of NIPS, pages 3464-3472, 2014.","title":"Zero-shot recognition with unreliable attributes","authors":"Dinesh Jayaraman, Kristen Grauman","year":"2014","venue":"Proc. of NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jayaraman et al., 2014] Dinesh Jayaraman, Fei Sha, and Kristen Grauman. Decorrelating semantic visual attributes by resisting the urge to share. In Proc. of CVPR, pages 1629-1636. IEEE, 2014.","title":"Decorrelating semantic visual attributes by resisting the urge to share","authors":"Dinesh Jayaraman, Fei Sha, Kristen Grauman","year":"2014","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kovashka et al., 2012] Adriana Kovashka, Devi Parikh, and Kristen Grauman. Whittlesearch: Image search with rel- ative attribute feedback. In Proc. of CVPR, pages 2973- 2980. IEEE, 2012.","title":"Whittlesearch: Image search with relative attribute feedback","authors":"Adriana Kovashka, Devi Parikh, Kristen Grauman","year":"2012","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Proc. of NIPS, pages 1097-1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"Proc. of NIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lampert et al., 2014] Christoph H Lampert, Hannes Nick- isch, and Stefan Harmeling. Attribute-based classification for zero-shot visual object categorization. IEEE TPAMI, 36(3):453-465, 2014.","title":"Attribute-based classification for zero-shot visual object categorization","authors":"Hannes Christoph H Lampert, Stefan Nickisch, Harmeling","year":"2014","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liang et al., 2015] Kongming Liang, Hong Chang, Shiguang Shan, and Xilin Chen. A unified multiplicative framework for attribute learning. In Proc. of ICCV, pages 2506-2514, 2015.","title":"A unified multiplicative framework for attribute learning","authors":"Liang","year":"2015","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liang et al., 2016] Kongming Liang, Hong Chang, Shiguang Shan, and Xilin Chen. Attribute conjunction learning with recurrent neural network. In Proc. of ECML-PKDD, pages 345-360. Springer, 2016.","title":"Attribute conjunction learning with recurrent neural network","authors":"Liang","year":"2016","venue":"Proc. of ECML-PKDD","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lu et al., 2016] Cewu Lu, Ranjay Krishna, Michael Bern- stein, and Li Fei-Fei. Visual relationship detection with language priors. In Proc. of ECCV, pages 852-869. Springer, 2016.","title":"Visual relationship detection with language priors","authors":"Lu","year":"2016","venue":"Proc. of ECCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lu, 2016] Yao Lu. Unsupervised learning on neural net- work outputs: with application in zero-shot learning. In Proc. of IJCAI, 2016.","title":"Unsupervised learning on neural network outputs: with application in zero-shot learning","authors":"Yao Lu","year":"2016","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/485"},{"position":21,"raw":"[Russakovsky and Fei-Fei, 2010] Olga Russakovsky and Li Fei-Fei. Attribute learning in large-scale datasets. In Proc. of ECCV Workshop, 2010.","title":"Attribute learning in large-scale datasets","authors":"Olga Russakovsky, Li Fei-Fei","year":"2010","venue":"Proc. of ECCV Workshop","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Russakovsky et al., 2015] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. Ima- geNet Large Scale Visual Recognition Challenge. IJCV, 115(3):211-252, 2015.","title":"Ima-geNet Large Scale Visual Recognition Challenge","authors":"Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C Berg, Li Fei-Fei","year":"2015","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Simonyan and Zisserman, 2014] K. Simonyan and A. Zis- serman. Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"K Simonyan, A Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sun et al., 2010] Yu-yin Sun, Yin Zhang, and Zhi-hua Zhou. Multi-label learning with weak label. In Proc. of AAAI. Citeseer, 2010.","title":"Multi-label learning with weak label","authors":"Yu-Yin Sun, Yin Zhang, Zhi-Hua Zhou","year":"2010","venue":"Proc. of AAAI. Citeseer","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang and Ji, 2013] Xiaoyang Wang and Qiang Ji. A unified probabilistic approach modeling relationships between at- tributes and objects. In Proc. of ICCV, 2013.","title":"A unified probabilistic approach modeling relationships between attributes and objects","authors":"Xiaoyang Wang, Qiang Ji","year":"2013","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wu et al., 2013] Lei Wu, Rong Jin, and Anil K Jain. Tag completion for image retrieval. IEEE TPAMI, 35(3):716- 727, 2013.","title":"Tag completion for image retrieval","authors":"Lei Wu, Rong Jin, Anil K Jain","year":"2013","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wu et al., 2015] Baoyuan Wu, Siwei Lyu, and Bernard Ghanem. Ml-mg: multi-label learning with missing labels using a mixed graph. In Proc. of ICCV, 2015.","title":"Ml-mg: multi-label learning with missing labels using a mixed graph","authors":"Wu","year":"2015","venue":"Proc. of ICCV","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[You et al., 2016] Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo. Image captioning with semantic attention. In Proc. of CVPR, pages 4651-4659, 2016.","title":"Image captioning with semantic attention","authors":"You","year":"2016","venue":"Proc. of CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Yu et al., 2014] Hsiang-Fu Yu, Prateek Jain, Purushottam Kar, and Inderjit S Dhillon. Large-scale multi-label learn- ing with missing labels. In Proc. of ICML, pages 593-601, 2014.","title":"Large-scale multi-label learning with missing labels","authors":"Hsiang-Fu Yu, Prateek Jain, Purushottam Kar, Inderjit S Dhillon","year":"2014","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhao and Guo, 2015] Feipeng Zhao and Yuhong Guo. Semi-supervised multi-label learning with incomplete la- bels. In Proc. of IJCAI, pages 4062-4068. AAAI Press, 2015.","title":"Semi-supervised multi-label learning with incomplete labels","authors":"Guo Zhao, Feipeng Zhao, Yuhong Guo","year":"2015","venue":"Proc. of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/570"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}