{"type":"ScholarlyArticle","title":"Harnessing Synthesized Abstraction Images to Improve Facial Attribute Recognition","authors":[{"name":"Keke He","slug":"keke-he","given":"Keke","family":"He","affiliation":"School of Computer Science , Shanghai Key Lab of Intelligent Information Processing , Fudan University"},{"name":"Yanwei Fu","slug":"yanwei-fu","given":"Yanwei","family":"Fu","affiliation":"School of Data Science , Fudan University"},{"name":"Wuhao Zhang","slug":"wuhao-zhang","given":"Wuhao","family":"Zhang","affiliation":"Shanghai Jiao Tong University"},{"name":"Chengjie Wang","slug":"chengjie-wang","given":"Chengjie","family":"Wang","affiliation":"Tencent Youtu Lab"},{"name":"Yu-Gang Jiang","slug":"yu-gang-jiang","given":"Yu-Gang","family":"Jiang","affiliation":"School of Computer Science , Shanghai Key Lab of Intelligent Information Processing , Fudan University"},{"name":"Feiyue Huang","slug":"feiyue-huang","given":"Feiyue","family":"Huang","affiliation":"Tencent Youtu Lab"},{"name":"Xiangyang Xue","slug":"xiangyang-xue","given":"Xiangyang","family":"Xue","affiliation":"School of Computer Science , Shanghai Key Lab of Intelligent Information Processing , Fudan University"}],"abstract":"Facial attribute recognition is an important and yet challenging research\r\ntopic. Different from most previous approaches which predict attributes\r\nonly based on the whole images, this paper leverages facial parts\r\nlocations for better attribute prediction. A facial abstraction image\r\nwhich contains both local facial parts and facial texture information\r\nis introduced. This abstraction image is generated by a Generative\r\nAdversarial Network (GAN). Then we build a dual-path facial attribute\r\nrecognition network to utilize features from the original face images\r\nand facial abstraction images. Empirically, the features of facial\r\nabstraction images are complementary to features of original face\r\nimages. With the facial parts localized by the abstraction images,\r\nour method improves facial attributes recognition, especially the\r\nattributes located on small face regions. Extensive evaluations conducted\r\non CelebA and LFWA benchmark datasets show that state-of-the-art performance\r\nis achieved.","doi":"10.24963/ijcai.2018/102","url":"https://ijcai.org/proceedings/2018/102","pdf":"https://ijcai.org/proceedings/2018/0102.pdf","markdown":"https://ijcai.org/proceedings/2018/102.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/102","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":102,"pages":{"first":733,"last":740},"keywords":["Machine Learning: Deep Learning","Computer Vision: Computer Vision","Computer Vision: Biometrics, Face and Gesture Recognition"],"videos":[],"references":[{"position":1,"raw":"[Bourdev et al., 2011] Lubomir Bourdev, Subhransu Maji, and Jitendra Malik. Describing people: A poselet-based approach to attribute classification. In ICCV. IEEE, 2011.","title":"Describing people: A poselet-based approach to attribute classification","authors":"Bourdev","year":"2011","venue":"ICCV. IEEE","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2016] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convo- lutional nets, atrous convolution, and fully connected crfs. arXiv:1606.00915, 2016.","title":"Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","authors":"Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, Alan L Yuille","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deng et al., 2014] Yubin Deng, Ping Luo, Chen Change Loy, and Xiaoou Tang. Pedestrian attribute recognition at far distance. In ACM MM, 2014.","title":"Pedestrian attribute recognition at far distance","authors":"Yubin Deng, Ping Luo, Chen Change Loy, Xiaoou Tang","year":"2014","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ding et al., 2017] Hui Ding, Hao Zhou, Shaohua Kevin Zhou, and Rama Chellappa. A deep cascade network for unaligned face attribute classification. In AAAI, 2017.","title":"A deep cascade network for unaligned face attribute classification","authors":"Hui Ding, Hao Zhou, Shaohua, Kevin Zhou, Rama Chellappa","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ehrlich et al., 2016] Max Ehrlich, Timothy J Shields, Timur Almaev, and Mohamed R Amer. Facial attributes classifi- cation using multi-task representation learning. In CVPR Workshops, 2016.","title":"Facial attributes classification using multi-task representation learning","authors":"Ehrlich","year":"2016","venue":"CVPR Workshops","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goodfellow et al., 2014] Ian Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial nets. In NIPS, 2014.","title":"Generative adversarial nets","authors":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Han et al., 2017] Hu Han, Anil K Jain, Shiguang Shan, and Xilin Chen. Heterogeneous face attribute estimation: A deep multi-task learning approach. TPAMI, 2017.","title":"Heterogeneous face attribute estimation: A deep multi-task learning approach","authors":"Hu Han, K Anil, Shiguang Jain, Xilin Shan, Chen","year":"2017","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hand and Chellappa, 2017] Emily M Hand and Rama Chel- lappa. Attributes for improved attributes: A multi-task net- work utilizing implicit and explicit relationships for facial attribute classification. In AAAI, 2017.","title":"Attributes for improved attributes: A multi-task network utilizing implicit and explicit relationships for facial attribute classification","authors":"M Emily, Rama Hand, Chellappa","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2015] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. arXiv:1512.03385, 2015.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hu et al., 2017] Guosheng Hu, Yang Hua, Yang Yuan, Zhi- hong Zhang, Zheng Lu, Sankha S Mukherjee, Timo- thy M Hospedales, Neil M Robertson, and Yongxin Yang. Attribute-enhanced face recognition with neural tensor fu- sion networks. In ICCV, 2017.","title":"Attribute-enhanced face recognition with neural tensor fusion networks","authors":"Guosheng Hu, Yang Hua, Yang Yuan, Zhihong Zhang, Zheng Lu, Timothy M Sankha S Mukherjee, Neil M Hospedales, Yongxin Robertson, Yang","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Huang et al., 2007] Gary B Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical report, 2007.","title":"Labeled faces in the wild: A database for studying face recognition in unconstrained environments","authors":"Manu Gary B Huang, Tamara Ramesh, Erik Berg, Learned-Miller","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ioffe and Szegedy, 2015] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep net- work training by reducing internal covariate shift. In ICML, 2015.","title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","authors":"Szegedy Ioffe, Sergey Ioffe, Christian Szegedy","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jampour et al., 2017] Mahdi Jampour, Chen Li, Lap-Fai Yu, Kun Zhou, Stephen Lin, and Horst Bischof. Face inpaint- ing based on high-level facial attributes. CVIU, 161:29 - 41, 2017.","title":"Face inpainting based on high-level facial attributes","authors":"Mahdi Jampour, Chen Li, Lap-Fai Yu, Kun Zhou, Stephen Lin, Horst Bischof","year":"2017","venue":"CVIU","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jia et al., 2014] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Gir-shick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv, 2014.","title":"Caffe: Convolutional architecture for fast feature embedding","authors":"Y Jia, E Shelhamer, J Donahue, S Karayev, J Long, R Gir-Shick, S Guadarrama, T Darrell","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kalayeh et al., 2017] Mahdi M Kalayeh, Boqing Gong, and Mubarak Shah. Improving facial attribute prediction using semantic segmentation. In CVPR, June 2017.","title":"Improving facial attribute prediction using semantic segmentation","authors":"Boqing Mahdi M Kalayeh, Mubarak Gong, Shah","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kumar et al., 2008] Neeraj Kumar, Peter Belhumeur, and Shree Nayar. Facetracer: A search engine for large col- lections of images with faces. In ECCV. Springer, 2008.","title":"Facetracer: A search engine for large collections of images with faces","authors":"Neeraj Kumar, Peter Belhumeur, Shree Nayar","year":"2008","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kumar et al., 2009] Neeraj Kumar, Alexander C Berg, Pe- ter N Belhumeur, and Shree K Nayar. Attribute and simile classifiers for face verification. In ICCV, pages 365-372. IEEE, 2009.","title":"Attribute and simile classifiers for face verification","authors":"Neeraj Kumar, Alexander C Berg, Peter N Belhumeur, Shree K Nayar","year":"2009","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Le et al., 2012] Vuong Le, Jonathan Brandt, Zhe Lin, Lubomir Bourdev, and Thomas S Huang. Interactive fa- cial feature localization. In ECCV. 2012.","title":"Interactive facial feature localization","authors":"Jonathan Vuong Le, Zhe Brandt, Lubomir Lin, Thomas S Bourdev, Huang","year":"2012","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2016] Dangwei Li, Zhang Zhang, Xiaotang Chen, Haibin Ling, and Kaiqi Huang. A richly annotated dataset for pedestrian attribute recognition. arXiv:1603.07054, 2016.","title":"A richly annotated dataset for pedestrian attribute recognition","authors":"Dangwei Li, Zhang Zhang, Xiaotang Chen, Haibin Ling, Kaiqi Huang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu et al., 2015a] Sifei Liu, Jimei Yang, Chang Huang, and Ming-Hsuan Yang. Multi-objective convolutional learning for face labeling. In CVPR, June 2015.","title":"Multi-objective convolutional learning for face labeling","authors":"Liu","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2015b] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV, 2015.","title":"Deep learning face attributes in the wild","authors":"Liu","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lu et al., 2017] Yongxi Lu, Abhishek Kumar, Shuangfei Zhai, Yu Cheng, Tara Javidi, and Rogerio Feris. Fully- adaptive feature sharing in multi-task networks with appli- cations in person attribute classification. In CVPR, 2017.","title":"Fullyadaptive feature sharing in multi-task networks with applications in person attribute classification","authors":"Lu","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Luo et al., 2012] Ping Luo, Xiaogang Wang, and Xiaoou Tang. Hierarchical face parsing via deep learning. In CVPR. IEEE, 2012.","title":"Hierarchical face parsing via deep learning","authors":"Ping Luo, Xiaogang Wang, Xiaoou Tang","year":"2012","venue":"CVPR. IEEE","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"Mo et al., ] Zhenyao Mo, John P Lewis, and Ulrich Neu- mann. Face inpainting with local linear representations. [Pathak et al., 2016]","title":"Face inpainting with local linear representations","authors":"Mo","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Con- text encoders: Feature learning by inpainting. In CVPR, 2016.","title":"Context encoders: Feature learning by inpainting","authors":"Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, Alexei A Efros","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Rudd et al., 2016] Ethan M Rudd, Manuel Günther, and Terrance E Boult. Moon: A mixed objective optimization network for the recognition of facial attributes. In ECCV. Springer, 2016.","title":"Moon: A mixed objective optimization network for the recognition of facial attributes","authors":"Ethan M Rudd, Manuel Günther, Terrance E Boult","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Siddiquie et al., 2011] Behjat Siddiquie, Rogerio S Feris, and Larry S Davis. Image ranking and retrieval based on multi-attribute queries. In CVPR. IEEE, 2011.","title":"Image ranking and retrieval based on multi-attribute queries","authors":"Behjat Siddiquie, Rogerio S Feris, Larry S Davis","year":"2011","venue":"CVPR. IEEE","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Smith et al., 2013] Brandon M Smith, Li Zhang, Jonathan Brandt, Zhe Lin, and Jianchao Yang. Exemplar-based face parsing. In CVPR, 2013.","title":"Exemplar-based face parsing","authors":"M Brandon, Li Smith, Jonathan Zhang, Zhe Brandt, Jianchao Lin, Yang","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2016] Jing Wang, Yu Cheng, and Roge- rio Schmidt Feris. Walk and learn: Facial attribute rep- resentation learning from egocentric video and contextual data. In CVPR, 2016.","title":"Walk and learn: Facial attribute representation learning from egocentric video and contextual data","authors":"Jing Wang, Yu Cheng, Rogerio Schmidt, Feris","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2017a] Ting-Chun Wang, Ming-Yu Liu, Jun- Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. High-resolution image synthesis and semantic manipula- tion with conditional gans. arXiv:1711.11585, 2017.","title":"High-resolution image synthesis and semantic manipulation with conditional gans","authors":"Wang","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wang et al., 2017b] Zhanxiong Wang, Keke He, Yanwei Fu, Rui Feng, Yu-Gang Jiang, and Xiangyang Xue. Multi- task deep neural network for joint face recognition and fa- cial attribute prediction. In ICMR. ACM, 2017.","title":"Multitask deep neural network for joint face recognition and facial attribute prediction","authors":"Wang","year":"2017","venue":"ICMR. ACM","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yeh et al., 2017] Raymond A Yeh, Chen Chen, Teck Yian Lim, Alexander G Schwing, Mark Hasegawa-Johnson, and Minh N Do. Semantic image inpainting with deep generative models. In CVPR, 2017.","title":"Semantic image inpainting with deep generative models","authors":"Raymond A Yeh, Chen Chen, Yian Teck, Alexander G Lim, Mark Schwing, Minh N Hasegawa-Johnson, Do","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Zhang et al., 2014] Ning Zhang, Manohar Paluri, Marc'Aurelio Ranzato, Trevor Darrell, and Lubomir Bourdev. Panda: Pose aligned networks for deep attribute modeling. In CVPR, 2014.","title":"Panda: Pose aligned networks for deep attribute modeling","authors":"Ning Zhang, Manohar Paluri, Marc'aurelio Ranzato, Trevor Darrell, Lubomir Bourdev","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhong et al., 2016] Yang Zhong, Josephine Sullivan, and Haibo Li. Face attribute prediction using off-the-shelf cnn features. In ICB. IEEE, 2016.","title":"Face attribute prediction using off-the-shelf cnn features","authors":"Yang Zhong, Josephine Sullivan, Haibo Li","year":"2016","venue":"ICB. IEEE","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}