{"type":"ScholarlyArticle","title":"Pedestrian Attribute Recognition by Joint Visual-semantic Reasoning and Knowledge Distillation","authors":[{"name":"Qiaozhe Li","slug":"qiaozhe-li","given":"Qiaozhe","family":"Li","affiliation":"University of Chinese Academy of Sciences"},{"name":"Xin Zhao","slug":"xin-zhao","given":"Xin","family":"Zhao","affiliation":"University of Chinese Academy of Sciences"},{"name":"Ran He","slug":"ran-he","given":"Ran","family":"He","affiliation":"University of Chinese Academy of Sciences"},{"name":"Kaiqi Huang","slug":"kaiqi-huang","given":"Kaiqi","family":"Huang","affiliation":"University of Chinese Academy of Sciences"}],"abstract":"Pedestrian attribute recognition in surveillance is a challenging task in computer vision due to significant pose variation, viewpoint change and poor image quality. To achieve effective recognition, this paper presents a graph-based global reasoning framework to jointly model potential visual-semantic relations of attributes and distill auxiliary human parsing knowledge to guide the relational learning. The reasoning framework models attribute groups on a graph and learns a projection function to adaptively assign local visual features to the nodes of the graph. After feature projection, graph convolution is utilized to perform global reasoning between the attribute groups to model their mutual dependencies. Then, the learned node features are projected back to visual space to facilitate knowledge transfer. An additional regularization term is proposed by distilling human parsing knowledge from a pre-trained teacher model to enhance feature representations. The proposed framework is verified on three large scale pedestrian attribute datasets including PETA, RAP, and PA-100k. Experiments show that our method achieves state-of-the-art results.","doi":"10.24963/ijcai.2019/117","url":"https://ijcai.org/proceedings/2019/117","pdf":"https://ijcai.org/proceedings/2019/0117.pdf","markdown":"https://ijcai.org/proceedings/2019/117.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/117","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Computer Vision","track":"Main track","number":117,"pages":{"first":833,"last":839},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Alp Guler et al., 2018] Riza Alp Guler, Natalia Neverova, and Iasonas Kokkinos. Densepose: Dense human pose estimation in the wild. In CVPR, pages 7297-7306, 2018.","title":"Densepose: Dense human pose estimation in the wild","authors":"Alp Guler","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2012] Huizhong Chen, Andrew Gallagher, and Bernd Girod. Describing clothing by semantic attributes. In ECCV, pages 609-623, 2012.","title":"Describing clothing by semantic attributes","authors":"Huizhong Chen, Andrew Gallagher, Bernd Girod","year":"2012","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018a] Xinlei Chen, Li-Jia Li, Li Fei-Fei, and Abhinav Gupta. Iterative visual reasoning beyond convo- lutions. In CVPR, pages 7239-7248, 2018.","title":"Iterative visual reasoning beyond convolutions","authors":"Chen","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2018b] Yunpeng Chen, Marcus Rohrbach, Zhicheng Yan, Shuicheng Yan, Jiashi Feng, and Yannis Kalantidis. Graph-based global reasoning networks. arXiv preprint arXiv:1811.12814, 2018.","title":"Graph-based global reasoning networks","authors":"Chen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Deng et al., 2014] Yubin Deng, Ping Luo, Chen Change Loy, and Xiaoou Tang. Pedestrian attribute recognition at far distance. In ACM Multimedia, pages 789-792, 2014.","title":"Pedestrian attribute recognition at far distance","authors":"Yubin Deng, Ping Luo, Chen Change Loy, Xiaoou Tang","year":"2014","venue":"ACM Multimedia","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gray and Tao, 2008] Douglas Gray and Hai Tao. Viewpoint invariant pedestrian recognition with an ensemble of local- ized features. In ECCV, pages 262-275, 2008.","title":"Viewpoint invariant pedestrian recognition with an ensemble of localized features","authors":"Tao ; Gray, Douglas Gray, Hai Tao","year":"2008","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeff Dean","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kalayeh et al., 2018] Mahdi M Kalayeh, Emrah Basaran, Muhittin Gökmen, Mustafa E Kamasak, and Mubarak Shah. Human semantic parsing for person re- identification. In CVPR, pages 1062-1071, 2018.","title":"Human semantic parsing for person reidentification","authors":"Emrah Mahdi M Kalayeh, Muhittin Basaran, Mustafa E Gökmen, Mubarak Kamasak, Shah","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kipf and Welling, 2017] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li and Gupta, 2018] Yin Li and Abhinav Gupta. Beyond grids: Learning graph representations for visual recogni- tion. In NIPS, pages 9245-9255, 2018.","title":"Beyond grids: Learning graph representations for visual recognition","authors":"Li, Gupta, Yin Li, Abhinav Gupta","year":"2018","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2015] Dangwei Li, Xiaotang Chen, and Kaiqi Huang. Multi-attribute learning for pedestrian attribute recognition in surveillance scenarios. In ACPR, 2015.","title":"Multi-attribute learning for pedestrian attribute recognition in surveillance scenarios","authors":"Li","year":"2015","venue":"ACPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li et al., 2016a] Dangwei Li, Zhang Zhang, Xiaotang Chen, Haibin Ling, and Kaiqi Huang. A richly annotated dataset for pedestrian attribute recognition. In arXiv preprint arXiv:1603.07054, 2016.","title":"A richly annotated dataset for pedestrian attribute recognition","authors":"Li","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2016b] Qiang Li, Maoying Qiao, Wei Bian, and Dacheng Tao. Conditional graphical lasso for multi-label image classification. In CVPR, pages 2977-2986, 2016.","title":"Conditional graphical lasso for multi-label image classification","authors":"Li","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Li et al., 2016c] Yining Li, Chen Huang, Chen Change Loy, and Xiaoou Tang. Human attribute recognition by deep hierarchical contexts. In ECCV, pages 684-700, 2016.","title":"Human attribute recognition by deep hierarchical contexts","authors":"Li","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Li et al., 2018] Dangwei Li, Xiaotang Chen, Zhang Zhang, and Kaiqi Huang. Pose guided deep model for pedestrian attribute recognition in surveillance scenarios. In ICME, pages 1-6, 2018.","title":"Pose guided deep model for pedestrian attribute recognition in surveillance scenarios","authors":"Li","year":"2018","venue":"ICME","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li et al., 2019] Qiaozhe Li, Xin Zhao, Ran He, and Kaiqi Huang. Visual-semantic graph reasoning for pedestrian attribute recognition. In AAAI, 2019.","title":"Visual-semantic graph reasoning for pedestrian attribute recognition","authors":"Qiaozhe Li, Xin Zhao, Ran He, Kaiqi Huang","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2017] Xihui Liu, Haiyu Zhao, Maoqing Tian, Lu Sheng, Jing Shao, Shuai Yi, Junjie Yan, and Xiaogang Wang. Hydraplus-net: Attentive deep features for pedes- trian analysis. In ICCV, pages 350-359, 2017.","title":"Hydraplus-net: Attentive deep features for pedestrian analysis","authors":"Liu","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2018] Pengze Liu, Xihui Liu, Junjie Yan, and Jing Shao. Localization guided learning for pedestrian at- tribute recognition. In BMVC, 2018.","title":"Localization guided learning for pedestrian attribute recognition","authors":"Liu","year":"2018","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ma et al., 2018] Chih-Yao Ma, Asim Kadav, Iain Melvin, Zsolt Kira, Ghassan AlRegib, and Hans Peter Graf. Attend and interact: Higher-order object interactions for video un- derstanding. In CVPR, pages 6790-6800, 2018.","title":"Attend and interact: Higher-order object interactions for video understanding","authors":"Chih-Yao Ma, Asim Kadav, Iain Melvin, Zsolt Kira, Ghassan Alregib, Hans Peter Graf","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sabour et al., 2017] Sara Sabour, Nicholas Frosst, and Ge- offrey E Hinton. Dynamic routing between capsules. In NIPS, pages 3856-3866, 2017.","title":"Dynamic routing between capsules","authors":"Sara Sabour, Nicholas Frosst, Geoffrey E Hinton","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sarafianos and Kakadiaris, 2018] Nikolaos Sarafianos and Ioannis A Kakadiaris. Deep imbalanced attribute classifi- cation using visual attention aggregation. In ECCV, pages 680-697, 2018.","title":"Deep imbalanced attribute classification using visual attention aggregation","authors":"Nikolaos Sarafianos, A Ioannis, Kakadiaris","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Sarfraz et al., 2017] M Saquib Sarfraz, Arne Schumann, Yan Wang, and Rainer Stiefelhagen. Deep view-sensitive pedestrian attribute inference in an end-to-end model. In BMVC, 2017.","title":"Deep view-sensitive pedestrian attribute inference in an end-to-end model","authors":"Arne Saquib Sarfraz, Yan Schumann, Rainer Wang, Stiefelhagen","year":"2017","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Siddiquie et al., 2011] Behjat Siddiquie, Rogerio S Feris, and Larry S Davis. Image ranking and retrieval based on multi-attribute queries. In CVPR, pages 801-808, 2011.","title":"Image ranking and retrieval based on multi-attribute queries","authors":"Behjat Siddiquie, Rogerio S Feris, Larry S Davis","year":"2011","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sudowe et al., 2015] Patrick Sudowe, Hannah Spitzer, and Bastian Leibe. Person attribute recognition with a jointly- trained holistic cnn model. In ICCV Workshops, 2015.","title":"Person attribute recognition with a jointlytrained holistic cnn model","authors":"Patrick Sudowe, Hannah Spitzer, Bastian Leibe","year":"2015","venue":"ICCV Workshops","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang and Gupta, 2018] Xiaolong Wang and Abhinav Gupta. Videos as space-time region graphs. In ECCV, pages 399-417, 2018.","title":"Videos as space-time region graphs","authors":"Wang, Gupta, Xiaolong Wang, Abhinav Gupta","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wang et al., 2016] Jiang Wang, Yi Yang, Junhua Mao, Zhi- heng Huang, Chang Huang, and Wei Xu. Cnn-rnn: A unified framework for multi-label image classification. In CVPR, pages 2285-2294, 2016.","title":"Cnn-rnn: A unified framework for multi-label image classification","authors":"Jiang Wang, Yi Yang, Junhua Mao, Zhiheng Huang, Chang Huang, Wei Xu","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wang et al., 2017] Jingya Wang, Xiatian Zhu, Shaogang Gong, and Wei Li. Attribute recognition by joint recur- rent learning of context and correlation. In ICCV, pages 531-540, 2017.","title":"Attribute recognition by joint recurrent learning of context and correlation","authors":"Wang","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2018] Jingya Wang, Xiatian Zhu, Shaogang Gong, and Wei Li. Transferable joint attribute-identity deep learning for unsupervised person re-identification. In CVPR, pages 2275-2284, 2018.","title":"Transferable joint attribute-identity deep learning for unsupervised person re-identification","authors":"Wang","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2014] Ning Zhang, Manohar Paluri, Marc'Aurelio Ranzato, Trevor Darrell, and Lubomir Bourdev. Panda: Pose aligned networks for deep attribute modeling. In CVPR, pages 1637-1644, 2014.","title":"Panda: Pose aligned networks for deep attribute modeling","authors":"Ning Zhang, Manohar Paluri, Marc'aurelio Ranzato, Trevor Darrell, Lubomir Bourdev","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhao et al., 2018] Xin Zhao, Liufang Sang, Guiguang Ding, Yuchen Guo, and Xiaoming Jin. Grouping attribute recog- nition for pedestrian with joint recurrent learning. In IJ- CAI, pages 3177-3183, 2018.","title":"Grouping attribute recognition for pedestrian with joint recurrent learning","authors":"Xin Zhao, Liufang Sang, Guiguang Ding, Yuchen Guo, Xiaoming Jin","year":"2018","venue":"IJ-CAI","doi":null,"url":null,"internal_url":"/proceedings/2018/441"},{"position":31,"raw":"[Zhu et al., 2013] Jianqing Zhu, Shengcai Liao, Zhen Lei, Dong Yi, and Stan Li. Pedestrian attribute classification in surveillance: Database and evaluation. In ICCV Work- shops, pages 331-338, 2013.","title":"Pedestrian attribute classification in surveillance: Database and evaluation","authors":"Jianqing Zhu, Shengcai Liao, Zhen Lei, Dong Yi, Stan Li","year":"2013","venue":"ICCV Workshops","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}