{"type":"ScholarlyArticle","title":"Self-Supervised Gait Encoding with Locality-Aware Attention for Person Re-Identification","authors":[{"name":"Haocong Rao","slug":"haocong-rao","given":"Haocong","family":"Rao","affiliation":"Shenzhen Institutes of Advanced Technology , Chinese Academy of Sciences; South China University of Technology; The Chinese University of Hong Kong , Hong Kong"},{"name":"Siqi Wang","slug":"siqi-wang","given":"Siqi","family":"Wang","affiliation":"National University of Defense Technology"},{"name":"Xiping Hu","slug":"xiping-hu","given":"Xiping","family":"Hu","affiliation":"Shenzhen Institutes of Advanced Technology , Chinese Academy of Sciences; Lanzhou University; The Chinese University of Hong Kong , Hong Kong"},{"name":"Mingkui Tan","slug":"mingkui-tan","given":"Mingkui","family":"Tan","affiliation":"South China University of Technology"},{"name":"Huang Da","slug":"huang-da","given":"Huang","family":"Da","affiliation":"National University of Defense Technology"},{"name":"Jun Cheng","slug":"jun-cheng","given":"Jun","family":"Cheng","affiliation":"Shenzhen Institutes of Advanced Technology , Chinese Academy of Sciences; The Chinese University of Hong Kong , Hong Kong"},{"name":"Bin Hu","slug":"bin-hu","given":"Bin","family":"Hu","affiliation":"Lanzhou University"}],"abstract":"Gait-based person re-identification (Re-ID) is valuable for safety-critical applications, and using only 3D skeleton data to extract discriminative gait features for person Re-ID is an emerging open topic. Existing methods either adopt hand-crafted features or learn gait features by traditional supervised learning paradigms. Unlike previous methods, we for the first time propose a generic gait encoding approach that can utilize unlabeled skeleton data to learn gait representations in a self-supervised manner. Specifically, we first propose to introduce self-supervision by learning to reconstruct input skeleton sequences in reverse order, which facilitates learning richer high-level semantics and better gait representations. Second, inspired by the fact that motion's continuity endows temporally adjacent skeletons with higher correlations (“locality”), we propose a locality-aware attention mechanism that encourages learning larger attention weights for temporally adjacent skeletons when reconstructing current skeleton, so as to learn locality when encoding gait. Finally, we propose Attention-based Gait Encodings (AGEs), which are built using context vectors learned by locality-aware attention, as final gait representations. AGEs are directly utilized to realize effective person Re-ID. Our approach typically improves existing skeleton-based methods by 10-20% Rank-1 accuracy, and it achieves comparable or even superior performance to multi-modal methods with extra RGB or depth information.","doi":"10.24963/ijcai.2020/125","url":"https://ijcai.org/proceedings/2020/125","pdf":"https://ijcai.org/proceedings/2020/0125.pdf","markdown":"https://ijcai.org/proceedings/2020/125.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/125","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":125,"pages":{"first":898,"last":905},"keywords":["Machine Learning: Deep Learning","Machine Learning: Unsupervised Learning","Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24035/file_24035.m3u8","length_sec":298,"provider":"hls"}],"references":[{"position":1,"raw":"[Aggarwal and Cai, 1999] Jake K Aggarwal and Quin Cai. Human motion analysis: A review. Computer vision and image understanding, 73(3):428-440, 1999.","title":"Human motion analysis: A review","authors":"Cai ; ] Aggarwal, K Jake, Quin Aggarwal, Cai","year":"1999","venue":"Computer vision and image understanding","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andersson and Araujo, 2015] Virginia Ortiz Andersson and Ricardo Matsumura Araujo. Person identification using anthropometric and gait data from kinect sensor. In AAAI, 2015.","title":"Person identification using anthropometric and gait data from kinect sensor","authors":"Andersson, Araujo, Virginia Ortiz Andersson, Ricardo Matsumura Araujo","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Barbosa et al., 2012] Igor Barros Barbosa, Marco Cristani, Alessio Del Bue, Loris Bazzani, and Vittorio Murino. Re- identification with rgb-d sensors. In ECCV, pages 433- 442. Springer, 2012.","title":"Reidentification with rgb-d sensors","authors":"Igor Barros Barbosa, Marco Cristani, Alessio Del Bue, Loris Bazzani, Vittorio Murino","year":"2012","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chunli and Kejun, 2010] Lin Chunli and Wang Kejun. A behavior classification based on enhanced gait energy im- age. In 2nd International Conference on Networking and Digital Society, volume 2, pages 589-592. IEEE, 2010.","title":"A behavior classification based on enhanced gait energy image","authors":"Lin Chunli, Wang Kejun","year":"2010","venue":"2nd International Conference on Networking and Digital Society","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Connor and Ross, 2018] Patrick Connor and Arun Ross. Biometric recognition by gait: A survey of modalities and features. Computer vision and image understanding, 167:1-27, 2018.","title":"Biometric recognition by gait: A survey of modalities and features","authors":"Connor, Ross ;, Patrick Connor, Arun Ross","year":"2018","venue":"Computer vision and image understanding","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cutting and Kozlowski, 1977] James E Cutting and Lynn T Kozlowski. Recognizing friends by their walk: Gait per- ception without familiarity cues. Bulletin of the psycho- nomic society, 9(5):353-356, 1977.","title":"Recognizing friends by their walk: Gait perception without familiarity cues","authors":"E James, Lynn T Cutting, Kozlowski","year":"1977","venue":"Bulletin of the psychonomic society","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gray and Tao, 2008] Douglas Gray and Hai Tao. Viewpoint invariant pedestrian recognition with an ensemble of local- ized features. In ECCV, pages 262-275. Springer, 2008.","title":"Viewpoint invariant pedestrian recognition with an ensemble of localized features","authors":"Tao ; Gray, Douglas Gray, Hai Tao","year":"2008","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Han et al., 2017] Fei Han, Brian Reily, William Hoff, and Hao Zhang. Space-time representation of people based on 3d skeletal data: A review. Computer Vision and Image Understanding, 158:85-105, 2017.","title":"Space-time representation of people based on 3d skeletal data: A review","authors":"Fei Han, Brian Reily, William Hoff, Hao Zhang","year":"2017","venue":"Computer Vision and Image Understanding","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Haque et al., 2016] Albert Haque, Alexandre Alahi, and Li Fei-Fei. Recurrent attention models for depth-based person identification. In CVPR, pages 1229-1238, 2016.","title":"Recurrent attention models for depth-based person identification","authors":"Albert Haque, Alexandre Alahi, Li Fei-Fei","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hasan and Babaguchi, 2016] Mohamed Hasan and Noborou Babaguchi. Long-term people reidentifica- tion using anthropometric signature. In 8th International Conference on Biometrics Theory, Applications and Systems, pages 1-6. IEEE, 2016.","title":"Long-term people reidentification using anthropometric signature","authors":"Mohamed Hasan, Noborou Babaguchi","year":"2016","venue":"8th International Conference on Biometrics Theory, Applications and Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Karianakis et al., 2018] Nikolaos Karianakis, Zicheng Liu, Yinpeng Chen, and Stefano Soatto. Reinforced temporal attention and split-rate transfer for depth-based person re- identification. In ECCV, pages 715-733, 2018.","title":"Reinforced temporal attention and split-rate transfer for depth-based person reidentification","authors":"Nikolaos Karianakis, Zicheng Liu, Yinpeng Chen, Stefano Soatto","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liao et al., 2020] Rijun Liao, Shiqi Yu, Weizhi An, and Yongzhen Huang. A model-based gait recognition method with body pose and human prior knowledge. Pattern Recognition, 98:107069, 2020.","title":"A model-based gait recognition method with body pose and human prior knowledge","authors":"Liao","year":"2020","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Luong et al., 2015] Thang Luong, Hieu Pham, and Christo- pher D. Manning. Effective approaches to attention-based neural machine translation. In EMNLP, pages 1412-1421, Lisbon, Portugal, September 2015. Association for Com- putational Linguistics.","title":"Effective approaches to attention-based neural machine translation","authors":"Thang Luong, Hieu Pham, Christopher D Manning","year":"2015","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Munaro et al., 2014a] Matteo Munaro, Alberto Basso, An- drea Fossati, Luc Van Gool, and Emanuele Menegatti. 3d reconstruction of freely moving persons for re- identification with a depth sensor. In ICRA, pages 4512- 4519. IEEE, 2014.","title":"3d reconstruction of freely moving persons for reidentification with a depth sensor","authors":"Munaro","year":"2014","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Munaro et al., 2014b] Matteo Munaro, Andrea Fossati, Al- berto Basso, Emanuele Menegatti, and Luc Van Gool. One-shot person re-identification with a consumer depth camera. In Person Re-Identification, pages 161-181. Springer, 2014.","title":"One-shot person re-identification with a consumer depth camera","authors":"Munaro","year":"2014","venue":"Person Re-Identification","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Munaro et al., 2014c] Matteo Munaro, Stefano Ghidoni, Deniz Tartaro Dizmen, and Emanuele Menegatti. A feature-based approach to people re-identification using skeleton keypoints. In ICRA, pages 5644-5651. IEEE, 2014.","title":"A feature-based approach to people re-identification using skeleton keypoints","authors":"Munaro","year":"2014","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Murray et al., 1964] M Pat Murray, A Bernard Drought, and Ross C Kory. Walking patterns of normal men. JBJS, 46(2):335-360, 1964.","title":"Walking patterns of normal men","authors":"Pat Murray, A Bernard Drought, Ross C Kory","year":"1964","venue":"JBJS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nambiar et al., 2019] Athira Nambiar, Alexandre Bernardino, and Jacinto C Nascimento. Gait-based person re-identification: A survey. ACM Computing Surveys (CSUR), 52(2):33, 2019.","title":"Gait-based person re-identification: A survey","authors":"Athira Nambiar, Alexandre Bernardino, Jacinto C Nascimento","year":"2019","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pala et al., 2019] Pietro Pala, Lorenzo Seidenari, Stefano Berretti, and Alberto Del Bimbo. Enhanced skeleton and face 3d data for person re-identification from depth cam- eras. Computers & Graphics, 79:69-80, 2019.","title":"Enhanced skeleton and face 3d data for person re-identification from depth cameras","authors":"Pietro Pala, Lorenzo Seidenari, Stefano Berretti, Alberto Del Bimbo","year":"2019","venue":"Computers & Graphics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sivapalan et al., 2011] Sabesan Sivapalan, Daniel Chen, Si- mon Denman, Sridha Sridharan, and Clinton Fookes. Gait energy volumes and frontal gait recognition using depth images. In 2011 International Joint Conference on Bio- metrics (IJCB), pages 1-6. IEEE, 2011.","title":"Gait energy volumes and frontal gait recognition using depth images","authors":"Sivapalan","year":"2011","venue":"2011 International Joint Conference on Biometrics (IJCB)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Vezzani et al., 2013] Roberto Vezzani, Davide Baltieri, and Rita Cucchiara. People reidentification in surveillance and forensics: A survey. ACM Computing Surveys (CSUR), 46(2):29, 2013.","title":"People reidentification in surveillance and forensics: A survey","authors":"Roberto Vezzani, Davide Baltieri, Rita Cucchiara","year":"2013","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Weston et al., 2015] Jason Weston, Sumit Chopra, and An- toine Bordes. Memory networks. In ICLR, 2015.","title":"Memory networks","authors":"Jason Weston, Sumit Chopra","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wu et al., 2017] Ancong Wu, Wei-Shi Zheng, and Jian- Huang Lai. Robust depth-based person re-identification. IEEE Transactions on Image Processing, 26(6):2588- 2603, 2017.","title":"Robust depth-based person re-identification","authors":"Ancong Wu, Wei-Shi Zheng, Jian-Huang Lai","year":"2017","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yoo et al., 2002] Jang-Hee Yoo, Mark S Nixon, and Chris J Harris. Extracting gait signatures based on anatomical knowledge. In Proceedings of BMVA Symposium on Ad- vancing Biometric Technologies, pages 596-606.","title":"Extracting gait signatures based on anatomical knowledge","authors":"Jang-Hee Yoo, Mark S Nixon, Chris J Harris","year":null,"venue":"Proceedings of BMVA Symposium on Advancing Biometric Technologies","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2019] Yuqi Zhang, Yongzhen Huang, Liang Wang, and Shiqi Yu. A comprehensive study on gait bio- metrics using a joint cnn-based method. Pattern Recogni- tion, 93:228-236, 2019.","title":"A comprehensive study on gait biometrics using a joint cnn-based method","authors":"Yuqi Zhang, Yongzhen Huang, Liang Wang, Shiqi Yu","year":"2019","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zheng et al., 2019] Wu Zheng, Lin Li, Zhaoxiang Zhang, Yan Huang, and Liang Wang. Relational network for skeleton-based action recognition. In ICME, pages 826- 831. IEEE, 2019.","title":"Relational network for skeleton-based action recognition","authors":"Wu Zheng, Lin Li, Zhaoxiang Zhang, Yan Huang, Liang Wang","year":"2019","venue":"ICME","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}