{"type":"ScholarlyArticle","title":"SimMC: Simple Masked Contrastive Learning of Skeleton Representations for Unsupervised Person Re-Identification","authors":[{"name":"Haocong Rao","slug":"haocong-rao","given":"Haocong","family":"Rao","affiliation":"School of Computer Science and Engineering , Nanyang Technological University Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY)"},{"name":"Chunyan Miao","slug":"chunyan-miao","given":"Chunyan","family":"Miao","affiliation":"School of Computer Science and Engineering , Nanyang Technological University Joint NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY)"}],"abstract":"Recent advances in skeleton-based person re-identification (re-ID) obtain impressive performance via either hand-crafted skeleton descriptors or skeleton representation learning with deep learning paradigms. However, they typically require skeletal pre-modeling and label information for training, which leads to limited applicability of these methods. In this paper, we focus on unsupervised skeleton-based person re-ID, and present a generic Simple Masked Contrastive learning (SimMC) framework to learn effective representations from unlabeled 3D skeletons for person re-ID. Specifically, to fully exploit skeleton features within each skeleton sequence, we first devise a masked prototype contrastive learning (MPC) scheme to cluster the most typical skeleton features (skeleton prototypes) from different subsequences randomly masked from raw sequences, and contrast the inherent similarity between skeleton features and different prototypes to learn discriminative skeleton representations without using any label. Then, considering that different subsequences within the same sequence usually enjoy strong correlations due to the nature of motion continuity, we propose the masked intra-sequence contrastive learning (MIC) to capture intra-sequence pattern consistency between subsequences, so as to encourage learning more effective skeleton representations for person re-ID. Extensive experiments validate that the proposed SimMC outperforms most state-of-the-art skeleton-based methods. We further show its scalability and efficiency in enhancing the performance of existing models. Our codes are available at https://github.com/Kali-Hac/SimMC.","doi":"10.24963/ijcai.2022/180","url":"https://ijcai.org/proceedings/2022/180","pdf":"https://ijcai.org/proceedings/2022/0180.pdf","markdown":"https://ijcai.org/proceedings/2022/180.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/180","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Computer Vision","track":"Main Track","number":180,"pages":{"first":1290,"last":1297},"keywords":["Computer Vision: Recognition (object detection, categorization)","Computer Vision: Other"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986447","length_sec":80,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984590","length_sec":343,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Andersson and Araujo, 2015] Virginia Ortiz Andersson and Ricardo Matsumura Araujo. Person identification using anthropometric and gait data from kinect sensor. In AAAI, 2015.","title":"Person identification using anthropometric and gait data from kinect sensor","authors":"Andersson, Araujo, Virginia Ortiz Andersson, Ricardo Matsumura Araujo","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barbosa et al., 2012] Igor Barros Barbosa, Marco Cristani, Alessio Del Bue, Loris Bazzani, and Vittorio Murino. Re- identification with rgb-d sensors. In ECCV, pages 433- 442. Springer, 2012.","title":"Reidentification with rgb-d sensors","authors":"Igor Barros Barbosa, Marco Cristani, Alessio Del Bue, Loris Bazzani, Vittorio Murino","year":"2012","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen and He, 2021] Xinlei Chen and Kaiming He. Explor- ing simple siamese representation learning. In CVPR, pages 15750-15758, 2021.","title":"Exploring simple siamese representation learning","authors":"Chen, He ;, Xinlei Chen, Kaiming He","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2020] Ting Chen, Simon Kornblith, Moham- mad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, 2020.","title":"A simple framework for contrastive learning of visual representations","authors":"Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ester et al., 1996] Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al. A density-based algorithm for discovering clusters in large spatial databases with noise. In KDD, volume 96, pages 226-231, 1996.","title":"A density-based algorithm for discovering clusters in large spatial databases with noise","authors":"Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu","year":"1996","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Farenzena et al., 2010] Michela Farenzena, Loris Bazzani, Alessandro Perina, Vittorio Murino, and Marco Cristani. Person re-identification by symmetry-driven accumulation of local features. In CVPR, pages 2360-2367. IEEE, 2010.","title":"Person re-identification by symmetry-driven accumulation of local features","authors":"Michela Farenzena, Loris Bazzani, Alessandro Perina, Vittorio Murino, Marco Cristani","year":"2010","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gray and Tao, 2008] Douglas Gray and Hai Tao. Viewpoint invariant pedestrian recognition with an ensemble of local- ized features. In ECCV, pages 262-275. Springer, 2008.","title":"Viewpoint invariant pedestrian recognition with an ensemble of localized features","authors":"Tao ; Gray, Douglas Gray, Hai Tao","year":"2008","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Han et al., 2017] Fei Han, Brian Reily, William Hoff, and Hao Zhang. Space-time representation of people based on 3d skeletal data: A review. Computer Vision and Image Understanding, 158:85-105, 2017.","title":"Space-time representation of people based on 3d skeletal data: A review","authors":"Fei Han, Brian Reily, William Hoff, Hao Zhang","year":"2017","venue":"Computer Vision and Image Understanding","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2020] Kaiming He, Haoqi Fan, Yuxin Wu, Sain- ing Xie, and Ross Girshick. Momentum contrast for un- supervised visual representation learning. In CVPR, pages 9729-9738, 2020.","title":"Momentum contrast for unsupervised visual representation learning","authors":"Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, Ross Girshick","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2021] Junnan Li, Pan Zhou, Caiming Xiong, and Steven Hoi. Prototypical contrastive learning of unsuper- vised representations. In ICLR, 2021.","title":"Prototypical contrastive learning of unsupervised representations","authors":"Li","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liao et al., 2020] Rijun Liao, Shiqi Yu, Weizhi An, and Yongzhen Huang. A model-based gait recognition method with body pose and human prior knowledge. Pattern Recognition, 98:107069, 2020.","title":"A model-based gait recognition method with body pose and human prior knowledge","authors":"Liao","year":"2020","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2015] Zheng Liu, Zhaoxiang Zhang, Qiang Wu, and Yunhong Wang. Enhancing person re-identification by integrating gait biometric. Neurocomputing, 168:1144- 1156, 2015.","title":"Enhancing person re-identification by integrating gait biometric","authors":"Zheng Liu, Zhaoxiang Zhang, Qiang Wu, Yunhong Wang","year":"2015","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Munaro et al., 2014a] Matteo Munaro, Andrea Fossati, Al- berto Basso, Emanuele Menegatti, and Luc Van Gool. One-shot person re-identification with a consumer depth camera. In Person Re-Identification, pages 161-181. Springer, 2014.","title":"One-shot person re-identification with a consumer depth camera","authors":"Munaro","year":"2014","venue":"Person Re-Identification","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Munaro et al., 2014b] Matteo Munaro, Stefano Ghidoni, Deniz Tartaro Dizmen, and Emanuele Menegatti. A feature-based approach to people re-identification using skeleton keypoints. In ICRA, pages 5644-5651. IEEE, 2014.","title":"A feature-based approach to people re-identification using skeleton keypoints","authors":"Munaro","year":"2014","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Murray et al., 1964] M Pat Murray, A Bernard Drought, and Ross C Kory. Walking patterns of normal men. Journal of Bone and Joint Surgery, 46(2):335-360, 1964.","title":"Walking patterns of normal men","authors":"Pat Murray, A Bernard Drought, Ross C Kory","year":"1964","venue":"Journal of Bone and Joint Surgery","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nambiar et al., 2017] Athira Nambiar, Alexandre Bernardino, Jacinto C Nascimento, and Ana Fred. Context-aware person re-identification in the wild via fu- sion of gait and anthropometric features. In International Conference on Automatic Face & Gesture Recognition, pages 973-980. IEEE, 2017.","title":"Context-aware person re-identification in the wild via fusion of gait and anthropometric features","authors":"Athira Nambiar, Alexandre Bernardino, Jacinto C Nascimento, Ana Fred","year":"2017","venue":"International Conference on Automatic Face & Gesture Recognition","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Oord et al., 2018] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03748, 2018.","title":"Representation learning with contrastive predictive coding","authors":"Oord","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pala et al., 2019] Pietro Pala, Lorenzo Seidenari, Stefano Berretti, and Alberto Del Bimbo. Enhanced skeleton and face 3d data for person re-identification from depth cam- eras. Computers & Graphics, 2019.","title":"Enhanced skeleton and face 3d data for person re-identification from depth cameras","authors":"Pietro Pala, Lorenzo Seidenari, Stefano Berretti, Alberto Del Bimbo","year":"2019","venue":"Computers & Graphics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Rao et al., 2020] Haocong Rao, Siqi Wang, Xiping Hu, Mingkui Tan, Huang Da, Jun Cheng, and Bin Hu. Self- supervised gait encoding with locality-aware attention for person re-identification. In IJCAI, volume 1, pages 898- 905, 2020.","title":"Selfsupervised gait encoding with locality-aware attention for person re-identification","authors":"Haocong Rao, Siqi Wang, Xiping Hu, Mingkui Tan, Huang Da, Jun Cheng, Bin Hu","year":"2020","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rao et al., 2021a] Haocong Rao, Xiping Hu, Jun Cheng, and Bin Hu. Sm-sge: A self-supervised multi-scale skele- ton graph encoding framework for person re-identification. In Proceedings of the 29th ACM international conference on Multimedia, 2021.","title":"Sm-sge: A self-supervised multi-scale skeleton graph encoding framework for person re-identification","authors":"Rao","year":"2021","venue":"Proceedings of the 29th ACM international conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rao et al., 2021b] Haocong Rao, Siqi Wang, Xiping Hu, Mingkui Tan, Yi Guo, Jun Cheng, Xinwang Liu, and Bin Hu. A self-supervised gait encoding approach with locality-awareness for 3d skeleton based person re- identification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.","title":"A self-supervised gait encoding approach with locality-awareness for 3d skeleton based person reidentification","authors":"Rao","year":"2021","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rao et al., 2021c] Haocong Rao, Shihao Xu, Xiping Hu, Jun Cheng, and Bin Hu. Augmented skeleton based con- trastive action learning with momentum lstm for unsuper- vised action recognition. Information Sciences, 569:90- 109, 2021.","title":"Augmented skeleton based contrastive action learning with momentum lstm for unsupervised action recognition","authors":"Rao","year":"2021","venue":"Information Sciences","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rao et al., 2021d] Haocong Rao, Shihao Xu, Xiping Hu, Jun Cheng, and Bin Hu. Multi-level graph encoding with structural-collaborative relation learning for skeleton- based person re-identification. In IJCAI, 2021.","title":"Multi-level graph encoding with structural-collaborative relation learning for skeletonbased person re-identification","authors":"Rao","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Van der Maaten and Hinton, 2008] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.","title":"Visualizing data using t-sne","authors":"Van Der Maaten, Laurens Hinton, Geoffrey Van Der Maaten, Hinton","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wu et al., 2018] Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non- parametric instance discrimination. In CVPR, pages 3733- 3742, 2018.","title":"Unsupervised feature learning via nonparametric instance discrimination","authors":"Zhirong Wu, Yuanjun Xiong, Stella X Yu, Dahua Lin","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Ye et al., 2021] Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven CH Hoi. Deep learn- ing for person re-identification: A survey and outlook. IEEE Transactions on Pattern Analysis and Machine In- telligence, 2021.","title":"Deep learning for person re-identification: A survey and outlook","authors":"Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, Steven Ch Hoi","year":"2021","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yu et al., 2006] Shiqi Yu, Daoliang Tan, and Tieniu Tan. A framework for evaluating the effect of view angle, cloth- ing and carrying condition on gait recognition. In ICPR, volume 4, pages 441-444. IEEE, 2006.","title":"A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition","authors":"Shiqi Yu, Daoliang Tan, Tieniu Tan","year":"2006","venue":"ICPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zheng et al., 2015] Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian. Scalable person re-identification: A benchmark. In ICCV, pages 1116-1124, 2015.","title":"Scalable person re-identification: A benchmark","authors":"Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, Qi Tian","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}