{"type":"ScholarlyArticle","title":"Co-Saliency Spatio-Temporal Interaction Network for Person Re-Identification in Videos","authors":[{"name":"Jiawei Liu","slug":"jiawei-liu","given":"Jiawei","family":"Liu","affiliation":"University of Science and Technology of China , China"},{"name":"Zheng-Jun Zha","slug":"zheng-jun-zha","given":"Zheng-Jun","family":"Zha","affiliation":"University of Science and Technology of China , China"},{"name":"Xierong Zhu","slug":"xierong-zhu","given":"Xierong","family":"Zhu","affiliation":"University of Science and Technology of China , China"},{"name":"Na Jiang","slug":"na-jiang","given":"Na","family":"Jiang","affiliation":"Capital Normal University , China"}],"abstract":"Person re-identification aims at identifying a certain pedestrian across non-overlapping camera networks. Video-based person re-identification approaches have gained significant attention recently, expanding image-based approaches by learning features from multiple frames. In this work, we propose a novel Co-Saliency Spatio-Temporal Interaction Network (CSTNet) for person re-identification in videos. It captures the common salient foreground regions among video frames and explores the spatial-temporal long-range context interdependency from such regions, towards learning discriminative pedestrian representation. Specifically, multiple co-saliency learning modules within CSTNet are designed to utilize the correlated information across video frames to extract the salient features from the task-relevant regions and suppress background interference. Moreover, multiple spatial-temporal interaction modules within CSTNet are proposed, which exploit the spatial and temporal long-range context interdependencies on such features and spatial-temporal information correlation, to enhance feature representation. Extensive experiments on two benchmarks have demonstrated the effectiveness of the proposed method.","doi":"10.24963/ijcai.2020/141","url":"https://ijcai.org/proceedings/2020/141","pdf":"https://ijcai.org/proceedings/2020/0141.pdf","markdown":"https://ijcai.org/proceedings/2020/141.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/141","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":141,"pages":{"first":1012,"last":1018},"keywords":["Computer Vision: Video: Events, Activities and Surveillance","Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24679/file_24679.m3u8","length_sec":298,"provider":"hls"}],"references":[{"position":1,"raw":"[Chen et al., 2018] Dapeng Chen, Hongsheng Li, Tong Xiao, Shuai Yi, and Xiaogang Wang. Video person re- identification with competitive snippet-similarity aggrega- tion and co-attentive snippet embedding. In CVPR, pages 1169-1178, 2018.","title":"Video person reidentification with competitive snippet-similarity aggregation and co-attentive snippet embedding","authors":"Dapeng Chen, Hongsheng Li, Tong Xiao, Shuai Yi, Xiaogang Wang","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Hermans et al., 2017] Alexander Hermans, Lucas Beyer, and Bastian Leibe. In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737, 2017.","title":"In defense of the triplet loss for person re-identification","authors":"Alexander Hermans, Lucas Beyer, Bastian Leibe","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Hou et al., 2019] Ruibing Hou, Bingpeng Ma, Hong Chang, Xinqian Gu, Shiguang Shan, and Xilin Chen. Vrstc: Occlusion-free video person re-identification. In CVPR, pages 7183-7192, 2019.","title":"Vrstc: Occlusion-free video person re-identification","authors":"Hou","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Li et al., 2018a] Shuang Li, Slawomir Bak, Peter Carr, and Xiaogang Wang. Diversity regularized spatiotemporal at- tention for video-based person re-identification. In CVPR, pages 369-378, 2018.","title":"Diversity regularized spatiotemporal attention for video-based person re-identification","authors":"Li","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Li et al., 2018b] Wei Li, Xiatian Zhu, and Shaogang Gong. Harmonious attention network for person re-identification. In CVPR, pages 2285-2294, 2018.","title":"Harmonious attention network for person re-identification","authors":"Li","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Li et al., 2019] Jianing Li, Jingdong Wang, Qi Tian, Wen Gao, and Shiliang Zhang. Global-local temporal represen- tations for video person re-identification. In ICCV, pages 3958-3967, 2019.","title":"Global-local temporal representations for video person re-identification","authors":"Jianing Li, Jingdong Wang, Qi Tian, Wen Gao, Shiliang Zhang","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Liao et al., 2018] Xingyu Liao, Lingxiao He, Zhouwang Yang, and Chi Zhang. Video-based person re- identification via 3d convolutional networks and non-local attention. In Asian Conference on Computer Vision, pages 620-634. Springer, 2018.","title":"Video-based person reidentification via 3d convolutional networks and non-local attention","authors":"Liao","year":"2018","venue":"Asian Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liu et al., 2016] Jiawei Liu, Zhengjun Zha, Qi Tian, Dong Liu, Ting Yao, Qiang Ling, and Tao Mei. Multi-scale triplet cnn for person re-identification. In ACM Interna- tional Conference on Multimedia, pages 192-196, 2016.","title":"Multi-scale triplet cnn for person re-identification","authors":"Jiawei Liu, Zhengjun Zha, Qi Tian, Dong Liu, Ting Yao, Qiang Ling, Tao Mei","year":"2016","venue":"ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2019a] Jiawei Liu, Zhengjun Zha, Di Chen, Richang Hong, and Meng Wang. Adaptive transfer net- work for cross-domain person re-identification. In CVPR, pages 7202-7211, 2019.","title":"Adaptive transfer network for cross-domain person re-identification","authors":"Liu","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2019b] Jiawei Liu, Zhengjun Zha, Xuejin Chen, Zilei Wang, and Yongdong Zhang. Dense 3d- convolutional neural network for person re-identification in videos. ACM Transactions on Multimedia Computing, Communications, and Applications, 15(1):8, 2019.","title":"Dense 3dconvolutional neural network for person re-identification in videos","authors":"Liu","year":"2019","venue":"ACM Transactions on Multimedia Computing, Communications, and Applications","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2019c] Yiheng Liu, Zhenxun Yuan, Wengang Zhou, and Houqiang Li. Spatial and temporal mutual pro- motion for video-based person re-identification. In AAAI, volume 33, pages 8786-8793, 2019.","title":"Spatial and temporal mutual promotion for video-based person re-identification","authors":"Liu","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[McLaughlin et al., 2016] Niall McLaughlin, Jesus Mar- tinez del Rincon, and Paul Miller. Recurrent convolu- tional network for video-based person re-identification. In CVPR, pages 1325-1334, 2016.","title":"Jesus Martinez del Rincon, and Paul Miller. Recurrent convolutional network for video-based person re-identification","authors":"Niall Mclaughlin","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Rao et al., 2019] Shivansh Rao, Peng Cao, Tanzila Rahman, Mrigank Rochan, and Yang Wang. Non-local attentive temporal network for video-based person re-identification. In IEEE International Conference on Advanced Video and Signal Based Surveillance, pages 1-8. IEEE, 2019.","title":"Non-local attentive temporal network for video-based person re-identification","authors":"Rao","year":"2019","venue":"IEEE International Conference on Advanced Video and Signal Based Surveillance","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Song et al., 2018] Chunfeng Song, Yan Huang, Wanli Ouyang, and Liang Wang. Mask-guided contrastive at- tention model for person re-identification. In CVPR, pages 1179-1188, 2018.","title":"Mask-guided contrastive attention model for person re-identification","authors":"Chunfeng Song, Yan Huang, Wanli Ouyang, Liang Wang","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Subramaniam et al., 2016] Arulkumar Subramaniam, Moitreya Chatterjee, and Anurag Mittal. Deep neural net- works with inexact matching for person re-identification. In Advances in Neural Information Processing Systems, pages 2667-2675, 2016.","title":"Deep neural networks with inexact matching for person re-identification","authors":"Subramaniam","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Szegedy et al., 2016] Christian Szegedy, Vincent Van- houcke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In CVPR, pages 2818-2826, 2016.","title":"Rethinking the inception architecture for computer vision","authors":"Szegedy","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2014] Taiqing Wang, Shaogang Gong, Xiatian Zhu, and Shengjin Wang. Person re-identification by video ranking. In ECCV, pages 688-703. Springer, 2014.","title":"Person re-identification by video ranking","authors":"Wang","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2018] Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. Non-local neural networks. In CVPR, pages 7794-7803, 2018.","title":"Non-local neural networks","authors":"Xiaolong Wang, Ross Girshick, Abhinav Gupta, Kaiming He","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xu et al., 2017] Shuangjie Xu, Yu Cheng, Kang Gu, Yang Yang, Shiyu Chang, and Pan Zhou. Jointly attentive spatial-temporal pooling networks for video-based person re-identification. In ICCV, pages 4733-4742, 2017.","title":"Jointly attentive spatial-temporal pooling networks for video-based person re-identification","authors":"Shuangjie Xu, Yu Cheng, Kang Gu, Yang Yang, Shiyu Chang, Pan Zhou","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang et al., 2012] Hanwang Zhang, Zheng-Jun Zha, Shuicheng Yan, Jingwen Bian, and Tat-Seng Chua. Attribute feedback. In ACM International Conference on Multimedia, pages 79-88, 2012.","title":"Attribute feedback","authors":"Hanwang Zhang, Zheng-Jun Zha, Shuicheng Yan, Jingwen Bian, Tat-Seng Chua","year":"2012","venue":"ACM International Conference on Multimedia","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhang et al., 2014] Hanwang Zhang, Zheng-Jun Zha, Yang Yang, Shuicheng Yan, and Tat-Seng Chua. Robust (semi) nonnegative graph embedding. IEEE Transactions on Im- age Processing, 23(7):2996-3012, 2014.","title":"Robust (semi) nonnegative graph embedding","authors":"Hanwang Zhang, Zheng-Jun Zha, Yang Yang, Shuicheng Yan, Tat-Seng Chua","year":"2014","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhao et al., 2017] Haiyu Zhao, Maoqing Tian, Shuyang Sun, Jing Shao, Junjie Yan, Shuai Yi, Xiaogang Wang, and Xiaoou Tang. Spindle net: Person re-identification with human body region guided feature decomposition and fu- sion. In CVPR, pages 1077-1085, 2017.","title":"Spindle net: Person re-identification with human body region guided feature decomposition and fusion","authors":"Haiyu Zhao, Maoqing Tian, Shuyang Sun, Jing Shao, Junjie Yan, Shuai Yi, Xiaogang Wang, Xiaoou Tang","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhao et al., 2019] Yiru Zhao, Xu Shen, Zhongming Jin, Hongtao Lu, and Xian-sheng Hua. Attribute-driven fea- ture disentangling and temporal aggregation for video per- son re-identification. In CVPR, pages 4913-4922, 2019.","title":"Attribute-driven feature disentangling and temporal aggregation for video person re-identification","authors":"Yiru Zhao, Xu Shen, Zhongming Jin, Hongtao Lu, Xian-Sheng Hua","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zheng et al., 2016] Liang Zheng, Zhi Bie, Yifan Sun, Jing- dong Wang, Chi Su, Shengjin Wang, and Qi Tian. Mars: A video benchmark for large-scale person re-identification. In ECCV, pages 868-884. Springer, 2016.","title":"Mars: A video benchmark for large-scale person re-identification","authors":"Liang Zheng, Zhi Bie, Yifan Sun, Jingdong Wang, Chi Su, Shengjin Wang, Qi Tian","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhou et al., 2017] Zhen Zhou, Yan Huang, Wei Wang, Liang Wang, and Tieniu Tan. See the forest for the trees: Joint spatial and temporal recurrent neural networks for video-based person re-identification. In CVPR, pages 4747-4756, 2017.","title":"See the forest for the trees: Joint spatial and temporal recurrent neural networks for video-based person re-identification","authors":"Zhen Zhou, Yan Huang, Wei Wang, Liang Wang, Tieniu Tan","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}