{"type":"ScholarlyArticle","title":"Compressed Self-Attention for Deep Metric Learning with Low-Rank Approximation","authors":[{"name":"Ziye Chen","slug":"ziye-chen","given":"Ziye","family":"Chen","affiliation":"School of Computer Science , Institute of Artificial Intelligence , and National Engineering Research Center for Multimedia Software , Wuhan University , China"},{"name":"Mingming Gong","slug":"mingming-gong","given":"Mingming","family":"Gong","affiliation":"School of Mathematics and Statistics , University of Melbourne , Australia {ziyechen , 2014301500136 ,"},{"name":"Lingjuan Ge","slug":"lingjuan-ge","given":"Lingjuan","family":"Ge","affiliation":"School of Computer Science , Institute of Artificial Intelligence , and National Engineering Research Center for Multimedia Software , Wuhan University , China"},{"name":"Bo Du","slug":"bo-du","given":"Bo","family":"Du","affiliation":"School of Computer Science , Institute of Artificial Intelligence , and National Engineering Research Center for Multimedia Software , Wuhan University , China"}],"abstract":"In this paper, we apply self-attention (SA) mechanism to boost the performance of deep metric learning. However, due to the pairwise similarity measurement, the cost of storing and manipulating the complete attention maps makes it infeasible for large inputs. To solve this problem, we propose\r\na compressed self-attention with low-rank approximation (CSALR) module, which significantly reduces the computation and memory costs without\r\nsacrificing the accuracy. In CSALR, the original attention map is decomposed into a landmark attention map and a combination coefficient map with\r\na small number of landmark feature vectors sampled from the input feature map by average pooling. Thanks to the efficiency of CSALR, we can apply CSALR to high-resolution shallow convolutional layers and implement a multi-head form of CSALR, which further boosts the performance. We evaluate the proposed CSALR on person reidentification which is a typical metric learning task. Extensive experiments shows the effectiveness and efficiency of CSALR in deep metric learning and its superiority over the baselines.","doi":"10.24963/ijcai.2020/285","url":"https://ijcai.org/proceedings/2020/285","pdf":"https://ijcai.org/proceedings/2020/0285.pdf","markdown":"https://ijcai.org/proceedings/2020/285.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/285","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":285,"pages":{"first":2058,"last":2064},"keywords":["Machine Learning: Deep Learning","Machine Learning: Deep Learning: Convolutional networks","Machine Learning: Classification"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24682/file_24682.m3u8","length_sec":287,"provider":"hls"}],"references":[{"position":1,"raw":"[Baker, 1977] Christopher TH Baker. The numerical treat- ment of integral equations. 1977.","title":"The numerical treatment of integral equations","authors":"T H Christopher, Baker","year":"1977","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2020] Ziye Chen, Yanwu Xu, Mingming Gong, Chaohui Wang, Bo Du, and Kun Zhang. Compressed self- attention for deep metric learning. 2020.","title":"Compressed selfattention for deep metric learning","authors":"Chen","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Deng et al., 2019] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4690-4699, 2019.","title":"Arcface: Additive angular margin loss for deep face recognition","authors":"Deng","year":"2019","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Fan et al., 2019] Xing Fan, Wei Jiang, Hao Luo, and Mengjuan Fei. Spherereid: Deep hypersphere manifold embedding for person re-identification. Journal of Vi- sual Communication and Image Representation, 60:51- 58, 2019.","title":"Spherereid: Deep hypersphere manifold embedding for person re-identification","authors":"Xing Fan, Wei Jiang, Hao Luo, Mengjuan Fei","year":"2019","venue":"Journal of Visual Communication and Image Representation","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fang et al., 2020a] Yixiang Fang, Xin Huang, Lu Qin, Ying Zhang, Wenjie Zhang, Reynold Cheng, and Xuemin Lin. A survey of community search over big graphs. 29(1):353-392, 2020.","title":"A survey of community search over big graphs","authors":"Fang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fang et al., 2020b] Yixiang Fang, Yixing Yang, Wenjie Zhang, Xuemin Lin, and Xin Cao. Effective and efficient community search over large heterogeneous information networks. 13(6):854-867, 2020.","title":"Effective and efficient community search over large heterogeneous information networks","authors":"Fang","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fu et al., 2019] Jun Fu, Jing Liu, Haijie Tian, Yong Li, Yongjun Bao, Zhiwei Fang, and Hanqing Lu. Dual at- tention network for scene segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3146-3154, 2019.","title":"Dual attention network for scene segmentation","authors":"Jun Fu, Jing Liu, Haijie Tian, Yong Li, Yongjun Bao, Zhiwei Fang, Hanqing Lu","year":"2019","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Han et al., 2018] Kai Han, Jianyuan Guo, Chao Zhang, and Mingjian Zhu. Attribute-aware attention model for fine- grained representation learning. pages 2040-2048, 2018.","title":"Attribute-aware attention model for finegrained representation learning","authors":"Kai Han, Jianyuan Guo, Chao Zhang, Mingjian Zhu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kalayeh et al., 2018] Mahdi M Kalayeh, Emrah Basaran, Muhittin Gökmen, Mustafa E Kamasak, and Mubarak Shah. Human semantic parsing for person re- identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1062- 1071, 2018.","title":"Human semantic parsing for person reidentification","authors":"Emrah Mahdi M Kalayeh, Muhittin Basaran, Mustafa E Gökmen, Mubarak Kamasak, Shah","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2018] Weiyang Liu, Rongmei Lin, Zhen Liu, Lixin Liu, Zhiding Yu, Bo Dai, and Le Song. Learn- ing towards minimum hyperspherical energy. In Advances in Neural Information Processing Systems, pages 6222- 6233, 2018.","title":"Learning towards minimum hyperspherical energy","authors":"Weiyang Liu, Rongmei Lin, Zhen Liu, Lixin Liu, Zhiding Yu, Bo Dai, Le Song","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Mishchuk et al., 2017] Anastasiia Mishchuk, Dmytro Mishkin, Filip Radenovic, and Jiri Matas. Working hard to know your neighbor's margins: Local descriptor learn- ing loss. In Advances in Neural Information Processing Systems, pages 4826-4837, 2017.","title":"Working hard to know your neighbor's margins: Local descriptor learning loss","authors":"Anastasiia Mishchuk, Dmytro Mishkin, Filip Radenovic, Jiri Matas","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ristani et al., 2016] Ergys Ristani, Francesco Solera, Roger Zou, Rita Cucchiara, and Carlo Tomasi. Performance mea- sures and a data set for multi-target, multi-camera track- ing. In European Conference on Computer Vision, pages 17-35. Springer, 2016.","title":"Performance measures and a data set for multi-target, multi-camera tracking","authors":"Ristani","year":"2016","venue":"European Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Si et al., 2018] Jianlou Si, Honggang Zhang, Chun-Guang Li, Jason Kuen, Xiangfei Kong, Alex C Kot, and Gang Wang. Dual attention matching network for context-aware feature sequence based person re-identification. In Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5363-5372, 2018.","title":"Dual attention matching network for context-aware feature sequence based person re-identification","authors":"Si","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Sun et al., 2018] Yifan Sun, Liang Zheng, Yi Yang, Qi Tian, and Shengjin Wang. Beyond part models: Person retrieval with refined part pooling (and a strong convolutional base- line). In Proceedings of the European Conference on Com- puter Vision (ECCV), pages 480-496, 2018.","title":"Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline)","authors":"Yifan Sun, Liang Zheng, Yi Yang, Qi Tian, Shengjin Wang","year":"2018","venue":"Proceedings of the European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing sys- tems, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Wang et al., 2018a] Cheng Wang, Qian Zhang, Chang Huang, Wenyu Liu, and Xinggang Wang. Mancs: A multi-task attentional network with curriculum sampling for person re-identification. In Proceedings of the Euro- pean Conference on Computer Vision (ECCV), pages 365- 381, 2018.","title":"Mancs: A multi-task attentional network with curriculum sampling for person re-identification","authors":"Wang","year":"2018","venue":"Proceedings of the European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Wang et al., 2018b] Xiaolong Wang, Ross Girshick, Abhi- nav Gupta, and Kaiming He. Non-local neural networks. In Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition, pages 7794-7803, 2018.","title":"Non-local neural networks","authors":"Wang","year":"2018","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Wang et al., 2018c] Zheng Wang, Xiang Bai, Mang Ye, and Shinichi Satoh. Incremental deep hidden attribute learn- ing. pages 72-80, 2018.","title":"Incremental deep hidden attribute learning","authors":"Wang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xu et al., 2019] Yanwu Xu, Mingming Gong, Tongliang Liu, Kayhan Batmanghelich, and Chaohui Wang. Ro- bust angular local descriptor learning. arXiv preprint arXiv:1901.07076, 2019.","title":"Robust angular local descriptor learning","authors":"Xu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ye et al., 2018] Mang Ye, Zheng Wang, Xiangyuan Lan, and Pong C Yuen. Visible thermal person re-identification via dual-constrained top-ranking. pages 1092-1099, 2018.","title":"Visible thermal person re-identification via dual-constrained top-ranking","authors":"Mang Ye, Zheng Wang, Xiangyuan Lan, Pong C Yuen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2018/152"},{"position":21,"raw":"[Zhang et al., 2018] Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augustus Odena. Self-attention generative adversarial networks. arXiv preprint arXiv:1805.08318, 2018.","title":"Self-attention generative adversarial networks","authors":"Han Zhang, Ian Goodfellow, Dimitris Metaxas, Augustus Odena","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zheng et al., 2015] Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian. Scalable person re-identification: A benchmark. In Proceedings of the IEEE international conference on computer vision, pages 1116-1124, 2015.","title":"Scalable person re-identification: A benchmark","authors":"Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, Qi Tian","year":"2015","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zheng et al., 2017] Zhedong Zheng, Liang Zheng, and Yi Yang. Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In Proceed- ings of the IEEE International Conference on Computer Vision, pages 3754-3762, 2017.","title":"Unlabeled samples generated by gan improve the person re-identification baseline in vitro","authors":"Zheng","year":"2017","venue":"Proceedings of the IEEE International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhong et al., 2017] Zhun Zhong, Liang Zheng, Donglin Cao, and Shaozi Li. Re-ranking person re-identification with k-reciprocal encoding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1318-1327, 2017.","title":"Re-ranking person re-identification with k-reciprocal encoding","authors":"Zhun Zhong, Liang Zheng, Donglin Cao, Shaozi Li","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}