{"type":"ScholarlyArticle","title":"Label-Attended Hashing for Multi-Label Image Retrieval","authors":[{"name":"Yanzhao Xie","slug":"yanzhao-xie","given":"Yanzhao","family":"Xie","affiliation":"Huazhong University of Science and Technology"},{"name":"Yu Liu","slug":"yu-liu","given":"Yu","family":"Liu","affiliation":"Huazhong University of Science and Technology"},{"name":"Yangtao Wang","slug":"yangtao-wang","given":"Yangtao","family":"Wang","affiliation":"Huazhong University of Science and Technology"},{"name":"Lianli Gao","slug":"lianli-gao","given":"Lianli","family":"Gao","affiliation":"The University of Electronic Science and Technology of China {yzxie , liu_yu ,"},{"name":"Peng Wang","slug":"peng-wang","given":"Peng","family":"Wang","affiliation":"Huazhong University of Science and Technology"},{"name":"Ke Zhou","slug":"ke-zhou","given":"Ke","family":"Zhou","affiliation":"Huazhong University of Science and Technology"}],"abstract":"For the multi-label image retrieval, the existing hashing algorithms neglect the dependency between objects and thus fail to capture the attention information in the feature extraction, which affects the precision of hash codes. To address this problem, we explore the inter-dependency between objects through their co-occurrence correlation from the label set and adopt Multi-modal Factorized Bilinear (MFB) pooling component so that the image representation learning can capture this attention information. We propose a Label-Attended Hashing (LAH) algorithm which enables an end-to-end hash model with inter-dependency feature extraction. LAH first combines Convolutional Neural Network (CNN) and Graph Convolution Network (GCN) to separately generate the image representation and label co-occurrence embeddings, then adopts MFB to fuse these two modal vectors, finally learns the hash function with a Cauchy distribution based loss function via back propagation. Extensive experiments on public multi-label datasets demonstrate that (1) LAH can achieve the state-of-the-art retrieval results and (2) the usage of co-occurrence relationship and MFB not only promotes the precision of hash codes but also accelerates the hash learning. GitHub address: https://github.com/IDSM-AI/LAH.","doi":"10.24963/ijcai.2020/133","url":"https://ijcai.org/proceedings/2020/133","pdf":"https://ijcai.org/proceedings/2020/0133.pdf","markdown":"https://ijcai.org/proceedings/2020/133.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/133","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":133,"pages":{"first":955,"last":962},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24108/file_24108.m3u8","length_sec":275,"provider":"hls"}],"references":[{"position":1,"raw":"[Amari, 1993] Shun-ichi Amari. Backpropagation and stochastic gradient descent method. Neurocomputing, 5(3):185-196, 1993.","title":"Shun-ichi Amari. Backpropagation and stochastic gradient descent method","authors":"Amari","year":"1993","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2018] Yue Cao, Mingsheng Long, Bin Liu, and Jianmin Wang. Deep cauchy hashing for hamming space retrieval. In CVPR, pages 1229-1237, 2018.","title":"Deep cauchy hashing for hamming space retrieval","authors":"Yue Cao, Mingsheng Long, Bin Liu, Jianmin Wang","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2019] Zhao-Min Chen, Xiu-Shen Wei, Peng Wang, and Yanwen Guo. Multi-label image recognition with graph convolutional networks. In CVPR, pages 5177- 5186, 2019.","title":"Multi-label image recognition with graph convolutional networks","authors":"Xiu-Shen Zhao-Min Chen, Peng Wei, Yanwen Wang, Guo","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Fei-Fei Li. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Fei-Fei Li","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Everingham et al., 2010] Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John Winn, and Andrew Zis- serman. The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88(2):303-338, jun 2010.","title":"The pascal visual object classes (voc) challenge","authors":"Mark Everingham, Luc Van Gool, K I Christopher, John Williams, Andrew Winn, Zisserman","year":"2010","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang et al., 2018] Chang-Qin Huang, Shang-Ming Yang, Yan Pan, and Hanjiang Lai. Object-location-aware hashing for multi-label image retrieval via automatic mask learn- ing. IEEE Trans. Cybernetics, 27(9):4490-4502, 2018.","title":"Object-location-aware hashing for multi-label image retrieval via automatic mask learning","authors":"Chang-Qin Huang, Shang-Ming Yang, Yan Pan, Hanjiang Lai","year":"2018","venue":"IEEE Trans. Cybernetics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huiskes and Lew, 2008] Mark J. Huiskes and Michael S. Lew. The MIR flickr retrieval evaluation. In SIGMM, pages 39-43, 2008.","title":"The MIR flickr retrieval evaluation","authors":"Mark J Huiskes, Michael S Lew","year":"2008","venue":"SIGMM","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Joulin et al., 2016] Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hervé Jégou, and Tomas Mikolov. Fasttext.zip: Compressing text classification models. CoRR, abs/1612.03651, 2016.","title":"Fasttext.zip: Compressing text classification models","authors":"Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hervé Jégou, Tomas Mikolov","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Lai et al., 2016] Hanjiang Lai, Pan Yan, Xiangbo Shu, Yun- chao Wei, and Shuicheng Yan. Instance-aware hashing for multi-label image retrieval. IEEE Trans. Image Process- ing, 25(6):2469-2479, 2016.","title":"Instance-aware hashing for multi-label image retrieval","authors":"Hanjiang Lai, Pan Yan, Xiangbo Shu, Yunchao Wei, Shuicheng Yan","year":"2016","venue":"IEEE Trans. Image Processing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2017] Yanghao Li, Naiyan Wang, Jiaying Liu, and Xiaodi Hou. Factorized bilinear models for image recog- nition. In ICCV, pages 2098-2106, 2017.","title":"Factorized bilinear models for image recognition","authors":"Yanghao Li, Naiyan Wang, Jiaying Liu, Xiaodi Hou","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge J. Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: com- mon objects in context. In ECCV, pages 740-755, 2014.","title":"Microsoft COCO: common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge J Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence Zitnick","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2019] Yu Liu, Jingkuan Song, Ke Zhou, Lingyu Yan, Li Liu, Fuhao Zou, and Ling Shao. Deep self-taught hashing for image retrieval. IEEE Trans. Cybernetics, 49(6):2229-2241, 2019.","title":"Deep self-taught hashing for image retrieval","authors":"Yu Liu, Jingkuan Song, Ke Zhou, Lingyu Yan, Li Liu, Fuhao Zou, Ling Shao","year":"2019","venue":"IEEE Trans. Cybernetics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2020] Yu Liu, Yangtao Wang, Ke Zhou, Yujuan Yang, and Yifei Liu. Semantic-aware data quality assess- ment for image big data. Future Gener. Comput. Syst., 102:53-65, 2020.","title":"Semantic-aware data quality assessment for image big data","authors":"Yu Liu, Yangtao Wang, Ke Zhou, Yujuan Yang, Yifei Liu","year":"2020","venue":"Future Gener. Comput. Syst","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ma et al., 2018] Cheng Ma, Zhixiang Chen, Jiwen Lu, and Jie Zhou. Rank-consistency multi-label deep hashing. In ICME, pages 1-6, 2018.","title":"Rank-consistency multi-label deep hashing","authors":"Cheng Ma, Zhixiang Chen, Jiwen Lu, Jie Zhou","year":"2018","venue":"ICME","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Kai Chen, Greg Cor- rado, and Jeffrey Dean. Efficient estimation of word rep- resentations in vector space. In ICLR, 2013.","title":"Efficient estimation of word representations in vector space","authors":"Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean","year":"2013","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pennington et al., 2014] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In EMNLP, pages 1532-1543, 2014.","title":"Glove: Global vectors for word representation","authors":"Jeffrey Pennington, Richard Socher, Christopher D Manning","year":"2014","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Song et al., 2017] Jingkuan Song, Tao He, Lianli Gao, Xing Xu, and Heng Tao Shen. Deep region hashing for ef- ficient large-scale instance search from images. CoRR, abs/1701.07901, 2017.","title":"Deep region hashing for efficient large-scale instance search from images","authors":"Song","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2017] Zhouxia Wang, Tianshui Chen, Guanbin Li, Ruijia Xu, and Liang Lin. Multi-label image recog- nition by recurrently discovering attentional regions. In ICCV, pages 464-472, 2017.","title":"Multi-label image recognition by recurrently discovering attentional regions","authors":"Wang","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Yang et al., 2019] Erkun Yang, Tongliang Liu, Cheng Deng, Wei Liu, and Dacheng Tao. Distillhash: Unsuper- vised deep hashing by distilling data pairs. In CVPR, pages 2946-2955, 2019.","title":"Distillhash: Unsupervised deep hashing by distilling data pairs","authors":"Yang","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yu et al., 2017] Zhou Yu, Jun Yu, Jianping Fan, and Dacheng Tao. Multi-modal factorized bilinear pooling with co-attention learning for visual question answering. In ICCV, pages 1839-1848, 2017.","title":"Multi-modal factorized bilinear pooling with co-attention learning for visual question answering","authors":"Zhou Yu, Jun Yu, Jianping Fan, Dacheng Tao","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zhao et al., 2015] Fang Zhao, Yongzhen Huang, Liang Wang, and Tieniu Tan. Deep semantic ranking based hash- ing for multi-label image retrieval. In CVPR, pages 1556- 1564, 2015.","title":"Deep semantic ranking based hashing for multi-label image retrieval","authors":"Fang Zhao, Yongzhen Huang, Liang Wang, Tieniu Tan","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhou et al., 2018] Xiang Zhou, Fumin Shen, Li Liu, Wei Liu, Liqiang Nie, Yang Yang, and Heng Tao Shen. Graph convolutional network hashing. IEEE transactions on cy- bernetics, 2018.","title":"Graph convolutional network hashing","authors":"Xiang Zhou, Fumin Shen, Li Liu, Wei Liu, Liqiang Nie, Yang Yang, Heng Tao Shen","year":"2018","venue":"IEEE transactions on cybernetics","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zou et al., 2019] Qin Zou, Zheng Zhang, Ling Cao, Long Chen, and Song Wang. Transductive zero-shot hashing for multi-label image retrieval. CoRR, abs/1911.07192, 2019.","title":"Transductive zero-shot hashing for multi-label image retrieval","authors":"Qin Zou, Zheng Zhang, Ling Cao, Long Chen, Song Wang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}