{"type":"ScholarlyArticle","title":"Graph Convolutional Network Hashing for Cross-Modal Retrieval","authors":[{"name":"Ruiqing Xu","slug":"ruiqing-xu","given":"Ruiqing","family":"Xu","affiliation":"School of Electronic Engineering , Xidian University"},{"name":"Chao Li","slug":"chao-li","given":"Chao","family":"Li","affiliation":"School of Electronic Engineering , Xidian University"},{"name":"Junchi Yan","slug":"junchi-yan","given":"Junchi","family":"Yan","affiliation":"Dept. of CSE & MoE Key Lab of Artificial Intelligence , Shanghai Jiao Tong University"},{"name":"Cheng Deng","slug":"cheng-deng","given":"Cheng","family":"Deng","affiliation":"School of Electronic Engineering , Xidian University"},{"name":"Xianglong Liu","slug":"xianglong-liu","given":"Xianglong","family":"Liu","affiliation":"Beihang University"}],"abstract":"Deep network based cross-modal retrieval has recently made significant progress. However, bridging modality gap to further enhance the retrieval accuracy still remains a crucial bottleneck. In this paper, we propose a Graph Convolutional Hashing (GCH) approach, which learns modality-unified binary codes via an affinity graph. An end-to-end deep architecture is constructed with three main components: a semantic encoder module, two feature encoding networks, and a graph convolutional network (GCN). We design a semantic encoder as a teacher module to guide the feature encoding process, a.k.a. student module, for semantic information exploiting. Furthermore, GCN is utilized to explore the inherent similarity structure among data points, which will help to generate discriminative hash codes. Extensive experiments on three benchmark datasets demonstrate that the proposed GCH outperforms the state-of-the-art methods.","doi":"10.24963/ijcai.2019/138","url":"https://ijcai.org/proceedings/2019/138","pdf":"https://ijcai.org/proceedings/2019/0138.pdf","markdown":"https://ijcai.org/proceedings/2019/138.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/138","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Computer Vision","track":"Main track","number":138,"pages":{"first":982,"last":988},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martín Abadi, Paul Barham, Jianmin Chen, et al. Tensorflow: A system for large-scale ma- chine learning. In 12th {USENIX} Symposium on Oper- ating Systems Design and Implementation ({OSDI} 16), pages 265-283, 2016.","title":"Tensorflow: A system for large-scale machine learning","authors":"Martín Abadi, Paul Barham, Jianmin Chen","year":"2016","venue":"12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bronstein et al., 2010] Michael M. Bronstein, Alexan- der M. Bronstein, Fabrice Michel, and Nikos Paragios. Data fusion through cross-modality metric learning using similarity-sensitive hashing. In CVPR, pages 3594-3601, 2010.","title":"Data fusion through cross-modality metric learning using similarity-sensitive hashing","authors":"M Michael, Alexander M Bronstein, Fabrice Bronstein, Nikos Michel, Paragios","year":"2010","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bruna et al., 2013] Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and lo- cally connected networks on graphs. arXiv preprint arXiv:1312.6203, 2013.","title":"Spectral networks and locally connected networks on graphs","authors":"Joan Bruna, Wojciech Zaremba, Arthur Szlam, Yann Lecun","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cao et al., 2016] Yue Cao, Mingsheng Long, Jianmin Wang, Qiang Yang, and Philip S. Yu. Deep visual- semantic hashing for cross-modal retrieval. In Proc. ACM SIGKDD Int. Conf. Knowl. Disc. Data Min., pages 1445- 1454, 2016.","title":"Deep visualsemantic hashing for cross-modal retrieval","authors":"Yue Cao, Mingsheng Long, Jianmin Wang, Qiang Yang, Philip S Yu","year":"2016","venue":"Proc. ACM SIGKDD Int. Conf. Knowl. Disc. Data Min","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cao et al., 2018] Yue Cao, Bin Liu, Mingsheng Long, and Jianmin Wang. Cross-modal hamming hashing. In ECCV, pages 202-218, 2018.","title":"Cross-modal hamming hashing","authors":"Yue Cao, Bin Liu, Mingsheng Long, Jianmin Wang","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chatfield et al., 2014] Ken Chatfield, Karen Simonyan, An- drea Vedaldi, and Andrew Zisserman. Return of the devil in the details: Delving deep into convolutional nets. arXiv preprint arXiv:1405.3531, 2014.","title":"Return of the devil in the details: Delving deep into convolutional nets","authors":"Ken Chatfield, Karen Simonyan, Andrea Vedaldi, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chua et al., 2009] Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yan-Tao Zheng. Nus- wide: A real-world web image database from national uni- versity of singapore. In Proc. of ACM Conf. on Image and Video Retrieval (CIVR'09), 2009.","title":"Nuswide: A real-world web image database from national university of singapore","authors":"Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, Yan-Tao Zheng","year":"2009","venue":"Proc. of ACM Conf. on Image and Video Retrieval (CIVR'09)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Deng et al., 2016] Cheng Deng, Xu Tang, Junchi Yan, Wei Liu, and Xinbo Gao. Discriminative dictionary learning with common label alignment for cross-modal retrieval. IEEE Trans. Multimed., 18(2):208-218, 2016.","title":"Discriminative dictionary learning with common label alignment for cross-modal retrieval","authors":"Cheng Deng, Xu Tang, Junchi Yan, Wei Liu, Xinbo Gao","year":"2016","venue":"IEEE Trans. Multimed","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Deng et al., 2018] Cheng Deng, Zhaojia Chen, Xianglong Liu, Xinbo Gao, and Dacheng Tao. Triplet-based deep hashing network for cross-modal retrieval. IEEE Trans. Image Process., 27(8):3893-3903, 2018.","title":"Triplet-based deep hashing network for cross-modal retrieval","authors":"Cheng Deng, Zhaojia Chen, Xianglong Liu, Xinbo Gao, Dacheng Tao","year":"2018","venue":"IEEE Trans. Image Process","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ding et al., 2014] Guiguang Ding, Yuchen Guo, and Jile Zhou. Collective matrix factorization hashing for multi- modal data. In CVPR, pages 2083-2090, 2014.","title":"Collective matrix factorization hashing for multimodal data","authors":"Ding","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Duvenaud et al., 2015] David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Tim- othy Hirzel, Alán Aspuru-Guzik, and Ryan P Adams. Convolutional networks on graphs for learning molecular fingerprints. In NeurIPS, pages 2224-2232, 2015.","title":"Convolutional networks on graphs for learning molecular fingerprints","authors":"Dougal David K Duvenaud, Jorge Maclaurin, Rafael Iparraguirre, Timothy Bombarell, Alán Hirzel, Ryan P Aspuru-Guzik, Adams","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang and Chen, 2017] Feihu Huang and Songcan Chen. Learning dynamic conditional gaussian graphical models. IEEE Transactions on Knowledge and Data Engineering, 30(4):703-716, 2017.","title":"Learning dynamic conditional gaussian graphical models","authors":"Chen ; Huang, Feihu Huang, Songcan Chen","year":"2017","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Huiskes and Lew, 2008] Mark J. Huiskes and Michael S. Lew. The mir flickr retrieval evaluation. In MIR '08: Pro- ceedings of the 2008 ACM International Conference on Multimedia Information Retrieval. ACM, 2008.","title":"The mir flickr retrieval evaluation","authors":"Mark J Huiskes, Michael S Lew","year":"2008","venue":"MIR '08: Proceedings of the 2008 ACM International Conference on Multimedia Information Retrieval","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jiang and Li, 2017] Qing-Yuan Jiang and Wu-Jun Li. Deep cross-modal hashing. In CVPR, pages 3232-3240, 2017.","title":"Deep cross-modal hashing","authors":"Li ; Qing-Yuan Jiang, Wu-Jun Jiang, Li","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.","title":"Semi-supervised classification with graph convolutional networks","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In NeurIPS, pages 1097- 1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2018] Chao Li, Cheng Deng, Ning Li, Wei Liu, Xinbo Gao, and Dacheng Tao. Self-supervised adversar- ial hashing networks for cross-modal retrieval. In CVPR, pages 4242-4251, 2018.","title":"Self-supervised adversarial hashing networks for cross-modal retrieval","authors":"Chao Li, Cheng Deng, Ning Li, Wei Liu, Xinbo Gao, Dacheng Tao","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2019] Chao Li, Cheng Deng, Lei Wang, De Xie, and Xianglong Liu. Coupled cyclegan: Unsupervised hashing network for cross-modal retrieval. arXiv preprint arXiv:1903.02149, 2019.","title":"Coupled cyclegan: Unsupervised hashing network for cross-modal retrieval","authors":"Chao Li, Cheng Deng, Lei Wang, De Xie, Xianglong Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Pi- otr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, pages 740-755. Springer, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lin et al., 2015] Zijia Lin, Guiguang Ding, Mingqing Hu, and Jianmin Wang. Semantics-preserving hashing for cross-view retrieval. In CVPR, 2015.","title":"Semantics-preserving hashing for cross-view retrieval","authors":"Lin","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shen et al., 2015] Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. Supervised discrete hashing. In CVPR, pages 37-45, 2015.","title":"Supervised discrete hashing","authors":"Fumin Shen, Chunhua Shen, Wei Liu, Heng Tao Shen","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2015] Di Wang, Xinbo Gao, Xiumei Wang, and Lihuo He. Semantic topic multimodal hashing for cross- media retrieval. In IJCAI, pages 3890-3896, 2015.","title":"Semantic topic multimodal hashing for crossmedia retrieval","authors":"Di Wang, Xinbo Gao, Xiumei Wang, Lihuo He","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yang et al., 2017] Erkun Yang, Cheng Deng, Wei Liu, Xi- anglong Liu, Dacheng Tao, and Xinbo Gao. Pairwise rela- tionship guided deep hashing for cross-modal retrieval. In AAAI, 2017.","title":"Pairwise relationship guided deep hashing for cross-modal retrieval","authors":"Erkun Yang, Cheng Deng, Wei Liu, Xianglong Liu, Dacheng Tao, Xinbo Gao","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang et al., 2018] Erkun Yang, Cheng Deng, Chao Li, Wei Liu, Jie Li, and Dacheng Tao. Shared predictive cross- modal deep quantization. IEEE Trans. Neural Netw. Learn. Syst., (99):1-12, 2018.","title":"Shared predictive crossmodal deep quantization","authors":"Erkun Yang, Cheng Deng, Chao Li, Wei Liu, Jie Li, Dacheng Tao","year":"2018","venue":"IEEE Trans. Neural Netw. Learn. Syst","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yang et al., 2019] Xu Yang, Cheng Deng, Feng Zheng, Junchi Yan, and Wei Liu. Deep spectral cluster- ing using dual autoencoder network. arXiv preprint arXiv:1904.13113, 2019.","title":"Deep spectral clustering using dual autoencoder network","authors":"Xu Yang, Cheng Deng, Feng Zheng, Junchi Yan, Wei Liu","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang and Li, 2014] Dongqing Zhang and Wu Jun Li. Large-scale supervised multimodal hashing with seman- tic correlation maximization. In AAAI, pages 2177-2183, 2014.","title":"Large-scale supervised multimodal hashing with semantic correlation maximization","authors":"Li ; Zhang, Dongqing Zhang, Wu Jun, Li","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}