{"type":"ScholarlyArticle","title":"TUCH: Turning Cross-view Hashing into Single-view Hashing via Generative Adversarial Nets","authors":[{"name":"Xin Zhao","slug":"xin-zhao","given":"Xin","family":"Zhao","affiliation":"† Tsinghua National Laboratory for Information Science and Technology (TNList) School of Software , Tsinghua University , Beijing 100084 , China"},{"name":"Guiguang Ding","slug":"guiguang-ding","given":"Guiguang","family":"Ding","affiliation":"† Tsinghua National Laboratory for Information Science and Technology (TNList) School of Software , Tsinghua University , Beijing 100084 , China"},{"name":"Yuchen Guo","slug":"yuchen-guo","given":"Yuchen","family":"Guo","affiliation":"† Tsinghua National Laboratory for Information Science and Technology (TNList) School of Software , Tsinghua University , Beijing 100084 , China"},{"name":"Jungong Han","slug":"jungong-han","given":"Jungong","family":"Han","affiliation":"‡ School of Computing & Communications , Lancaster University , UK"},{"name":"Yue Gao","slug":"yue-gao","given":"Yue","family":"Gao","affiliation":"† Tsinghua National Laboratory for Information Science and Technology (TNList) School of Software , Tsinghua University , Beijing 100084 , China"}],"abstract":"Cross-view retrieval, which focuses on searching images as response to text queries or vice versa, has received increasing attention recently. Cross-view hashing is to efficiently solve the cross-view retrieval problem with binary hash codes. Most existing works on cross-view hashing exploit multi-view embedding method to tackle this problem, which inevitably causes the information loss in both image and text domains. Inspired by the Generative Adversarial Nets (GANs), this paper presents a new model that is able to Turn Cross-view Hashing into single-view hashing (TUCH), thus enabling the information of image to be preserved as much as possible. TUCH is a novel deep architecture that integrates a language model network T for text feature extraction, a generator network G to generate fake images from text feature and a hashing network H for learning hashing functions to generate compact binary codes. Our architecture effectively unifies joint generative adversarial learning and cross-view hashing. Extensive empirical evidence shows that our TUCH approach achieves state-of-the-art results, especially on text to image retrieval, based on image-sentences datasets, i.e. standard IAPRTC-12 and large-scale Microsoft COCO.","doi":"10.24963/ijcai.2017/491","url":"https://ijcai.org/proceedings/2017/491","pdf":"https://ijcai.org/proceedings/2017/0491.pdf","markdown":"https://ijcai.org/proceedings/2017/491.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/491","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":491,"pages":{"first":3511,"last":3517},"keywords":["Machine Learning: Feature Selection/Construction","Natural Language Processing: Information Retrieval","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Bronstein et al., 2010] Michael M. Bronstein, Alexander M. Bron- stein, Fabrice Michel, and Nikos Paragios. Data fusion through cross-modality metric learning using similarity-sensitive hash- ing. In CVPR, 2010.","title":"Data fusion through cross-modality metric learning using similarity-sensitive hashing","authors":"M Michael, Alexander M Bronstein, Fabrice Bronstein, Nikos Michel, Paragios","year":"2010","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2016a] Yue Cao, Mingsheng Long, Jianmin Wang, Qiang Yang, and Philip S. Yu. Deep visual-semantic hashing for cross-modal retrieval. In KDD, 2016.","title":"Deep visual-semantic hashing for cross-modal retrieval","authors":"Cao","year":"2016","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cao et al., 2016b] Yue Cao, Mingsheng Long, Jianmin Wang, Han Zhu, and Qingfu Wen. Deep quantization network for efficient image retrieval. In AAAI, 2016.","title":"Deep quantization network for efficient image retrieval","authors":"Cao","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Denton et al., 2015] Emily L. Denton, Soumith Chintala, Arthur Szlam, and Rob Fergus. Deep generative image models using a laplacian pyramid of adversarial networks. In NIPS, 2015.","title":"Deep generative image models using a laplacian pyramid of adversarial networks","authors":"Emily L Denton, Soumith Chintala, Arthur Szlam, Rob Fergus","year":"2015","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ding et al., 2016] Guiguang Ding, Yuchen Guo, Jile Zhou, and Yue Gao. Large-scale cross-modality search via collective ma- trix factorization hashing. TIP, pages 5427-5440, 2016.","title":"Large-scale cross-modality search via collective matrix factorization hashing","authors":"Ding","year":"2016","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Feng et al., 2014] Fangxiang Feng, Xiaojie Wang, and Ruifan Li. Cross-modal retrieval with correspondence autoencoder. In Mul- timedia, 2014.","title":"Cross-modal retrieval with correspondence autoencoder","authors":"Feng","year":"2014","venue":"Multimedia","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gionis et al., 1999] Aristides Gionis, Piotr Indyk, and Rajeev Mot- wani. Similarity search in high dimensions via hashing. In VLDB, 1999.","title":"Similarity search in high dimensions via hashing","authors":"Gionis","year":"1999","venue":"VLDB","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gong et al., 2013] Yunchao Gong, Svetlana Lazebnik, Albert Gor- do, and Florent Perronnin. Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. PAMI, 2013.","title":"Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval","authors":"Gong","year":"2013","venue":"PAMI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goodfellow et al., 2014] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversari- al nets. In NIPS, 2014.","title":"Generative adversarial nets","authors":"Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C Courville, Yoshua Bengio","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Grubinger et al., 2006] Michael Grubinger, Paul Clough, Henning Müller, and Thomas Deselaers. The iapr tc-12 benchmark: A new evaluation resource for visual information systems. In Inter- national workshop ontoImage, volume 5, 2006.","title":"The iapr tc-12 benchmark: A new evaluation resource for visual information systems","authors":"Michael Grubinger, Paul Clough, Henning Müller, Thomas Deselaers","year":"2006","venue":"International workshop ontoImage","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Guo et al., 2016] Yuchen Guo, Guiguang Ding, Jungong Han, and Xiaoming Jin. Robust iterative quantization for efficient ℓp-norm similarity search. In IJCAI2016, pages 3382-3388, 2016.","title":"Robust iterative quantization for efficient ℓp-norm similarity search","authors":"Yuchen Guo, Guiguang Ding, Jungong Han, Xiaoming Jin","year":"2016","venue":"IJCAI2016","doi":null,"url":null,"internal_url":"/proceedings/2016/478"},{"position":12,"raw":"[He et al., 2013] Kaiming He, Fang Wen, and Jian Sun. K-means hashing: An affinity-preserving quantization method for learning binary compact codes. In CVPR, 2013.","title":"K-means hashing: An affinity-preserving quantization method for learning binary compact codes","authors":"Kaiming He, Fang Wen, Jian Sun","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kong and Li, 2012] Weihao Kong and Wu-Jun Li. Isotropic hash- ing. In NIPS, 2012.","title":"Weihao Kong and Wu-Jun Li. Isotropic hashing","authors":"Li Kong","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kumar and Udupa, 2011] Shaishav Kumar and Raghavendra Udu- pa. Learning hash functions for cross-view similarity search. In IJCAI, 2011.","title":"Shaishav Kumar and Raghavendra Udupa. Learning hash functions for cross-view similarity search","authors":"Udupa Kumar","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lai et al., 2015] Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. Simultaneous feature learning and hash coding with deep neural networks. In CVPR, 2015.","title":"Simultaneous feature learning and hash coding with deep neural networks","authors":"Hanjiang Lai, Yan Pan, Ye Liu, Shuicheng Yan","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: common objects in con- text. In ECCV, 2014.","title":"Microsoft COCO: common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge J Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence Zitnick","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lin et al., 2015] Zijia Lin, Guiguang Ding, Mingqing Hu, and Jianmin Wang. Semantics-preserving hashing for cross-view re- trieval. In CVPR, 2015.","title":"Semantics-preserving hashing for cross-view retrieval","authors":"Lin","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lin et al., 2016] Zijia Lin, Guiguang Ding, Jungong Han, and Jianmin Wang. Cross-view retrieval via probability-based semantics-preserving hashing. Transactions on Cybernetics, 2016.","title":"Cross-view retrieval via probability-based semantics-preserving hashing","authors":"Lin","year":"2016","venue":"Transactions on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu and Lu, 2016] Shicong Liu and Hongtao Lu. Accurate deep representation quantization with gradient snapping layer for sim- ilarity search. CoRR, 2016.","title":"Accurate deep representation quantization with gradient snapping layer for similarity search","authors":"Lu ; Liu, Shicong Liu, Hongtao Lu","year":"2016","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu et al., 2012] Wei Liu, Jun Wang, Rongrong Ji, Yu-Gang Jiang, and Shih-Fu Chang. Supervised hashing with kernels. In CVPR, 2012.","title":"Supervised hashing with kernels","authors":"Wei Liu, Jun Wang, Rongrong Ji, Yu-Gang Jiang, Shih-Fu Chang","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2014] Wei Liu, Cun Mu, Sanjiv Kumar, and Shih-Fu Chang. Discrete graph hashing. In NIPS, 2014.","title":"Discrete graph hashing","authors":"Wei Liu, Cun Mu, Sanjiv Kumar, Shih-Fu Chang","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Masci et al., 2014] Jonathan Masci, Michael M. Bronstein, Alexander M. Bronstein, and Jürgen Schmidhuber. Multimodal similarity-preserving hashing. PAMI, 2014.","title":"Multimodal similarity-preserving hashing","authors":"Jonathan Masci, Michael M Bronstein, Alexander M Bronstein, Jürgen Schmidhuber","year":"2014","venue":"PAMI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Radford et al., 2015] Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep con- volutional generative adversarial networks. CoRR, 2015.","title":"Unsupervised representation learning with deep convolutional generative adversarial networks","authors":"Radford","year":"2015","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Reed et al., 2016] Scott E. Reed, Zeynep Akata, Xinchen Yan, La- janugen Logeswaran, Bernt Schiele, and Honglak Lee. Genera- tive adversarial text to image synthesis. In ICML, 2016.","title":"Generative adversarial text to image synthesis","authors":"Scott E Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, Honglak Lee","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Shen et al., 2015a] Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. Supervised discrete hashing. In CVPR, 2015.","title":"Supervised discrete hashing","authors":"Shen","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Shen et al., 2015b] Fumin Shen, Chunhua Shen, Qinfeng Shi, An- ton van den Hengel, Zhenmin Tang, and Heng Tao Shen. Hashing on nonlinear manifolds. TIP, 2015. [Smeulders et al., 2000]","title":"Hashing on nonlinear manifolds","authors":"Shen","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"Arnold W. M. Smeulders, Marcel Worring, Simone Santini, Amarnath Gupta, and Ramesh C. Jain. Content- based image retrieval at the end of the early years. PAMI, 2000.","title":"Contentbased image retrieval at the end of the early years","authors":"W M Arnold, Marcel Smeulders, Simone Worring, Amarnath Santini, Ramesh C Gupta, Jain","year":"2000","venue":"PAMI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Song et al., 2013] Jingkuan Song, Yang Yang, Yi Yang, Zi Huang, and Heng Tao Shen. Inter-media hashing for large-scale retrieval from heterogeneous data sources. In SIGMOD, 2013.","title":"Inter-media hashing for large-scale retrieval from heterogeneous data sources","authors":"Jingkuan Song, Yang Yang, Yi Yang, Zi Huang, Heng Tao Shen","year":"2013","venue":"SIGMOD","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wang et al., 2016] Jun Wang, Wei Liu, Sanjiv Kumar, and Shih- Fu Chang. Learning to hash for indexing big data -A survey. Proceedings of the IEEE, 2016.","title":"Learning to hash for indexing big data -A survey","authors":"Jun Wang, Wei Liu, Sanjiv Kumar, Shih-Fu Chang","year":"2016","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wei et al., 2014] Ying Wei, Yangqiu Song, Yi Zhen, Bo Liu, and Qiang Yang. Scalable heterogeneous translated hashing. In KDD, 2014.","title":"Scalable heterogeneous translated hashing","authors":"Ying Wei, Yangqiu Song, Yi Zhen, Bo Liu, Qiang Yang","year":"2014","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Weiss et al., 2008] Yair Weiss, Antonio Torralba, and Robert Fer- gus. Spectral hashing. In NIPS, 2008.","title":"Spectral hashing","authors":"Yair Weiss, Antonio Torralba, Robert Fergus","year":"2008","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wu et al., 2015] Botong Wu, Qiang Yang, Wei-Shi Zheng, Yizhou Wang, and Jingdong Wang. Quantized correlation hashing for fast cross-modal search. In IJCAI, 2015.","title":"Quantized correlation hashing for fast cross-modal search","authors":"Botong Wu, Qiang Yang, Wei-Shi Zheng, Yizhou Wang, Jingdong Wang","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/554"},{"position":34,"raw":"[Xia et al., 2014] Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, and Shuicheng Yan. Supervised hashing for image retrieval via image representation learning. In AAAI, 2014.","title":"Supervised hashing for image retrieval via image representation learning","authors":"Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, Shuicheng Yan","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Xu et al., 2013] Bin Xu, Jiajun Bu, Yue Lin, Chun Chen, Xiaofei He, and Deng Cai. Harmonious hashing. In IJCAI, 2013.","title":"Harmonious hashing","authors":"Bin Xu, Jiajun Bu, Yue Lin, Chun Chen, Xiaofei He, Deng Cai","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhang and Li, 2014] Dongqing Zhang and Wu-Jun Li. Large-scale supervised multimodal hashing with semantic correlation maxi- mization. In AAAI, 2014.","title":"Large-scale supervised multimodal hashing with semantic correlation maximization","authors":"Li ; Zhang, Dongqing Zhang, Wu-Jun Li","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhang et al., 2014] Peichao Zhang, Wei Zhang, Wu-Jun Li, and Minyi Guo. Supervised hashing with latent factor models. In SIGIR, 2014.","title":"Supervised hashing with latent factor models","authors":"Peichao Zhang, Wei Zhang, Wu-Jun Li, Minyi Guo","year":"2014","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhen and Yeung, 2012] Yi Zhen and Dit-Yan Yeung. Co- regularized hashing for multimodal data. In NIPS, 2012.","title":"Coregularized hashing for multimodal data","authors":"Yeung ; ] Zhen, Dit-Yan Yi Zhen, Yeung","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhu et al., 2016] Han Zhu, Mingsheng Long, Jianmin Wang, and Yue Cao. Deep hashing network for efficient similarity retrieval. In AAAI, 2016.","title":"Deep hashing network for efficient similarity retrieval","authors":"Han Zhu, Mingsheng Long, Jianmin Wang, Yue Cao","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}