{"type":"ScholarlyArticle","title":"Unsupervised Deep Hashing via Binary Latent Factor Models for Large-scale Cross-modal Retrieval","authors":[{"name":"Gengshen Wu","slug":"gengshen-wu","given":"Gengshen","family":"Wu","affiliation":"Lancaster University , Lancaster , LA1 4YW , UK"},{"name":"Zijia Lin","slug":"zijia-lin","given":"Zijia","family":"Lin","affiliation":"Microsoft Research , Beijing , 100080 , China"},{"name":"Jungong Han","slug":"jungong-han","given":"Jungong","family":"Han","affiliation":"Lancaster University , Lancaster , LA1 4YW , UK"},{"name":"Li Liu","slug":"li-liu","given":"Li","family":"Liu","affiliation":"Inception Institute of Artificial Intelligence , Abu Dhabi , UAE"},{"name":"Guiguang Ding","slug":"guiguang-ding","given":"Guiguang","family":"Ding","affiliation":"Tsinghua University , Beijing , 100084 , China"},{"name":"Baochang Zhang","slug":"baochang-zhang","given":"Baochang","family":"Zhang","affiliation":"Beihang University , Beijing , 100083 , China"},{"name":"Jialie Shen","slug":"jialie-shen","given":"Jialie","family":"Shen","affiliation":"Northumbria University , Newcastle Upon Tyne , NE1 8ST , UK"}],"abstract":"Despite its great success, matrix factorization based cross-modality hashing suffers from two problems: 1) there is no engagement between feature learning and binarization; and 2) most existing methods impose the relaxation strategy by discarding the discrete constraints when learning the hash function, which usually yields suboptimal solutions. In this paper, we propose a novel multimodal hashing framework, referred as Unsupervised Deep Cross-Modal Hashing (UDCMH), for multimodal data search in a self-taught manner via integrating deep learning and matrix factorization with binary latent factor models. On one hand, our unsupervised deep learning framework enables the feature learning to be jointly optimized with the binarization. On the other hand, the hashing system based on the binary latent factor models can generate unified binary codes by solving a discrete-constrained objective function directly with no need for a relaxation step. Moreover, novel Laplacian constraints are incorporated into the objective function, which allow to preserve not only the nearest neighbors that are commonly considered in the literature but also the farthest neighbors of data, even if the semantic labels are not available. Extensive experiments on multiple datasets highlight the superiority of the proposed framework over several state-of-the-art baselines.","doi":"10.24963/ijcai.2018/396","url":"https://ijcai.org/proceedings/2018/396","pdf":"https://ijcai.org/proceedings/2018/0396.pdf","markdown":"https://ijcai.org/proceedings/2018/396.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/396","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":396,"pages":{"first":2854,"last":2860},"keywords":["Machine Learning: Deep Learning","Machine Learning: Classification","Machine Learning: Data Mining","Machine Learning: Feature Selection ; Learning Sparse Models"],"videos":[],"references":[{"position":1,"raw":"[Bottou, 2010] Léon Bottou. Large-scale machine learn- ing with stochastic gradient descent. In Proceedings of COMPSTAT'2010, pages 177-186. Springer, 2010.","title":"Large-scale machine learning with stochastic gradient descent","authors":"Léon Bottou, Bottou","year":"2010","venue":"Proceedings of COMPSTAT'2010","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2017] Yue Cao, Mingsheng Long, Jianmin Wang, and Shichen Liu. Collective deep quantization for efficient cross-modal retrieval. In AAAI, pages 3974-3980, 2017.","title":"Collective deep quantization for efficient cross-modal retrieval","authors":"Yue Cao, Mingsheng Long, Jianmin Wang, Shichen Liu","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chua et al., 2009] Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. Nus- wide: a real-world web image database from national uni- versity of singapore. In Proceedings of the ACM interna- tional conference on image and video retrieval, page 48. ACM, 2009.","title":"Nuswide: a real-world web image database from national university of singapore","authors":"Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, Yantao Zheng","year":"2009","venue":"Proceedings of the ACM international conference on image and video retrieval","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ding et al., 2014] Guiguang Ding, Yuchen Guo, and Jile Zhou. Collective matrix factorization hashing for multi- modal data. In CVPR, pages 2075-2082, 2014.","title":"Collective matrix factorization hashing for multimodal data","authors":"Ding","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Huiskes and Lew, 2008] Mark J Huiskes and Michael S Lew. The mir flickr retrieval evaluation. In ACM Multi- media, pages 39-43. ACM, 2008.","title":"The mir flickr retrieval evaluation","authors":"J Mark, Michael S Huiskes, Lew","year":"2008","venue":"ACM Multimedia","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Jia et al., 2014] Yangqing Jia, Evan Shelhamer, Jeff Don- ahue, Sergey Karayev, Jonathan Long, Ross Girshick, Ser- gio Guadarrama, and Trevor Darrell. Caffe: Convolutional architecture for fast feature embedding. In ACM Multime- dia, pages 675-678. ACM, 2014.","title":"Caffe: Convolutional architecture for fast feature embedding","authors":"Jia","year":"2014","venue":"ACM Multimedia","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jiang and Li, 2016] Qing-Yuan Jiang and Wu-Jun Li. Deep cross-modal hashing. arXiv preprint arXiv:1602.02255, 2016.","title":"Deep cross-modal hashing","authors":"Li ; Qing-Yuan Jiang, Wu-Jun Jiang, Li","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural in- formation processing systems, pages 1097-1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kumar and Udupa, 2011] Shaishav Kumar and Raghaven- dra Udupa. Learning hash functions for cross-view sim- ilarity search. In IJCAI, volume 22, page 1360, 2011.","title":"Shaishav Kumar and Raghavendra Udupa. Learning hash functions for cross-view similarity search","authors":"Udupa Kumar","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2017a] Xuelong Li, Di Hu, and Feiping Nie. Deep binary reconstruction for cross-modal hashing. arXiv preprint arXiv:1708.05127, 2017.","title":"Deep binary reconstruction for cross-modal hashing","authors":"Li","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2017b] Xuelong Li, Di Hu, and Feiping Nie. Large graph hashing with spectral rotation. In AAAI, pages 2203- 2209, 2017.","title":"Large graph hashing with spectral rotation","authors":"Li","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lin et al., 2015] Zijia Lin, Guiguang Ding, Mingqing Hu, and Jianmin Wang. Semantics-preserving hashing for cross-view retrieval. In CVPR, pages 3864-3872, 2015.","title":"Semantics-preserving hashing for cross-view retrieval","authors":"Lin","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2011] Wei Liu, Jun Wang, Sanjiv Kumar, and Shih-Fu Chang. Hashing with graphs. In ICML, pages 1-8, 2011.","title":"Hashing with graphs","authors":"Wei Liu, Jun Wang, Sanjiv Kumar, Shih-Fu Chang","year":"2011","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2015] Li Liu, Mengyang Yu, and Ling Shao. Pro- jection bank: From high-dimensional data to medium- length binary codes. In ICCV, pages 2821-2829, 2015.","title":"Projection bank: From high-dimensional data to mediumlength binary codes","authors":"Li Liu, Mengyang Yu, Ling Shao","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2016] Hong Liu, Rongrong Ji, Yongjian Wu, and Gang Hua. Supervised matrix factorization for cross- modality hashing. arXiv preprint arXiv:1603.05572, 2016.","title":"Supervised matrix factorization for crossmodality hashing","authors":"Liu","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2017a] Li Liu, Ling Shao, Fumin Shen, and Mengyang Yu. Discretely coding semantic rank orders for supervised image hashing. In CVPR, pages 5140-5149. IEEE, 2017.","title":"Discretely coding semantic rank orders for supervised image hashing","authors":"Liu","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2017b] Li Liu, Fumin Shen, Yuming Shen, Xi- anglong Liu, and Ling Shao. Deep sketch hashing: Fast free-hand sketch-based image retrieval. In CVPR, pages 2862-2871, 2017.","title":"Deep sketch hashing: Fast free-hand sketch-based image retrieval","authors":"Liu","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2017c] Li Liu, Mengyang Yu, and Ling Shao. La- tent structure preserving hashing. IJCV, 122(3):439-457, 2017.","title":"Latent structure preserving hashing","authors":"Liu","year":"2017","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Long et al., 2017] Yang Long, Li Liu, Ling Shao, Fumin Shen, Guiguang Ding, and Jungong Han. From zero-shot learning to conventional supervised classification: Unseen visual data synthesis. 2017. [Rasiwasia et al., 2010]","title":"From zero-shot learning to conventional supervised classification: Unseen visual data synthesis","authors":"Yang Long, Li Liu, Ling Shao, Fumin Shen, Guiguang Ding, Jungong Han","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"Nikhil Rasiwasia, Jose Costa Pereira, Emanuele Coviello, Gabriel Doyle, Gert RG Lanckriet, Roger Levy, and Nuno Vasconcelos. A new approach to cross-modal multimedia retrieval. In ACM Multimedia, pages 251-260. ACM, 2010.","title":"A new approach to cross-modal multimedia retrieval","authors":"Nikhil Rasiwasia, Jose Costa Pereira, Emanuele Coviello, Gabriel Doyle, Gert Rg Lanckriet, Roger Levy, Nuno Vasconcelos","year":"2010","venue":"ACM Multimedia","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rumelhart et al., 1986] David E Rumelhart, Geoffrey E Hinton, James L McClelland, et al. A general framework for parallel distributed processing. Parallel distributed processing: Explorations in the microstructure of cogni- tion, 1:45-76, 1986.","title":"A general framework for parallel distributed processing","authors":"Geoffrey E David E Rumelhart, James L Hinton, Mcclelland","year":"1986","venue":"Parallel distributed processing: Explorations in the microstructure of cognition","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shen et al., 2015] Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. Supervised discrete hashing. In CVPR, pages 37-45, 2015.","title":"Supervised discrete hashing","authors":"Fumin Shen, Chunhua Shen, Wei Liu, Heng Tao Shen","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Song et al., 2013] Jingkuan Song, Yang Yang, Yi Yang, Zi Huang, and Heng Tao Shen. Inter-media hashing for large-scale retrieval from heterogeneous data sources. In ACM SIGMOD ICMD, pages 785-796. ACM, 2013.","title":"Inter-media hashing for large-scale retrieval from heterogeneous data sources","authors":"Song","year":"2013","venue":"ACM SIGMOD ICMD","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2016] Di Wang, Xinbo Gao, Xiumei Wang, Li- huo He, and Bo Yuan. Multimodal discriminative binary embedding for large-scale cross-modal retrieval. IEEE Transactions on Image Processing, 25(10):4540-4554, 2016.","title":"Multimodal discriminative binary embedding for large-scale cross-modal retrieval","authors":"Di Wang, Xinbo Gao, Xiumei Wang, Lihuo He, Bo Yuan","year":"2016","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2017] Di Wang, Quan Wang, and Xinbo Gao. Robust and flexible discrete hashing for cross-modal sim- ilarity search. IEEE Transactions on Circuits and Systems for Video Technology, 2017.","title":"Robust and flexible discrete hashing for cross-modal similarity search","authors":"Di Wang, Quan Wang, Xinbo Gao","year":"2017","venue":"IEEE Transactions on Circuits and Systems for Video Technology","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wu et al., 2017] Gengshen Wu, Li Liu, Yuchen Guo, Guiguang D-ing, Jungong Han, Jialie Shen, and Ling Shao. Unsupervised deep video hashing with balanced ro- tation. IJCAI, 2017.","title":"Unsupervised deep video hashing with balanced rotation","authors":"Gengshen Wu, Li Liu, Yuchen Guo, D-Ing Guiguang, Jungong Han, Jialie Shen, Ling Shao","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/429"},{"position":27,"raw":"[Zhou et al., 2014] Jile Zhou, Guiguang Ding, and Yuchen Guo. Latent semantic sparse hashing for cross-modal sim- ilarity search. In ACM SIGIR conference on Research and development in information retrieval, pages 415-424. ACM, 2014.","title":"Latent semantic sparse hashing for cross-modal similarity search","authors":"Zhou","year":"2014","venue":"ACM SIGIR conference on Research and development in information retrieval","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhu et al., 2013] Xiaofeng Zhu, Zi Huang, Heng Tao Shen, and Xin Zhao. Linear cross-modal hashing for efficient multimedia search. In ACM Multimedia, pages 143-152. ACM, 2013.","title":"Linear cross-modal hashing for efficient multimedia search","authors":"Xiaofeng Zhu, Zi Huang, Heng Tao Shen, Xin Zhao","year":"2013","venue":"ACM Multimedia","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}