{"type":"ScholarlyArticle","title":"Locality Constrained Deep Supervised Hashing for Image Retrieval","authors":[{"name":"Hao Zhu","slug":"hao-zhu","given":"Hao","family":"Zhu","affiliation":"School of Information Science and Technology , ShanghaiTech University"},{"name":"Shenghua Gao","slug":"shenghua-gao","given":"Shenghua","family":"Gao","affiliation":"School of Information Science and Technology , ShanghaiTech University"}],"abstract":"Deep Convolutional Neural Network (DCNN) based deep hashing has shown its success for fast and accurate image retrieval, however directly minimizing the quantization error in deep hashing will change the distribution of DCNN features, and consequently change the similarity between the query and the retrieved images in hashing. In this paper, we propose a novel Locality-Constrained Deep Supervised Hashing. By simultaneously learning discriminative DCNN features and preserving the similarity between image pairs, the hash codes of our scheme preserves the distribution of DCNN features thus favors the accurate image retrieval.The contributions of this paper are two-fold: i) Our analysis shows that minimizing quantization error in deep hashing makes the features less discriminative which is not desirable for image retrieval; ii) We propose a Locality-Constrained Deep Supervised Hashing which preserves the similarity between image pairs in hashing.Extensive experiments on the CIFARA-10 and NUS-WIDE datasets show that our method significantly boosts the accuracy of image retrieval, especially on the CIFAR-10 dataset, the improvement is usually more than 6% in terms of the MAP measurement. Further, our method demonstrates 10 times faster than state-of-the-art methods in the training phase.","doi":"10.24963/ijcai.2017/499","url":"https://ijcai.org/proceedings/2017/499","pdf":"https://ijcai.org/proceedings/2017/0499.pdf","markdown":"https://ijcai.org/proceedings/2017/499.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/499","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning S-Z","track":"Main track","number":499,"pages":{"first":3567,"last":3573},"keywords":["Robotics and Vision: Robotics and Vision","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Andoni and Indyk, 2006] Alexandr Andoni and Piotr Indyk. Near- optimal hashing algorithms for approximate nearest neighbor in high dimensions. In FOCS, pages 459-468. IEEE, 2006.","title":"Nearoptimal hashing algorithms for approximate nearest neighbor in high dimensions","authors":"Indyk Andoni, Alexandr Andoni, Piotr Indyk","year":"2006","venue":"FOCS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chatfield et al., 2014] K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman. Return of the devil in the details: Delving deep into convolutional nets. In BMVC, 2014.","title":"Return of the devil in the details: Delving deep into convolutional nets","authors":"K Chatfield, K Simonyan, A Vedaldi, A Zisserman","year":"2014","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chopra et al., 2005] Sumit Chopra, Raia Hadsell, and Yann Le- Cun. Learning a similarity metric discriminatively, with appli- cation to face verification. In CVPR, volume 1, pages 539-546. IEEE, 2005.","title":"Learning a similarity metric discriminatively, with application to face verification","authors":"Sumit Chopra, Raia Hadsell, Yann Le-Cun","year":"2005","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chua et al., 2009] Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. Nus-wide: a real- world web image database from national university of singapore. In ICIVR, page 48. ACM, 2009.","title":"Nus-wide: a realworld web image database from national university of singapore","authors":"Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, Yantao Zheng","year":"2009","venue":"ICIVR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Erin Liong et al., 2015] Venice Erin Liong, Jiwen Lu, Gang Wang, Pierre Moulin, and Jie Zhou. Deep hashing for compact binary codes learning. In CVPR, pages 2475-2483, 2015.","title":"Deep hashing for compact binary codes learning","authors":"Erin Liong","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gionis et al., 1999] Aristides Gionis, Piotr Indyk, Rajeev Mot- wani, et al. Similarity search in high dimensions via hashing. In VLDB, volume 99, pages 518-529, 1999.","title":"Similarity search in high dimensions via hashing","authors":"Gionis","year":"1999","venue":"VLDB","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gong and Lazebnik, 2011] Yunchao Gong and Svetlana Lazebnik. Iterative quantization: A procrustean approach to learning binary codes. In CVPR, pages 817-824. IEEE, 2011.","title":"Iterative quantization: A procrustean approach to learning binary codes","authors":"Lazebnik Gong, Yunchao Gong, Svetlana Lazebnik","year":"2011","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gong et al., 2013] Yunchao Gong, Svetlana Lazebnik, Albert Gordo, and Florent Perronnin. Iterative quantization: A pro- crustean approach to learning binary codes for large-scale image retrieval. PAMI, 35(12):2916-2929, 2013.","title":"Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval","authors":"Gong","year":"2013","venue":"PAMI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Krizhevsky and Hinton, 2009] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Master's thesis, Department of Computer Science, University of Toronto, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Ge- offrey E Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, pages 1097-1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lai et al., 2015] Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. Simultaneous feature learning and hash coding with deep neural networks. In CVPR, pages 3270-3278, 2015.","title":"Simultaneous feature learning and hash coding with deep neural networks","authors":"Hanjiang Lai, Yan Pan, Ye Liu, Shuicheng Yan","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2016] Wu-Jun Li, Sheng Wang, and Wang-Cheng Kang. Feature learning based deep supervised hashing with pairwise la- bels. AAAI, 2016.","title":"Feature learning based deep supervised hashing with pairwise labels","authors":"Wu-Jun Li, Sheng Wang, Wang-Cheng Kang","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":"/proceedings/2016/245"},{"position":14,"raw":"[Lin et al., 2013] Guosheng Lin, Chunhua Shen, David Suter, and Anton van den Hengel. A general two-step approach to learning- based hashing. In ICCV, pages 2552-2559, 2013.","title":"A general two-step approach to learningbased hashing","authors":"Guosheng Lin, Chunhua Shen, David Suter, Anton Van Den, Hengel","year":"2013","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2014] Guosheng Lin, Chunhua Shen, Qinfeng Shi, An- ton van den Hengel, and David Suter. Fast supervised hashing with decision trees for high-dimensional data. In CVPR, pages 1963-1970, 2014.","title":"Fast supervised hashing with decision trees for high-dimensional data","authors":"Guosheng Lin, Chunhua Shen, Qinfeng Shi, Anton Van Den, David Hengel, Suter","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2012] Wei Liu, Jun Wang, Rongrong Ji, Yu-Gang Jiang, and Shih-Fu Chang. Supervised hashing with kernels. In CVPR, pages 2074-2081. IEEE, 2012.","title":"Supervised hashing with kernels","authors":"Wei Liu, Jun Wang, Rongrong Ji, Yu-Gang Jiang, Shih-Fu Chang","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2016] Haomiao Liu, Ruiping Wang, Shiguang Shan, and Xilin Chen. Deep supervised hashing for fast image retrieval. In CVPR, June 2016.","title":"Deep supervised hashing for fast image retrieval","authors":"Liu","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Norouzi and Blei, 2011] Mohammad Norouzi and David M Blei. Minimal loss hashing for compact binary codes. In ICML, pages 353-360, 2011.","title":"Minimal loss hashing for compact binary codes","authors":"Blei Norouzi, Mohammad Norouzi, David M Blei","year":"2011","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Raginsky and Lazebnik, 2009] Maxim Raginsky and Svetlana Lazebnik. Locality-sensitive binary codes from shift-invariant kernels. In NIPS, pages 1509-1517, 2009.","title":"Maxim Raginsky and Svetlana Lazebnik. Locality-sensitive binary codes from shift-invariant kernels","authors":"Lazebnik Raginsky","year":"2009","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Russakovsky et al., 2015] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Ima- genet large scale visual recognition challenge. IJCV, 115(3):211- 252, 2015.","title":"Imagenet large scale visual recognition challenge","authors":"Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein","year":"2015","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Salakhutdinov and Hinton, 2009] Ruslan Salakhutdinov and Geof- frey Hinton. Semantic hashing. IJAR, 50(7):969-978, 2009.","title":"Ruslan Salakhutdinov and Geoffrey Hinton","authors":"Hinton Salakhutdinov","year":"2009","venue":"IJAR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Schroff et al., 2015] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recog- nition and clustering. In CVPR, pages 815-823, 2015.","title":"Facenet: A unified embedding for face recognition and clustering","authors":"Schroff","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shen et al., 2015] Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. Supervised discrete hashing. In CVPR, pages 37-45, 2015.","title":"Supervised discrete hashing","authors":"Fumin Shen, Chunhua Shen, Wei Liu, Heng Tao Shen","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale im- age recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sun et al., 2014] Yi Sun, Xiaogang Wang, and Xiaoou Tang. Deep learning face representation from predicting 10,000 classes. In CVPR, pages 1891-1898, 2014.","title":"Deep learning face representation from predicting 10,000 classes","authors":"Yi Sun, Xiaogang Wang, Xiaoou Tang","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Torralba et al., 2008] Antonio Torralba, Robert Fergus, and Yair Weiss. Small codes and large image databases for recognition. In CVPR, pages 1-8. IEEE, 2008.","title":"Small codes and large image databases for recognition","authors":"Torralba","year":"2008","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Vedaldi and Lenc, 2015] Andrea Vedaldi and Karel Lenc. Matcon- vnet: Convolutional neural networks for matlab. In ACM MM, pages 689-692. ACM, 2015.","title":"Matconvnet: Convolutional neural networks for matlab","authors":"Andrea Vedaldi, Karel Lenc","year":"2015","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wang et al., 2010] Jun Wang, Sanjiv Kumar, and Shih-Fu Chang. Sequential projection learning for hashing with compact codes. In ICML, pages 1127-1134, 2010.","title":"Sequential projection learning for hashing with compact codes","authors":"Jun Wang, Sanjiv Kumar, Shih-Fu Chang","year":"2010","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wang et al., 2015] Qifan Wang, Zhiwei Zhang, and Luo Si. Rank- ing preserving hashing for fast similarity search. In AAAI, pages 3911-3917, 2015.","title":"Ranking preserving hashing for fast similarity search","authors":"Qifan Wang, Zhiwei Zhang, Luo Si","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":"/proceedings/2015/549"},{"position":30,"raw":"[Weiss et al., 2008] Yair Weiss, Antonio Torralba, and Rob Fergus. Spectral hashing. In NIPS, pages 1753-1760, 2008.","title":"Spectral hashing","authors":"Yair Weiss, Antonio Torralba, Rob Fergus","year":"2008","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xia et al., 2014] Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, and Shuicheng Yan. Supervised hashing for image retrieval via image representation learning. In AAAI, volume 1, page 2, 2014.","title":"Supervised hashing for image retrieval via image representation learning","authors":"Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, Shuicheng Yan","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Yao et al., 2016] Ting Yao, Fuchen Long, Tao Mei, and Yong Rui. Deep semantic-preserving and ranking-based hashing for image retrieval. AAAI, 2016.","title":"Deep semantic-preserving and ranking-based hashing for image retrieval","authors":"Ting Yao, Fuchen Long, Tao Mei, Yong Rui","year":"2016","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2016/553"},{"position":33,"raw":"[Zeiler and Fergus, 2014] Matthew D Zeiler and Rob Fergus. Vi- sualizing and understanding convolutional networks. In ECCV, pages 818-833. Springer, 2014.","title":"Visualizing and understanding convolutional networks","authors":"D Matthew, Rob Zeiler, Fergus","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zhang et al., 2014] Peichao Zhang, Wei Zhang, Wu-Jun Li, and Minyi Guo. Supervised hashing with latent factor models. In SIGIR, pages 173-182. ACM, 2014.","title":"Supervised hashing with latent factor models","authors":"Zhang","year":"2014","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Zhu et al., 2016] Han Zhu, Mingsheng Long, Jianmin Wang, and Yue Cao. Deep hashing network for efficient similarity retrieval. In AAAI, 2016.","title":"Deep hashing network for efficient similarity retrieval","authors":"Han Zhu, Mingsheng Long, Jianmin Wang, Yue Cao","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhuang et al., 2016] B. Zhuang, G. Lin, C. Shen, and I. Reid. Fast training of triplet-based deep binary embedding networks. In CVPR, 2016.","title":"Fast training of triplet-based deep binary embedding networks","authors":"B Zhuang, G Lin, C Shen, I Reid","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}