{"type":"ScholarlyArticle","title":"Deep Recurrent Quantization for Generating Sequential Binary Codes","authors":[{"name":"Jingkuan Song","slug":"jingkuan-song","given":"Jingkuan","family":"Song","affiliation":"Center for Future Media , University of Electronic Science and Technology of China"},{"name":"Xiaosu Zhu","slug":"xiaosu-zhu","given":"Xiaosu","family":"Zhu","affiliation":"Center for Future Media , University of Electronic Science and Technology of China"},{"name":"Lianli Gao","slug":"lianli-gao","given":"Lianli","family":"Gao","affiliation":"Center for Future Media , University of Electronic Science and Technology of China"},{"name":"Xin-Shun Xu","slug":"xin-shun-xu","given":"Xin-Shun","family":"Xu","affiliation":"Shandong University"},{"name":"Wu Liu","slug":"wu-liu","given":"Wu","family":"Liu","affiliation":"JD AI Research"},{"name":"Heng Tao Shen","slug":"heng-tao-shen","given":"Heng Tao","family":"Shen","affiliation":"Center for Future Media , University of Electronic Science and Technology of China"}],"abstract":"Quantization has been an effective technology in ANN (approximate nearest neighbour) search due to its high accuracy and fast search speed. To meet the requirement of different applications, there is always a trade-off between retrieval accuracy and\r\nspeed, reflected by variable code lengths. However, to encode the dataset into different code lengths, existing methods need to train several models, where each model can only produce a specific code length. This incurs a considerable training time cost, and largely reduces the flexibility of quantization methods to be deployed in real applications. To address this issue, we propose a Deep Recurrent Quantization (DRQ) architecture which can generate sequential binary codes. To the end, when the\r\nmodel is trained, a sequence of binary codes can be generated and the code length can be easily controlled by adjusting the number of recurrent iterations. A shared codebook and a scalar factor is designed to be the learnable weights in the deep recurrent quantization block, and the whole framework can be trained in an end-to-end manner. As far as we know, this is the first quantization method that can be trained once and generate sequential binary codes. Experimental results on the benchmark datasets show that our model achieves comparable or even better performance compared with the state-of-the-art for image retrieval. But it requires significantly less number of parameters and training times. Our code is published online: https://github.com/cfm-uestc/DRQ.","doi":"10.24963/ijcai.2019/128","url":"https://ijcai.org/proceedings/2019/128","pdf":"https://ijcai.org/proceedings/2019/0128.pdf","markdown":"https://ijcai.org/proceedings/2019/128.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/128","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Computer Vision","track":"Main track","number":128,"pages":{"first":912,"last":918},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Cao et al., 2016] Yue Cao, Mingsheng Long, Jianmin Wang, Han Zhu, and Qingfu Wen. Deep quantization net- work for efficient image retrieval. In AAAI, pages 3457- 3463, 2016.","title":"Deep quantization network for efficient image retrieval","authors":"Yue Cao, Mingsheng Long, Jianmin Wang, Han Zhu, Qingfu Wen","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2017] Yue Cao, Mingsheng Long, Jianmin Wang, and Shichen Liu. Deep visual-semantic quanti- zation for efficient image retrieval. In CVPR, volume 2, 2017.","title":"Deep visual-semantic quantization for efficient image retrieval","authors":"Yue Cao, Mingsheng Long, Jianmin Wang, Shichen Liu","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ge et al., 2013] Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. Optimized product quantization for approxi- mate nearest neighbor search. In CVPR, pages 2946-2953, 2013.","title":"Optimized product quantization for approximate nearest neighbor search","authors":"Ge","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gong et al., 2013] Yunchao Gong, Svetlana Lazebnik, Al- bert Gordo, and Florent Perronnin. Iterative quantization: A procrustean approach to learning binary codes for large- scale image retrieval. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 35(12):2916-2929, 2013.","title":"Iterative quantization: A procrustean approach to learning binary codes for largescale image retrieval","authors":"Gong","year":"2013","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Jegou et al., 2011] Herve Jegou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neigh- bor search. IEEE transactions on pattern analysis and ma- chine intelligence, 33(1):117-128, 2011.","title":"Product quantization for nearest neighbor search","authors":"Jegou","year":"2011","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Kulis and Darrell, 2009] Brian Kulis and Trevor Darrell. Learning to hash with binary reconstructive embeddings. In NIPS, pages 1042-1050, 2009.","title":"Learning to hash with binary reconstructive embeddings","authors":"Brian Kulis, Trevor Darrell","year":"2009","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Lai et al., 2015a] Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. Simultaneous feature learning and hash coding with deep neural networks. In CVPR, pages 3270- 3278, 2015.","title":"Simultaneous feature learning and hash coding with deep neural networks","authors":"Lai","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lai et al., 2015b] Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. Simultaneous feature learning and hash coding with deep neural networks. In CVPR, pages 3270- 3278, 2015.","title":"Simultaneous feature learning and hash coding with deep neural networks","authors":"Lai","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2015] Wu-Jun Li, Sheng Wang, and Wang-Cheng Kang. Feature learning based deep supervised hashing with pairwise labels. arXiv preprint arXiv:1511.03855, 2015.","title":"Feature learning based deep supervised hashing with pairwise labels","authors":"Wu-Jun Li, Sheng Wang, Wang-Cheng Kang","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Li et al., 2017] Qi Li, Zhenan Sun, Ran He, and Tieniu Tan. Deep supervised discrete hashing. In NIPS, pages 2482- 2491, 2017.","title":"Deep supervised discrete hashing","authors":"Qi Li, Zhenan Sun, Ran He, Tieniu Tan","year":"2017","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lin et al., 2014] Guosheng Lin, Chunhua Shen, Qinfeng Shi, Anton Van den Hengel, and David Suter. Fast su- pervised hashing with decision trees for high-dimensional data. In CVPR, pages 1963-1970, 2014.","title":"Fast supervised hashing with decision trees for high-dimensional data","authors":"Guosheng Lin, Chunhua Shen, Qinfeng Shi, Anton Van Den, David Hengel, Suter","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2012] Wei Liu, Jun Wang, Rongrong Ji, Yu-Gang Jiang, and Shih-Fu Chang. Supervised hashing with ker- nels. In CVPR, pages 2074-2081, 2012.","title":"Supervised hashing with kernels","authors":"Wei Liu, Jun Wang, Rongrong Ji, Yu-Gang Jiang, Shih-Fu Chang","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu et al., 2016] Haomiao Liu, Ruiping Wang, Shiguang Shan, and Xilin Chen. Deep supervised hashing for fast image retrieval. In CVPR, pages 2064-2072, 2016.","title":"Deep supervised hashing for fast image retrieval","authors":"Liu","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2018] Bin Liu, Yue Cao, Mingsheng Long, Jian- min Wang, and Jingdong Wang. Deep triplet quantization. In ACM MM, pages 755-763. ACM, 2018.","title":"Deep triplet quantization","authors":"Bin Liu, Yue Cao, Mingsheng Long, Jianmin Wang, Jingdong Wang","year":"2018","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Martinez et al., 2014] Julieta Martinez, Holger H Hoos, and James J Little. Stacked quantizers for compositional vector compression. arXiv preprint arXiv:1411.2173, 2014.","title":"Stacked quantizers for compositional vector compression","authors":"Julieta Martinez, James J Holger H Hoos, Little","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Salakhutdinov and Hinton, 2009] Ruslan Salakhutdinov and Geoffrey Hinton. Semantic hashing. International Journal of Approximate Reasoning, 50(7):969-978, 2009.","title":"Ruslan Salakhutdinov and Geoffrey Hinton","authors":"Hinton Salakhutdinov","year":"2009","venue":"International Journal of Approximate Reasoning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shen et al., 2015] Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. Supervised discrete hashing. In CVPR, pages 37-45, 2015.","title":"Supervised discrete hashing","authors":"Fumin Shen, Chunhua Shen, Wei Liu, Heng Tao Shen","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Song et al., 2018a] Jingkuan Song, Lianli Gao, Li Liu, Xi- aofeng Zhu, and Nicu Sebe. Quantization-based hash- ing: a general framework for scalable image and video retrieval. Pattern Recognition, 75:175 -187, 2018.","title":"Quantization-based hashing: a general framework for scalable image and video retrieval","authors":"Song","year":"2018","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Song et al., 2018b] Jingkuan Song, Tao He, Lianli Gao, Xing Xu, Alan Hanjalic, and Heng Tao Shen. Binary gen- erative adversarial networks for image retrieval. In AAAI, pages 394-401, 2018.","title":"Binary generative adversarial networks for image retrieval","authors":"Song","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Song et al., 2018c] Jingkuan Song, Hanwang Zhang, Xi- angpeng Li, Lianli Gao, Meng Wang, and Richang Hong. Self-supervised video hashing with hierarchical binary auto-encoder. IEEE Transactions on Image Processing, 27(7):3210-3221, 2018.","title":"Self-supervised video hashing with hierarchical binary auto-encoder","authors":"Song","year":"2018","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Wang et al., 2016] Xiaofang Wang, Yi Shi, and Kris M Ki- tani. Deep supervised hashing with triplet labels. In ACCV, pages 70-84. Springer, 2016.","title":"Deep supervised hashing with triplet labels","authors":"Xiaofang Wang, Yi Shi, Kris M Kitani","year":"2016","venue":"ACCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Xia et al., 2014] Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, and Shuicheng Yan. Supervised hashing for image re- trieval via image representation learning. In AAAI, page 2, 2014.","title":"Supervised hashing for image retrieval via image representation learning","authors":"Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, Shuicheng Yan","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yu et al., 2018] Tan Yu, Junsong Yuan, Chen Fang, and Hailin Jin. Product quantization network for fast image retrieval. In ECCV, pages 191-206. Springer, 2018.","title":"Product quantization network for fast image retrieval","authors":"Tan Yu, Junsong Yuan, Chen Fang, Hailin Jin","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang et al., 2014] Peichao Zhang, Wei Zhang, Wu-Jun Li, and Minyi Guo. Supervised hashing with latent factor models. In ACM SIGIR, pages 173-182. ACM, 2014.","title":"Supervised hashing with latent factor models","authors":"Peichao Zhang, Wei Zhang, Wu-Jun Li, Minyi Guo","year":"2014","venue":"ACM SIGIR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhang et al., 2015] Ruimao Zhang, Liang Lin, Rui Zhang, Wangmeng Zuo, and Lei Zhang. Bit-scalable deep hash- ing with regularized similarity learning for image retrieval and person re-identification. IEEE Transactions on Image Processing, 24(12):4766-4779, 2015.","title":"Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification","authors":"Ruimao Zhang, Liang Lin, Rui Zhang, Wangmeng Zuo, Lei Zhang","year":"2015","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang et al., 2016] Ziming Zhang, Yuting Chen, and Venkatesh Saligrama. Efficient training of very deep neu- ral networks for supervised hashing. In CVPR, pages 1487-1495, 2016.","title":"Efficient training of very deep neural networks for supervised hashing","authors":"Ziming Zhang, Yuting Chen, Venkatesh Saligrama","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhao et al., 2015] Fang Zhao, Yongzhen Huang, Liang Wang, and Tieniu Tan. Deep semantic ranking based hash- ing for multi-label image retrieval. In CVPR, pages 1556- 1564, 2015.","title":"Deep semantic ranking based hashing for multi-label image retrieval","authors":"Fang Zhao, Yongzhen Huang, Liang Wang, Tieniu Tan","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhu et al., 2016] Han Zhu, Mingsheng Long, Jianmin Wang, and Yue Cao. Deep hashing network for efficient similarity retrieval. In AAAI, pages 2415-2421, 2016.","title":"Deep hashing network for efficient similarity retrieval","authors":"Han Zhu, Mingsheng Long, Jianmin Wang, Yue Cao","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}