{"type":"ScholarlyArticle","title":"Beyond Product Quantization: Deep Progressive Quantization for Image Retrieval","authors":[{"name":"Lianli Gao","slug":"lianli-gao","given":"Lianli","family":"Gao","affiliation":"Center for Future Media , University of Electronic Science and Technology of China"},{"name":"Xiaosu Zhu","slug":"xiaosu-zhu","given":"Xiaosu","family":"Zhu","affiliation":"Center for Future Media , University of Electronic Science and Technology of China"},{"name":"Jingkuan Song","slug":"jingkuan-song","given":"Jingkuan","family":"Song","affiliation":"Center for Future Media , University of Electronic Science and Technology of China"},{"name":"Zhou Zhao","slug":"zhou-zhao","given":"Zhou","family":"Zhao","affiliation":"Zhejiang University"},{"name":"Heng Tao Shen","slug":"heng-tao-shen","given":"Heng Tao","family":"Shen","affiliation":"Center for Future Media , University of Electronic Science and Technology of China"}],"abstract":"Product Quantization (PQ) has long been a mainstream for generating an exponentially large codebook at very low memory/time cost. Despite its success, PQ is still tricky for the decomposition of high-dimensional vector space, and the retraining of model is usually unavoidable when the code length changes. In this work, we propose a deep progressive quantization (DPQ) model, as an alternative to PQ, for large scale image retrieval. DPQ learns the quantization codes sequentially and approximates the original feature space progressively. Therefore, we can train the quantization codes with different code lengths simultaneously. Specifically, we first utilize the label information for guiding the learning of visual features, and then apply several quantization blocks to progressively approach the visual features. Each quantization block is designed to be a layer of a convolutional neural network, and the whole framework can be trained in an end-to-end manner. Experimental results on the benchmark datasets show that our model significantly outperforms the state-of-the-art for image retrieval. Our model is trained once for different code lengths and therefore requires less computation time. Additional ablation\r\nstudy demonstrates the effect of each component of our proposed model. Our code is released at https://github.com/cfm-uestc/DPQ.","doi":"10.24963/ijcai.2019/102","url":"https://ijcai.org/proceedings/2019/102","pdf":"https://ijcai.org/proceedings/2019/0102.pdf","markdown":"https://ijcai.org/proceedings/2019/102.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/102","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Computer Vision","track":"Main track","number":102,"pages":{"first":723,"last":729},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Arandjelovic et al., 2016] Relja Arandjelovic, Petr Gronat, Akihiko Torii, Tomas Pajdla, and Josef Sivic. Netvlad: Cnn architecture for weakly supervised place recognition. In CVPR, pages 5297-5307, 2016.","title":"Netvlad: Cnn architecture for weakly supervised place recognition","authors":"Relja Arandjelovic, Petr Gronat, Akihiko Torii, Tomas Pajdla, Josef Sivic","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cao et al., 2016] Yue Cao, Mingsheng Long, Jianmin Wang, Han Zhu, and Qingfu Wen. Deep quantization net- work for efficient image retrieval. In AAAI, pages 3457- 3463, 2016.","title":"Deep quantization network for efficient image retrieval","authors":"Yue Cao, Mingsheng Long, Jianmin Wang, Han Zhu, Qingfu Wen","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cao et al., 2017] Yue Cao, Mingsheng Long, Jianmin Wang, and Shichen Liu. Deep visual-semantic quanti- zation for efficient image retrieval. In CVPR, volume 2, 2017.","title":"Deep visual-semantic quantization for efficient image retrieval","authors":"Yue Cao, Mingsheng Long, Jianmin Wang, Shichen Liu","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chua et al., 2009] Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. Nus- wide: a real-world web image database from national uni- versity of singapore. In CIVR, page 48, 2009.","title":"Nuswide: a real-world web image database from national university of singapore","authors":"Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, Yantao Zheng","year":"2009","venue":"CIVR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, pages 248-255, 2009.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ge et al., 2013] Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. Optimized product quantization for approxi- mate nearest neighbor search. In CVPR, pages 2946-2953, 2013.","title":"Optimized product quantization for approximate nearest neighbor search","authors":"Ge","year":"2013","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gersho and Gray, 2012] Allen Gersho and Robert M Gray. Vector quantization and signal compression, volume 159. Springer Science & Business Media, 2012.","title":"Vector quantization and signal compression","authors":"Allen Gersho, Robert M Gray","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gionis et al., 1999] Aristides Gionis, Piotr Indyk, Rajeev Motwani, et al. Similarity search in high dimensions via hashing. In VLDB, volume 99, pages 518-529, 1999.","title":"Similarity search in high dimensions via hashing","authors":"Gionis","year":"1999","venue":"VLDB","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gong et al., 2013] Yunchao Gong, Svetlana Lazebnik, Al- bert Gordo, and Florent Perronnin. Iterative quantization: A procrustean approach to learning binary codes for large- scale image retrieval. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 35(12):2916-2929, 2013.","title":"Iterative quantization: A procrustean approach to learning binary codes for largescale image retrieval","authors":"Gong","year":"2013","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jegou et al., 2011] Herve Jegou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neigh- bor search. IEEE transactions on pattern analysis and ma- chine intelligence, 33(1):117-128, 2011.","title":"Product quantization for nearest neighbor search","authors":"Jegou","year":"2011","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Klein and Wolf, 2017] Benjamin Klein and Lior Wolf. In defense of product quantization. arXiv preprint arXiv:1711.08589, 2017.","title":"In defense of product quantization","authors":"Benjamin Klein, Lior Wolf","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Krizhevsky and Hinton, 2009] Alex Krizhevsky and Geof- frey Hinton. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky, Geoffrey Hinton","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Krizhevsky et al., 2012] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, pages 1097- 1105, 2012.","title":"Imagenet classification with deep convolutional neural networks","authors":"Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton","year":"2012","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kulis and Darrell, 2009] Brian Kulis and Trevor Darrell. Learning to hash with binary reconstructive embeddings. In NIPS, pages 1042-1050, 2009.","title":"Learning to hash with binary reconstructive embeddings","authors":"Brian Kulis, Trevor Darrell","year":"2009","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lai et al., 2015] Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. Simultaneous feature learning and hash coding with deep neural networks. In CVPR, pages 3270- 3278, 2015.","title":"Simultaneous feature learning and hash coding with deep neural networks","authors":"Lai","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lew et al., 2006] Michael S Lew, Nicu Sebe, Chabane Djer- aba, and Ramesh Jain. Content-based multimedia informa- tion retrieval: State of the art and challenges. ACM Trans- actions on Multimedia Computing, Communications, and Applications (TOMM), 2(1):1-19, 2006.","title":"Content-based multimedia information retrieval: State of the art and challenges","authors":"Nicu Michael S Lew, Chabane Sebe, Ramesh Djeraba, Jain","year":"2006","venue":"ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2012] Wei Liu, Jun Wang, Rongrong Ji, Yu-Gang Jiang, and Shih-Fu Chang. Supervised hashing with ker- nels. In CVPR, pages 2074-2081, 2012.","title":"Supervised hashing with kernels","authors":"Wei Liu, Jun Wang, Rongrong Ji, Yu-Gang Jiang, Shih-Fu Chang","year":"2012","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2016] Haomiao Liu, Ruiping Wang, Shiguang Shan, and Xilin Chen. Deep supervised hashing for fast image retrieval. In CVPR, pages 2064-2072, 2016.","title":"Deep supervised hashing for fast image retrieval","authors":"Liu","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Martinez et al., 2014] Julieta Martinez, Holger H Hoos, and James J Little. Stacked quantizers for compositional vector compression. arXiv preprint arXiv:1411.2173, 2014.","title":"Stacked quantizers for compositional vector compression","authors":"Julieta Martinez, James J Holger H Hoos, Little","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shakhnarovich et al., 2008] Gregory Shakhnarovich, Trevor Darrell, and Piotr Indyk. Nearest-neighbor meth- ods in learning and vision. IEEE Transactions on Neural Networks, 19(2):377, 2008.","title":"Nearest-neighbor methods in learning and vision","authors":"Gregory Shakhnarovich, Trevor Darrell, Piotr Indyk","year":"2008","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shen et al., 2015] Fumin Shen, Chunhua Shen, Wei Liu, and Heng Tao Shen. Supervised discrete hashing. In CVPR, pages 37-45, 2015.","title":"Supervised discrete hashing","authors":"Fumin Shen, Chunhua Shen, Wei Liu, Heng Tao Shen","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Song et al., 2018a] Jingkuan Song, Lianli Gao, Li Liu, Xi- aofeng Zhu, and Nicu Sebe. Quantization-based hash- ing: a general framework for scalable image and video retrieval. Pattern Recognition, 75:175 -187, 2018.","title":"Quantization-based hashing: a general framework for scalable image and video retrieval","authors":"Song","year":"2018","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Song et al., 2018b] Jingkuan Song, Tao He, Lianli Gao, Xing Xu, Alan Hanjalic, and Heng Tao Shen. Binary gen- erative adversarial networks for image retrieval. In AAAI, pages 394-401, 2018.","title":"Binary generative adversarial networks for image retrieval","authors":"Song","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Song et al., 2018c] Jingkuan Song, Hanwang Zhang, Xi- angpeng Li, Lianli Gao, Meng Wang, and Richang Hong. Self-supervised video hashing with hierarchical binary auto-encoder. IEEE Transactions on Image Processing, 27(7):3210-3221, 2018.","title":"Self-supervised video hashing with hierarchical binary auto-encoder","authors":"Song","year":"2018","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wang et al., 2018] Jingdong Wang, Ting Zhang, Nicu Sebe, Heng Tao Shen, et al. A survey on learning to hash. IEEE transactions on pattern analysis and machine intelligence, 40(4):769-790, 2018.","title":"A survey on learning to hash","authors":"Jingdong Wang, Ting Zhang, Nicu Sebe, Heng Tao Shen","year":"2018","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xia et al., 2014] Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, and Shuicheng Yan. Supervised hashing for image retrieval via image representation learning. In AAAI, vol- ume 1, page 2, 2014.","title":"Supervised hashing for image retrieval via image representation learning","authors":"Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, Shuicheng Yan","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2015] Ting Zhang, Guo-Jun Qi, Jinhui Tang, and Jingdong Wang. Sparse composite quantization. In CVPR, pages 4548-4556, 2015.","title":"Sparse composite quantization","authors":"Ting Zhang, Guo-Jun Qi, Jinhui Tang, Jingdong Wang","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhu et al., 2016] Han Zhu, Mingsheng Long, Jianmin Wang, and Yue Cao. Deep hashing network for efficient similarity retrieval. In AAAI, pages 2415-2421, 2016.","title":"Deep hashing network for efficient similarity retrieval","authors":"Han Zhu, Mingsheng Long, Jianmin Wang, Yue Cao","year":"2016","venue":"AAAI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}