Deep Multiple Instance Hashing for Object-based Image Retrieval

Deep Multiple Instance Hashing for Object-based Image Retrieval

Wanqing Zhao, Ziyu Guan, Hangzai Luo, Jinye Peng, Jianping Fan

Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence
Main track. Pages 3504-3510. https://doi.org/10.24963/ijcai.2017/490

Multi-keyword query is widely supported in text search engines. However, an analogue in image retrieval systems, multi-object query, is rarely studied. Meanwhile, traditional object-based image retrieval methods often involve multiple steps separately and need expensive location labeling for detecting objects. In this work, we propose a weakly-supervised Deep Multiple Instance Hashing (DMIH) framework for object-based image retrieval. DMIH integrates object detection and hashing learning on the basis of a popular CNN model to build the end-to-end relation between a raw image and the binary hashing codes of multiple objects in it. Specifically, we cast the object detection of each object class as a binary multiple instance learning problem where instances are object proposals extracted from multi-scale convolutional feature maps. For hashing training, we sample image pairs to learn their semantic relationships in terms of hash codes of the most probable proposals for owned labels as guided by object predictors. The two objectives benefit each other in learning. DMIH outperforms state-of-the-arts on public benchmarks for object-based image retrieval and achieves promising results for multi-object queries.
Keywords:
Machine Learning: Multi-instance/Multi-label/Multi-view learning
Machine Learning: Deep Learning