{"type":"ScholarlyArticle","title":"Extensible Cross-Modal Hashing","authors":[{"name":"Tian-yi Chen","slug":"tian-yi-chen","given":"Tian-yi","family":"Chen","affiliation":"School of Computer Science and Technology , University of Science and Technology of China , China"},{"name":"Lan Zhang","slug":"lan-zhang","given":"Lan","family":"Zhang","affiliation":"School of Computer Science and Technology , University of Science and Technology of China , China; School of Data Science , University of Science and Technology of China , China"},{"name":"Shi-cong Zhang","slug":"shi-cong-zhang","given":"Shi-cong","family":"Zhang","affiliation":"School of Computer Science and Technology , University of Science and Technology of China , China"},{"name":"Zi-long Li","slug":"zi-long-li","given":"Zi-long","family":"Li","affiliation":"School of Information Science and Engineering , Northeastern University , China"},{"name":"Bai-chuan Huang","slug":"bai-chuan-huang","given":"Bai-chuan","family":"Huang","affiliation":"Department of Physics , University of California Berkeley , USA"}],"abstract":"Cross-modal hashing (CMH) models are introduced to significantly reduce the cost of large-scale cross-modal data retrieval systems. In many real-world applications, however, data of new categories arrive continuously, which requires the model has good extensibility. That is the model should be updated to accommodate data of new categories but still retain good performance for the old categories with minimum computation cost. Unfortunately, existing CMH methods fail to satisfy the extensibility requirements. In this work, we propose a novel extensible cross-modal hashing (ECMH) to enable highly efficient and low-cost model extension. Our proposed ECMH has several desired features: 1) it has good forward compatibility, so there is no need to update old hash codes; 2) the ECMH model is extended to support new data categories using only new data by a well-designed ``weak constraint incremental learning'' algorithm, which saves up to 91\\% time cost comparing with retraining the model with both new and old data; 3) the extended model achieves high precision and recall on both old and new tasks. Our extensive experiments show the effectiveness of our design.","doi":"10.24963/ijcai.2019/292","url":"https://ijcai.org/proceedings/2019/292","pdf":"https://ijcai.org/proceedings/2019/0292.pdf","markdown":"https://ijcai.org/proceedings/2019/292.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/292","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":292,"pages":{"first":2109,"last":2115},"keywords":["Machine Learning Applications: Big data ; Scalability","Machine Learning: Classification","Machine Learning: Transfer, Adaptation, Multi-task Learning","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Baltrusaitis et al., 2018] Tadas Baltrusaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal machine learning: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41:423-443, 2018.","title":"Multimodal machine learning: A survey and taxonomy","authors":"Tadas Baltrusaitis, Chaitanya Ahuja, Louis-Philippe Morency","year":"2018","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bottou, 2010] Léon Bottou. Large-scale machine learn- ing with stochastic gradient descent. In Yves Lecheval- lier and Gilbert Saporta, editors, Proceedings of COMP- STAT'2010, pages 177-186, Heidelberg, 2010. Physica- Verlag HD. Bronstein et al., 2010]","title":"Large-scale machine learning with stochastic gradient descent","authors":"Léon Bottou, Bottou","year":"2010","venue":"Proceedings of COMP-STAT'2010","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"Michael M. Bronstein, Alexan- der M. Bronstein, Fabrice Michel, and Nikos Paragios. Data fusion through cross-modality metric learning us- ing similarity-sensitive hashing. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 3594-3601, 2010.","title":"Data fusion through cross-modality metric learning using similarity-sensitive hashing","authors":"M Michael, Alexander M Bronstein, Fabrice Bronstein, Nikos Michel, Paragios","year":"2010","venue":"IEEE Computer Society Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2018] Zhen-Duo Chen, Wan-Jin Yu, Chuan- Xiang Li, Liqiang Nie, and Xin-Shun Xu. Dual deep neu- ral networks cross-modal hashing. In AAAI Conference on Artificial Intelligence, pages 274-281, 2018.","title":"Dual deep neural networks cross-modal hashing","authors":"Zhen-Duo Chen, Wan-Jin Yu, Chuan-Xiang Li, Liqiang Nie, Xin-Shun Xu","year":"2018","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cortes and Vapnik, 1995] Corinna Cortes and Vladimir Vapnik. Support-vector networks. Mach. Learn., 20(3):273-297, September 1995.","title":"Support-vector networks","authors":"Corinna Cortes, Vladimir Vapnik","year":"1995","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Guo et al., 2010] Gongde Guo, J Huang, and Lifei Chen. Knn model based incremental learning algorithm. Pattern Recognition and Artificial Intell., 23:701-707, 10 2010.","title":"Knn model based incremental learning algorithm","authors":"Guo","year":"2010","venue":"Pattern Recognition and Artificial Intell","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hinton et al., 2015] Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural net- work. In arXiv: Machine Learning, 2015.","title":"Distilling the knowledge in a neural network","authors":"Geoffrey Hinton, Oriol Vinyals, Jeffrey Dean","year":"2015","venue":"arXiv: Machine Learning","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Huiskes and Lew, 2008] Mark J Huiskes and Michael S Lew. The mir flickr retrieval evaluation. In Proceedings of the 1st ACM international conference on Multimedia in- formation retrieval, pages 39-43. ACM, 2008.","title":"The mir flickr retrieval evaluation","authors":"J Mark, Michael S Huiskes, Lew","year":"2008","venue":"Proceedings of the 1st ACM international conference on Multimedia information retrieval","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Iyyer et al., 2015] Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. Deep unordered com- position rivals syntactic methods for text classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Vol- ume 1: Long Papers), pages 1681-1691. Association for Computational Linguistics, July 2015.","title":"Deep unordered composition rivals syntactic methods for text classification","authors":"Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, Hal Daumé, Iii","year":"2015","venue":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jiang and Li, 2017] Q. Jiang and W. Li. Deep cross-modal hashing. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3270-3278, July 2017.","title":"Deep cross-modal hashing","authors":"Li ; Jiang, ] Q Jiang, W Li","year":"2017","venue":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Joshi and Kulkarni, 2012] Prachi Joshi and Parag Kulkarni. Incremental learning: Areas and methods-a survey. vol- ume 2, page 43. Academy & Industry Research Collabo- ration Center (AIRCC), 2012.","title":"Incremental learning: Areas and methods-a survey","authors":"Prachi Joshi, Parag Kulkarni","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Li and Hoiem, 2018] Zhizhong Li and Derek Hoiem. Learning without forgetting. pages 614-629, 2018.","title":"Learning without forgetting","authors":"Hoiem ; Li, Zhizhong Li, Derek Hoiem","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Li et al., 2018] Chao Li, Cheng Deng, Ning Li, Wei Liu, Xinbo Gao, and Dacheng Tao. Self-supervised adversar- ial hashing networks for cross-modal retrieval. IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4242-4251, 2018.","title":"Self-supervised adversarial hashing networks for cross-modal retrieval","authors":"Chao Li, Cheng Deng, Ning Li, Wei Liu, Xinbo Gao, Dacheng Tao","year":"2018","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vi- sion -ECCV 2014, pages 740-755, Cham, 2014. Springer International Publishing.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence Zitnick","year":"2014","venue":"Computer Vision -ECCV 2014","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2015] K. Lin, H. Yang, J. Hsiao, and C. Chen. Deep learning of binary hash codes for fast image retrieval. In 2015 IEEE Conference on Computer Vision and Pat- tern Recognition Workshops (CVPRW), pages 27-35, June 2015.","title":"Deep learning of binary hash codes for fast image retrieval","authors":"K Lin, H Yang, J Hsiao, C Chen","year":"2015","venue":"IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Razavian et al., 2014] A. S. Razavian, H. Azizpour, J. Sul- livan, and S. Carlsson. Cnn features off-the-shelf: An as- tounding baseline for recognition. In 2014 IEEE Confer- ence on Computer Vision and Pattern Recognition Work- shops, pages 512-519, June 2014.","title":"Cnn features off-the-shelf: An astounding baseline for recognition","authors":"A S Razavian, H Azizpour, J Sullivan, S Carlsson","year":"2014","venue":"IEEE Conference on Computer Vision and Pattern Recognition Workshops","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shen et al., 2015] Fumin Shen, Chunhua Shen, Qinfeng Shi, Anton van den Hengel, Zhenmin Tang, and Heng Tao Shen. Hashing on nonlinear manifolds. IEEE Transactions on Image Processing, 24:1839-1851, 2015.","title":"Hashing on nonlinear manifolds","authors":"Fumin Shen, Chunhua Shen, Qinfeng Shi, Anton Van Den, Zhenmin Hengel, Heng Tao Tang, Shen","year":"2015","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Simonyan and Zisserman, 2015] K. Simonyan and A. Zis- serman. Very deep convolutional networks for large-scale image recognition. In international conference on learn- ing representations, 2015.","title":"Very deep convolutional networks for large-scale image recognition","authors":"K Simonyan, A Zisserman","year":"2015","venue":"international conference on learning representations","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Wang et al., 2016] Jun Wang, Wei Liu, Sanjiv Kumar, and Shih-Fu Chang. Learning to hash for indexing big data a survey. arXiv: Learning, 104:34-57, 2016.","title":"Learning to hash for indexing big data a survey","authors":"Jun Wang, Wei Liu, Sanjiv Kumar, Shih-Fu Chang","year":"2016","venue":"arXiv: Learning","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Wu et al., 2015] Botong Wu, Qiang Yang, Wei-Shi Zheng, Yizhou Wang, and Jingdong Wang. Quantized correla- tion hashing for fast cross-modal search. In Proceedings of the 24th International Conference on Artificial Intelli- gence, IJCAI'15, pages 3946-3952. AAAI Press, 2015.","title":"Quantized correlation hashing for fast cross-modal search","authors":"Botong Wu, Qiang Yang, Wei-Shi Zheng, Yizhou Wang, Jingdong Wang","year":"2015","venue":"Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI'15","doi":null,"url":null,"internal_url":"/proceedings/2015/554"},{"position":21,"raw":"[Wu et al., 2018] Gengshen Wu, Zijia Lin, Jungong Han, Li Liu, Guiguang Ding, Baochang Zhang, and Jialie Shen. Unsupervised deep hashing via binary latent factor mod- els for large-scale cross-modal retrieval. In IJCAI, pages 2854-2860, 7 2018.","title":"Unsupervised deep hashing via binary latent factor models for large-scale cross-modal retrieval","authors":"Gengshen Wu, Zijia Lin, Jungong Han, Li Liu, Guiguang Ding, Baochang Zhang, Jialie Shen","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/396"},{"position":22,"raw":"[Zhang et al., 2014] Peichao Zhang, Wei Zhang, Wu-Jun Li, and Minyi Guo. Supervised hashing with latent factor models. In Proceedings of the 37th international ACM SIGIR conference on Research & development in informa- tion retrieval, pages 173-182. ACM, 2014.","title":"Supervised hashing with latent factor models","authors":"Peichao Zhang, Wei Zhang, Wu-Jun Li, Minyi Guo","year":"2014","venue":"Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhen and Yeung, 2012] Yi Zhen and Dit-Yan Yeung. A probabilistic model for multimodal hash function learn- ing. In Proceedings of the 18th ACM SIGKDD interna- tional conference on Knowledge discovery and data min- ing, pages 940-948. ACM, 2012.","title":"A probabilistic model for multimodal hash function learning","authors":"Yeung ; ] Zhen, Dit-Yan Yi Zhen, Yeung","year":"2012","venue":"Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}