{"type":"ScholarlyArticle","title":"Set and Rebase: Determining the Semantic Graph Connectivity for Unsupervised Cross-Modal Hashing","authors":[{"name":"Weiwei Wang","slug":"weiwei-wang","given":"Weiwei","family":"Wang","affiliation":"School of Computer Science and Engineering , Nanjing University of Science and Technology , China"},{"name":"Yuming Shen","slug":"yuming-shen","given":"Yuming","family":"Shen","affiliation":"Inception Institute of Artificial Intelligence (IIAI) , Abu Dhabi , United Arab Emirates"},{"name":"Haofeng Zhang","slug":"haofeng-zhang","given":"Haofeng","family":"Zhang","affiliation":"School of Computer Science and Engineering , Nanjing University of Science and Technology , China"},{"name":"Yazhou Yao","slug":"yazhou-yao","given":"Yazhou","family":"Yao","affiliation":"School of Computer Science and Engineering , Nanjing University of Science and Technology , China"},{"name":"Li Liu","slug":"li-liu","given":"Li","family":"Liu","affiliation":"Inception Institute of Artificial Intelligence (IIAI) , Abu Dhabi , United Arab Emirates"}],"abstract":"The label-free nature of unsupervised cross-modal hashing hinders models from exploiting the exact semantic data similarity. Existing research typically simulates the semantics by a heuristic geometric prior in the original feature space. However, this introduces heavy bias into the model as the original features are not fully representing the underlying multi-view data relations. To address the problem above, in this paper, we propose a novel unsupervised hashing method called Semantic-Rebased Cross-modal Hashing (SRCH). A novel ‘Set-and-Rebase’ process is defined to initialize and update the cross-modal similarity graph of training data. In particular, we set the graph according to the intra-modal feature geometric basis and then alternately rebase it to update the edges within according to the hashing results. We develop an alternating optimization routine to rebase the graph and train the hashing auto-encoders with closed-form solutions so that the overall framework is efficiently trained. Our experimental results on benchmarked datasets demonstrate the superiority of our model against state-of-the-art algorithms.","doi":"10.24963/ijcai.2020/119","url":"https://ijcai.org/proceedings/2020/119","pdf":"https://ijcai.org/proceedings/2020/0119.pdf","markdown":"https://ijcai.org/proceedings/2020/119.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/119","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Computer Vision","track":"Main track","number":119,"pages":{"first":853,"last":859},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24415/file_24415.m3u8","length_sec":271,"provider":"hls"}],"references":[{"position":1,"raw":"[Bronstein et al., 2010] Michael M Bronstein, Alexander M Bronstein, Fabrice Michel, and Nikos Paragios. Data fusion through cross-modality metric learning using similarity-sensitive hashing. In CVPR, pages 3594-3601, 2010.","title":"Data fusion through cross-modality metric learning using similarity-sensitive hashing","authors":"Alexander M Michael M Bronstein, Fabrice Bronstein, Nikos Michel, Paragios","year":"2010","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Cer et al., 2018] Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Con- stant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, et al. Universal sentence encoder. arXiv preprint arXiv:1803.11175, 2018.","title":"Universal sentence encoder","authors":"Daniel Cer, Yinfei Yang, Sheng-Yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chao et al., 2019] Li Chao, Deng Cheng, Wang Lei, Xie De, and Liu Xianglong. Coupled cyclegan: Unsupervised hashing network for cross-modal retrieval. arXiv preprint arXiv:1903.02149, 2019.","title":"Coupled cyclegan: Unsupervised hashing network for cross-modal retrieval","authors":"Li Chao, Deng Cheng, Wang Lei, Xie De, Liu Xianglong","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chua et al., 2009] Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. Nus- wide: a real-world web image database from national uni- versity of singapore. In CIVR, 2009.","title":"Nuswide: a real-world web image database from national university of singapore","authors":"Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, Yantao Zheng","year":"2009","venue":"CIVR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ding et al., 2014] Guiguang Ding, Yuchen Guo, and Jile Zhou. Collective matrix factorization hashing for multi- modal data. In CVPR, pages 2083-2090, 2014.","title":"Collective matrix factorization hashing for multimodal data","authors":"Ding","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Erin Liong et al., 2017] Venice Erin Liong, Jiwen Lu, Yap- Peng Tan, and Jie Zhou. Cross-modal deep variational hashing. In ICCV, pages 4077-4085, 2017.","title":"Cross-modal deep variational hashing","authors":"Erin Liong","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gong et al., 2013] Yunchao Gong, Svetlana Lazebnik, Al- bert Gordo, and Florent Perronnin. Iterative quantization: A procrustean approach to learning binary codes for large- scale image retrieval. IEEE TPAMI, 35(12):2916-2929, 2013.","title":"Iterative quantization: A procrustean approach to learning binary codes for largescale image retrieval","authors":"Gong","year":"2013","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hu et al., 2018] Di Hu, Feiping Nie, and Xuelong Li. Dis- crete spectral hashing for efficient similarity retrieval. IEEE TIP, 28(3):1080-1091, 2018.","title":"Discrete spectral hashing for efficient similarity retrieval","authors":"Di Hu, Feiping Nie, Xuelong Li","year":"2018","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Huiskes and Lew, 2008] Mark Huiskes and Michael Lew. The mir flickr retrieval evaluation. In ICMIR, pages 39- 43. ACM, 2008.","title":"The mir flickr retrieval evaluation","authors":"Lew ; Huiskes, Mark Huiskes, Michael Lew","year":"2008","venue":"ICMIR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Irie et al., 2015] Go Irie, Hiroyuki Arai, and Yukinobu Taniguchi. Alternating co-quantization for cross-modal hashing. In ICCV, pages 1886-1894, 2015.","title":"Alternating co-quantization for cross-modal hashing","authors":"Irie","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jian et al., 2018] Zhang Jian, Peng Yuxin, and Yuan Mingkuan. Unsupervised generative adversarial cross- modal hashing. In AAAI, pages 539-546, 2018.","title":"Unsupervised generative adversarial crossmodal hashing","authors":"Zhang Jian, Yuan Peng Yuxin, Mingkuan","year":"2018","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jiang and Li, 2017] Qing-Yuan Jiang and Wu-Jun Li. Deep cross-modal hashing. In CVPR, pages 3232-3240, 2017.","title":"Deep cross-modal hashing","authors":"Li ; Qing-Yuan Jiang, Wu-Jun Jiang, Li","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kumar and Udupa, 2011] Shaishav Kumar and Raghaven- dra Udupa. Learning hash functions for cross-view sim- ilarity search. In IJCAI, pages 1360-1365, 2011.","title":"Shaishav Kumar and Raghavendra Udupa. Learning hash functions for cross-view similarity search","authors":"Udupa Kumar","year":"2011","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In ECCV, pages 740-755, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liu et al., 2011] Wei Liu, Jun Wang, Sanjiv Kumar, and Shih-Fu Chang. Hashing with graphs. In ICML, 2011.","title":"Hashing with graphs","authors":"Wei Liu, Jun Wang, Sanjiv Kumar, Shih-Fu Chang","year":"2011","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liu et al., 2014] Wei Liu, Cun Mu, Sanjiv Kumar, and Shih- Fu Chang. Discrete graph hashing. In NeurIPS, pages 3419-3427, 2014.","title":"Discrete graph hashing","authors":"Wei Liu, Cun Mu, Sanjiv Kumar, Shih-Fu Chang","year":"2014","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2017] Li Liu, Fumin Shen, Yuming Shen, Xiang- long Liu, and Ling Shao. Deep sketch hashing: Fast free- hand sketch-based image retrieval. In CVPR, pages 2862- 2871, 2017.","title":"Deep sketch hashing: Fast freehand sketch-based image retrieval","authors":"Li Liu, Fumin Shen, Yuming Shen, Xianglong Liu, Ling Shao","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Long et al., 2016] Mingsheng Long, Yue Cao, Jianmin Wang, and Philip S Yu. Composite correlation quanti- zation for efficient multimodal retrieval. In ACM SIGIR, pages 579-588, 2016.","title":"Composite correlation quantization for efficient multimodal retrieval","authors":"Mingsheng Long, Yue Cao, Jianmin Wang, Philip S Yu","year":"2016","venue":"ACM SIGIR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Peng et al., 2019] Xi Peng, Zhenyu Huang, Jiancheng Lv, Hongyuan Zhu, and Joey Tianyi Zhou. Comic: Multi-view clustering without parameter selection. In ICML, pages 5092-5101, 2019.","title":"Comic: Multi-view clustering without parameter selection","authors":"Peng","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Rasiwasia et al., 2010] Nikhil Rasiwasia, Jose Costa Pereira, Emanuele Coviello, Gabriel Doyle, Gert RG Lanckriet, Roger Levy, and Nuno Vasconcelos. A new approach to cross-modal multimedia retrieval. In ACM MM, pages 251-260, 2010.","title":"A new approach to cross-modal multimedia retrieval","authors":"Nikhil Rasiwasia, Jose Costa Pereira, Emanuele Coviello, Gabriel Doyle, Gert Rg Lanckriet, Roger Levy, Nuno Vasconcelos","year":"2010","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rastegari et al., 2013] Mohammad Rastegari, Jonghyun Choi, Shobeir Fakhraei, Daume Hal, and Larry Davis. Pre- dictable dual-view hashing. In ICML, pages 1328-1336, 2013.","title":"Predictable dual-view hashing","authors":"Mohammad Rastegari, Jonghyun Choi, Shobeir Fakhraei, Daume Hal, Larry Davis","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shah and Koltun, 2017] Sohil Atul Shah and Vladlen Koltun. Robust continuous clustering. PNAS, 114(37):9814-9819, 2017.","title":"Robust continuous clustering","authors":"Atul Sohil, Vladlen Shah, Koltun","year":"2017","venue":"PNAS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shen et al., 2017] Yuming Shen, Li Liu, Ling Shao, and Jingkuan Song. Deep binaries: Encoding semantic-rich cues for efficient textual-visual cross retrieval. In ICCV, pages 4097-4106, 2017.","title":"Deep binaries: Encoding semantic-rich cues for efficient textual-visual cross retrieval","authors":"Yuming Shen, Li Liu, Ling Shao, Jingkuan Song","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shen et al., 2018] Fumin Shen, Yan Xu, Li Liu, Yang Yang, Zi Huang, and Heng Tao Shen. Unsupervised deep hashing with similarity-adaptive and discrete optimization. IEEE TPAMI, 40(12):3034-3044, 2018.","title":"Unsupervised deep hashing with similarity-adaptive and discrete optimization","authors":"Fumin Shen, Yan Xu, Li Liu, Yang Yang, Zi Huang, Heng Tao Shen","year":"2018","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Song et al., 2013] Jingkuan Song, Yang Yang, Yi Yang, Zi Huang, and Heng Tao Shen. Inter-media hashing for large-scale retrieval from heterogeneous data sources. In ACM SIGMOD, pages 785-796, 2013.","title":"Inter-media hashing for large-scale retrieval from heterogeneous data sources","authors":"Song","year":"2013","venue":"ACM SIGMOD","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Su et al., 2018] Shupeng Su, Chao Zhang, Kai Han, and Yonghong Tian. Greedy hash: Towards fast optimization for accurate hash coding in cnn. In NeurIPS, pages 798- 807, 2018.","title":"Greedy hash: Towards fast optimization for accurate hash coding in cnn","authors":"Su","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tang et al., 2016] Jun Tang, Ke Wang, and Ling Shao. Su- pervised matrix factorization hashing for cross-modal re- trieval. IEEE TIP, 25(7):3157-3166, 2016.","title":"Supervised matrix factorization hashing for cross-modal retrieval","authors":"Jun Tang, Ke Wang, Ling Shao","year":"2016","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wu et al., 2015] Botong Wu, Qiang Yang, Wei-Shi Zheng, Yizhou Wang, and Jingdong Wang. Quantized correlation hashing for fast cross-modal search. In IJCAI, pages 3946- 3952, 2015.","title":"Quantized correlation hashing for fast cross-modal search","authors":"Botong Wu, Qiang Yang, Wei-Shi Zheng, Yizhou Wang, Jingdong Wang","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/554"},{"position":29,"raw":"[Zhang and Li, 2014] Dongqing Zhang and Wu-Jun Li. Large-scale supervised multimodal hashing with seman- tic correlation maximization. In AAAI, pages 2177-2183, 2014.","title":"Large-scale supervised multimodal hashing with semantic correlation maximization","authors":"Li ; Zhang, Dongqing Zhang, Wu-Jun Li","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang and Wang, 2016] Ting Zhang and Jingdong Wang. Collaborative quantization for cross-modal similarity search. In CVPR, pages 2036-2045, 2016.","title":"Collaborative quantization for cross-modal similarity search","authors":"Ting Zhang, Jingdong Wang","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2018a] Haofeng Zhang, Li Liu, Yang Long, and Ling Shao. Unsupervised deep hashing with pseudo labels for scalable image retrieval. IEEE TIP, 27(4):1626- 1638, 2018.","title":"Unsupervised deep hashing with pseudo labels for scalable image retrieval","authors":"Zhang","year":"2018","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhang et al., 2018b] Zheng Zhang, Li Liu, Fumin Shen, Heng Tao Shen, and Ling Shao. Binary multi-view clus- tering. IEEE TPAMI, 41(7):1774-1782, 2018.","title":"Binary multi-view clustering","authors":"Zhang","year":"2018","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}