{"type":"ScholarlyArticle","title":"Image-text Retrieval: A Survey on Recent Research and Development","authors":[{"name":"Min Cao","slug":"min-cao","given":"Min","family":"Cao","affiliation":"Soochow University"},{"name":"Shiping Li","slug":"shiping-li","given":"Shiping","family":"Li","affiliation":"Soochow University"},{"name":"Juntao Li","slug":"juntao-li","given":"Juntao","family":"Li","affiliation":"Soochow University"},{"name":"Liqiang Nie","slug":"liqiang-nie","given":"Liqiang","family":"Nie","affiliation":"Shandong University"},{"name":"Min Zhang","slug":"min-zhang","given":"Min","family":"Zhang","affiliation":"Soochow University; Harbin Institute of Technology , Shenzhen"}],"abstract":"In the past few years, cross-modal image-text retrieval (ITR) has experienced increased interest in the research community due to its excellent research value and broad real-world application. It is designed for the scenarios where the queries are from one modality and the retrieval galleries from another modality. This paper presents a comprehensive and up-to-date survey on the ITR approaches from four perspectives. By dissecting an ITR system into two processes: feature extraction and feature alignment, we summarize the recent advance of the ITR approaches from these two perspectives. On top of this, the efficiency-focused study on the ITR system is introduced as the third perspective. To keep pace with the times, we also provide a pioneering overview of the cross-modal pre-training ITR approaches as the fourth perspective. Finally, we outline the common benchmark datasets and evaluation metric for ITR, and conduct the accuracy comparison among the representative ITR approaches. Some critical yet less studied issues are discussed at the end of the paper.","doi":"10.24963/ijcai.2022/759","url":"https://ijcai.org/proceedings/2022/759","pdf":"https://ijcai.org/proceedings/2022/0759.pdf","markdown":"https://ijcai.org/proceedings/2022/759.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/759","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Survey Track","track":"Survey Track","number":759,"pages":{"first":5410,"last":5417},"keywords":["Survey Track: -"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987302","length_sec":88,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985445","length_sec":548,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Baltrušaitis et al., 2018] Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal machine learning: A survey and taxonomy. TPAMI, 2018.","title":"Multimodal machine learning: A survey and taxonomy","authors":"Tadas Baltrušaitis, Chaitanya Ahuja, Louis-Philippe Morency","year":"2018","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chen et al., 2020a] Hui Chen, Guiguang Ding, Xudong Liu, Zijia Lin, Ji Liu, and Jungong Han. Imram: Iterative matching with recurrent attention memory for cross-modal image-text retrieval. In CVPR, 2020.","title":"Imram: Iterative matching with recurrent attention memory for cross-modal image-text retrieval","authors":"Chen","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2020b] Jianan Chen, Lu Zhang, Cong Bai, and Kidiyo Kpalma. Review of recent deep learning based methods for image-text retrieval. In MIPR, 2020.","title":"Review of recent deep learning based methods for image-text retrieval","authors":"Chen","year":"2020","venue":"MIPR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2020c] Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. Uniter: Universal image-text representation learning. In ECCV, 2020.","title":"Uniter: Universal image-text representation learning","authors":"Chen","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chun et al., 2021] Sanghyuk Chun, Seong Joon Oh, Rafael Sampaio de Rezende, Yannis Kalantidis, and Diane Larlus. Probabilistic embeddings for cross-modal retrieval. In CVPR, 2021.","title":"Probabilistic embeddings for cross-modal retrieval","authors":"Chun","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cui et al., 2021] Yuhao Cui, Zhou Yu, Chunqi Wang, Zhongzhou Zhao, Ji Zhang, Meng Wang, and Jun Yu. Rosita: Enhancing vision-and-language semantic align- ments via cross-and intra-modal knowledge integration. In ACM MM, 2021.","title":"Rosita: Enhancing vision-and-language semantic alignments via cross-and intra-modal knowledge integration","authors":"Yuhao Cui, Zhou Yu, Chunqi Wang, Zhongzhou Zhao, Ji Zhang, Meng Wang, Jun Yu","year":"2021","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Diao et al., 2021] Haiwen Diao, Ying Zhang, Lin Ma, and Huchuan Lu. Similarity reasoning and filtration for image- text matching. In AAAI, 2021.","title":"Similarity reasoning and filtration for imagetext matching","authors":"Diao","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dou et al., 2021] Zi-Yi Dou, Yichong Xu, Zhe Gan, Jian- feng Wang, Shuohang Wang, Lijuan Wang, Chenguang Zhu, Zicheng Liu, Michael Zeng, et al. An empirical study of training end-to-end vision-and-language transformers. arXiv preprint arXiv:2111.02387, 2021.","title":"An empirical study of training end-to-end vision-and-language transformers","authors":"Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang, Lijuan Wang, Chenguang Zhu, Zicheng Liu, Michael Zeng","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Faghri et al., 2017] Fartash Faghri, David J Fleet, Jamie Ryan Kiros, and Sanja Fidler. Vse++: Im- proving visual-semantic embeddings with hard negatives. In BMVC, 2017.","title":"Vse++: Improving visual-semantic embeddings with hard negatives","authors":"Faghri","year":"2017","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Frome et al., 2013] Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc'Aurelio Ranzato, and Tomas Mikolov. Devise: A deep visual-semantic em- bedding model. In NeurIPS, 2013.","title":"Devise: A deep visual-semantic embedding model","authors":"Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc'aurelio Ranzato, Tomas Mikolov","year":"2013","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gan et al., 2022] Zhe Gan, Yen-Chun Chen, Linjie Li, Tian- long Chen, Yu Cheng, Shuohang Wang, and Jingjing Liu. Playing lottery tickets with vision and language. In AAAI, 2022.","title":"Playing lottery tickets with vision and language","authors":"Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Gao et al., 2020] Dehong Gao, Linbo Jin, Ben Chen, Minghui Qiu, Peng Li, Yi Wei, Yi Hu, and Hao Wang. Fashionbert: Text and image matching with adaptive loss for cross-modal retrieval. In SIGIR, 2020.","title":"Fashionbert: Text and image matching with adaptive loss for cross-modal retrieval","authors":"Dehong Gao, Linbo Jin, Ben Chen, Minghui Qiu, Peng Li, Yi Wei, Yi Hu, Hao Wang","year":"2020","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hu et al., 2021] Peng Hu, Xi Peng, Hongyuan Zhu, Liangli Zhen, and Jie Lin. Learning cross-modal retrieval with noisy labels. In CVPR, 2021.","title":"Learning cross-modal retrieval with noisy labels","authors":"Peng Hu, Xi Peng, Hongyuan Zhu, Liangli Zhen, Jie Lin","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Huang et al., 2020] Zhicheng Huang, Zhaoyang Zeng, Bei Liu, Dongmei Fu, and Jianlong Fu. Pixel-bert: Aligning image pixels with text by deep multi-modal transformers. arXiv preprint arXiv:2004.00849, 2020.","title":"Pixel-bert: Aligning image pixels with text by deep multi-modal transformers","authors":"Zhicheng Huang, Zhaoyang Zeng, Bei Liu, Dongmei Fu, Jianlong Fu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Huang et al., 2021] Zhicheng Huang, Zhaoyang Zeng, Yu- pan Huang, Bei Liu, Dongmei Fu, and Jianlong Fu. See- ing out of the box: End-to-end pre-training for vision- language representation learning. In CVPR, 2021.","title":"Seeing out of the box: End-to-end pre-training for visionlanguage representation learning","authors":"Huang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Huo et al., 2021] Yuqi Huo, Manli Zhang, Guangzhen Liu, Haoyu Lu, Yizhao Gao, Guoxing Yang, Jingyuan Wen, Heng Zhang, Baogui Xu, Weihao Zheng, et al. Wenlan: Bridging vision and language by large-scale multi-modal pre-training. arXiv preprint arXiv:2103.06561, 2021.","title":"Wenlan: Bridging vision and language by large-scale multi-modal pre-training","authors":"Yuqi Huo, Manli Zhang, Guangzhen Liu, Haoyu Lu, Yizhao Gao, Guoxing Yang, Jingyuan Wen, Heng Zhang, Baogui Xu, Weihao Zheng","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Ji et al., 2019] Zhong Ji, Haoran Wang, Jungong Han, and Yanwei Pang. Saliency-guided attention network for image-sentence matching. In ICCV, 2019.","title":"Saliency-guided attention network for image-sentence matching","authors":"Zhong Ji, Haoran Wang, Jungong Han, Yanwei Pang","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ji et al., 2021] Zhong Ji, Kexin Chen, and Haoran Wang. Step-wise hierarchical alignment network for image-text matching. In IJCAI, 2021.","title":"Step-wise hierarchical alignment network for image-text matching","authors":"Zhong Ji, Kexin Chen, Haoran Wang","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2021/106"},{"position":19,"raw":"[Jia et al., 2021] Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text su- pervision. In ICML, 2021.","title":"Scaling up visual and vision-language representation learning with noisy text supervision","authors":"Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, Tom Duerig","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kim et al., 2021] Wonjae Kim, Bokyung Son, and Ildoo Kim. Vilt: Vision-and-language transformer without con- volution or region supervision. In ICML, 2021.","title":"Vilt: Vision-and-language transformer without convolution or region supervision","authors":"Kim","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lee et al., 2018] Kuang-Huei Lee, Xi Chen, Gang Hua, Houdong Hu, and Xiaodong He. Stacked cross attention for image-text matching. In ECCV, 2018.","title":"Stacked cross attention for image-text matching","authors":"Kuang-Huei Lee, Xi Chen, Gang Hua, Houdong Hu, Xiaodong He","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Li et al., 2018] Chao Li, Cheng Deng, Ning Li, Wei Liu, Xinbo Gao, and Dacheng Tao. Self-supervised adversar- ial hashing networks for cross-modal retrieval. In CVPR, 2018.","title":"Self-supervised adversarial hashing networks for cross-modal retrieval","authors":"Chao Li, Cheng Deng, Ning Li, Wei Liu, Xinbo Gao, Dacheng Tao","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Li et al., 2019a] Kunpeng Li, Yulun Zhang, Kai Li, Yuanyuan Li, and Yun Fu. Visual semantic reasoning for image-text matching. In ICCV, 2019.","title":"Visual semantic reasoning for image-text matching","authors":"Li","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Li et al., 2019b] Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557, 2019.","title":"Visualbert: A simple and performant baseline for vision and language","authors":"Li","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Li et al., 2020a] Gen Li, Nan Duan, Yuejian Fang, Ming Gong, and Daxin Jiang. Unicoder-vl: A universal encoder for vision and language by cross-modal pre-training. In AAAI, 2020.","title":"Unicoder-vl: A universal encoder for vision and language by cross-modal pre-training","authors":"Li","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Li et al., 2020b] Wei Li, Can Gao, Guocheng Niu, Xinyan Xiao, Hao Liu, Jiachen Liu, Hua Wu, and Haifeng Wang. Unimo: Towards unified-modal understanding and gener- ation via cross-modal contrastive learning. arXiv preprint arXiv:2012.15409, 2020.","title":"Unimo: Towards unified-modal understanding and generation via cross-modal contrastive learning","authors":"Li","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Li et al., 2020c] Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et al. Oscar: Object-semantics aligned pre-training for vision-language tasks. In ECCV, 2020.","title":"Oscar: Object-semantics aligned pre-training for vision-language tasks","authors":"Li","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Li et al., 2021] Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distillation. In NeurIPS, 2021.","title":"Align before fuse: Vision and language representation learning with momentum distillation","authors":"Li","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Liu et al., 2019] Chunxiao Liu, Zhendong Mao, An-An Liu, Tianzhu Zhang, Bin Wang, and Yongdong Zhang. Focus your attention: A bidirectional focal attention network for image-text matching. In ACM MM, 2019.","title":"Focus your attention: A bidirectional focal attention network for image-text matching","authors":"Chunxiao Liu, Zhendong Mao, An-An Liu, Tianzhu Zhang, Bin Wang, Yongdong Zhang","year":"2019","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Liu et al., 2021] Yongfei Liu, Chenfei Wu, Shao-yen Tseng, Vasudev Lal, Xuming He, and Nan Duan. Kd-vlp: Im- proving end-to-end vision-and-language pretraining with object knowledge distillation. In EMNLP, 2021.","title":"Kd-vlp: Improving end-to-end vision-and-language pretraining with object knowledge distillation","authors":"Yongfei Liu, Chenfei Wu, Shao-Yen, Vasudev Tseng, Xuming Lal, Nan He, Duan","year":"2021","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Lu et al., 2019] Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolin- guistic representations for vision-and-language tasks. In NeurIPS, 2019.","title":"Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks","authors":"Lu","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Mafla et al., 2021] Andrés Mafla, Rafael S Rezende, Lluis Gomez, Diane Larlus, and Dimosthenis Karatzas. Stacmr: scene-text aware cross-modal retrieval. In WACV, pages 2220-2230, 2021.","title":"Stacmr: scene-text aware cross-modal retrieval","authors":"Andrés Mafla, Lluis Rafael S Rezende, Diane Gomez, Dimosthenis Larlus, Karatzas","year":"2021","venue":"WACV","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Miech et al., 2021] Antoine Miech, Jean-Baptiste Alayrac, Ivan Laptev, Josef Sivic, and Andrew Zisserman. Thinking fast and slow: Efficient text-to-visual retrieval with trans- formers. In CVPR, 2021.","title":"Thinking fast and slow: Efficient text-to-visual retrieval with transformers","authors":"Antoine Miech, Jean-Baptiste Alayrac, Ivan Laptev, Josef Sivic, Andrew Zisserman","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Qi et al., 2020] Di Qi, Lin Su, Jia Song, Edward Cui, Taroon Bharti, and Arun Sacheti. Imagebert: Cross-modal pre-training with large-scale weak-supervised image-text data. arXiv preprint arXiv:2001.07966, 2020.","title":"Imagebert: Cross-modal pre-training with large-scale weak-supervised image-text data","authors":"Di Qi, Lin Su, Jia Song, Edward Cui, Taroon Bharti, Arun Sacheti","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Qu et al., 2021] Leigang Qu, Meng Liu, Jianlong Wu, Zan Gao, and Liqiang Nie. Dynamic modality interaction mod- eling for image-text retrieval. In SIGIR, 2021.","title":"Dynamic modality interaction modeling for image-text retrieval","authors":"Qu","year":"2021","venue":"SIGIR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Ren et al., 2021] Shuhuai Ren, Junyang Lin, Guangxiang Zhao, Rui Men, An Yang, Jingren Zhou, Xu Sun, and Hongxia Yang. Learning relation alignment for calibrated cross-modal retrieval. In ACL-IJCNLP, 2021.","title":"Learning relation alignment for calibrated cross-modal retrieval","authors":"Ren","year":"2021","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Sarafianos et al., 2019] Nikolaos Sarafianos, Xiang Xu, and Ioannis A Kakadiaris. Adversarial representation learning for text-to-image matching. In ICCV, 2019.","title":"Adversarial representation learning for text-to-image matching","authors":"Nikolaos Sarafianos, Xiang Xu, Ioannis A Kakadiaris","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Song and Soleymani, 2019] Yale Song and Mohammad So- leymani. Polysemous visual-semantic embedding for cross-modal retrieval. In CVPR, 2019.","title":"Yale Song and Mohammad Soleymani. Polysemous visual-semantic embedding for cross-modal retrieval","authors":"Soleymani Song","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Uppal et al., 2022] Shagun Uppal, Sarthak Bhagat, Deva- manyu Hazarika, Navonil Majumder, Soujanya Poria, Roger Zimmermann, and Amir Zadeh. Multimodal re- search in vision and language: A review of current and emerging trends. In Information Fusion, 2022.","title":"Multimodal research in vision and language: A review of current and emerging trends","authors":"Shagun Uppal, Sarthak Bhagat, Devamanyu Hazarika, Navonil Majumder, Soujanya Poria, Roger Zimmermann, Amir Zadeh","year":"2022","venue":"Information Fusion","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Wang et al., 2018] Liwei Wang, Yin Li, Jing Huang, and Svetlana Lazebnik. Learning two-branch neural networks for image-text matching tasks. In TPAMI, 2018.","title":"Learning two-branch neural networks for image-text matching tasks","authors":"Liwei Wang, Yin Li, Jing Huang, Svetlana Lazebnik","year":"2018","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Wang et al., 2019] Zihao Wang, Xihui Liu, Hongsheng Li, Lu Sheng, Junjie Yan, Xiaogang Wang, and Jing Shao. Camp: Cross-modal adaptive message passing for text- image retrieval. In ICCV, 2019.","title":"Camp: Cross-modal adaptive message passing for textimage retrieval","authors":"Wang","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Wang et al., 2020] Haoran Wang, Ying Zhang, Zhong Ji, Yanwei Pang, and Lin Ma. Consensus-aware visual- semantic embedding for image-text matching. In ECCV, 2020.","title":"Consensus-aware visualsemantic embedding for image-text matching","authors":"Haoran Wang, Ying Zhang, Zhong Ji, Yanwei Pang, Lin Ma","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Wei et al., 2020] Xi Wei, Tianzhu Zhang, Yan Li, Yongdong Zhang, and Feng Wu. Multi-modality cross attention net- work for image and sentence matching. In CVPR, 2020.","title":"Multi-modality cross attention network for image and sentence matching","authors":"Wei","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wu et al., 2019a] Hao Wu, Jiayuan Mao, Yufeng Zhang, Yuning Jiang, Lei Li, Weiwei Sun, and Wei-Ying Ma. Uni- fied visual-semantic embeddings: Bridging vision and lan- guage with structured meaning representations. In CVPR, 2019.","title":"Unified visual-semantic embeddings: Bridging vision and language with structured meaning representations","authors":"Wu","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Wu et al., 2019b] Yiling Wu, Shuhui Wang, Guoli Song, and Qingming Huang. Learning fragment self-attention embeddings for image-text matching. In ACM MM, 2019.","title":"Learning fragment self-attention embeddings for image-text matching","authors":"Wu","year":"2019","venue":"ACM MM","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Xu et al., 2021] Haiyang Xu, Ming Yan, Chenliang Li, Bin Bi, Songfang Huang, Wenming Xiao, and Fei Huang. E2e- vlp: End-to-end vision-language pre-training enhanced by visual learning. In ACL-IJCNLP, 2021.","title":"E2evlp: End-to-end vision-language pre-training enhanced by visual learning","authors":"Haiyang Xu, Ming Yan, Chenliang Li, Bin Bi, Songfang Huang, Wenming Xiao, Fei Huang","year":"2021","venue":"ACL-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Xue et al., 2021] Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, and Jiebo Luo. Probing inter-modality: Visual parsing with self-attention for vision-and-language pre-training. In NeurIPS, 2021.","title":"Probing inter-modality: Visual parsing with self-attention for vision-and-language pre-training","authors":"Yupan Hongwei Xue, Bei Huang, Houwen Liu, Jianlong Peng, Houqiang Fu, Jiebo Li, Luo","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Yang et al., 2017] Erkun Yang, Cheng Deng, Wei Liu, Xi- anglong Liu, Dacheng Tao, and Xinbo Gao. Pairwise rela- tionship guided deep hashing for cross-modal retrieval. In AAAI, 2017.","title":"Pairwise relationship guided deep hashing for cross-modal retrieval","authors":"Erkun Yang, Cheng Deng, Wei Liu, Xianglong Liu, Dacheng Tao, Xinbo Gao","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Yao et al., 2022] Lewei Yao, Runhui Huang, Lu Hou, Guan- song Lu, Minzhe Niu, Hang Xu, Xiaodan Liang, Zhenguo Li, Xin Jiang, and Chunjing Xu. Filip: Fine-grained inter- active language-image pre-training. In ICLR, 2022.","title":"Filip: Fine-grained interactive language-image pre-training","authors":"Yao","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Yu et al., 2021a] Fei Yu, Jiji Tang, Weichong Yin, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. Ernie-vil: Knowl- edge enhanced vision-language representations through scene graph. In AAAI, 2021.","title":"Ernie-vil: Knowledge enhanced vision-language representations through scene graph","authors":"Yu","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Yu et al., 2021b] Jun Yu, Hao Zhou, Yibing Zhan, and Dacheng Tao. Deep graph-neighbor coherence preserving network for unsupervised cross-modal hashing. In AAAI, 2021.","title":"Deep graph-neighbor coherence preserving network for unsupervised cross-modal hashing","authors":"Yu","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Zhang et al., 2018] Xi Zhang, Hanjiang Lai, and Jiashi Feng. Attention-aware deep adversarial hashing for cross- modal retrieval. In ECCV, 2018.","title":"Attention-aware deep adversarial hashing for crossmodal retrieval","authors":"Xi Zhang, Hanjiang Lai, Jiashi Feng","year":"2018","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Zhang et al., 2020] Qi Zhang, Zhen Lei, Zhaoxiang Zhang, and Stan Z Li. Context-aware attention network for image- text retrieval. In CVPR, 2020.","title":"Context-aware attention network for imagetext retrieval","authors":"Qi Zhang, Zhen Lei, Zhaoxiang Zhang, Stan Z Li","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Zhang et al., 2021] Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, and Jianfeng Gao. Vinvl: Revisiting visual representations in vision-language models. In CVPR, 2021.","title":"Vinvl: Revisiting visual representations in vision-language models","authors":"Zhang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Zheng et al., 2020] Zhedong Zheng, Liang Zheng, Michael Garrett, Yi Yang, Mingliang Xu, and Yi-Dong Shen. Dual- path convolutional image-text embeddings with instance loss. In TOMM, 2020. Zhuge et al., 2021] Mingchen Zhuge, Dehong Gao, Deng- Ping Fan, Linbo Jin, Ben Chen, Haoming Zhou, Minghui Qiu, and Ling Shao. Kaleido-bert: Vision-language pre- training on fashion domain. In CVPR, 2021.","title":"Dualpath convolutional image-text embeddings with instance loss","authors":"Zhedong Zheng, Liang Zheng, Michael Garrett, Yi Yang, Mingliang Xu, Yi-Dong Shen, ; Zhuge","year":"2020","venue":"TOMM","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}