{"type":"ScholarlyArticle","title":"Answer Mining from a Pool of Images: Towards Retrieval-Based Visual Question Answering","authors":[{"name":"Abhirama Subramanyam Penamakuri","slug":"abhirama-subramanyam-penamakuri","given":"Abhirama Subramanyam","family":"Penamakuri","affiliation":"Indian Institute of Technology Jodhpur"},{"name":"Manish Gupta","slug":"manish-gupta","given":"Manish","family":"Gupta","affiliation":"Microsoft"},{"name":"Mithun Das Gupta","slug":"mithun-das-gupta","given":"Mithun Das","family":"Gupta","affiliation":"Microsoft"},{"name":"Anand Mishra","slug":"anand-mishra","given":"Anand","family":"Mishra","affiliation":"Indian Institute of Technology Jodhpur"}],"abstract":"We study visual question answering in a setting where the answer has to be mined from a pool of relevant and irrelevant images given as a context. For such a setting, a model must first retrieve relevant images from the pool and answer the question from these retrieved images. We refer to this problem as retrieval-based visual question answering (or RETVQA in short). The RETVQA is distinctively different and more challenging than the traditionally-studied Visual Question Answering (VQA), where a given question has to be answered with a single relevant image in context. Towards solving the RETVQA task, we propose a unified Multi Image BART (MI-BART) that takes a question and retrieved images using our relevance encoder for free-form fluent answer generation. Further, we introduce the largest dataset in this space, namely RETVQA, which has the following salient features: multi-image and retrieval requirement for VQA, metadata-independent questions over a pool of heterogeneous images, expecting a mix of classification-oriented and open-ended generative answers. Our proposed framework achieves an accuracy of 76.5% and a fluency of 79.3% on the proposed dataset, namely RETVQA and also outperforms state-of-the-art methods by 4.9% and 11.8% on the image segment of the publicly available WebQA dataset on the accuracy and fluency metrics, respectively.","doi":"10.24963/ijcai.2023/146","url":"https://ijcai.org/proceedings/2023/146","pdf":"https://ijcai.org/proceedings/2023/0146.pdf","markdown":"https://ijcai.org/proceedings/2023/146.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/146","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Computer Vision","track":"Main Track","number":146,"pages":{"first":1312,"last":1321},"keywords":["Machine Learning: ML: Multi-modal learning","Computer Vision: CV: Applications","Computer Vision: CV: Vision and language"],"videos":[],"references":[{"position":1,"raw":"[Ahmad et al., 2019] Amin Ahmad, Noah Constant, Yinfei Yang, and Daniel Cer. Reqa: An evaluation for end-to- end answer retrieval models. In Proceedings of the 2nd Workshop on Machine Reading for Question Answering, pages 137-146, 2019.","title":"Reqa: An evaluation for end-toend answer retrieval models","authors":"Amin Ahmad, Noah Constant, Yinfei Yang, Daniel Cer","year":"2019","venue":"Proceedings of the 2nd Workshop on Machine Reading for Question Answering","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andreas et al., 2016] Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. Neural module networks. In CVPR, pages 39-48, 2016.","title":"Neural module networks","authors":"Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Dan Klein","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Antol et al., 2015] Stanislaw Antol, Aishwarya Agrawal, Ji- asen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zit- nick, and Devi Parikh. Vqa: Visual question answering. In ICCV, pages 2425-2433, 2015.","title":"Vqa: Visual question answering","authors":"Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, Devi Parikh","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bansal et al., 2020] Ankan Bansal, Yuting Zhang, and Rama Chellappa. Visual question answering on image sets. In ECCV, pages 51-67, 2020.","title":"Visual question answering on image sets","authors":"Ankan Bansal, Yuting Zhang, Rama Chellappa","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chang et al., 2022] Yingshan Chang, Mridu Narang, Hisami Suzuki, Guihong Cao, Jianfeng Gao, and Yonatan Bisk. Webqa: Multihop and multimodal qa. In CVPR, pages 16495-16504, 2022.","title":"Webqa: Multihop and multimodal qa","authors":"Chang","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2020] Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. Uniter: Universal image-text representation learning. In ECCV, pages 104-120, 2020.","title":"Uniter: Universal image-text representation learning","authors":"Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, Jingjing Liu","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cho et al., 2021] Jaemin Cho, Jie Lei, Hao Tan, and Mohit Bansal. Unifying vision-and-language tasks via text gen- eration. In ICML, pages 1931-1942, 2021.","title":"Unifying vision-and-language tasks via text generation","authors":"Jaemin Cho, Jie Lei, Hao Tan, Mohit Bansal","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understand- ing. In NAACL-HLT, pages 4171-4186, 2019.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fukui et al., 2016] Akira Fukui, Dong Huk Park, Daylen Yang, Anna Rohrbach, Trevor Darrell, and Marcus Rohrbach. Multimodal compact bilinear pooling for vi- sual question answering and visual grounding. In EMNLP, pages 457-468, 2016.","title":"Multimodal compact bilinear pooling for visual question answering and visual grounding","authors":"Akira Fukui, Dong Huk Park, Daylen Yang, Anna Rohrbach, Trevor Darrell, Marcus Rohrbach","year":"2016","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gao et al., 2015] Haoyuan Gao, Junhua Mao, Jie Zhou, Zhi- heng Huang, Lei Wang, and Wei Xu. Are you talking to a machine? dataset and methods for multilingual image question. In NeurIPS, volume 28, pages 2296-2304, 2015.","title":"Are you talking to a machine? dataset and methods for multilingual image question","authors":"Gao","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Goyal et al., 2017] Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the V in VQA matter: Elevating the role of image under- standing in visual question answering. In CVPR, pages 6904-6913, 2017.","title":"Making the V in VQA matter: Elevating the role of image understanding in visual question answering","authors":"Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, Devi Parikh","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Guo et al., 2023] Dalu Guo, Chang Xu, and Dacheng Tao. Bilinear graph networks for visual question answering. IEEE Transactions on Neural Networks and Learning Sys- tems, 34(2):1023-1034, 2023.","title":"Bilinear graph networks for visual question answering","authors":"Dalu Guo, Chang Xu, Dacheng Tao","year":"2023","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hsu et al., 2021] Chao-Chun Hsu, Eric Lind, Luca Sol- daini, and Alessandro Moschitti. Answer generation for retrieval-based question answering systems. In ACL/IJCNLP, pages 4276-4282, 2021.","title":"Answer generation for retrieval-based question answering systems","authors":"Hsu","year":"2021","venue":"ACL/IJCNLP","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hu et al., 2017] Ronghang Hu, Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Kate Saenko. Learning to reason: End-to-end module networks for visual question answering. In ICCV, pages 804-813, 2017.","title":"Learning to reason: End-to-end module networks for visual question answering","authors":"Hu","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Johnson et al., 2017] Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for com- positional language and elementary visual reasoning. In CVPR, pages 2901-2910, 2017.","title":"Clevr: A diagnostic dataset for compositional language and elementary visual reasoning","authors":"Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, Ross Girshick","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kembhavi et al., 2017] Aniruddha Kembhavi, Minjoon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, and Hannaneh Hajishirzi. Are you smarter than a sixth grader? textbook question answering for multimodal machine comprehension. In CVPR, pages 4999-5007, 2017.","title":"Are you smarter than a sixth grader? textbook question answering for multimodal machine comprehension","authors":"Aniruddha Kembhavi, Minjoon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, Hannaneh Hajishirzi","year":"2017","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kim et al., 2021] Wonjae Kim, Bokyung Son, and Ildoo Kim. Vilt: Vision-and-language transformer without con- volution or region supervision. In ICML, pages 5583- 5594, 2021.","title":"Vilt: Vision-and-language transformer without convolution or region supervision","authors":"Kim","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kingma and Ba, 2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.","title":"Adam: A method for stochastic optimization","authors":"P Diederik, Jimmy Kingma, Ba","year":"2015","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Krishna et al., 2017] Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. IJCV, 123(1):32-73, 2017.","title":"Visual genome: Connecting language and vision using crowdsourced dense image annotations","authors":"Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma","year":"2017","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lei et al., 2018] Jie Lei, Licheng Yu, Mohit Bansal, and Tamara L Berg. Tvqa: Localized, compositional video question answering. In EMNLP, pages 1369-1379, 2018.","title":"Tvqa: Localized, compositional video question answering","authors":"Jie Lei, Licheng Yu, Mohit Bansal, Tamara L Berg","year":"2018","venue":"EMNLP","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lei et al., 2020] Jie Lei, Licheng Yu, Tamara Berg, and Mo- hit Bansal. Tvqa+: Spatio-temporal grounding for video question answering. In ACL, pages 8211-8225, 2020.","title":"Tvqa+: Spatio-temporal grounding for video question answering","authors":"Jie Lei, Licheng Yu, Tamara Berg, Mohit Bansal","year":"2020","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Li et al., 2019] Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. Visualbert: A simple and performant baseline for vision and language. CoRR, 2019.","title":"Visualbert: A simple and performant baseline for vision and language","authors":"Li","year":"2019","venue":"CoRR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Li et al., 2020] Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et al. Oscar: Object-semantics aligned pre-training for vision-language tasks. In ECCV, pages 121-137, 2020.","title":"Oscar: Object-semantics aligned pre-training for vision-language tasks","authors":"Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei","year":"2020","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Com- mon objects in context. In ECCV, pages 740-755, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Lu et al., 2016] Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. Hierarchical question-image co-attention for visual question answering. In NeurIPS, volume 29, pages 289-297, 2016.","title":"Hierarchical question-image co-attention for visual question answering","authors":"Lu","year":"2016","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Lu et al., 2020] Jiasen Lu, Vedanuj Goswami, Marcus Rohrbach, Devi Parikh, and Stefan Lee. 12-in-1: Multi- task vision and language representation learning. In CVPR, pages 10437-10446, 2020.","title":"12-in-1: Multitask vision and language representation learning","authors":"Lu","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Malinowski et al., 2015] Mateusz Malinowski, Marcus Rohrbach, and Mario Fritz. Ask your neurons: A neural- based approach to answering questions about images. In ICCV, pages 1-9, 2015.","title":"Ask your neurons: A neuralbased approach to answering questions about images","authors":"Mateusz Malinowski, Marcus Rohrbach, Mario Fritz","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Marino et al., 2019] Kenneth Marino, Mohammad Raste- gari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge. In CVPR, pages 3195-3204, 2019.","title":"Ok-vqa: A visual question answering benchmark requiring external knowledge","authors":"Kenneth Marino, Mohammad Rastegari, Ali Farhadi, Roozbeh Mottaghi","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Noh et al., 2016] Hyeonwoo Noh, Paul Hongsuck Seo, and Bohyung Han. Image question answering using convolu- tional neural network with dynamic parameter prediction. In CVPR, pages 30-38, 2016.","title":"Image question answering using convolutional neural network with dynamic parameter prediction","authors":"Noh","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Paszke et al., 2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high- performance deep learning library. In NeurIPS, volume 32, pages 8024-8035, 2019.","title":"Pytorch: An imperative style, highperformance deep learning library","authors":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga","year":"2019","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Rajpurkar et al., 2018] Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don't know: Unanswerable questions for squad. In ACL, pages 784-789, 2018.","title":"Know what you don't know: Unanswerable questions for squad","authors":"Pranav Rajpurkar, Robin Jia, Percy Liang","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Ren et al., 2015a] Mengye Ren, Ryan Kiros, and Richard Zemel. Exploring models and data for image question an- swering. In NeurIPS, volume 28, pages 2953-2961, 2015.","title":"Exploring models and data for image question answering","authors":"Ren","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Ren et al., 2015b] Shaoqing Ren, Kaiming He, Ross B. Gir- shick, and Jian Sun. Faster R-CNN: towards real-time ob- ject detection with region proposal networks. In NeurIPS, volume 28, pages 91-99, 2015.","title":"Faster R-CNN: towards real-time object detection with region proposal networks","authors":"Ren","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Shah et al., 2019] Sanket Shah, Anand Mishra, Naganand Yadati, and Partha Pratim Talukdar. Kvqa: Knowledge- aware visual question answering. In AAAI, pages 8876- 8884, 2019.","title":"Kvqa: Knowledgeaware visual question answering","authors":"Sanket Shah, Anand Mishra, Naganand Yadati, Partha Pratim Talukdar","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Shih et al., 2016] Kevin J Shih, Saurabh Singh, and Derek Hoiem. Where to look: Focus regions for visual question answering. In CVPR, pages 4613-4621, 2016.","title":"Where to look: Focus regions for visual question answering","authors":"Kevin J Shih, Saurabh Singh, Derek Hoiem","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Singh et al., 2019] Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In CVPR, pages 8317-8326, 2019.","title":"Towards vqa models that can read","authors":"Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, Marcus Rohrbach","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Singh et al., 2021] Hrituraj Singh, Anshul Nasery, Denil Mehta, Aishwarya Agarwal, Jatin Lamba, and Bal- aji Vasan Srinivasan. Mimoqa: Multimodal input multi- modal output question answering. In NAACL-HLT, pages 5317-5332, 2021.","title":"Mimoqa: Multimodal input multimodal output question answering","authors":"Singh","year":"2021","venue":"NAACL-HLT","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Suhr et al., 2019] Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Huajun Bai, and Yoav Artzi. A corpus for rea- soning about natural language grounded in photographs. In ACL, pages 6418-6428, 2019.","title":"A corpus for reasoning about natural language grounded in photographs","authors":"Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Huajun Bai, Yoav Artzi","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Talmor et al., 2021] Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, and Jonathan Berant. Multimodalqa: complex question answering over text, tables and images. In ICLR (Poster), 2021.","title":"Multimodalqa: complex question answering over text, tables and images","authors":"Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, Jonathan Berant","year":"2021","venue":"ICLR (Poster)","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Tan and Bansal, 2019] Hao Tan and Mohit Bansal. Lxmert: Learning cross-modality encoder representations from transformers. In EMNLP-IJCNLP, pages 5100-5111, 2019.","title":"Lxmert: Learning cross-modality encoder representations from transformers","authors":"Hao Tan, Mohit Bansal","year":"2019","venue":"EMNLP-IJCNLP","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Tapaswi et al., 2016] Makarand Tapaswi, Yukun Zhu, Rainer Stiefelhagen, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. Movieqa: Understanding stories in movies through question-answering. In CVPR, pages 4631-4640, 2016.","title":"Movieqa: Understanding stories in movies through question-answering","authors":"Makarand Tapaswi, Yukun Zhu, Rainer Stiefelhagen, Antonio Torralba, Raquel Urtasun, Sanja Fidler","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Tito et al., 2021] Rubèn Tito, Dimosthenis Karatzas, and Ernest Valveny. Document collection visual question an- swering. In ICDAR, pages 778-792, 2021.","title":"Rubèn Tito, Dimosthenis Karatzas, and Ernest Valveny. Document collection visual question answering","authors":"Tito","year":"2021","venue":"ICDAR","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, volume 30, pages 5998-6008, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wang et al., 2022] Zirui Wang, Jiahui Yu, Adams Wei Yu, Zihang Dai, Yulia Tsvetkov, and Yuan Cao. Simvlm: Sim- ple visual language model pretraining with weak supervi- sion. In ICLR (Poster), 2022.","title":"Simvlm: Simple visual language model pretraining with weak supervision","authors":"Wang","year":"2022","venue":"ICLR (Poster)","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Wolf et al., 2020] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Fun- towicz, Joe Davison, et al. Transformers: State-of-the-art natural language processing. In EMNLP: System Demon- strations, pages 38-45, 2020.","title":"Transformers: State-of-the-art natural language processing","authors":"Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison","year":"2020","venue":"EMNLP: System Demonstrations","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Xiong et al., 2016] Caiming Xiong, Stephen Merity, and Richard Socher. Dynamic memory networks for visual and textual question answering. In ICML, pages 2397-2406, 2016.","title":"Dynamic memory networks for visual and textual question answering","authors":"Xiong","year":"2016","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Yang et al., 2016] Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, and Alex Smola. Stacked attention net- works for image question answering. In CVPR, pages 21- 29, 2016.","title":"Stacked attention networks for image question answering","authors":"Yang","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Yuan et al., 2021] Weizhe Yuan, Graham Neubig, and Pengfei Liu. Bartscore: Evaluating generated text as text generation. In NeurIPS, volume 34, pages 27263-27277, 2021.","title":"Bartscore: Evaluating generated text as text generation","authors":"Yuan","year":"2021","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Zhou et al., 2020] Luowei Zhou, Hamid Palangi, Lei Zhang, Houdong Hu, Jason J. Corso, and Jianfeng Gao. Unified vision-language pre-training for image captioning and VQA. In AAAI, pages 13041-13049, 2020.","title":"Unified vision-language pre-training for image captioning and VQA","authors":"Luowei Zhou, Hamid Palangi, Lei Zhang, Houdong Hu, Jason J Corso, Jianfeng Gao","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Zhu et al., 2016] Yuke Zhu, Oliver Groth, Michael Bern- stein, and Li Fei-Fei. Visual7w: Grounded question an- swering in images. In CVPR, pages 4995-5004, 2016.","title":"Visual7w: Grounded question answering in images","authors":"Yuke Zhu, Oliver Groth, Michael Bernstein, Li Fei-Fei","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}