{"type":"ScholarlyArticle","title":"Proposal-free One-stage Referring Expression via Grid-Word Cross-Attention","authors":[{"name":"Wei Suo","slug":"wei-suo","given":"Wei","family":"Suo","affiliation":""},{"name":"MengYang Sun","slug":"mengyang-sun","given":"MengYang","family":"Sun","affiliation":""},{"name":"Peng Wang","slug":"peng-wang","given":"Peng","family":"Wang","affiliation":""},{"name":"Qi Wu","slug":"qi-wu","given":"Qi","family":"Wu","affiliation":""}],"abstract":"Referring Expression Comprehension (REC) has become one of the most important tasks in visual reasoning, since it is an essential step for many vision-and-language tasks such as visual question answering. However, it has not been widely used in many downstream tasks because it suffers 1) two-stage methods exist heavy computation cost and inevitable error accumulation, and 2) one-stage methods have to depend on lots of hyper-parameters (such as anchors) to generate bounding box. In this paper, we present a proposal-free one-stage (PFOS) model that is able to regress the region-of-interest from the image, based on a textual query, in an end-to-end manner. Instead of using the dominant anchor proposal fashion, we directly take the dense-grid of image as input for a cross-attention transformer that learns grid-word correspondences. The final bounding box is predicted directly from the image without the time-consuming anchor selection process that previous methods suffer. Our model achieves the state-of-the-art performance on four referring expression datasets with higher efficiency, comparing to previous best one-stage and two-stage methods.","doi":"10.24963/ijcai.2021/143","url":"https://ijcai.org/proceedings/2021/143","pdf":"https://ijcai.org/proceedings/2021/0143.pdf","markdown":"https://ijcai.org/proceedings/2021/143.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/143","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Computer Vision II","track":"Main Track","number":143,"pages":{"first":1032,"last":1038},"keywords":["Computer Vision: Language and Vision","Computer Vision: Structural and Model-Based Approaches, Knowledge Representation and Reasoning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}