{"type":"ScholarlyArticle","title":"Local Representation is Not Enough: Soft Point-Wise Transformer for Descriptor and Detector of Local Features","authors":[{"name":"Zihao Wang","slug":"zihao-wang","given":"Zihao","family":"Wang","affiliation":""},{"name":"Xueyi Li","slug":"xueyi-li","given":"Xueyi","family":"Li","affiliation":""},{"name":"Zhen Li","slug":"zhen-li","given":"Zhen","family":"Li","affiliation":""}],"abstract":"Significant progress has been witnessed for the descriptor and detector of local features, but there still exist several challenging and intractable limitations, such as insufficient localization accuracy and non-discriminative description, especially in repetitive- or blank-texture regions, which haven't be well addressed. The coarse feature representation and limited receptive field are considered as the main issues for these limitations. To address these issues, we propose a novel Soft Point-Wise Transformer for Descriptor and Detector, simultaneously mining long-range intrinsic and cross-scale dependencies of local features. Furthermore, our model leverages the distinct transformers based on the soft point-wise attention, substantially decreasing the memory and computation complexity, especially for high-resolution feature maps. In addition, multi-level decoder is constructed to guarantee the high detection accuracy and discriminative description. Extensive experiments demonstrate that our model outperforms the existing state-of-the-art methods on the image matching and visual localization benchmarks.","doi":"10.24963/ijcai.2021/159","url":"https://ijcai.org/proceedings/2021/159","pdf":"https://ijcai.org/proceedings/2021/0159.pdf","markdown":"https://ijcai.org/proceedings/2021/159.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/159","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Computer Vision II","track":"Main Track","number":159,"pages":{"first":1150,"last":1156},"keywords":["Computer Vision: Recognition: Detection, Categorization, Indexing, Matching, Retrieval, Semantic Interpretation","Computer Vision: 2D and 3D Computer Vision"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}