---
title: "Multimodal Image Matching Based on Cross-Modality Completion Pre-training"
authors:
  - "Meng Yang"
  - "Fan Fan"
  - "Jun Huang"
  - "Yong Ma"
  - "Xiaoguang Mei"
  - "Zhanchuan Cai"
  - "Jiayi Ma"
doi: "10.24963/ijcai.2025/246"
url: "https://ijcai.org/proceedings/2025/246"
pdf: "https://ijcai.org/proceedings/2025/0246.pdf"
venue: "Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence"
year: 2025
pages: "2206-2214"
track: "Main Track"
keywords:
  - "Computer Vision: CV: Transfer, low-shot, semi- and un- supervised learning"
  - "Computer Vision: CV: Machine learning for vision"
  - "Computer Vision: CV: Multimodal learning"
  - "Computer Vision: CV: Low-level Vision"
---

# Multimodal Image Matching Based on Cross-Modality Completion Pre-training

Meng Yang, Fan Fan, Jun Huang, Yong Ma, Xiaoguang Mei, Zhanchuan Cai, Jiayi Ma

*Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence*, pages 2206-2214. DOI: [10.24963/ijcai.2025/246](https://doi.org/10.24963/ijcai.2025/246)

## Abstract

The differences in imaging devices cause multimodal images to have modal differences and geometric distortions, complicating the matching task. Deep learning-based matching methods struggle with multimodal images due to the lack of large annotated multimodal datasets. To address these challenges, we propose XCP-Match based on cross-modality completion pre-training. XCP-Match has two phases. (1) Self-supervised cross-modality completion pre-training based on real multimodal image dataset. We develop a novel pre-training model to learn cross-modal semantic features. The pre-training uses masked image modeling method for cross-modality completion, and introduces an attention-weighted contrastive loss to emphasize matching in overlapping areas. (2) Supervised fine-tuning for multimodal image matching based on the augmented MegaDepth dataset. XCP-Match constructs a complete matching framework to overcome geometric distortions and achieve precise matching. Two-phase training encourages the model to learn deep cross-modal semantic information, improving adaptation to modal differences without needing large annotated datasets. Experiments demonstrate that XCP-Match outperforms existing algorithms on public datasets.