UniCT Depth: Event-Image Fusion Based Monocular Depth Estimation with Convolution-Compensated ViT Dual SA Block

Jing, Luoxi; Shi, Dianxi; Liu, Zhe; Jin, Songchang; Qiu, Chunping; Qiao, Ziteng; Li, Yuxian; Xia, Jianqiang

doi:10.24963/ijcai.2025/144

UniCT Depth: Event-Image Fusion Based Monocular Depth Estimation with Convolution-Compensated ViT Dual SA Block

Luoxi Jing, Dianxi Shi, Zhe Liu, Songchang Jin, Chunping Qiu, Ziteng Qiao, Yuxian Li, Jianqiang Xia

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 1287-1295.

https://doi.org/10.24963/ijcai.2025/144

Keywords

Computer Vision: CV: Low-level Vision
Computer Vision: CV: Scene analysis and understanding
Machine Learning: ML: Multi-modal learning
Computer Vision: CV: Multimodal learning

Abstract

Depth estimation plays a crucial role in 3D scene understanding and is extensively used in a wide range of vision tasks. Image-based methods struggle in challenging scenarios, while event cameras offer high dynamic range and temporal resolution but face difficulties with sparse data. Combining event and image data provides significant advantages, yet effective integration remains challenging. Existing CNN-based fusion methods struggle with occlusions and depth disparities due to limited receptive fields, while Transformer-based fusion methods often lack deep modality interaction. To address these issues, we propose UniCT Depth, an event-image fusion method that unifies CNNs and Transformers to model local and global features. We propose the Convolution-compensated ViT Dual SA (CcViT-DA) Block, designed for the encoder, which integrates Context Modeling Self-Attention (CMSA) to capture spatial dependencies and Modal Fusion Self-Attention (MFSA) for effective cross-modal fusion. Furthermore, we design the tailored Detail Compensation Convolution (DCC) Block to improve texture details and enhances edge representations. Extensive experiments show that UniCT Depth outperforms existing image, event, and fusion-based monocular depth estimation methods across key metrics.

Luoxi Jing, Dianxi Shi, Zhe Liu, Songchang Jin, Chunping Qiu, Ziteng Qiao, Yuxian Li, and Jianqiang Xia. 2025. UniCT Depth: Event-Image Fusion Based Monocular Depth Estimation with Convolution-Compensated ViT Dual SA Block. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25. IJCAI, 1287–1295. https://doi.org/10.24963/ijcai.2025/144

@inproceedings{ijcai2025p144,
  title     = {UniCT Depth: Event-Image Fusion Based Monocular Depth Estimation with Convolution-Compensated ViT Dual SA Block},
  author    = {Jing, Luoxi and Shi, Dianxi and Liu, Zhe and Jin, Songchang and Qiu, Chunping and Qiao, Ziteng and Li, Yuxian and Xia, Jianqiang},
  booktitle = {Proceedings of the Thirty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-25}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {James Kwok},
  pages     = {1287--1295},
  year      = {2025},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2025/144},
  url       = {https://doi.org/10.24963/ijcai.2025/144},
}

TY  - CPAPER
TI  - UniCT Depth: Event-Image Fusion Based Monocular Depth Estimation with Convolution-Compensated ViT Dual SA Block
AU  - Jing, Luoxi
AU  - Shi, Dianxi
AU  - Liu, Zhe
AU  - Jin, Songchang
AU  - Qiu, Chunping
AU  - Qiao, Ziteng
AU  - Li, Yuxian
AU  - Xia, Jianqiang
T2  - Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2025
SP  - 1287
EP  - 1295
DO  - 10.24963/ijcai.2025/144
UR  - https://doi.org/10.24963/ijcai.2025/144
ER  -