{"type":"ScholarlyArticle","title":"MSVIT: Improving Spiking Vision Transformer Using Multi-scale Attention Fusion","authors":[{"name":"Wei Hua","slug":"wei-hua","given":"Wei","family":"Hua","affiliation":""},{"name":"Chenlin Zhou","slug":"chenlin-zhou","given":"Chenlin","family":"Zhou","affiliation":""},{"name":"Jibin Wu","slug":"jibin-wu","given":"Jibin","family":"Wu","affiliation":""},{"name":"Yansong Chua","slug":"yansong-chua","given":"Yansong","family":"Chua","affiliation":""},{"name":"Yangyang Shu","slug":"yangyang-shu","given":"Yangyang","family":"Shu","affiliation":""}],"abstract":"The combination of Spiking Neural Networks (SNNs) with Vision Transformer architectures has attracted significant attention due to the great potential for energy-efficient and high-performance computing paradigms. However, a substantial performance gap still exists between SNN-based and ANN-based transformer architectures. While existing methods propose spiking self-attention mechanisms that are successfully combined with SNNs, the overall architectures proposed by these methods suffer from a bottleneck in effectively extracting features from different image scales. In this paper, we address this issue and propose MSVIT, a novel spike-driven Transformer architecture, which firstly uses multi-scale spiking attention (MSSA) to enrich the capability of spiking attention blocks. We validate our approach across various main data sets. The experimental results indicate that our MSVIT outperforms existing SNN-based models, positioning itself as a state-of-the-art solution among NN-transformer architectures. The codes are available at https://github.com/Nanhu-AI-Lab/MSViT.","doi":"10.24963/ijcai.2025/601","url":"https://ijcai.org/proceedings/2025/601","pdf":"https://ijcai.org/proceedings/2025/0601.pdf","markdown":"https://ijcai.org/proceedings/2025/601.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/601","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Machine Learning","track":"Main Track","number":601,"pages":{"first":5399,"last":5407},"keywords":["Computer Vision: CV: Efficiency and Optimization","Machine Learning: ML: Multi-modal learning","Computer Vision: CV: Representation learning","Machine Learning: ML: Attention models"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}