CoLA-Former: Graph Transformer Using Communal Linear Attention for Lightweight Sequential Recommendation

Zhao, Zhongying; Zhang, Jinyu; Jia, Chuanxu; Li, Chao; Yu, Yanwei; Zeng, Qingtian

doi:10.24963/ijcai.2025/410

CoLA-Former: Graph Transformer Using Communal Linear Attention for Lightweight Sequential Recommendation

Zhongying Zhao, Jinyu Zhang, Chuanxu Jia, Chao Li, Yanwei Yu, Qingtian Zeng

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 3689-3697.

https://doi.org/10.24963/ijcai.2025/410

Keywords

Data Mining: DM: Recommender systems
Data Mining: DM: Mining graphs

Abstract

Graph Transformer has shown great promise in capturing the dynamics of user preferences for sequential recommendations. However, the self-attention mechanism within its structure is of quadratic complexity, posing challenges for deployment on devices with limited resources. To this end, we propose a Communal Linear Attention-enhanced Graph TransFormer for lightweight sequential recommendation, namely CoLA-Former. Specifically, we introduce a Communal Linear Attention (CoLAttention) mechanism. It utilizes low-rank yet reusable communal units to calculate the global correlations on sequential graphs. The weights from the units are also made communal across different training batches, enabling inter-batch global weighting. Moreover, we devise a low-rank approximation component. It utilizes weights distillation to reduce the scale of the trainable parameters in the Graph Transformer network. Extensive experimental results on three real-world datasets demonstrate that the proposed CoLA-Former significantly outperforms twelve state-of-the-art methods in accuracy and efficiency. The datasets and codes are available at https://github.com/ZZY-GraphMiningLab/CoLA_Former.

Zhongying Zhao, Jinyu Zhang, Chuanxu Jia, Chao Li, Yanwei Yu, and Qingtian Zeng. 2025. CoLA-Former: Graph Transformer Using Communal Linear Attention for Lightweight Sequential Recommendation. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25. IJCAI, 3689–3697. https://doi.org/10.24963/ijcai.2025/410

@inproceedings{ijcai2025p410,
  title     = {CoLA-Former: Graph Transformer Using Communal Linear Attention for Lightweight Sequential Recommendation},
  author    = {Zhao, Zhongying and Zhang, Jinyu and Jia, Chuanxu and Li, Chao and Yu, Yanwei and Zeng, Qingtian},
  booktitle = {Proceedings of the Thirty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-25}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {James Kwok},
  pages     = {3689--3697},
  year      = {2025},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2025/410},
  url       = {https://doi.org/10.24963/ijcai.2025/410},
}

TY  - CPAPER
TI  - CoLA-Former: Graph Transformer Using Communal Linear Attention for Lightweight Sequential Recommendation
AU  - Zhao, Zhongying
AU  - Zhang, Jinyu
AU  - Jia, Chuanxu
AU  - Li, Chao
AU  - Yu, Yanwei
AU  - Zeng, Qingtian
T2  - Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2025
SP  - 3689
EP  - 3697
DO  - 10.24963/ijcai.2025/410
UR  - https://doi.org/10.24963/ijcai.2025/410
ER  -