---
title: "CoLA-Former: Graph Transformer Using Communal Linear Attention for Lightweight Sequential Recommendation"
authors:
  - "Zhongying Zhao"
  - "Jinyu Zhang"
  - "Chuanxu Jia"
  - "Chao Li"
  - "Yanwei Yu"
  - "Qingtian Zeng"
doi: "10.24963/ijcai.2025/410"
url: "https://ijcai.org/proceedings/2025/410"
pdf: "https://ijcai.org/proceedings/2025/0410.pdf"
venue: "Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence"
year: 2025
pages: "3689-3697"
track: "Main Track"
keywords:
  - "Data Mining: DM: Recommender systems"
  - "Data Mining: DM: Mining graphs"
---

# CoLA-Former: Graph Transformer Using Communal Linear Attention for Lightweight Sequential Recommendation

Zhongying Zhao, Jinyu Zhang, Chuanxu Jia, Chao Li, Yanwei Yu, Qingtian Zeng

*Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence*, pages 3689-3697. DOI: [10.24963/ijcai.2025/410](https://doi.org/10.24963/ijcai.2025/410)

## Abstract

Graph Transformer has shown great promise in capturing the dynamics of user preferences for sequential recommendations. However, the self-attention mechanism within its structure is of quadratic complexity, posing challenges for deployment on devices with limited resources. To this end, we propose a Communal Linear Attention-enhanced Graph TransFormer for lightweight sequential recommendation, namely CoLA-Former. Specifically, we introduce a Communal Linear Attention (CoLAttention) mechanism. It utilizes low-rank yet reusable communal units to calculate the global correlations on sequential graphs. The weights from the units are also made communal across different training batches, enabling inter-batch global weighting. Moreover, we devise a low-rank approximation component. It utilizes weights distillation to reduce the scale of the trainable parameters in the Graph Transformer network. Extensive experimental results on three real-world datasets demonstrate that the proposed CoLA-Former significantly outperforms twelve state-of-the-art methods in accuracy and efficiency. The datasets and codes are available at https://github.com/ZZY-GraphMiningLab/CoLA_Former.