{"type":"ScholarlyArticle","title":"CoLA-Former: Graph Transformer Using Communal Linear Attention for Lightweight Sequential Recommendation","authors":[{"name":"Zhongying Zhao","slug":"zhongying-zhao","given":"Zhongying","family":"Zhao","affiliation":""},{"name":"Jinyu Zhang","slug":"jinyu-zhang","given":"Jinyu","family":"Zhang","affiliation":""},{"name":"Chuanxu Jia","slug":"chuanxu-jia","given":"Chuanxu","family":"Jia","affiliation":""},{"name":"Chao Li","slug":"chao-li","given":"Chao","family":"Li","affiliation":""},{"name":"Yanwei Yu","slug":"yanwei-yu","given":"Yanwei","family":"Yu","affiliation":""},{"name":"Qingtian Zeng","slug":"qingtian-zeng","given":"Qingtian","family":"Zeng","affiliation":""}],"abstract":"Graph Transformer has shown great promise in capturing the dynamics of user preferences for sequential recommendations. However, the self-attention mechanism within its structure is of quadratic complexity, posing challenges for deployment on devices with limited resources. To this end, we propose a Communal Linear Attention-enhanced Graph TransFormer for lightweight sequential recommendation, namely CoLA-Former. Specifically, we introduce a Communal Linear Attention (CoLAttention) mechanism. It utilizes low-rank yet reusable communal units to calculate the global correlations on sequential graphs. The weights from the units are also made communal across different training batches, enabling inter-batch global weighting. Moreover, we devise a low-rank approximation component. It utilizes weights distillation to reduce the scale of the trainable parameters in the Graph Transformer network. Extensive experimental results on three real-world datasets demonstrate that the proposed CoLA-Former significantly outperforms twelve state-of-the-art methods in accuracy and efficiency. The datasets and codes are available at https://github.com/ZZY-GraphMiningLab/CoLA_Former.","doi":"10.24963/ijcai.2025/410","url":"https://ijcai.org/proceedings/2025/410","pdf":"https://ijcai.org/proceedings/2025/0410.pdf","markdown":"https://ijcai.org/proceedings/2025/410.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/410","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Data Mining","track":"Main Track","number":410,"pages":{"first":3689,"last":3697},"keywords":["Data Mining: DM: Recommender systems","Data Mining: DM: Mining graphs"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}