{"type":"ScholarlyArticle","title":"DNFormer: Differential Attention for Graph Transformers","authors":[{"name":"Zizhen Wang","slug":"zizhen-wang","given":"Zizhen","family":"Wang","affiliation":""},{"name":"Dongxiao He","slug":"dongxiao-he","given":"Dongxiao","family":"He","affiliation":""},{"name":"Zhizhi Yu","slug":"zhizhi-yu","given":"Zhizhi","family":"Yu","affiliation":""},{"name":"Kuntharrgyal Khysru","slug":"kuntharrgyal-khysru","given":"Kuntharrgyal","family":"Khysru","affiliation":""},{"name":"Weixiong Zhang","slug":"weixiong-zhang","given":"Weixiong","family":"Zhang","affiliation":""}],"abstract":"Graph Transformers (GTs) have emerged as a powerful paradigm for graph representation learning, leveraging attention mechanisms to enable flexible information exchange. Recent GTs often adopt local attention to restrict computation to neighborhoods, thereby reducing costs and enhancing scalability. However, local attention computation poses a serious issue that significantly reduces the efficacy of the overall approaches – because computation is restricted to local neighborhoods, node pairs that share topological patterns receive similar attention scores, thereby reducing attention’s discriminative power. Consequently, GT-based models typically overemphasize common structural motifs but fail to capture node-specific properties. To address this problem, we propose DNFormer, a novel GT grounded in differential modeling. We introduce a differential local attention mechanism in DNFormer that computes attention scores as differences across multiple topology-aware similarity measures, rather than relying on a single score. This mechanism suppresses shared topological patterns among neighbors and emphasizes relative node-to-node distinctions. As a result, DNFormer mitigates structural dominance in attention aggregation, better capturing node-specific relational patterns while preserving local structural properties. Extensive experiments confirm DNFormer's superior performance over representative GT baselines.","doi":"10.24963/ijcai.2026/344","url":"https://ijcai.org/proceedings/2026/344","pdf":"https://ijcai.org/proceedings/2026/0344.pdf","markdown":"https://ijcai.org/proceedings/2026/344.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/344","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Data Mining","track":"Main Track","number":344,"pages":{"first":3097,"last":3105},"keywords":["Data Mining: Mining graphs"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}