Efficient and Exact Global Attention on Latent Summaries for Knowledge Graph Reasoning

Abstract

Capturing global context through attention is essential for reasoning over knowledge graphs, especially when relevant entities are distant or disconnected. To scale attention to large graphs, recent methods replace Softmax with kernel feature mappings, reducing computational complexity to linear in the number of nodes. While efficient, these approximations tend to produce overly smooth attention scores, which can reduce discrimination between correct triplets and hard negative samples. Moreover, they exhibit scale-dependent shifts in attention entropy, making them sensitive to changes in graph size during inductive inference. In this paper, we introduce LaGR, a novel approach for integrating global information in knowledge graph reasoning. Rather than approximating interactions among all nodes, LaGR compresses the graph into a fixed, compact set of latent summaries and applies exact self-attention within this latent space. This change yields scale-invariant attention and stable performance across diverse data settings. In addition, we propose a node-adaptive residual fusion mechanism that dynamically balances local and global information at the node level, leading to more expressive representations. Extensive experiments on both transductive and inductive benchmarks show that LaGR substantially outperforms state-of-the-art baselines, demonstrating that exact attention over latent summaries is an efficient and effective way to capture global context in knowledge graph reasoning. Our implementation is available at https://github.com/XMU-KG/LaGR.