---
title: "Going Beyond the Edge: Distributed Inference of Transformer Models on Ultra-Low-Power Wireless Devices"
authors:
  - "Alexander Gräfe"
  - "Ding Huo"
  - "Vincent de Bakker"
  - "Johannes Berger"
  - "Marco Zimmerling"
  - "Sebastian Trimpe"
doi: "10.24963/ijcai.2026/14"
url: "https://ijcai.org/proceedings/2026/14"
pdf: "https://ijcai.org/proceedings/2026/0014.pdf"
venue: "Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence"
year: 2026
pages: "118-126"
track: "Main Track"
keywords:
  - "Agent-based and Multi-agent Systems: Agent communication"
  - "Agent-based and Multi-agent Systems: Engineering methods, platforms, languages and tools"
---

# Going Beyond the Edge: Distributed Inference of Transformer Models on Ultra-Low-Power Wireless Devices

Alexander Gräfe, Ding Huo, Vincent de Bakker, Johannes Berger, Marco Zimmerling, Sebastian Trimpe

*Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence*, pages 118-126. DOI: [10.24963/ijcai.2026/14](https://doi.org/10.24963/ijcai.2026/14)

## Abstract

Transformer models are rapidly becoming a cornerstone of modern Internet of Things (IoT) applications, yet their computational and memory demands far exceed the capabilities of a single typical ultra-low-power IoT device. We present CATS, a framework for distributed transformer inference on ultra-low-power wireless devices, enabling multiple devices to collaboratively execute models far larger than what a single device can sustain. At its core, CATS is a communication-aware distributed transformer inference scheme co-designed across transformer partitioning, wireless communication and training. It employs SomeGather, a new pruned communication primitive that selectively broadcasts activation columns to reduce communication bandwidth and RAM usage without sacrificing model accuracy. Building on SomeGather, we design a partitioning method that exploits this primitive for efficient model parallelism. To cope with unreliable wireless communication, CATS employs message-dropout during training, which mimics packet losses and yields models that are robust to message loss during inference. In real-world experiments, we show that CATS brings distributed transformer inference to ultra-low-power wireless devices for the first time, with deployments on up to 16 devices that collaboratively execute transformer models up to 14 times larger than what a single device can run.