---
title: "DoMoE: Domain-Aware Semantic Expert Prediction for Efficient MoE Inference Under Expert Offloading"
authors:
  - "Yao Mu"
  - "Fahao Chen"
  - "Wenbin Zhu"
  - "Mengying Zhao"
  - "Zhaoyan Shen"
  - "Dongxiao Yu"
doi: "10.24963/ijcai.2026/657"
url: "https://ijcai.org/proceedings/2026/657"
pdf: "https://ijcai.org/proceedings/2026/0657.pdf"
venue: "Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence"
year: 2026
pages: "5901-5909"
track: "Main Track"
keywords:
  - "Natural Language Processing: Language models"
  - "Planning and Scheduling: Planning algorithms"
  - "Planning and Scheduling: Scheduling"
---

# DoMoE: Domain-Aware Semantic Expert Prediction for Efficient MoE Inference Under Expert Offloading

Yao Mu, Fahao Chen, Wenbin Zhu, Mengying Zhao, Zhaoyan Shen, Dongxiao Yu

*Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence*, pages 5901-5909. DOI: [10.24963/ijcai.2026/657](https://doi.org/10.24963/ijcai.2026/657)

## Abstract

Mixture-of-Experts (MoE) large language models improve inference efficiency through sparse expert activation, but deployment on resource-constrained devices remains challenging due to the large expert parameter footprint. Expert offloading mitigates this issue by loading experts on demand, yet its effectiveness critically depends on accurate and efficient expert prediction: inaccurate predictions incur redundant expert transfers, while overly expensive predictors negate latency benefits. Existing trajectory-based methods suffer from low accuracy, whereas semantic-based approaches incur prohibitive similarity-matching overhead. We propose DoMoE, a domain-aware MoE inference system that exploits domain locality in inference workloads. DoMoE organizes routing information into domain-specific expert routing tables, restricts semantic matching to domain-relevant tokens and explicitly balances prediction accuracy against prediction overhead. Experiments show that DoMoE achieves a 1.32 X average throughput improvement and a 1.22 X increase in expert hit ratio across multiple MoE models and workloads, enabling efficient and accurate expert routing for practical inference.