{"type":"ScholarlyArticle","title":"DoMoE: Domain-Aware Semantic Expert Prediction for Efficient MoE Inference Under Expert Offloading","authors":[{"name":"Yao Mu","slug":"yao-mu","given":"Yao","family":"Mu","affiliation":""},{"name":"Fahao Chen","slug":"fahao-chen","given":"Fahao","family":"Chen","affiliation":""},{"name":"Wenbin Zhu","slug":"wenbin-zhu","given":"Wenbin","family":"Zhu","affiliation":""},{"name":"Mengying Zhao","slug":"mengying-zhao","given":"Mengying","family":"Zhao","affiliation":""},{"name":"Zhaoyan Shen","slug":"zhaoyan-shen","given":"Zhaoyan","family":"Shen","affiliation":""},{"name":"Dongxiao Yu","slug":"dongxiao-yu","given":"Dongxiao","family":"Yu","affiliation":""}],"abstract":"Mixture-of-Experts (MoE) large language models improve inference efficiency through sparse expert activation, but deployment on resource-constrained devices remains challenging due to the large expert parameter footprint. Expert offloading mitigates this issue by loading experts on demand, yet its effectiveness critically depends on accurate and efficient expert prediction: inaccurate predictions incur redundant expert transfers, while overly expensive predictors negate latency benefits. Existing trajectory-based methods suffer from low accuracy, whereas semantic-based approaches incur prohibitive similarity-matching overhead. We propose DoMoE, a domain-aware MoE inference system that exploits domain locality in inference workloads. DoMoE organizes routing information into domain-specific expert routing tables, restricts semantic matching to domain-relevant tokens and explicitly balances prediction accuracy against prediction overhead. Experiments show that DoMoE achieves a 1.32 X average throughput improvement and a 1.22 X increase in expert hit ratio across multiple MoE models and workloads, enabling efficient and accurate expert routing for practical inference.","doi":"10.24963/ijcai.2026/657","url":"https://ijcai.org/proceedings/2026/657","pdf":"https://ijcai.org/proceedings/2026/0657.pdf","markdown":"https://ijcai.org/proceedings/2026/657.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/657","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Natural Language Processing","track":"Main Track","number":657,"pages":{"first":5901,"last":5909},"keywords":["Natural Language Processing: Language models","Planning and Scheduling: Planning algorithms","Planning and Scheduling: Scheduling"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}