Low-Latency Real-Time Audio Game Commentary System via LLM-Based Parallel Text Generation

Kawamatsu, Ryota; Afzal, Anum; Saito, Yuki; Takamichi, Shinnosuke; Neubig, Graham; Sudoh, Katsuhito; Takamura, Hiroya; Ishigaki, Tatsuya

doi:10.24963/ijcai.2026/972

Low-Latency Real-Time Audio Game Commentary System via LLM-Based Parallel Text Generation

Ryota Kawamatsu, Anum Afzal, Yuki Saito, Shinnosuke Takamichi, Graham Neubig, Katsuhito Sudoh, Hiroya Takamura, Tatsuya Ishigaki

Download BibTeX Cite

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. Demo Track. Pages 8427-8431.

https://doi.org/10.24963/ijcai.2026/972

Keywords

AI: Multidisciplinary Topics and Applications
AI: Natural Language Processing

Abstract

We present a low-latency real-time audio game commentary system that generates spoken commentary directly from live gameplay video. In this end-to-end setting, a key bottleneck is accumulated waiting time; conventional pipelines capture frames, generate text, and synthesize speech sequentially for each utterance, and do not request the next generation until speech playback has completed. This strict sequentiality causes long and unnatural silence between utterances. To address this latency bottleneck, our system runs text generation in parallel with speech playback and buffers multiple candidate utterances ahead of time, enabling immediate synthesis at playback boundaries. Experiments on fast-paced game videos show that our parallel design reduces the mean inter-utterance silence from 9.6 seconds to 0.3 seconds compared to sequential baselines. It also improves similarity to professional speaking--silence timing patterns by over 40 %, and a user study with 120 experienced game players confirms significantly improved perceived speaking rhythm. Our demo video is available at: https://youtu.be/pmrRUlvav8M.

Ryota Kawamatsu, Anum Afzal, Yuki Saito, Shinnosuke Takamichi, Graham Neubig, Katsuhito Sudoh, Hiroya Takamura, and Tatsuya Ishigaki. 2026. Low-Latency Real-Time Audio Game Commentary System via LLM-Based Parallel Text Generation. In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26. IJCAI, 8427–8431. https://doi.org/10.24963/ijcai.2026/972

@inproceedings{ijcai2026p972,
  title     = {Low-Latency Real-Time Audio Game Commentary System via LLM-Based Parallel Text Generation},
  author    = {Kawamatsu, Ryota and Afzal, Anum and Saito, Yuki and Takamichi, Shinnosuke and Neubig, Graham and Sudoh, Katsuhito and Takamura, Hiroya and Ishigaki, Tatsuya},
  booktitle = {Proceedings of the Thirty-Fifth International Joint Conference on
               Artificial Intelligence, {IJCAI-26}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Diego Calvanese},
  pages     = {8427--8431},
  year      = {2026},
  month     = {8},
  note      = {Demo Track},
  doi       = {10.24963/ijcai.2026/972},
  url       = {https://doi.org/10.24963/ijcai.2026/972},
}

TY  - CPAPER
TI  - Low-Latency Real-Time Audio Game Commentary System via LLM-Based Parallel Text Generation
AU  - Kawamatsu, Ryota
AU  - Afzal, Anum
AU  - Saito, Yuki
AU  - Takamichi, Shinnosuke
AU  - Neubig, Graham
AU  - Sudoh, Katsuhito
AU  - Takamura, Hiroya
AU  - Ishigaki, Tatsuya
T2  - Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2026
SP  - 8427
EP  - 8431
DO  - 10.24963/ijcai.2026/972
UR  - https://doi.org/10.24963/ijcai.2026/972
ER  -