Uncertainty-Guided Adaptive Conservative Offline Reinforcement Learning for Safer Mechanical Ventilation

Liu, Huidong; Yu, Hang; Zhang, Qiyang; Dou, Jiarui; Long, Xianlei; Shi, Jiantao; Gu, Fuqiang

doi:10.24963/ijcai.2026/755

Uncertainty-Guided Adaptive Conservative Offline Reinforcement Learning for Safer Mechanical Ventilation

Huidong Liu, Hang Yu, Qiyang Zhang, Jiarui Dou, Xianlei Long, Jiantao Shi, Fuqiang Gu

Download BibTeX Cite

Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence. AI and Health. Pages 6788-6796.

https://doi.org/10.24963/ijcai.2026/755

Keywords

Precision medicine: Precision medicine
Medical diagnosis: Medical diagnosis
Clinical decision support system: Clinical decision support system
Treatment recommendation: Treatment recommendation

Abstract

Mechanical ventilation (MV) is essential in intensive care units (ICUs), yet conventional protocols lack personalization and risk harmful over- or under-ventilation. Offline reinforcement learning (ORL) enables policy optimization from retrospective clinical data without unsafe online interaction, but existing methods are highly sensitive to distributional shift and out-of-distribution (OOD) actions, limiting their reliability in complex clinical settings. To address these challenges, We propose UBER-CQL (Uncertainty-Balanced Exploration and Robust Conservative Q-Learning), a robust ORL algorithm for safe decision-making under dataset shift. UBER-CQL integrates heteroscedastic Bayesian neural networks with conservative Q-learning to model posterior Q-value uncertainty, which is used to adaptively penalize unreliable high-risk actions while maintaining performance within the data support. We further design numerically stable objectives for conservative Bayesian value estimation. Experiments on in-distribution and OOD subsets of MIMIC-III and eICU demonstrate that UBER-CQL outperforms state-of-the-art ORL and clinician baselines, producing safer and more effective MV strategies.

Huidong Liu, Hang Yu, Qiyang Zhang, Jiarui Dou, Xianlei Long, Jiantao Shi, and Fuqiang Gu. 2026. Uncertainty-Guided Adaptive Conservative Offline Reinforcement Learning for Safer Mechanical Ventilation. In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26. IJCAI, 6788–6796. https://doi.org/10.24963/ijcai.2026/755

@inproceedings{ijcai2026p755,
  title     = {Uncertainty-Guided Adaptive Conservative Offline Reinforcement Learning for Safer Mechanical Ventilation},
  author    = {Liu, Huidong and Yu, Hang and Zhang, Qiyang and Dou, Jiarui and Long, Xianlei and Shi, Jiantao and Gu, Fuqiang},
  booktitle = {Proceedings of the Thirty-Fifth International Joint Conference on
               Artificial Intelligence, {IJCAI-26}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Diego Calvanese},
  pages     = {6788--6796},
  year      = {2026},
  month     = {8},
  note      = {AI and Health},
  doi       = {10.24963/ijcai.2026/755},
  url       = {https://doi.org/10.24963/ijcai.2026/755},
}

TY  - CPAPER
TI  - Uncertainty-Guided Adaptive Conservative Offline Reinforcement Learning for Safer Mechanical Ventilation
AU  - Liu, Huidong
AU  - Yu, Hang
AU  - Zhang, Qiyang
AU  - Dou, Jiarui
AU  - Long, Xianlei
AU  - Shi, Jiantao
AU  - Gu, Fuqiang
T2  - Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26
PB  - International Joint Conferences on Artificial Intelligence Organization
PY  - 2026
SP  - 6788
EP  - 6796
DO  - 10.24963/ijcai.2026/755
UR  - https://doi.org/10.24963/ijcai.2026/755
ER  -