Proceedings

Self-Consistent Model-based Adaptation for Visual Reinforcement Learning

Xinning Zhou, Chengyang Ying, Yao Feng, Hang Su, Jun Zhu

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 7191-7199.

https://doi.org/10.24963/ijcai.2025/800

Keywords

  • Machine Learning: ML: Reinforcement learning
  • Machine Learning: ML: Robustness
  • Machine Learning: ML: Model-based and model learning reinforcement learning

Abstract

Visual reinforcement learning agents typically face serious performance declines in real-world applications caused by visual distractions. Existing methods rely on fine-tuning the policy's representations with hand-crafted augmentations. In this work, we propose Self-Consistent Model-based Adaptation (SCMA), a novel method that fosters robust adaptation without modifying the policy. By transferring cluttered observations to clean ones with a denoising model, SCMA can mitigate distractions for various policies as a plug-and-play enhancement. To optimize the denoising model in an unsupervised manner, we derive an unsupervised distribution matching objective with a theoretical analysis of its optimality. We further present a practical algorithm to optimize the objective by estimating the distribution of clean observations with a pre-trained world model. Extensive experiments on multiple visual generalization benchmarks and real robot data demonstrate that SCMA effectively boosts performance across various distractions and exhibits better sample efficiency.

Xinning Zhou, ChengYang Ying, Yao Feng, Hang Su, and Jun Zhu. 2025. Self-Consistent Model-based Adaptation for Visual Reinforcement Learning. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25. IJCAI, 7191–7199. https://doi.org/10.24963/ijcai.2025/800
Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards