{"type":"ScholarlyArticle","title":"Self-Consistent Model-based Adaptation for Visual Reinforcement Learning","authors":[{"name":"Xinning Zhou","slug":"xinning-zhou","given":"Xinning","family":"Zhou","affiliation":""},{"name":"Chengyang Ying","slug":"chengyang-ying","given":"ChengYang","family":"Ying","affiliation":""},{"name":"Yao Feng","slug":"yao-feng","given":"Yao","family":"Feng","affiliation":""},{"name":"Hang Su","slug":"hang-su","given":"Hang","family":"Su","affiliation":""},{"name":"Jun Zhu","slug":"jun-zhu","given":"Jun","family":"Zhu","affiliation":""}],"abstract":"Visual reinforcement learning agents typically face serious performance declines in real-world applications caused by visual distractions. Existing methods rely on fine-tuning the policy's representations with hand-crafted augmentations. In this work, we propose Self-Consistent Model-based Adaptation (SCMA), a novel method that fosters robust adaptation without modifying the policy. By transferring cluttered observations to clean ones with a denoising model, SCMA can mitigate distractions for various policies as a plug-and-play enhancement. To optimize the denoising model in an unsupervised manner, we derive an unsupervised distribution matching objective with a theoretical analysis of its optimality. We further present a practical algorithm to optimize the objective by estimating the distribution of clean observations with a pre-trained world model. Extensive experiments on multiple visual generalization benchmarks and real robot data demonstrate that SCMA effectively boosts performance across various distractions and exhibits better sample efficiency.","doi":"10.24963/ijcai.2025/800","url":"https://ijcai.org/proceedings/2025/800","pdf":"https://ijcai.org/proceedings/2025/0800.pdf","markdown":"https://ijcai.org/proceedings/2025/800.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/800","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Machine Learning","track":"Main Track","number":800,"pages":{"first":7191,"last":7199},"keywords":["Machine Learning: ML: Reinforcement learning","Machine Learning: ML: Robustness","Machine Learning: ML: Model-based and model learning reinforcement learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}