{"type":"ScholarlyArticle","title":"DisPIM: Distilling PreTrained Image Models for Generalizable Visuo-Motor Control","authors":[{"name":"Haitao Wang","slug":"haitao-wang","given":"Haitao","family":"Wang","affiliation":""},{"name":"Hejun Wu","slug":"hejun-wu","given":"Hejun","family":"Wu","affiliation":""}],"abstract":"We introduce DisPIM, a framework that leverages pretrained image models (PIMs) for visuo-motor control. Applying PIMs to visuo-motor control faces a big difficulty due to the distribution shift between the distribution of visual environmental states and that of the pretraining datasets. Due to such a distribution shift, fine-tuning PIMs specifically for visuo-motor control may hurt the generalizability of PIMs, while adding additional tunable parameters for specific actions apparently lead to high computational costs. DisPIM addresses these challenges using a novel feature distillation approach, which obtains a compact model that not only inherit the generalization capability of PIMs but also acquire task-specific skills for visuo-motor control. This good for both sides is mainly achieved by means of a target Q-ensemble mechanism, which is inspired by double Q-learning. This Q-ensemble mechanism can adaptively adjust the distillation rate, so as to balance the objective of generalization and task-specific ability during training. With this balancing mechanism, DisPIM achieves both task-specific and generalizable control requiring a low computation cost. Across a series of algorithms, task domains, and evaluation metrics in both simulation and real robot, our DisPIM demonstrates significant improvements in generalization and overall performance with low computational overhead.","doi":"10.24963/ijcai.2025/978","url":"https://ijcai.org/proceedings/2025/978","pdf":"https://ijcai.org/proceedings/2025/0978.pdf","markdown":"https://ijcai.org/proceedings/2025/978.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/978","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Robotics","track":"Main Track","number":978,"pages":{"first":8796,"last":8804},"keywords":["Robotics: ROB: Behavior and control","Robotics: ROB: Robotics and vision","Robotics: ROB: Manipulation"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}