{"type":"ScholarlyArticle","title":"Multi-granularity Knowledge Transfer for Continual Reinforcement Learning","authors":[{"name":"Chaofan Pan","slug":"chaofan-pan","given":"Chaofan","family":"Pan","affiliation":""},{"name":"Lingfei Ren","slug":"lingfei-ren","given":"Lingfei","family":"Ren","affiliation":""},{"name":"Yihui Feng","slug":"yihui-feng","given":"Yihui","family":"Feng","affiliation":""},{"name":"Linbo Xiong","slug":"linbo-xiong","given":"Linbo","family":"Xiong","affiliation":""},{"name":"Wei Wei","slug":"wei-wei","given":"Wei","family":"Wei","affiliation":""},{"name":"Yonghao Li","slug":"yonghao-li","given":"Yonghao","family":"Li","affiliation":""},{"name":"Xin Yang","slug":"xin-yang","given":"Xin","family":"Yang","affiliation":""}],"abstract":"Continual reinforcement learning (CRL) empowers RL agents with the ability to learn a sequence of tasks, accumulating knowledge learned in the past and using the knowledge for problemsolving or future task learning. However, existing methods often focus on transferring fine-grained knowledge across similar tasks, which neglects the multi-granularity structure of human cognitive control, resulting in insufficient knowledge transfer across diverse tasks. To enhance coarse-grained knowledge transfer, we propose a novel framework called MT-Core (as shorthand for Multi-granularity knowledge Transfer for Continual reinforcement learning). MT-Core has a key characteristic of multi-granularity policy learning: 1) a coarsegrained policy formulation for utilizing the powerful reasoning ability of the large language model (LLM) to set goals, and 2) a fine-grained policy learning through RL which is oriented by the goals. We also construct a new policy library (knowledge base) to store policies that can be retrieved for multi-granularity knowledge transfer. Experimental results demonstrate the superiority of the proposed MT-Core in handling diverse CRL tasks versus popular baselines.","doi":"10.24963/ijcai.2025/669","url":"https://ijcai.org/proceedings/2025/669","pdf":"https://ijcai.org/proceedings/2025/0669.pdf","markdown":"https://ijcai.org/proceedings/2025/669.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/669","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Machine Learning","track":"Main Track","number":669,"pages":{"first":6012,"last":6020},"keywords":["Natural Language Processing: NLP: Language models","Machine Learning: ML: Learnware/model reuse/transfer learning","Machine Learning: ML: Online learning","Machine Learning: ML: Multi-task and transfer learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}