{"type":"ScholarlyArticle","title":"Efficient and Stable Offline-to-online Reinforcement Learning via Continual Policy Revitalization","authors":[{"name":"Rui Kong","slug":"rui-kong","given":"Rui","family":"Kong","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , China School of Artificial Intelligence ,"},{"name":"Chenyang Wu","slug":"chenyang-wu","given":"Chenyang","family":"Wu","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , China School of Artificial Intelligence ,"},{"name":"Chen-Xiao Gao","slug":"chen-xiao-gao","given":"Chen-Xiao","family":"Gao","affiliation":"National Key Laboratory for Novel Software Technology , Nanjing University , China School of Artificial Intelligence ,"},{"name":"Zongzhang Zhang","slug":"zongzhang-zhang","given":"Zongzhang","family":"Zhang","affiliation":"Nanjing University , China"},{"name":"Ming Li","slug":"ming-li","given":"Ming","family":"Li","affiliation":"Nanjing University , China"}],"abstract":"In offline Reinforcement Learning (RL), the pre-trained policies are utilized for initialization and subsequent online fine-tuning. However, existing methods suffer from instability and low sample efficiency compared to pure online learning. This paper identifies these limitations stemming from direct policy initialization using offline-trained policy models. We propose Continual Policy Revitalization (CPR) as a novel efficient, stable fine-tuning method. CPR incorporates a periodic policy revitalization technique, restoring the overtrained policy network to full learning capacity while ensuring stable initial performance. This approach enables fine-tuning without being adversely affected by low-quality pre-trained policies. In contrast to previous research, CPR initializes the new policy with an adaptive policy constraint in policy optimization. Such optimization keeps the new policy close to behavior policy constructed from historical policies. This contributes to stable policy improvement and optimal converged performance. Practically, CPR can seamlessly integrate into existing offline RL algorithms with minimal modification. We empirically validate the effectiveness of our method through extensive experiments, demonstrating substantial improvements in learning stability and efficiency compared to previous approaches. Our code is available at https://github.com/LAMDA-RL/CPR.","doi":"10.24963/ijcai.2024/477","url":"https://ijcai.org/proceedings/2024/477","pdf":"https://ijcai.org/proceedings/2024/0477.pdf","markdown":"https://ijcai.org/proceedings/2024/477.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/477","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":477,"pages":{"first":4317,"last":4325},"keywords":["Machine Learning: ML: Reinforcement learning","Machine Learning: ML: Offline reinforcement learning"],"videos":[],"references":[{"position":1,"raw":"[Abbas et al., 2023] Zaheer Abbas, Rosie Zhao, Joseph Mo- dayil, Adam White, and Marlos C. Machado. Loss of plasticity in continual deep reinforcement learning. arXiv preprint arXiv:2303.07507, 2023.","title":"Loss of plasticity in continual deep reinforcement learning","authors":"Zaheer Abbas, Rosie Zhao, Joseph Modayil, Adam White, Marlos C Machado","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Agarwal et al., 2022] Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron C. Courville, and Marc G. Bellemare. Reincarnating reinforcement learning: Reusing prior computation to accelerate progress. In Ad- vances in Neural Information Processing Systems, pages 28955-28971, 2022.","title":"Reincarnating reinforcement learning: Reusing prior computation to accelerate progress","authors":"Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron C Courville, Marc G Bellemare","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Beeson and Montana, 2022] Alex Beeson and Giovanni Montana. Improving TD3-BC: Relaxed policy constraint for offline learning and stable online fine-tuning. In Offline RL Workshop: Offline RL as a \"Launchpad\", 2022.","title":"Improving TD3-BC: Relaxed policy constraint for offline learning and stable online fine-tuning","authors":"Alex Beeson, Giovanni Montana","year":"2022","venue":"Offline RL Workshop: Offline RL as a \"Launchpad","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brockman et al., 2016] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. OpenAI Gym, 2016. Erhan et al., 2010]","title":"OpenAI Gym","authors":"Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, Wojciech Zaremba","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"Dumitru Erhan, Aaron C. Courville, Yoshua Bengio, and Pascal Vincent. Why does unsuper- vised pre-training help deep learning? In International Conference on Artificial Intelligence and Statistics, pages 201-208, 2010.","title":"Why does unsupervised pre-training help deep learning","authors":"Dumitru Erhan, Aaron C Courville, Yoshua Bengio, Pascal Vincent","year":"2010","venue":"International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Fu et al., 2020] Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4RL: datasets for deep data-driven reinforcement learning. arXiv preprint arXiv:2004.07219, 2020.","title":"D4RL: datasets for deep data-driven reinforcement learning","authors":"Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, Sergey Levine","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Fujimoto and Gu, 2021] Scott Fujimoto and Shixi- ang Shane Gu. A minimalist approach to offline re- inforcement learning. In Advances in Neural Information Processing Systems, pages 20132-20145, 2021.","title":"A minimalist approach to offline reinforcement learning","authors":"Gu Fujimoto, Scott Fujimoto, Shixiang Shane Gu","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fujimoto et al., 2018] Scott Fujimoto, Herke van Hoof, and David Meger. Addressing function approximation error in actor-critic methods. In International Conference on Machine Learning, pages 1582-1591, 2018.","title":"Addressing function approximation error in actor-critic methods","authors":"Scott Fujimoto, Herke Van Hoof, David Meger","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Fujimoto et al., 2019] Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without exploration. In International Conference on Ma- chine Learning, pages 2052-2062, 2019.","title":"Off-policy deep reinforcement learning without exploration","authors":"Scott Fujimoto, David Meger, Doina Precup","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gao and Rui, 2023] Chen-Xiao Gao and Kong Rui. OfflineRL-Lib: Benchmarked Implementations of Offline RL Algorithms, February 2023.","title":"OfflineRL-Lib: Benchmarked Implementations of Offline RL Algorithms","authors":"Chen-Xiao Gao, Kong Rui","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gao et al., 2024] Chen-Xiao Gao, Chenyang Wu, Mingjun Cao, Rui Kong, Zongzhang Zhang, and Yang Yu. ACT: Empowering decision transformer with dynamic program- ming via advantage conditioning. In AAAI Conference on Artificial Intelligence, pages 12127-12135, 2024.","title":"ACT: Empowering decision transformer with dynamic programming via advantage conditioning","authors":"Chen-Xiao Gao, Chenyang Wu, Mingjun Cao, Rui Kong, Zongzhang Zhang, Yang Yu","year":"2024","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Haarnoja et al., 2018] Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference on Machine Learning, pages 1856-1865, 2018.","title":"Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor","authors":"Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, Sergey Levine","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Harris et al., 2020] Charles R. Harris, K. Jarrod Millman, Stéfan van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. Array programming with NumPy. Nature, 585:357-362, 2020.","title":"Array programming with NumPy","authors":"Charles R Harris, K Jarrod Millman, Stéfan Van Der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H Van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández Del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, Travis E Oliphant","year":"2020","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kostrikov et al., 2022] Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with im- plicit Q-learning. In International Conference on Learning Representations, 2022.","title":"Offline reinforcement learning with implicit Q-learning","authors":"Kostrikov","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kumar et al., 2019] Aviral Kumar, Justin Fu, Matthew Soh, George Tucker, and Sergey Levine. Stabilizing off-policy Q-learning via bootstrapping error reduction. In Advances in Neural Information Processing Systems, pages 11761- 11771, 2019.","title":"Stabilizing off-policy Q-learning via bootstrapping error reduction","authors":"Aviral Kumar, Justin Fu, Matthew Soh, George Tucker, Sergey Levine","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lai et al., 2020] Kwei-Herng Lai, Daochen Zha, Yuening Li, and Xia Hu. Dual policy distillation. In International Joint Conference on Artificial Intelligence, pages 3146- 3152, 2020.","title":"Dual policy distillation","authors":"Kwei-Herng Lai, Daochen Zha, Yuening Li, Xia Hu","year":"2020","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lee et al., 2021] Seunghyun Lee, Younggyo Seo, Kimin Lee, Pieter Abbeel, and Jinwoo Shin. Offline-to-online re- inforcement learning via balanced replay and pessimistic q-ensemble. In Conference on Robot Learning, pages 1702-1712, 2021.","title":"Offline-to-online reinforcement learning via balanced replay and pessimistic q-ensemble","authors":"Seunghyun Lee, Younggyo Seo, Kimin Lee, Pieter Abbeel, Jinwoo Shin","year":"2021","venue":"Conference on Robot Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Levine et al., 2020] Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tu- torial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643, 2020.","title":"Offline reinforcement learning: Tutorial, review, and perspectives on open problems","authors":"Sergey Levine, Aviral Kumar, George Tucker, Justin Fu","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2023] Jianxiong Li, Xiao Hu, Haoran Xu, Jingjing Liu, Xianyuan Zhan, and Ya-Qin Zhang. PROTO: Iterative policy regularized offline-to-online reinforcement learn- ing. arXiv preprint arXiv:2305.15669, 2023.","title":"PROTO: Iterative policy regularized offline-to-online reinforcement learning","authors":"Jianxiong Li, Xiao Hu, Haoran Xu, Jingjing Liu, Xianyuan Zhan, Ya-Qin Zhang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mnih et al., 2016] Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asyn- chronous methods for deep reinforcement learning. In International Conference on Machine Learning, pages 1928-1937, 2016.","title":"Asynchronous methods for deep reinforcement learning","authors":"Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P Lillicrap, Tim Harley, David Silver, Koray Kavukcuoglu","year":"2016","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nair et al., 2020] Ashvin Nair, Murtaza Dalal, Abhishek Gupta, and Sergey Levine. Accelerating online rein- forcement learning with offline datasets. arXiv preprint arxiv:2006.09359, 2020.","title":"Accelerating online reinforcement learning with offline datasets","authors":"Ashvin Nair, Murtaza Dalal, Abhishek Gupta, Sergey Levine","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Nakamoto et al., 2023] Mitsuhiko Nakamoto, Yuexiang Zhai, Anikait Singh, Max Sobol Mark, Yi Ma, Chelsea Finn, Aviral Kumar, and Sergey Levine. Cal-QL: Calibrated offline RL pre-training for efficient online fine-tuning. arXiv preprint arXiv:2303.05479, 2023.","title":"Cal-QL: Calibrated offline RL pre-training for efficient online fine-tuning","authors":"Mitsuhiko Nakamoto, Yuexiang Zhai, Anikait Singh, Max Sobol Mark, Yi Ma, Chelsea Finn, Aviral Kumar, Sergey Levine","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Nikishin et al., 2022] Evgenii Nikishin, Max Schwarzer, Pierluca D'Oro, Pierre-Luc Bacon, and Aaron C. Courville. The primacy bias in deep reinforcement learn- ing. In International Conference on Machine Learning, pages 16828-16847, 2022.","title":"The primacy bias in deep reinforcement learning","authors":"Evgenii Nikishin, Max Schwarzer, D' Pierluca, Pierre-Luc Oro, Aaron C Bacon, Courville","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Pomerleau, 1991] Dean A Pomerleau. Efficient training of artificial neural networks for autonomous navigation. Neu- ral Computation, 3(1):88-97, 1991.","title":"Efficient training of artificial neural networks for autonomous navigation","authors":"A Dean, Pomerleau","year":"1991","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Ran et al., 2023] Yuhang Ran, Yi-Chen Li, Fuxiang Zhang, Zongzhang Zhang, and Yang Yu. Policy regularization with dataset constraint for offline reinforcement learning. In International Conference on Machine Learning, pages 28701-28717, 2023.","title":"Policy regularization with dataset constraint for offline reinforcement learning","authors":"Yuhang Ran, Yi-Chen Li, Fuxiang Zhang, Zongzhang Zhang, Yang Yu","year":"2023","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.","title":"Proximal policy optimization algorithms","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Tarasov et al., 2022] Denis Tarasov, Alexander Nikulin, Dmitry Akimov, Vladislav Kurenkov, and Sergey Kolesnikov. CORL: Research-oriented deep offline rein- forcement learning library. In Offline RL Workshop: Of- fline RL as a \"Launchpad\", 2022.","title":"CORL: Research-oriented deep offline reinforcement learning library","authors":"Denis Tarasov, Alexander Nikulin, Dmitry Akimov, Vladislav Kurenkov, Sergey Kolesnikov","year":"2022","venue":"Offline RL Workshop: Offline RL as a \"Launchpad","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Todorov et al., 2012] Emanuel Todorov, Tom Erez, and Yu- val Tassa. MuJoCo: A physics engine for model-based control. In IEEE/RSJ International Conference on Intelli- gent Robots and Systems, pages 5026-5033, 2012.","title":"MuJoCo: A physics engine for model-based control","authors":"Emanuel Todorov, Tom Erez, Yuval Tassa","year":"2012","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Uchendu et al., 2022] Ikechukwu Uchendu, Ted Xiao, Yao Lu, Banghua Zhu, Mengyuan Yan, Joséphine Simon, Matthew Bennice, Chuyuan Fu, Cong Ma, Jiantao Jiao, Sergey Levine, and Karol Hausman. Jump-start reinforce- ment learning. arXiv preprint arXiv:2204.02372, 2022.","title":"Jump-start reinforcement learning","authors":"Ikechukwu Uchendu, Ted Xiao, Yao Lu, Banghua Zhu, Mengyuan Yan, Joséphine Simon, Matthew Bennice, Chuyuan Fu, Cong Ma, Jiantao Jiao, Sergey Levine, Karol Hausman","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Wu and Zhang, 2023] Chenyang Wu and Zongzhang Zhang. Surfing information: The challenge of intelligent decision-making. Intelligent Computing, 2:0041, 2023.","title":"Surfing information: The challenge of intelligent decision-making","authors":"Zhang ; ] Chenyang Wu, Zongzhang Wu, Zhang","year":"2023","venue":"Intelligent Computing","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Xu et al., 2021] Tengyu Xu, Yingbin Liang, and Guanghui Lan. CRPO: A new approach for safe reinforcement learn- ing with convergence guarantee. In International Confer- ence on Machine Learning, pages 11480-11491, 2021.","title":"CRPO: A new approach for safe reinforcement learning with convergence guarantee","authors":"Xu","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Ye et al., 2021] Weirui Ye, Shaohuai Liu, Thanard Kuru- tach, Pieter Abbeel, and Yang Gao. Mastering Atari games with limited data. In Advances in Neural Information Pro- cessing Systems, pages 25476-25488, 2021.","title":"Mastering Atari games with limited data","authors":"Ye","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yu and Zhang, 2023] Zishun Yu and Xinhua Zhang. Actor- critic alignment for offline-to-online reinforcement learn- ing. In International Conference on Machine Learning, pages 40452-40474, 2023.","title":"Actorcritic alignment for offline-to-online reinforcement learning","authors":"Yu, Zhang, Zishun Yu, Xinhua Zhang","year":"2023","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Yu, 2018] Yang Yu. Towards sample efficient reinforcement learning. In International Joint Conference on Artificial Intelligence, pages 5739-5743, 2018.","title":"Towards sample efficient reinforcement learning","authors":"Yang Yu","year":"2018","venue":"International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2018/820"},{"position":35,"raw":"[Zhang et al., 2023] Haichao Zhang, Wei Xu, and Haonan Yu. Policy expansion for bridging offline-to-online rein- forcement learning. In International Conference on Learn- ing Representations, 2023.","title":"Policy expansion for bridging offline-to-online reinforcement learning","authors":"Zhang","year":"2023","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Zhang et al., 2024] Yinmin Zhang, Jie Liu, Chuming Li, Yazhe Niu, Yaodong Yang, Yu Liu, and Wanli Ouyang. A perspective of Q-value estimation on offline-to-online reinforcement learning. In AAAI Conference on Artificial Intelligence, pages 16908-16916, 2024.","title":"A perspective of Q-value estimation on offline-to-online reinforcement learning","authors":"Yinmin Zhang, Jie Liu, Chuming Li, Yazhe Niu, Yaodong Yang, Yu Liu, Wanli Ouyang","year":"2024","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhao et al., 2022] Yi Zhao, Rinu Boney, Alexander Ilin, Juho Kannala, and Joni Pajarinen. Adaptive behavior cloning regularization for stable offline-to-online rein- forcement learning. In European Symposium on Artificial Neural Networks, 2022.","title":"Adaptive behavior cloning regularization for stable offline-to-online reinforcement learning","authors":"Yi Zhao, Rinu Boney, Alexander Ilin, Juho Kannala, Joni Pajarinen","year":"2022","venue":"European Symposium on Artificial Neural Networks","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhou et al., 2022] Hattie Zhou, Ankit Vani, Hugo Larochelle, and Aaron C. Courville. Fortuitous for- getting in connectionist networks. In International Conference on Learning Representations, 2022.","title":"Fortuitous forgetting in connectionist networks","authors":"Hattie Zhou, Ankit Vani, Hugo Larochelle, Aaron C Courville","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhou et al., 2024] Renzhe Zhou, Chen-Xiao Gao, Zongzhang Zhang, and Yang Yu. Generalizable task representation learning for offline meta-reinforcement learning with data limitations. In AAAI Conference on Artificial Intelligence, pages 17132-17140, 2024.","title":"Generalizable task representation learning for offline meta-reinforcement learning with data limitations","authors":"Zhou","year":"2024","venue":"AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}