{"type":"ScholarlyArticle","title":"A Conservative Approach for Few-Shot Transfer in Off-Dynamics Reinforcement Learning","authors":[{"name":"Paul Daoudi","slug":"paul-daoudi","given":"Paul","family":"Daoudi","affiliation":"Huawei Noah's Ark Lab"},{"name":"Christophe Prieur","slug":"christophe-prieur","given":"Christophe","family":"Prieur","affiliation":"GIPSA Lab"},{"name":"Bogdan Robu","slug":"bogdan-robu","given":"Bogdan","family":"Robu","affiliation":"GIPSA Lab"},{"name":"Merwan Barlier","slug":"merwan-barlier","given":"Merwan","family":"Barlier","affiliation":"Huawei Noah's Ark Lab"},{"name":"Ludovic Dos Santos","slug":"ludovic-dos-santos","given":"Ludovic","family":"Dos Santos","affiliation":"Criteo AI Lab"}],"abstract":"Off-dynamics Reinforcement Learning (ODRL) seeks to transfer a policy from a source environment to a target environment characterized by distinct yet similar dynamics. In this context, traditional RL agents depend excessively on the dynamics of the source environment, resulting in the discovery of policies that excel in this environment but fail to provide reasonable performance in the target one. In the few-shot framework, a limited number of transitions from the target environment are introduced to facilitate a more effective transfer. Addressing this challenge, we propose an innovative approach inspired by recent advancements in Imitation Learning and Conservative RL algorithms. This method introduces a penalty to regulate the trajectories generated by the source-trained policy. We evaluate our method across various environments representing diverse off-dynamics conditions, where access to the target environment is extremely limited. These experiments include high-dimensional systems relevant to real-world applications. Across most tested scenarios, our proposed method demonstrates performance improvements compared to existing baselines.","doi":"10.24963/ijcai.2024/430","url":"https://ijcai.org/proceedings/2024/430","pdf":"https://ijcai.org/proceedings/2024/0430.pdf","markdown":"https://ijcai.org/proceedings/2024/430.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/430","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":430,"pages":{"first":3890,"last":3898},"keywords":["Machine Learning: ML: Reinforcement learning"],"videos":[],"references":[{"position":1,"raw":"[Abbeel et al., 2006] Pieter Abbeel, Morgan Quigley, and Andrew Y Ng. Using inaccurate models in reinforcement learning. In Proceedings of ICML, pages 1-8, 2006.","title":"Using inaccurate models in reinforcement learning","authors":"Pieter Abbeel, Morgan Quigley, Andrew Y Ng","year":"2006","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Abdullah et al., 2019] Mohammed Amin Abdullah, Hang Ren, Haitham Bou Ammar, Vladimir Milenkovic, Rui Luo, Mingtian Zhang, and Jun Wang. Wasserstein robust reinforcement learning. arXiv preprint arXiv:1907.13196, 2019.","title":"Wasserstein robust reinforcement learning","authors":"Mohammed Amin, Abdullah, Hang Ren, Haitham Bou Ammar, Vladimir Milenkovic, Rui Luo, Mingtian Zhang, Jun Wang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Achiam et al., 2017] Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. Constrained policy optimiza- tion. In Proceedings of ICML, pages 22-31. PMLR, 2017.","title":"Constrained policy optimization","authors":"Achiam","year":"2017","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Arndt et al., 2020] Karol Arndt, Murtaza Hazara, Ali Ghadirzadeh, and Ville Kyrki. Meta reinforcement learn- ing for sim-to-real domain adaptation. In Proceedings of ICRA, pages 2725-2731. IEEE, 2020.","title":"Meta reinforcement learning for sim-to-real domain adaptation","authors":"Karol Arndt, Murtaza Hazara, Ali Ghadirzadeh, Ville Kyrki","year":"2020","venue":"Proceedings of ICRA","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Babaeizadeh et al., 2016] Mohammad Babaeizadeh, Iuri Frosio, Stephen Tyree, Jason Clemons, and Jan Kautz. Reinforcement learning through asynchronous advantage actor-critic on a gpu. In Proceedings of ICLR, 2016.","title":"Reinforcement learning through asynchronous advantage actor-critic on a gpu","authors":"Mohammad Babaeizadeh, Iuri Frosio, Stephen Tyree, Jason Clemons, Jan Kautz","year":"2016","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Barreto et al., 2017] André Barreto, Will Dabney, Rémi Munos, Jonathan J Hunt, Tom Schaul, Hado P van Hasselt, and David Silver. Successor features for transfer in rein- forcement learning. Proceedings of NeurIPS, 30, 2017.","title":"Successor features for transfer in reinforcement learning","authors":"André Barreto, Will Dabney, Rémi Munos, Jonathan J Hunt, Tom Schaul, David Hado P Van Hasselt, Silver","year":"2017","venue":"Proceedings of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Barth-Maron et al., 2018] Gabriel Barth-Maron, Matthew W Hoffman, David Budden, Will Dabney, Dan Horgan, TB Dhruva, Alistair Muldal, Nicolas Heess, and Timothy Lillicrap. Distributed distributional deterministic policy gradients. In Proceedings of ICLR, 2018.","title":"Distributed distributional deterministic policy gradients","authors":"Gabriel Barth-Maron, Matthew W Hoffman, David Budden, Will Dabney, Dan Horgan, Alistair Dhruva, Nicolas Muldal, Timothy Heess, Lillicrap","year":"2018","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Brockman et al., 2016] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01540, 2016.","title":"Openai gym","authors":"Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, Wojciech Zaremba","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Corander et al., 2021] Jukka Corander, Ulpu Remes, and Timo Koski. On the jensen-shannon divergence and the variation distance for categorical probability distributions. Kybernetika, 2021.","title":"On the jensen-shannon divergence and the variation distance for categorical probability distributions","authors":"Jukka Corander, Ulpu Remes, Timo Koski","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"Coumans and Bai, 2016 2021]","title":null,"authors":"Bai Coumans","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"Erwin Coumans and Yunfei Bai. Pybullet, a python module for physics simulation for games, robotics and machine learning. http://pybullet.org, 2016-2021.","title":"Pybullet, a python module for physics simulation for games, robotics and machine learning","authors":"Erwin Coumans, Yunfei Bai","year":null,"venue":null,"doi":null,"url":"http://pybullet.org,2016-2021","internal_url":null},{"position":12,"raw":"[Csiszar and Körner, 1981] Imre Csiszar and J Körner. Cod- ing theorems for discrete memoryless systems. In Infor- mation Theory. Akadémiai Kiadó (Publishing House of the Hungarian Academy of Sciences), 1981.","title":"Coding theorems for discrete memoryless systems","authors":"Körner Csiszar, Imre Csiszar, J Körner","year":"1981","venue":"Information Theory","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Dadashi et al., 2021] Robert Dadashi, Léonard Hussenot, Matthieu Geist, and Olivier Pietquin. Primal wasserstein imitation learning. Proceedings of ICLR, 2021.","title":"Primal wasserstein imitation learning","authors":"Robert Dadashi, Léonard Hussenot, Matthieu Geist, Olivier Pietquin","year":"2021","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Desai et al., 2020] Siddharth Desai, Ishan Durugkar, Haresh Karnan, Garrett Warnell, Josiah Hanna, and Peter Stone. An imitation from observation approach to transfer learning with dynamics mismatch. Proceedings of NeurIPS, 33:3917-3929, 2020.","title":"An imitation from observation approach to transfer learning with dynamics mismatch","authors":"Siddharth Desai, Ishan Durugkar, Haresh Karnan, Garrett Warnell, Josiah Hanna, Peter Stone","year":"2020","venue":"Proceedings of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Dulac-Arnold et al., 2021] Gabriel Dulac-Arnold, Nir Levine, Daniel J Mankowitz, Jerry Li, Cosmin Paduraru, Sven Gowal, and Todd Hester. Challenges of real-world reinforcement learning: definitions, benchmarks and analysis. Machine Learning, 110(9):2419-2468, 2021.","title":"Challenges of real-world reinforcement learning: definitions, benchmarks and analysis","authors":"Gabriel Dulac-Arnold, Nir Levine, Daniel J Mankowitz, Jerry Li, Cosmin Paduraru, Sven Gowal, Todd Hester","year":"2021","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Eysenbach et al., 2021] Benjamin Eysenbach, Swapnil Asawa, Shreyas Chaudhari, Sergey Levine, and Ruslan Salakhutdinov. Off-dynamics reinforcement learning: Training for transfer with domain classifiers. Proceedings of ICLR, 2021.","title":"Off-dynamics reinforcement learning: Training for transfer with domain classifiers","authors":"Benjamin Eysenbach, Swapnil Asawa, Shreyas Chaudhari, Sergey Levine, Ruslan Salakhutdinov","year":"2021","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Fu et al., 2017] Justin Fu, Katie Luo, and Sergey Levine. Learning robust rewards with adversarial inverse rein- forcement learning. Proceedings of ICLR, 2017.","title":"Learning robust rewards with adversarial inverse reinforcement learning","authors":"Justin Fu, Katie Luo, Sergey Levine","year":"2017","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Fu et al., 2020] Justin Fu, Mohammad Norouzi, Ofir Nachum, George Tucker, Alexander Novikov, Mengjiao Yang, Michael R Zhang, Yutian Chen, Aviral Kumar, Cos- min Paduraru, et al. Benchmarks for deep off-policy eval- uation. In Proceedings of ICLR, 2020.","title":"Benchmarks for deep off-policy evaluation","authors":"Justin Fu, Mohammad Norouzi, Ofir Nachum, George Tucker, Alexander Novikov, Mengjiao Yang, Yutian Michael R Zhang, Aviral Chen, Cosmin Kumar, Paduraru","year":"2020","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Gamrian and Goldberg, 2019] Shani Gamrian and Yoav Goldberg. Transfer learning for related reinforcement learning tasks via image-to-image translation. In Proceed- ings of ICML, pages 2063-2072. PMLR, 2019.","title":"Transfer learning for related reinforcement learning tasks via image-to-image translation","authors":"Goldberg Gamrian, Shani Gamrian, Yoav Goldberg","year":"2019","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Gangwani, 2021] Tanmay Gangwani. Airl code. https:// github.com/tgangwani/RL-Indirect-imitation/tree/master, 2021.","title":"Airl code","authors":"Tanmay Gangwani","year":null,"venue":null,"doi":null,"url":"https://github.com/tgangwani/RL-Indirect-imitation/tree/master,2021","internal_url":null},{"position":21,"raw":"[Garcıa and Fernández, 2015] Javier Garcıa and Fernando Fernández. A comprehensive survey on safe reinforcement learning. JMLR, 16(1):1437-1480, 2015.","title":"Javier Garcıa and Fernando Fernández. A comprehensive survey on safe reinforcement learning","authors":"Fernández Garcıa","year":"2015","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Geng, 2021] Xinyang (Young) Geng. Code for conserva- tive q learning for offline reinforcement learning. https: //github.com/young-geng/CQL, 2021.","title":"Code for conservative q learning for offline reinforcement learning","authors":"; Geng, ( Xinyang, ) Young, Geng","year":"2021","venue":null,"doi":null,"url":"https://github.com/young-geng/CQL,2021","internal_url":null},{"position":23,"raw":"[Ghasemipour et al., 2020] Seyed Kamyar Seyed Ghasemipour, Richard Zemel, and Shixiang Gu. A divergence minimization perspective on imitation learning methods. In Proceedings of CoRL, pages 1259-1277.","title":"A divergence minimization perspective on imitation learning methods","authors":"Seyed Kamyar, Seyed Ghasemipour, Richard Zemel, Shixiang Gu","year":null,"venue":"Proceedings of CoRL","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"PMLR, 2020.","title":null,"authors":"Pmlr","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Hanna et al., 2021] Josiah P Hanna, Siddharth Desai, Haresh Karnan, Garrett Warnell, and Peter Stone. Grounded action transformation for sim-to-real reinforce- ment learning. Machine Learning, 110(9):2469-2499, 2021.","title":"Grounded action transformation for sim-to-real reinforcement learning","authors":"Siddharth Josiah P Hanna, Haresh Desai, Garrett Karnan, Peter Warnell, Stone","year":"2021","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Ho and Ermon, 2016] Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. Proceedings of NeurIPS, 29, 2016.","title":"Generative adversarial imitation learning","authors":"Jonathan Ho, Stefano Ermon","year":"2016","venue":"Proceedings of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Höfer et al., 2021] Sebastian Höfer, Kostas Bekris, Ankur Handa, Juan Camilo Gamboa, Melissa Mozifian, Florian Golemo, Chris Atkeson, Dieter Fox, Ken Goldberg, John Leonard, et al. Sim2real in robotics and automation: Ap- plications and challenges. IEEE transactions on automa- tion science and engineering, 18(2):398-400, 2021.","title":"Sim2real in robotics and automation: Applications and challenges","authors":"Sebastian Höfer, Kostas Bekris, Ankur Handa, Juan Camilo Gamboa, Melissa Mozifian, Florian Golemo, Chris Atkeson, Dieter Fox, Ken Goldberg, John Leonard","year":"2021","venue":"IEEE transactions on automation science and engineering","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Hussein et al., 2017] Ahmed Hussein, Mohamed Medhat Gaber, Eyad Elyan, and Chrisina Jayne. Imitation learn- ing: A survey of learning methods. ACM Computing Sur- veys (CSUR), 50(2):1-35, 2017.","title":"Imitation learning: A survey of learning methods","authors":"Ahmed Hussein, Mohamed Medhat Gaber, Eyad Elyan, Chrisina Jayne","year":"2017","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Jakobi et al., 1995] Nick Jakobi, Phil Husbands, and Inman Harvey. Noise and the reality gap: The use of simulation in evolutionary robotics. In Advances in Artificial Life: Third European Conference on Artificial Life, pages 704- 720. Springer, 1995.","title":"Noise and the reality gap: The use of simulation in evolutionary robotics","authors":"Nick Jakobi, Phil Husbands, Inman Harvey","year":"1995","venue":"Advances in Artificial Life: Third European Conference on Artificial Life","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Ju et al., 2022] Hao Ju, Rongshun Juan, Randy Gomez, Keisuke Nakamura, and Guangliang Li. Transferring pol- icy of deep reinforcement learning from simulation to real- ity for robotics. Nature Machine Intelligence, pages 1-11, 2022.","title":"Transferring policy of deep reinforcement learning from simulation to reality for robotics","authors":"Hao Ju, Rongshun Juan, Randy Gomez, Keisuke Nakamura, Guangliang Li","year":"2022","venue":"Nature Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Kakade and Langford, 2002] Sham Kakade and John Lang- ford. Approximately optimal approximate reinforcement learning. In Proceedings of ICML, pages 267-274, 2002.","title":"Sham Kakade and John Langford. Approximately optimal approximate reinforcement learning","authors":"Langford Kakade","year":"2002","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Kang et al., 2022] Katie Kang, Paula Gradu, Jason J Choi, Michael Janner, Claire Tomlin, and Sergey Levine. Lya- punov density models: Constraining distribution shift in learning-based control. In Proceedings of ICML, pages 10708-10733. PMLR, 2022.","title":"Lyapunov density models: Constraining distribution shift in learning-based control","authors":"Katie Kang, Paula Gradu, Jason J Choi, Michael Janner, Claire Tomlin, Sergey Levine","year":"2022","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Koos et al., 2012] Sylvain Koos, Jean-Baptiste Mouret, and Stéphane Doncieux. The transferability approach: Cross- ing the reality gap in evolutionary robotics. IEEE Transac- tions on Evolutionary Computation, 17(1):122-145, 2012.","title":"The transferability approach: Crossing the reality gap in evolutionary robotics","authors":"Jean-Baptiste Sylvain Koos, Stéphane Mouret, Doncieux","year":"2012","venue":"IEEE Transactions on Evolutionary Computation","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Kostrikov, 2018] Ilya Kostrikov. Pytorch rl agents. https:// github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail, 2018.","title":"Ilya Kostrikov. Pytorch rl agents","authors":"Kostrikov","year":"2018","venue":null,"doi":null,"url":"https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail","internal_url":null},{"position":35,"raw":"[Kumar et al., 2020] Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. Proceedings of NeurIPS, 33:1179-1191, 2020.","title":"Conservative q-learning for offline reinforcement learning","authors":"Aviral Kumar, Aurick Zhou, George Tucker, Sergey Levine","year":"2020","venue":"Proceedings of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Kumar et al., 2021] Ashish Kumar, Zipeng Fu, Deepak Pathak, and Jitendra Malik. RMA: rapid motor adapta- tion for legged robots. In Robotics: Science and Systems, 2021.","title":"RMA: rapid motor adaptation for legged robots","authors":"Ashish Kumar, Zipeng Fu, Deepak Pathak, Jitendra Malik","year":"2021","venue":"Robotics: Science and Systems","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Mordatch et al., 2015] Igor Mordatch, Kendall Lowrey, and Emanuel Todorov. Ensemble-cio: Full-body dynamic mo- tion planning that transfers to physical humanoids. In Pro- ceedings of IROS, pages 5307-5314. IEEE, 2015.","title":"Ensemble-cio: Full-body dynamic motion planning that transfers to physical humanoids","authors":"Mordatch","year":"2015","venue":"Proceedings of IROS","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Moskovitz et al., 2021] Ted Moskovitz, Michael Arbel, Fer- enc Huszar, and Arthur Gretton. Efficient wasserstein nat- ural gradients for reinforcement learning. Proceedings of ICLR, 2021.","title":"Efficient wasserstein natural gradients for reinforcement learning","authors":"Ted Moskovitz, Michael Arbel, Ferenc Huszar, Arthur Gretton","year":"2021","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Muratore et al., 2019] Fabio Muratore, Michael Gienger, and Jan Peters. Assessing transferability from simulation to reality for reinforcement learning. IEEE transactions on pattern analysis and machine intelligence, 43(4):1172- 1183, 2019.","title":"Assessing transferability from simulation to reality for reinforcement learning","authors":"Fabio Muratore, Michael Gienger, Jan Peters","year":"2019","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Nachum et al., 2019] Ofir Nachum, Yinlam Chow, Bo Dai, and Lihong Li. Dualdice: Behavior-agnostic estimation of discounted stationary distribution corrections. Proceed- ings of NeurIPS, 32, 2019.","title":"Dualdice: Behavior-agnostic estimation of discounted stationary distribution corrections","authors":"Ofir Nachum, Yinlam Chow, Bo Dai, Lihong Li","year":"2019","venue":"Proceedings of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Niu et al., 2022a] Haoyi Niu, Shubham Sharma, Yiwen Qiu, Ming Li, Guyue Zhou, Jianming HU, and Xianyuan Zhan. H2o code. https://github.com/t6-thu/H2O, 2022.","title":"H2o code","authors":"Niu","year":"2022","venue":null,"doi":null,"url":"https://github.com/t6-thu/H2O","internal_url":null},{"position":42,"raw":"[Niu et al., 2022b] Haoyi Niu, Shubham Sharma, Yiwen Qiu, Ming Li, Guyue Zhou, Jianming Hu, and Xianyuan Zhan. When to trust your simulator: Dynamics-aware hy- brid offline-and-online reinforcement learning. Proceed- ings of NeurIPS, 2022.","title":"When to trust your simulator: Dynamics-aware hybrid offline-and-online reinforcement learning","authors":"Niu","year":"2022","venue":"Proceedings of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Pacchiano et al., 2020] Aldo Pacchiano, Jack Parker- Holder, Yunhao Tang, Krzysztof Choromanski, Anna Choromanska, and Michael Jordan. Learning to score behaviors for guided policy optimization. In Proceedings of ICML, pages 7445-7454. PMLR, 2020.","title":"Learning to score behaviors for guided policy optimization","authors":"Aldo Pacchiano, Jack Parker-Holder, Yunhao Tang, Krzysztof Choromanski, Anna Choromanska, Michael Jordan","year":"2020","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Raffin, 2020] Antonin Raffin. Rl baselines3 zoo. https:// github.com/DLR-RM/rl-baselines3-zoo, 2020.","title":"Rl baselines3 zoo","authors":"Antonin Raffin","year":null,"venue":null,"doi":null,"url":"https://github.com/DLR-RM/rl-baselines3-zoo,2020","internal_url":null},{"position":45,"raw":"[Schulman et al., 2015] John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In Proceedings of ICML, pages 1889-1897. PMLR, 2015.","title":"Trust region policy optimization","authors":"John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, Philipp Moritz","year":"2015","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.","title":"Proximal policy optimization algorithms","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Tan et al., 2018] Jie Tan, Tingnan Zhang, Erwin Coumans, Atil Iscen, Yunfei Bai, Danijar Hafner, Steven Bohez, and Vincent Vanhoucke. Sim-to-real: Learning agile locomo- tion for quadruped robots. Robotics: Science and Systems XIV, 2018.","title":"Sim-to-real: Learning agile locomotion for quadruped robots","authors":"Jie Tan, Tingnan Zhang, Erwin Coumans, Atil Iscen, Yunfei Bai, Danijar Hafner, Steven Bohez, Vincent Vanhoucke","year":"2018","venue":"Robotics: Science and Systems","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Terpin et al., 2022] Antonio Terpin, Nicolas Lanzetti, Batuhan Yardim, Florian Dorfler, and Giorgia Ramponi. Trust region policy optimization with optimal transport discrepancies: Duality and algorithm for continuous actions. Proceedings of NeurIPS, 35:19786-19797, 2022.","title":"Trust region policy optimization with optimal transport discrepancies: Duality and algorithm for continuous actions","authors":"Terpin","year":"2022","venue":"Proceedings of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Tessler et al., 2019] Chen Tessler, Yonathan Efroni, and Shie Mannor. Action robust reinforcement learning and applications in continuous control. In Proceedings of ICML, pages 6215-6224. PMLR, 2019.","title":"Action robust reinforcement learning and applications in continuous control","authors":"Chen Tessler, Yonathan Efroni, Shie Mannor","year":"2019","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Torabi et al., 2019] Faraz Torabi, Garrett Warnell, and Pe- ter Stone. Recent advances in imitation learning from ob- servation. In Proceedings of IJCAI, pages 6325-6331. ij- cai.org, 2019.","title":"Recent advances in imitation learning from observation","authors":"Faraz Torabi, Garrett Warnell, Peter Stone","year":"2019","venue":"Proceedings of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/882"},{"position":51,"raw":"[Touati et al., 2020] Ahmed Touati, Amy Zhang, Joelle Pineau, and Pascal Vincent. Stable policy optimization via off-policy divergence regularization. In Proceedings of UAI, pages 1328-1337. PMLR, 2020.","title":"Stable policy optimization via off-policy divergence regularization","authors":"Ahmed Touati, Amy Zhang, Joelle Pineau, Pascal Vincent","year":"2020","venue":"Proceedings of UAI","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Weiss et al., 2016] Karl Weiss, Taghi M Khoshgoftaar, and DingDing Wang. A survey of transfer learning. Journal of Big data, 3(1):1-40, 2016.","title":"A survey of transfer learning","authors":"Karl Weiss, Taghi M Khoshgoftaar, Dingding Wang","year":"2016","venue":"Journal of Big data","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Yu et al., 2018] Wenhao Yu, C Karen Liu, and Greg Turk. Policy transfer with strategy optimization. In Proceedings of ICLR, 2018.","title":"Policy transfer with strategy optimization","authors":"Wenhao Yu, Karen Liu, Greg Turk","year":"2018","venue":"Proceedings of ICLR","doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Yu et al., 2020] Wenhao Yu, Jie Tan, Yunfei Bai, Erwin Coumans, and Sehoon Ha. Learning fast adaptation with meta strategy optimization. IEEE Robotics and Automa- tion Letters, 5(2):2950-2957, 2020.","title":"Learning fast adaptation with meta strategy optimization","authors":"Wenhao Yu, Jie Tan, Yunfei Bai, Erwin Coumans, Sehoon Ha","year":"2020","venue":"IEEE Robotics and Automation Letters","doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Zhu et al., 2018] Shaojun Zhu, Andrew Kimmel, Kostas E. Bekris, and Abdeslam Boularias. Fast model identifica- tion via physics engines for data-efficient policy search. In Proceedings of IJCAI, pages 3249-3256. ijcai.org, 2018.","title":"Fast model identification via physics engines for data-efficient policy search","authors":"Shaojun Zhu, Andrew Kimmel, Kostas E Bekris, Abdeslam Boularias","year":"2018","venue":"Proceedings of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/451"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}