{"type":"ScholarlyArticle","title":"A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents","authors":[{"name":"Haoyi Niu","slug":"haoyi-niu","given":"Haoyi","family":"Niu","affiliation":"Tsinghua University"},{"name":"Jianming Hu","slug":"jianming-hu","given":"Jianming","family":"Hu","affiliation":"Tsinghua University"},{"name":"Guyue Zhou","slug":"guyue-zhou","given":"Guyue","family":"Zhou","affiliation":"Tsinghua University"},{"name":"Xianyuan Zhan","slug":"xianyuan-zhan","given":"Xianyuan","family":"Zhan","affiliation":"Tsinghua University; Shanghai Artificial Intelligence Laboratory"}],"abstract":"The burgeoning fields of robot learning and embodied AI have triggered an increasing demand for large quantities of data. However, collecting sufficient unbiased data from the target domain remains a challenge due to costly data collection processes and stringent safety requirements. Consequently, researchers often resort to data from easily accessible source domains, such as simulation and laboratory environments, for cost-effective data acquisition and rapid model iteration. Nevertheless, the environments and embodiments of these source domains can be quite different from their target domain counterparts, underscoring the need for effective cross-domain policy transfer approaches. In this paper, we conduct a systematic review of existing cross-domain policy transfer methods. Through a nuanced categorization of domain gaps, we encapsulate the overarching insights and design considerations of each problem setting. We also provide a high-level discussion about the key methodologies used in cross-domain policy transfer problems. Lastly, we summarize the open challenges that lie beyond the capabilities of current paradigms and discuss potential future directions in this field.","doi":"10.24963/ijcai.2024/906","url":"https://ijcai.org/proceedings/2024/906","pdf":"https://ijcai.org/proceedings/2024/0906.pdf","markdown":"https://ijcai.org/proceedings/2024/906.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/906","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Survey Track","track":"Survey Track","number":906,"pages":{"first":8197,"last":8206},"keywords":["Robotics: General","Machine Learning: ML: Multi-task and transfer learning","Robotics: ROB: Learning in robotics"],"videos":[],"references":[{"position":1,"raw":"[Andrychowicz et al., 2020] Marcin Andrychowicz, Bowen Baker, Maciek Chociej, et al. Learning dexterous in-hand manipulation. IJRR, 2020.","title":"Learning dexterous in-hand manipulation","authors":"Marcin Andrychowicz, Bowen Baker, Maciek Chociej","year":"2020","venue":"IJRR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bewley et al., 2019] Alex Bewley, Jessica Rigley, Yuxuan Liu, et al. Learning to drive from simulation without real world labels. In ICRA, 2019.","title":"Learning to drive from simulation without real world labels","authors":"Alex Bewley, Jessica Rigley, Yuxuan Liu","year":"2019","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cao and Sadigh, 2021] Zhangjie Cao and Dorsa Sadigh. Learning from imperfect demonstrations from agents with varying dynamics. RAL, 2021.","title":"Learning from imperfect demonstrations from agents with varying dynamics","authors":"Sadigh Cao, Zhangjie Cao, Dorsa Sadigh","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cao et al., 2021] Zhangjie Cao, Yilun Hao, Mengxi Li, and Dorsa Sadigh. Learning feasibility to imitate demonstra- tors with different dynamics. In CoRL, 2021.","title":"Learning feasibility to imitate demonstrators with different dynamics","authors":"Cao","year":"2021","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chebotar et al., 2019] Yevgen Chebotar, Ankur Handa, Viktor Makoviychuk, Miles Macklin, Jan Issac, Nathan Ratliff, and Dieter Fox. Closing the sim-to-real loop: Adapting simulation randomization with real world expe- rience. In ICRA, 2019.","title":"Closing the sim-to-real loop: Adapting simulation randomization with real world experience","authors":"Yevgen Chebotar, Ankur Handa, Viktor Makoviychuk, Miles Macklin, Jan Issac, Nathan Ratliff, Dieter Fox","year":"2019","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2023] Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, and Hongyang Li. End- to-end autonomous driving: Challenges and frontiers. arXiv, 2023.","title":"Endto-end autonomous driving: Challenges and frontiers","authors":"Li Chen, Penghao Wu, Kashyap Chitta, Bernhard Jaeger, Andreas Geiger, Hongyang Li","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Choi et al., 2023] Wonje Choi, Woo Kyung Kim, Se- ungHyun Kim, and Honguk Woo. Efficient policy adap- tation with contrastive prompt ensemble for embodied agents. In NeurIPS, 2023.","title":"Efficient policy adaptation with contrastive prompt ensemble for embodied agents","authors":"Choi","year":"2023","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Christiano et al., 2016] Paul Christiano, Zain Shah, Igor Mordatch, et al. Transfer from simulation to real world through learning deep inverse dynamics model. arXiv, 2016.","title":"Transfer from simulation to real world through learning deep inverse dynamics model","authors":"Paul Christiano, Zain Shah, Igor Mordatch","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Desai et al., 2020] Siddharth Desai, Ishan Durugkar, Haresh Karnan, et al. An imitation from observation approach to transfer learning with dynamics mismatch. NeurIPS, 2020.","title":"An imitation from observation approach to transfer learning with dynamics mismatch","authors":"Siddharth Desai, Ishan Durugkar, Haresh Karnan","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Du et al., 2021] Yuqing Du, Olivia Watkins, Trevor Darrell, Pieter Abbeel, and Deepak Pathak. Auto-tuned sim-to-real transfer. In ICRA, 2021.","title":"Auto-tuned sim-to-real transfer","authors":"Yuqing Du, Olivia Watkins, Trevor Darrell, Pieter Abbeel, Deepak Pathak","year":"2021","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Duan et al., 2022] Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan. A survey of embodied ai: From simulators to research tasks. IEEE Transactions on Emerging Topics in Computational Intelligence, 2022.","title":"A survey of embodied ai: From simulators to research tasks","authors":"Duan","year":"2022","venue":"IEEE Transactions on Emerging Topics in Computational Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Dwibedi et al., 2019] Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre Sermanet, and Andrew Zisser- man. Temporal cycle-consistency learning. In CVPR, 2019.","title":"Temporal cycle-consistency learning","authors":"Dwibedi","year":"2019","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Eysenbach et al., 2020] Benjamin Eysenbach, Shreyas Chaudhari, Swapnil Asawa, Sergey Levine, and Ruslan Salakhutdinov. Off-dynamics reinforcement learning: Training for transfer with domain classifiers. In ICLR, 2020.","title":"Off-dynamics reinforcement learning: Training for transfer with domain classifiers","authors":"Benjamin Eysenbach, Shreyas Chaudhari, Swapnil Asawa, Sergey Levine, Ruslan Salakhutdinov","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Finn et al., 2017] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, 2017.","title":"Model-agnostic meta-learning for fast adaptation of deep networks","authors":"Finn","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Franzmeyer et al., 2022] Tim Franzmeyer, Philip Torr, and João F Henriques. Learn what matters: cross-domain im- itation learning with task-relevant embeddings. NeurIPS, 2022.","title":"Learn what matters: cross-domain imitation learning with task-relevant embeddings","authors":"Tim Franzmeyer, Philip Torr, João F Henriques","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Gangwani and Peng, 2020] Tanmay Gangwani and Jian Peng. State-only imitation with transition dynamics mis- match. In ICLR, 2020.","title":"State-only imitation with transition dynamics mismatch","authors":"Tanmay Gangwani, Jian Peng","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gupta et al., 2017] Abhishek Gupta, Coline Devin, YuXuan Liu, Pieter Abbeel, and Sergey Levine. Learning invariant feature spaces to transfer skills with reinforcement learn- ing. In ICLR, 2017.","title":"Learning invariant feature spaces to transfer skills with reinforcement learning","authors":"Abhishek Gupta, Coline Devin, Yuxuan Liu, Pieter Abbeel, Sergey Levine","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Gupta et al., 2021a] Agrim Gupta, Linxi Fan, Surya Gan- guli, and Li Fei-Fei. Metamorph: Learning universal con- trollers with transformers. In ICLR, 2021.","title":"Agrim Gupta, Linxi Fan","authors":"Gupta","year":"2021","venue":"Metamorph: Learning universal controllers with transformers","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Gupta et al., 2021b] Agrim Gupta, Silvio Savarese, Surya Ganguli, and Li Fei-Fei. Embodied intelligence via learn- ing and evolution. Nature communications, 2021.","title":"Embodied intelligence via learning and evolution","authors":"Gupta","year":"2021","venue":"Nature communications","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Hanna and Stone, 2017] Josiah Hanna and Peter Stone. Grounded action transformation for robot learning in sim- ulation. In AAAI, 2017.","title":"Grounded action transformation for robot learning in simulation","authors":"Stone Hanna, Josiah Hanna, Peter Stone","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Hejna et al., 2020a] Joey Hejna, Pieter Abbeel, and Lerrel Pinto. Task-agnostic morphology evolution. In ICLR, 2020.","title":"Task-agnostic morphology evolution","authors":"Hejna","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Hejna et al., 2020b] Joey Hejna, Lerrel Pinto, and Pieter Abbeel. Hierarchically decoupled imitation for morpho- logical transfer. In ICML, 2020.","title":"Hierarchically decoupled imitation for morphological transfer","authors":"Hejna","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Jiang et al., 2020] Shengyi Jiang, Jingcheng Pang, and Yang Yu. Offline imitation learning with a misspecified simulator. NeurIPS, 2020.","title":"Offline imitation learning with a misspecified simulator","authors":"Shengyi Jiang, Jingcheng Pang, Yang Yu","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Kar et al., 2019] Amlan Kar, Aayush Prakash, Ming-Yu Liu, Eric Cameracci, Justin Yuan, Matt Rusiniak, David Acuna, Antonio Torralba, and Sanja Fidler. Meta-sim: Learning to generate synthetic datasets. In ICCV, 2019.","title":"Meta-sim: Learning to generate synthetic datasets","authors":"Amlan Kar, Aayush Prakash, Ming-Yu Liu, Eric Cameracci, Justin Yuan, Matt Rusiniak, David Acuna, Antonio Torralba, Sanja Fidler","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Katdare et al., 2023] Pulkit Katdare, Nan Jiang, and Kather- ine Rose Driggs-Campbell. Marginalized importance sam- pling for off-environment policy evaluation. In CoRL, 2023.","title":"Marginalized importance sampling for off-environment policy evaluation","authors":"Pulkit Katdare, Nan Jiang, Katherine Rose Driggs-Campbell","year":"2023","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Kim et al., 2020] Kuno Kim, Yihong Gu, Jiaming Song, Shengjia Zhao, and Stefano Ermon. Domain adaptive im- itation learning. In ICML, 2020.","title":"Domain adaptive imitation learning","authors":"Kuno Kim, Yihong Gu, Jiaming Song, Shengjia Zhao, Stefano Ermon","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Kolev and Todorov, 2015] Svetoslav Kolev and Emanuel Todorov. Physically consistent state estimation and sys- tem identification for contacts. In Humanoids, 2015.","title":"Physically consistent state estimation and system identification for contacts","authors":"Svetoslav Kolev, Emanuel Todorov","year":"2015","venue":"Humanoids","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Laskin et al., 2020] Misha Laskin, Kimin Lee, Adam Stooke, et al. Reinforcement learning with augmented data. NeurIPS, 2020.","title":"Reinforcement learning with augmented data","authors":"Misha Laskin, Kimin Lee, Adam Stooke","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Lee et al., 2022] Alex X Lee, Coline Manon Devin, Yuxi- ang Zhou, et al. Beyond pick-and-place: Tackling robotic stacking of diverse shapes. In CoRL, 2022.","title":"Beyond pick-and-place: Tackling robotic stacking of diverse shapes","authors":"Alex X Lee, Coline Manon Devin, Yuxiang Zhou","year":"2022","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Li et al., 2023] Jianxiong Li, Xiao Hu, Haoran Xu, Jingjing Liu, Xianyuan Zhan, and Ya-Qin Zhang. Proto: Iterative policy regularized offline-to-online reinforcement learn- ing. arXiv, 2023.","title":"Proto: Iterative policy regularized offline-to-online reinforcement learning","authors":"Jianxiong Li, Xiao Hu, Haoran Xu, Jingjing Liu, Xianyuan Zhan, Ya-Qin Zhang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Li et al., 2024] Jianxiong Li, Jinliang Zheng, Yinan Zheng, Liyuan Mao, Xiao Hu, Sijie Cheng, Haoyi Niu, et al. De- cisionnce: Embodied multimodal representations via im- plicit preference learning. In ICML, 2024.","title":"Decisionnce: Embodied multimodal representations via implicit preference learning","authors":"Jianxiong Li, Jinliang Zheng, Yinan Zheng, Liyuan Mao, Xiao Hu, Sijie Cheng, Haoyi Niu","year":"2024","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Liu et al., 2018] YuXuan Liu, Abhishek Gupta, Pieter Abbeel, and Sergey Levine. Imitation from observation: Learning to imitate behaviors from raw video via context translation. In ICRA, 2018.","title":"Imitation from observation: Learning to imitate behaviors from raw video via context translation","authors":"Liu","year":"2018","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Liu et al., 2020] Fangchen Liu, Zhan Ling, Tongzhou Mu, and Hao Su. State alignment-based imitation learning. In ICLR, 2020.","title":"State alignment-based imitation learning","authors":"Fangchen Liu, Zhan Ling, Tongzhou Mu, Hao Su","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Liu et al., 2022] Jinxin Liu, Zhang Hongyin, and Donglin Wang. Dara: Dynamics-aware reward augmentation in of- fline reinforcement learning. In ICLR, 2022.","title":"Dara: Dynamics-aware reward augmentation in offline reinforcement learning","authors":"Jinxin Liu, Zhang Hongyin, Donglin Wang","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Ljung, 1998] Lennart Ljung. System identification. In Sig- nal analysis and prediction. Springer, 1998.","title":"System identification","authors":"Lennart Ljung","year":"1998","venue":"Signal analysis and prediction","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Ma et al., 2022] Yecheng Jason Ma, Shagun Sodhani, Di- nesh Jayaraman, Osbert Bastani, Vikash Kumar, and Amy Zhang. Vip: Towards universal visual reward and repre- sentation via value-implicit pre-training. In ICLR, 2022.","title":"Vip: Towards universal visual reward and representation via value-implicit pre-training","authors":"Jason Yecheng, Shagun Ma, Dinesh Sodhani, Osbert Jayaraman, Vikash Bastani, Amy Kumar, Zhang","year":"2022","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Ma et al., 2023] Yecheng Jason Ma, Vikash Kumar, Amy Zhang, Osbert Bastani, and Dinesh Jayaraman. Liv: Language-image representations and rewards for robotic control. In ICML, 2023.","title":"Liv: Language-image representations and rewards for robotic control","authors":"Jason Yecheng, Vikash Ma, Amy Kumar, Osbert Zhang, Dinesh Bastani, Jayaraman","year":"2023","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Mehta et al., 2020] Bhairav Mehta, Manfred Diaz, Florian Golemo, Christopher J Pal, and Liam Paull. Active domain randomization. In CoRL, 2020.","title":"Active domain randomization","authors":"Bhairav Mehta, Manfred Diaz, Florian Golemo, Christopher J Pal, Liam Paull","year":"2020","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Mueller et al., 2018] Matthias Mueller, Alexey Dosovitskiy, Bernard Ghanem, and Vladlen Koltun. Driving policy transfer via modularity and abstraction. In CoRL, 2018.","title":"Driving policy transfer via modularity and abstraction","authors":"Matthias Mueller, Alexey Dosovitskiy, Bernard Ghanem, Vladlen Koltun","year":"2018","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Muratore et al., 2021] Fabio Muratore, Christian Eilers, Michael Gienger, and Jan Peters. Data-efficient domain randomization with bayesian optimization. RAL, 2021.","title":"Data-efficient domain randomization with bayesian optimization","authors":"Fabio Muratore, Christian Eilers, Michael Gienger, Jan Peters","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Nagabandi et al., 2018] Anusha Nagabandi, Ignasi Clavera, Simin Liu, et al. Learning to adapt in dynamic, real- world environments through meta-reinforcement learning. In ICLR, 2018.","title":"Learning to adapt in dynamic, realworld environments through meta-reinforcement learning","authors":"Anusha Nagabandi, Ignasi Clavera, Simin Liu","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Nguyen-Tuong and Peters, 2011] Duy Nguyen-Tuong and Jan Peters. Model learning for robot control: a survey. Cognitive processing, 2011.","title":"Model learning for robot control: a survey","authors":"Nguyen-Tuong, Duy Peters, Jan Nguyen-Tuong, Peters","year":"2011","venue":"Cognitive processing","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Niu et al., 2022] Haoyi Niu, Shubham Sharma, Yiwen Qiu, Ming Li, Guyue Zhou, Jianming HU, and Xianyuan Zhan. When to trust your simulator: Dynamics-aware hybrid offline-and-online reinforcement learning. In NeurIPS, 2022.","title":"When to trust your simulator: Dynamics-aware hybrid offline-and-online reinforcement learning","authors":"Haoyi Niu, Shubham Sharma, Yiwen Qiu, Ming Li, Guyue Zhou, H U Jianming, Xianyuan Zhan","year":"2022","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Niu et al., 2023] Haoyi Niu, Tianying Ji, Bingqi Liu, Haocheng Zhao, Xiangyu Zhu, Jianying Zheng, Pengfei Huang, Guyue Zhou, Jianming Hu, and Xianyuan Zhan. H2o+: An improved framework for hybrid offline-and- online rl with dynamics gaps. arXiv, 2023.","title":"H2o+: An improved framework for hybrid offline-andonline rl with dynamics gaps","authors":"Niu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Octo Model Team et al., 2023] Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, et al. Octo: An open- source generalist robot policy, 2023.","title":"Octo Model Team","authors":"Octo Model, Team","year":"2023","venue":"Octo: An opensource generalist robot policy","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Open X-Embodiment and others, 2023] Open X- Embodiment et al. Open x-embodiment: Robotic learning datasets and RT-x models. arXiv, 2023.","title":"Open X-Embodiment et al. Open x-embodiment: Robotic learning datasets and RT-x models","authors":null,"year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Pan et al., 2017] Xinlei Pan, Yurong You, Ziyan Wang, and Cewu Lu. Virtual to real reinforcement learning for au- tonomous driving. BMVC, 2017.","title":"Virtual to real reinforcement learning for autonomous driving","authors":"Pan","year":"2017","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Pathak et al., 2018] Deepak Pathak, Parsa Mahmoudieh, Michael Luo, et al. Zero-shot visual imitation. In ICLR, 2018.","title":"Zero-shot visual imitation","authors":"Deepak Pathak, Parsa Mahmoudieh, Michael Luo","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Peng et al., 2018] Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel. Sim-to-real trans- fer of robotic control with dynamics randomization. In ICRA, 2018.","title":"Sim-to-real transfer of robotic control with dynamics randomization","authors":"Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, Pieter Abbeel","year":"2018","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Pertsch et al., 2022] Karl Pertsch, Ruta Desai, Vikash Ku- mar, et al. Cross-domain transfer via semantic skill imita- tion. In CoRL, 2022.","title":"Cross-domain transfer via semantic skill imitation","authors":"Karl Pertsch, Ruta Desai, Vikash Kumar","year":"2022","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Radford et al., 2021] Alec Radford, Jong Wook Kim, Chris Hallacy, et al. Learning transferable visual models from natural language supervision. In ICML, 2021.","title":"Learning transferable visual models from natural language supervision","authors":"Alec Radford, Jong Wook Kim, Chris Hallacy","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Radosavovic et al., 2021] Ilija Radosavovic, Xiaolong Wang, Lerrel Pinto, and Jitendra Malik. State-only imitation learning for dexterous manipulation. In IROS, 2021.","title":"State-only imitation learning for dexterous manipulation","authors":"Ilija Radosavovic, Xiaolong Wang, Lerrel Pinto, Jitendra Malik","year":"2021","venue":"IROS","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Rajeswaran et al., 2017] Aravind Rajeswaran, Sarvjeet Ghotra, Balaraman Ravindran, and Sergey Levine. EPOpt: Learning robust neural network policies using model ensembles. In ICLR, 2017.","title":"EPOpt: Learning robust neural network policies using model ensembles","authors":"Aravind Rajeswaran, Sarvjeet Ghotra, Balaraman Ravindran, Sergey Levine","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Rakelly et al., 2019] Kate Rakelly, Aurick Zhou, Chelsea Finn, Sergey Levine, and Deirdre Quillen. Efficient off- policy meta-reinforcement learning via probabilistic con- text variables. In ICML, 2019.","title":"Efficient offpolicy meta-reinforcement learning via probabilistic context variables","authors":"Rakelly","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Ramos et al., 2019] Fabio Ramos, Rafael Possas, and Di- eter Fox. Bayessim: adaptive domain randomization via probabilistic inference for robotics simulators. In RSS, 2019.","title":"Bayessim: adaptive domain randomization via probabilistic inference for robotics simulators","authors":"Fabio Ramos, Rafael Possas, Dieter Fox","year":"2019","venue":"RSS","doi":null,"url":null,"internal_url":null},{"position":56,"raw":"[Rao et al., 2020] Kanishka Rao, Chris Harris, Alex Irpan, Sergey Levine, Julian Ibarz, and Mohi Khansari. Rl- cyclegan: Reinforcement learning aware simulation-to- real. In CVPR, 2020.","title":"Rlcyclegan: Reinforcement learning aware simulation-toreal","authors":"Kanishka Rao, Chris Harris, Alex Irpan, Sergey Levine, Julian Ibarz, Mohi Khansari","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":57,"raw":"[Raychaudhuri et al., 2021] Dripta S Raychaudhuri, Sujoy Paul, Jeroen Vanbaar, and Amit K Roy-Chowdhury. Cross- domain imitation from observations. In ICML, 2021.","title":"Crossdomain imitation from observations","authors":"Sujoy Dripta S Raychaudhuri, Jeroen Paul, Amit K Roy-Chowdhury Vanbaar","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":58,"raw":"[Reed et al., 2022] Scott Reed, Konrad Zolna, Emilio Parisotto, et al. A generalist agent. TMLR, 2022.","title":"A generalist agent","authors":"Scott Reed, Konrad Zolna, Emilio Parisotto","year":"2022","venue":"TMLR","doi":null,"url":null,"internal_url":null},{"position":59,"raw":"[Sadeghi et al., 2018] Fereshteh Sadeghi, Alexander Toshev, Eric Jang, and Sergey Levine. Sim2real viewpoint invari- ant visual servoing by recurrent control. In CVPR, 2018.","title":"Sim2real viewpoint invariant visual servoing by recurrent control","authors":"Fereshteh Sadeghi, Alexander Toshev, Eric Jang, Sergey Levine","year":"2018","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":60,"raw":"[Salhotra et al., 2023] Gautam Salhotra, I-Chun Arthur Liu, and Gaurav S. Sukhatme. Learning robot manipulation from cross-morphology demonstration. In CoRL, 2023.","title":"Learning robot manipulation from cross-morphology demonstration","authors":"Gautam Salhotra, I-Chun Arthur Liu, Gaurav S Sukhatme","year":"2023","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":61,"raw":"[Sermanet et al., 2018] Pierre Sermanet, Corey Lynch, Yev- gen Chebotar, et al. Time-contrastive networks: Self- supervised learning from video. ICRA, 2018.","title":"Time-contrastive networks: Selfsupervised learning from video","authors":"Sermanet","year":"2018","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":62,"raw":"[Sharma et al., 2019] Pratyusha Sharma, Deepak Pathak, and Abhinav Gupta. Third-person visual imitation learn- ing via decoupled hierarchical controller. NeurIPS, 2019.","title":"Third-person visual imitation learning via decoupled hierarchical controller","authors":"Pratyusha Sharma, Deepak Pathak, Abhinav Gupta","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":63,"raw":"[Smith et al., 2023] Laura Smith, Yunhao Cao, and Sergey Levine. Grow your limits: Continuous improvement with real-world rl for robotic locomotion. arXiv, 2023.","title":"Grow your limits: Continuous improvement with real-world rl for robotic locomotion","authors":"Laura Smith, Yunhao Cao, Sergey Levine","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":64,"raw":"[Stadie et al., 2016] Bradly C Stadie, Pieter Abbeel, and Ilya Sutskever. Third person imitation learning. In ICLR, 2016.","title":"Third person imitation learning","authors":"Pieter Bradly C Stadie, Ilya Abbeel, Sutskever","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":65,"raw":"[Tan et al., 2016] Jie Tan, Zhaoming Xie, Byron Boots, and C Karen Liu. Simulation-based design of dynamic con- trollers for humanoid balancing. In IROS, 2016.","title":"Simulation-based design of dynamic controllers for humanoid balancing","authors":"Jie Tan, Zhaoming Xie, Byron Boots, Karen Liu","year":"2016","venue":"IROS","doi":null,"url":null,"internal_url":null},{"position":66,"raw":"[Tobin et al., 2017] Josh Tobin, Rachel Fong, Alex Ray, et al. Domain randomization for transferring deep neu- ral networks from simulation to the real world. In IROS, 2017.","title":"Domain randomization for transferring deep neural networks from simulation to the real world","authors":"Josh Tobin, Rachel Fong, Alex Ray","year":"2017","venue":"IROS","doi":null,"url":null,"internal_url":null},{"position":67,"raw":"[Triess et al., 2021] Larissa T Triess, Mariella Dreissig, Christoph B Rist, and J Marius Zöllner. A survey on deep domain adaptation for lidar perception. In IV, 2021.","title":"A survey on deep domain adaptation for lidar perception","authors":"Mariella Larissa T Triess, Christoph B Dreissig, Marius Rist, Zöllner","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":68,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, et al. Attention is all you need. In NeurIPS, 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":69,"raw":"[Vuong et al., 2019] Quan Vuong, Sharad Vikram, Hao Su, Sicun Gao, and Henrik I Christensen. How to pick the domain randomization parameters for sim-to-real transfer of reinforcement learning policies? arXiv, 2019.","title":"How to pick the domain randomization parameters for sim-to-real transfer of reinforcement learning policies?","authors":"Quan Vuong, Sharad Vikram, Hao Su, Sicun Gao, Henrik I Christensen","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":70,"raw":"[Walke et al., 2023] Homer Rich Walke, Kevin Black, Tony Z Zhao, et al. Bridgedata v2: A dataset for robot learning at scale. In CoRL, 2023.","title":"Bridgedata v2: A dataset for robot learning at scale","authors":"Walke","year":"2023","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":71,"raw":"[Wang et al., 2022a] Guan Wang, Haoyi Niu, Desheng Zhu, Jianming Hu, Xianyuan Zhan, and Guyue Zhou. A versa- tile and efficient reinforcement learning approach for au- tonomous driving. NeurIPS ML4AD Workshop, 2022.","title":"A versatile and efficient reinforcement learning approach for autonomous driving","authors":"Wang","year":"2022","venue":"NeurIPS ML4AD Workshop","doi":null,"url":null,"internal_url":null},{"position":72,"raw":"[Wang et al., 2022b] Zihan Wang, Zhangjie Cao, Yilun Hao, and Dorsa Sadigh. Weakly supervised correspondence learning. In ICRA, 2022.","title":"Weakly supervised correspondence learning","authors":"Wang","year":"2022","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":73,"raw":"[Wang et al., 2023] Zidan Wang, Takeru Oba, Takuma Yoneda, Rui Shen, Matthew Walter, and Bradly C Stadie. Cold diffusion on the replay buffer: Learning to plan from known good states. In CoRL, 2023.","title":"Cold diffusion on the replay buffer: Learning to plan from known good states","authors":"Zidan Wang, Takeru Oba, Takuma Yoneda, Rui Shen, Matthew Walter, Bradly C Stadie","year":"2023","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":74,"raw":"[Xu et al., 2023a] Kang Xu, Chenjia Bai, Xiaoteng Ma, Dong Wang, Bin Zhao, Zhen Wang, Xuelong Li, and Wei Li. Cross-domain policy adaptation via value-guided data filtering. In NeurIPS, 2023.","title":"Cross-domain policy adaptation via value-guided data filtering","authors":"Xu","year":"2023","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":75,"raw":"[Xu et al., 2023b] Mengda Xu, Zhenjia Xu, Cheng Chi, Manuela Veloso, and Shuran Song. Xskill: Cross embod- iment skill discovery. In CoRL, 2023.","title":"Xskill: Cross embodiment skill discovery","authors":"Xu","year":"2023","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":76,"raw":"[Xue et al., 2023] Zhenghai Xue, Qingpeng Cai, Shuchang Liu, Dong Zheng, Peng Jiang, Kun Gai, and Bo An. State regularized policy optimization on data with dynamics shift. In NeurIPS, 2023.","title":"State regularized policy optimization on data with dynamics shift","authors":"Xue","year":"2023","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":77,"raw":"[Yang et al., 2023] Jonathan Heewon Yang, Dorsa Sadigh, and Chelsea Finn. Polybot: Training one policy across robots while embracing variability. In CoRL, 2023.","title":"Polybot: Training one policy across robots while embracing variability","authors":"Jonathan Heewon, Yang, Dorsa Sadigh, Chelsea Finn","year":"2023","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":78,"raw":"[Yu et al., 2017] Wenhao Yu, Jie Tan, C Karen Liu, and Greg Turk. Preparing for the unknown: Learning a universal policy with online system identification. arXiv, 2017.","title":"Preparing for the unknown: Learning a universal policy with online system identification","authors":"Wenhao Yu, Jie Tan, Karen Liu, Greg Turk","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":79,"raw":"[Yu et al., 2018] Tianhe Yu, Chelsea Finn, Sudeep Dasari, et al. One-shot imitation from observing humans via domain-adaptive meta-learning. In RSS, 2018.","title":"One-shot imitation from observing humans via domain-adaptive meta-learning","authors":"Yu","year":"2018","venue":"RSS","doi":null,"url":null,"internal_url":null},{"position":80,"raw":"[Yu et al., 2023] Chen Yu, Weinan Zhang, Hang Lai, Zheng Tian, Laurent Kneip, and Jun Wang. Multi-embodiment legged robot control as a sequence modeling problem. In ICRA, 2023.","title":"Multi-embodiment legged robot control as a sequence modeling problem","authors":"Chen Yu, Weinan Zhang, Hang Lai, Zheng Tian, Laurent Kneip, Jun Wang","year":"2023","venue":"ICRA","doi":null,"url":null,"internal_url":null},{"position":81,"raw":"[Yue et al., 2019] Xiangyu Yue, Yang Zhang, Sicheng Zhao, Alberto Sangiovanni-Vincentelli, Kurt Keutzer, and Bo- qing Gong. Domain randomization and pyramid consis- tency: Simulation-to-real generalization without accessing target domain data. In ICCV, 2019.","title":"Domain randomization and pyramid consistency: Simulation-to-real generalization without accessing target domain data","authors":"Xiangyu Yue, Yang Zhang, Sicheng Zhao, Alberto Sangiovanni-Vincentelli, Kurt Keutzer, Boqing Gong","year":"2019","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":82,"raw":"[Zakka et al., 2021] Kevin Zakka, Andy Zeng, Pete Flo- rence, et al. Xirl: Cross-embodiment inverse reinforce- ment learning. In CoRL, 2021.","title":"Xirl: Cross-embodiment inverse reinforcement learning","authors":"Kevin Zakka, Andy Zeng, Pete Florence","year":"2021","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":83,"raw":"[Zhang et al., 2019] Jingwei Zhang, Lei Tai, Peng Yun, et al. Vr-goggles for robots: Real-to-sim domain adaptation for visual control. RAL, 2019.","title":"Vr-goggles for robots: Real-to-sim domain adaptation for visual control","authors":"Jingwei Zhang, Lei Tai, Peng Yun","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":84,"raw":"[Zhang et al., 2020] Qiang Zhang, Tete Xiao, Alexei A Efros, Lerrel Pinto, and Xiaolong Wang. Learning cross- domain correspondence for control with dynamics cycle- consistency. In ICLR, 2020.","title":"Learning crossdomain correspondence for control with dynamics cycleconsistency","authors":"Qiang Zhang, Tete Xiao, Alexei A Efros, Lerrel Pinto, Xiaolong Wang","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":85,"raw":"[Zhu et al., 2017] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image transla- tion using cycle-consistent adversarial networks. In ICCV, 2017.","title":"Unpaired image-to-image translation using cycle-consistent adversarial networks","authors":"Zhu","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":86,"raw":"[Zintgraf et al., 2019] Luisa Zintgraf, Kyriacos Shiarlis, Maximilian Igl, et al. Varibad: A very good method for bayes-adaptive deep rl via meta-learning. In ICLR, 2019.","title":"Varibad: A very good method for bayes-adaptive deep rl via meta-learning","authors":"Luisa Zintgraf, Kyriacos Shiarlis, Maximilian Igl","year":"2019","venue":"ICLR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}