{"type":"ScholarlyArticle","title":"CROP: Towards Distributional-Shift Robust Reinforcement Learning Using Compact Reshaped Observation Processing","authors":[{"name":"Philipp Altmann","slug":"philipp-altmann","given":"Philipp","family":"Altmann","affiliation":"LMU Munich"},{"name":"Fabian Ritz","slug":"fabian-ritz","given":"Fabian","family":"Ritz","affiliation":"LMU Munich"},{"name":"Leonard Feuchtinger","slug":"leonard-feuchtinger","given":"Leonard","family":"Feuchtinger","affiliation":"LMU Munich"},{"name":"Jonas Nüßlein","slug":"jonas-nu-lein","given":"Jonas","family":"Nüßlein","affiliation":"LMU Munich"},{"name":"Claudia Linnhoff-Popien","slug":"claudia-linnhoff-popien","given":"Claudia","family":"Linnhoff-Popien","affiliation":"LMU Munich"},{"name":"Thomy Phan","slug":"thomy-phan","given":"Thomy","family":"Phan","affiliation":"LMU Munich"}],"abstract":"The safe application of reinforcement learning (RL) requires generalization from limited training data to unseen scenarios. Yet, fulfilling tasks under changing circumstances is a key challenge in RL. Current state-of-the-art approaches for generalization apply data augmentation techniques to increase the diversity of training data. Even though this prevents overfitting to the training environment(s), it hinders policy optimization. Crafting a suitable observation, only containing crucial information, has been shown to be a challenging task itself. To improve data efficiency and generalization capabilities, we propose Compact Reshaped Observation Processing (CROP) to reduce the state information used for policy optimization. By providing only relevant information, overfitting to a specific training layout is precluded and generalization to unseen environments is improved. We formulate three CROPs that can be applied to fully observable observation- and action-spaces and provide methodical foundation. We empirically show the improvements of CROP in a distributionally shifted safety gridworld. We furthermore provide benchmark comparisons to full observability and data-augmentation in two different-sized procedurally generated mazes.","doi":"10.24963/ijcai.2023/380","url":"https://ijcai.org/proceedings/2023/380","pdf":"https://ijcai.org/proceedings/2023/0380.pdf","markdown":"https://ijcai.org/proceedings/2023/380.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/380","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":380,"pages":{"first":3414,"last":3422},"keywords":["Machine Learning: ML: Deep reinforcement learning","Machine Learning: ML: Robustness","AI Ethics, Trust, Fairness: ETF: Safety and robustness"],"videos":[],"references":[{"position":1,"raw":"[Agarwal et al., 2021] Rishabh Agarwal, Marlos C. Machado, Pablo Samuel Castro, and Marc G. Belle- mare. Contrastive behavioral similarity embeddings for generalization in reinforcement learning. In International Conference on Learning Representations, 2021.","title":"Contrastive behavioral similarity embeddings for generalization in reinforcement learning","authors":"Rishabh Agarwal, Marlos C Machado, Pablo Samuel Castro, Marc G Bellemare","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Amodei et al., 2016] Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in ai safety. arXiv preprint arXiv:1606.06565, 2016.","title":"Concrete problems in ai safety","authors":"Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, Dan Mané","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bartlett et al., 2020] Peter L Bartlett, Philip M Long, Gábor Lugosi, and Alexander Tsigler. Benign overfitting in lin- ear regression. Proceedings of the National Academy of Sciences, 117(48):30063-30070, 2020.","title":"Benign overfitting in linear regression","authors":"Philip M Peter L Bartlett, Gábor Long, Alexander Lugosi, Tsigler","year":"2020","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bishop and Nasrabadi, 2006] Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning, volume 4. Springer, 2006.","title":"Pattern recognition and machine learning","authors":"M Christopher, Bishop, M Nasser, Nasrabadi","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Brockman et al., 2016] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01540, 2016.","title":"Openai gym","authors":"Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, Wojciech Zaremba","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2014] Bor-Chun Chen, Chu-Song Chen, and Winston H Hsu. Cross-age reference coding for age- invariant face recognition and retrieval. In European conference on computer vision, pages 768-783. Springer, 2014.","title":"Cross-age reference coding for ageinvariant face recognition and retrieval","authors":"Chu-Song Bor-Chun Chen, Winston H Chen, Hsu","year":"2014","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2020] Pengguang Chen, Shu Liu, Hengshuang Zhao, and Jiaya Jia. Gridmask data augmentation. arXiv preprint arXiv:2001.04086, 2020.","title":"Gridmask data augmentation","authors":"Pengguang Chen, Shu Liu, Hengshuang Zhao, Jiaya Jia","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Choi et al., 2019] Jinyoung Choi, Kyungsik Park, Minsu Kim, and Sangok Seok. Deep reinforcement learning of navigation in a complex and crowded environment with a limited field of view. In 2019 International Conference on Robotics and Automation (ICRA), pages 5993-6000. IEEE, 2019.","title":"Deep reinforcement learning of navigation in a complex and crowded environment with a limited field of view","authors":"Jinyoung Choi, Kyungsik Park, Minsu Kim, Sangok Seok","year":"2019","venue":"2019 International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cobbe et al., 2020] Karl Cobbe, Chris Hesse, Jacob Hilton, and John Schulman. Leveraging procedural generation to benchmark reinforcement learning. In International con- ference on machine learning, pages 2048-2056. PMLR, 2020.","title":"Leveraging procedural generation to benchmark reinforcement learning","authors":"Karl Cobbe, Chris Hesse, Jacob Hilton, John Schulman","year":"2020","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ghosh et al., 2021] Dibya Ghosh, Jad Rahme, Aviral Ku- mar, Amy Zhang, Ryan P Adams, and Sergey Levine. Why generalization in rl is difficult: Epistemic pomdps and implicit partial observability. Advances in Neural In- formation Processing Systems, 34:25502-25515, 2021.","title":"Why generalization in rl is difficult: Epistemic pomdps and implicit partial observability","authors":"Dibya Ghosh, Jad Rahme, Aviral Kumar, Amy Zhang, Ryan P Adams, Sergey Levine","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gisslén et al., 2021] Linus Gisslén, Andy Eakins, Camilo Gordillo, Joakim Bergdahl, and Konrad Tollmar. Adver- sarial reinforcement learning for procedural content gener- ation. In 2021 IEEE Conference on Games (CoG), pages 1-8. IEEE, 2021.","title":"Adversarial reinforcement learning for procedural content generation","authors":"Gisslén","year":"2021","venue":"IEEE Conference on Games (CoG)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hendrycks et al., 2021] Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical anal- ysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8340-8349, 2021.","title":"The many faces of robustness: A critical analysis of out-of-distribution generalization","authors":"Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo","year":"2021","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jaakkola et al., 1994] Tommi Jaakkola, Satinder Singh, and Michael Jordan. Reinforcement learning algorithm for partially observable markov decision problems. Advances in neural information processing systems, 7, 1994.","title":"Reinforcement learning algorithm for partially observable markov decision problems","authors":"Tommi Jaakkola, Satinder Singh, Michael Jordan","year":"1994","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kaelbling et al., 1998] Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains. Artificial intel- ligence, 101(1-2):99-134, 1998.","title":"Planning and acting in partially observable stochastic domains","authors":"Leslie Pack, Kaelbling Michael L Littman, Anthony R Cassandra","year":"1998","venue":"Artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Karystinos and Pados, 2000] George N Karystinos and Dimitrios A Pados. On overfitting, generalization, and randomly expanded training sets. IEEE Transactions on Neural Networks, 11(5):1050-1057, 2000.","title":"On overfitting, generalization, and randomly expanded training sets","authors":"N George, Dimitrios A Karystinos, Pados","year":"2000","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Laskin et al., 2020] Misha Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, and Aravind Srinivas. Reinforcement learning with augmented data. Advances in neural information processing systems, 33:19884-19895, 2020.","title":"Reinforcement learning with augmented data","authors":"Misha Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, Aravind Srinivas","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Leike et al., 2017] Jan Leike, Miljan Martic, Victoria Krakovna, Pedro A Ortega, Tom Everitt, Andrew Lefrancq, Laurent Orseau, and Shane Legg. Ai safety grid- worlds. arXiv preprint arXiv:1711.09883, 2017.","title":"Ai safety gridworlds","authors":"Leike","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Malinin et al., 2021] Andrey Malinin, Neil Band, Ger- man Chesnokov, Yarin Gal, Mark JF Gales, Alexey Noskov, Andrey Ploskonosov, Liudmila Prokhorenkova, Ivan Provilkov, Vatsal Raina, et al. Shifts: A dataset of real distributional shift across multiple large-scale tasks. arXiv preprint arXiv:2107.07455, 2021.","title":"Shifts: A dataset of real distributional shift across multiple large-scale tasks","authors":"Andrey Malinin, Neil Band, German Chesnokov, Yarin Gal, J F Mark, Alexey Gales, Andrey Noskov, Liudmila Ploskonosov, Ivan Prokhorenkova, Provilkov, Raina Vatsal","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mazoure et al., 2022] Bogdan Mazoure, Ahmed M Ahmed, R Devon Hjelm, Andrey Kolobov, and Patrick MacAlpine. Cross-trajectory representation learning for zero-shot gen- eralization in RL. In International Conference on Learn- ing Representations, 2022.","title":"Cross-trajectory representation learning for zero-shot generalization in RL","authors":"Bogdan Mazoure, Ahmed M Ahmed, Devon Hjelm, Andrey Kolobov, Patrick Macalpine","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mnih et al., 2016] Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asyn- chronous Methods for Deep Reinforcement Learning. In International Conference on Machine Learning, 2016.","title":"Asynchronous Methods for Deep Reinforcement Learning","authors":"Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, Koray Kavukcuoglu","year":"2016","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Najar and Chetouani, 2021] Anis Najar and Mohamed Chetouani. Reinforcement learning with human advice: a survey. Frontiers in Robotics and AI, 8:584075, 2021.","title":"Reinforcement learning with human advice: a survey","authors":"Chetouani Najar, Anis Najar, Mohamed Chetouani","year":"2021","venue":"Frontiers in Robotics and AI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Pimentel et al., 2014] Marco AF Pimentel, David A Clifton, Lei Clifton, and Lionel Tarassenko. A review of novelty detection. Signal processing, 99:215-249, 2014.","title":"A review of novelty detection","authors":"A F Marco, David A Pimentel, Lei Clifton, Lionel Clifton, Tarassenko","year":"2014","venue":"Signal processing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Pinto et al., 2017] Lerrel Pinto, James Davidson, Rahul Sukthankar, and Abhinav Gupta. Robust adversarial re- inforcement learning. In International Conference on Ma- chine Learning, pages 2817-2826. PMLR, 2017.","title":"Robust adversarial reinforcement learning","authors":"Pinto","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Puterman, 1990] Martin L Puterman. Markov decision pro- cesses. Handbooks in operations research and manage- ment science, 2:331-434, 1990.","title":"Markov decision processes","authors":"Martin L Puterman","year":"1990","venue":"Handbooks in operations research and management science","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Quinonero-Candela et al., 2008] Joaquin Quinonero- Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. Dataset shift in machine learning. Mit Press, 2008.","title":"Dataset shift in machine learning","authors":"Quinonero-Candela","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Raffin et al., 2021] Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22(268):1-8, 2021.","title":"Stable-baselines3: Reliable reinforcement learning implementations","authors":"Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, Noah Dormann","year":"2021","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Raileanu et al., 2021] Roberta Raileanu, Maxwell Gold- stein, Denis Yarats, Ilya Kostrikov, and Rob Fergus. Automatic data augmentation for generalization in rein- forcement learning. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, vol- ume 34, pages 5402-5415.","title":"Automatic data augmentation for generalization in reinforcement learning","authors":"Roberta Raileanu, Maxwell Goldstein, Denis Yarats, Ilya Kostrikov, Rob Fergus","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"Curran Associates, Inc., 2021.","title":null,"authors":"Curran Associates, Inc","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Ramanan et al., 2021] Nandini Ramanan, Rasool Tah- masbi, Marjorie Sayer, Deokwoo Jung, Shalini Hemachandran, and Claudionor Nunes Coelho Jr. Real-time drift detection on time-series data. arXiv preprint arXiv:2110.06383, 2021.","title":"Real-time drift detection on time-series data","authors":"Rasool Nandini Ramanan, Marjorie Tahmasbi, Deokwoo Sayer, Shalini Jung, Claudionor Hemachandran, Coelho Nunes","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Raskutti et al., 2014] Garvesh Raskutti, Martin J Wain- wright, and Bin Yu. Early stopping and non-parametric regression: an optimal data-dependent stopping rule. The Journal of Machine Learning Research, 15(1):335-366, 2014.","title":"Early stopping and non-parametric regression: an optimal data-dependent stopping rule","authors":"Raskutti","year":"2014","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Prox- imal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.","title":"Proximal policy optimization algorithms","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Shorten and Khoshgoftaar, 2019] Connor Shorten and Taghi M Khoshgoftaar. A survey on image data augmen- tation for deep learning. Journal of big data, 6(1):1-48, 2019.","title":"A survey on image data augmentation for deep learning","authors":"Connor Shorten, M Taghi, Khoshgoftaar","year":"2019","venue":"Journal of big data","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Spaan, 2012] Matthijs TJ Spaan. Partially observable markov decision processes. In Reinforcement Learning, pages 387-414. Springer, 2012.","title":"Partially observable markov decision processes","authors":"T J Matthijs, Spaan","year":"2012","venue":"Reinforcement Learning","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Srivastava et al., 2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdi- nov. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning re- search, 15(1):1929-1958, 2014.","title":"Dropout: a simple way to prevent neural networks from overfitting","authors":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov","year":"2014","venue":"The journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Sutton and Barto, 2018] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.","title":"Reinforcement learning: An introduction","authors":"S Richard, Andrew G Sutton, Barto","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Sutton et al., 2000] Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. Policy Gradient Methods for Reinforcement Learning with Function Ap- proximation. In S. Solla, T. Leen, and K. Müller, editors, Advances in Neural Information Processing Systems, vol- ume 12, pages 1057-1063. MIT Press, 2000.","title":"Policy Gradient Methods for Reinforcement Learning with Function Approximation","authors":"David A Richard S Sutton, Mcallester, P Satinder, Yishay Singh, Mansour","year":"2000","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Thulasidasan et al., 2021] Sunil Thulasidasan, Sushil Thapa, Sayera Dhaubhadel, Gopinath Chennupati, Tan- moy Bhattacharya, and Jeff Bilmes. An effective baseline for robustness to distributional shift. In 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), pages 278-285. IEEE, 2021.","title":"An effective baseline for robustness to distributional shift","authors":"Sushil Sunil Thulasidasan, Sayera Thapa, Gopinath Dhaubhadel, Tanmoy Chennupati, Jeff Bhattacharya, Bilmes","year":"2021","venue":"20th IEEE International Conference on Machine Learning and Applications (ICMLA)","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Tobin et al., 2017] Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural net- works from simulation to the real world. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pages 23-30. IEEE, 2017.","title":"Domain randomization for transferring deep neural networks from simulation to the real world","authors":"Tobin","year":"2017","venue":"IEEE/RSJ international conference on intelligent robots and systems (IROS)","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Vlassis et al., 2012] Nikos Vlassis, Michael L Littman, and David Barber. On the computational complexity of stochastic controller optimization in pomdps. ACM Trans- actions on Computation Theory (TOCT), 4(4):1-8, 2012.","title":"On the computational complexity of stochastic controller optimization in pomdps","authors":"Nikos Vlassis, David Michael L Littman, Barber","year":"2012","venue":"ACM Transactions on Computation Theory (TOCT)","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Wang et al., 2020] Yaqing Wang, Quanming Yao, James T Kwok, and Lionel M Ni. Generalizing from a few ex- amples: A survey on few-shot learning. ACM computing surveys (csur), 53(3):1-34, 2020.","title":"Generalizing from a few examples: A survey on few-shot learning","authors":"Wang","year":"2020","venue":"ACM computing surveys (csur)","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Yarats et al., 2021] Denis Yarats, Ilya Kostrikov, and Rob Fergus. Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. In International Conference on Learning Representations, 2021.","title":"Image augmentation is all you need: Regularizing deep reinforcement learning from pixels","authors":"Denis Yarats, Ilya Kostrikov, Rob Fergus","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Yarats et al., 2022] Denis Yarats, Rob Fergus, Alessandro Lazaric, and Lerrel Pinto. Mastering visual continuous control: Improved data-augmented reinforcement learn- ing. In International Conference on Learning Represen- tations, 2022.","title":"Mastering visual continuous control: Improved data-augmented reinforcement learning","authors":"Denis Yarats, Rob Fergus, Alessandro Lazaric, Lerrel Pinto","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Ying, 2019] Xue Ying. An overview of overfitting and its solutions. Journal of Physics: Conference Series, 1168(2):022022, 2019.","title":"An overview of overfitting and its solutions","authors":"Xue Ying","year":"2019","venue":"Journal of Physics: Conference Series","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zhang et al., 2021] Amy Zhang, Rowan Thomas McAllis- ter, Roberto Calandra, Yarin Gal, and Sergey Levine. Learning invariant representations for reinforcement learn- ing without reconstruction. In International Conference on Learning Representations, 2021.","title":"Learning invariant representations for reinforcement learning without reconstruction","authors":"Amy Zhang, Thomas Rowan, Roberto Mcallister, Yarin Calandra, Sergey Gal, Levine","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Zhao et al., 2020] Wenshuai Zhao, Jorge Peña Queralta, and Tomi Westerlund. Sim-to-real transfer in deep rein- forcement learning for robotics: a survey. In 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pages 737-744. IEEE, 2020.","title":"Sim-to-real transfer in deep reinforcement learning for robotics: a survey","authors":"Zhao","year":"2020","venue":"IEEE Symposium Series on Computational Intelligence (SSCI)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}