{"type":"ScholarlyArticle","title":"Curriculum Learning in Reinforcement Learning","authors":[{"name":"Sanmit Narvekar","slug":"sanmit-narvekar","given":"Sanmit","family":"Narvekar","affiliation":"Department of Computer Science University of Texas at Austin"}],"abstract":"Transfer learning in reinforcement learning is an area of research that seeks to speed up or improve learning of a complex target task, by leveraging knowledge from one or more source tasks. This thesis will extend the concept of transfer learning to curriculum learning, where the goal is to design a sequence of source tasks for an agent to train on, such that final performance or learning speed is improved. We discuss completed work on this topic, including methods for semi-automatically generating source tasks tailored to an agent and the characteristics of a target domain, and automatically sequencing such tasks into a curriculum. Finally, we also present ideas for future work.","doi":"10.24963/ijcai.2017/757","url":"https://ijcai.org/proceedings/2017/757","pdf":"https://ijcai.org/proceedings/2017/0757.pdf","markdown":"https://ijcai.org/proceedings/2017/757.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/757","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Doctoral Consortium","track":"Doctoral Consortium","number":757,"pages":{"first":5195,"last":5196},"keywords":["Artificial Intelligence: machine learning","Artificial Intelligence: agents and multi-agent systems","Artificial Intelligence: artificial intelligence"],"videos":[],"references":[{"position":1,"raw":"[Fachantidis et al., 2013] Anestis Fachantidis, Ioannis Parta- las, Grigorios Tsoumakas, and Ioannis Vlahavas. Transfer- ring task models in reinforcement learning agents. Neuro- computing, 107:23-32, 2013.","title":"Transferring task models in reinforcement learning agents","authors":"Anestis Fachantidis, Ioannis Partalas, Grigorios Tsoumakas, Ioannis Vlahavas","year":"2013","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Fernández et al., 2010] Fernando Fernández, Javier García, and Manuela Veloso. Probabilistic policy reuse for inter- task transfer learning. Robotics and Autonomous Systems, 58(7):866 -871, 2010. Advances in Autonomous Robots for Service and Entertainment.","title":"Probabilistic policy reuse for intertask transfer learning","authors":"Fernando Fernández, Javier García, Manuela Veloso","year":"2010","venue":"Robotics and Autonomous Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Lazaric et al., 2008] A. Lazaric, M. Restelli, and A. Bonar- ini. Transfer of samples in batch reinforcement learning. In Proceedings of the Twenty-Fifth Annual International Conference on Machine Learning (ICML-2008), Helsinki, Finalnd, July 2008.","title":"Transfer of samples in batch reinforcement learning","authors":"A Lazaric, M Restelli, A Bonarini","year":"2008","venue":"Proceedings of the Twenty-Fifth Annual International Conference on Machine Learning (ICML-2008)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Lazaric, 2011] A. Lazaric. Transfer in reinforcement learn- ing: a framework and a survey. In M. Wiering and M. van Otterlo, editors, Reinforcement Learning: State of the Art. Springer, 2011.","title":"Transfer in reinforcement learning: a framework and a survey","authors":"A Lazaric","year":"2011","venue":"Reinforcement Learning: State of the Art","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Narvekar et al., 2016] Sanmit Narvekar, Jivko Sinapov, Matteo Leonetti, and Peter Stone. Source task creation for curriculum learning. In Proceedings of the 15th Interna- tional Conference on Autonomous Agents and Multiagent Systems (AAMAS 2016), May 2016.","title":"Source task creation for curriculum learning","authors":"Narvekar","year":"2016","venue":"Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2016)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Narvekar et al., 2017] Sanmit Narvekar, Jivko Sinapov, and Peter Stone. Autonomous task sequencing for customized curriculum design in reinforcement learning. In Proceed- ings of the 26th International Joint Conference on Artifi- cial Intelligence (IJCAI), August 2017.","title":"Autonomous task sequencing for customized curriculum design in reinforcement learning","authors":"Narvekar","year":"2017","venue":"Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2017/353"},{"position":7,"raw":"[Soni and Singh, 2006] Vishal Soni and Satinder Singh. Us- ing homomorphisms to transfer options across continuous reinforcement learning domains. In Proceedings of Amer- ican Association for Artificial Intelligence (AAAI), 2006.","title":"Using homomorphisms to transfer options across continuous reinforcement learning domains","authors":"Vishal Soni, Satinder Singh","year":"2006","venue":"Proceedings of American Association for Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Taylor and Stone, 2009] Matthew E. Taylor and Peter Stone. Transfer learning for reinforcement learning do- mains: A survey. Journal of Machine Learning Research, 10(1):1633-1685, 2009.","title":"Transfer learning for reinforcement learning domains: A survey","authors":"Matthew E Taylor, Peter Stone","year":"2009","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}