{"type":"ScholarlyArticle","title":"Proximal Curriculum with Task Correlations for Deep Reinforcement Learning","authors":[{"name":"Georgios Tzannetos","slug":"georgios-tzannetos","given":"Georgios","family":"Tzannetos","affiliation":"Max Planck Institute for Software Systems"},{"name":"Parameswaran Kamalaruban","slug":"parameswaran-kamalaruban","given":"Parameswaran","family":"Kamalaruban","affiliation":"The Alan Turing Institute"},{"name":"Adish Singla","slug":"adish-singla","given":"Adish","family":"Singla","affiliation":"Max Planck Institute for Software Systems"}],"abstract":"Curriculum design for reinforcement learning (RL) can speed up an agent's learning process and help it learn to perform well on complex tasks. However, existing techniques typically require domain-specific hyperparameter tuning, involve expensive optimization procedures for task selection, or are suitable only for specific learning objectives. In this work, we consider curriculum design in contextual multi-task settings where the agent's final performance is measured w.r.t. a target distribution over complex tasks. We base our curriculum design on the Zone of Proximal Development concept, which has proven to be effective in accelerating the learning process of RL agents for uniform distribution over all tasks. We propose a novel curriculum, ProCuRL-Target, that effectively balances the need for selecting tasks that are not too difficult for the agent while progressing the agent's learning toward the target distribution via leveraging task correlations. We theoretically justify the task selection strategy of ProCuRL-Target by analyzing a simple learning setting with REINFORCE learner model. Our experimental results across various domains with challenging target task distributions affirm the effectiveness of our curriculum strategy over state-of-the-art baselines in accelerating the training process of deep RL agents.","doi":"10.24963/ijcai.2024/556","url":"https://ijcai.org/proceedings/2024/556","pdf":"https://ijcai.org/proceedings/2024/0556.pdf","markdown":"https://ijcai.org/proceedings/2024/556.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/556","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":556,"pages":{"first":5027,"last":5036},"keywords":["Machine Learning: ML: Reinforcement learning","Planning and Scheduling: PS: Markov decisions processes"],"videos":[],"references":[{"position":1,"raw":"[Andrychowicz and others, 2017] Marcin Andrychowicz et al. Hindsight Experience Replay. In NeurIPS, 2017.","title":"Hindsight Experience Replay","authors":"Marcin Andrychowicz","year":"2017","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Baranes and Oudeyer, 2013] Adrien Baranes and Pierre- Yves Oudeyer. Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots. Robotics and Autonomous Systems, 61(1):49-73, 2013.","title":"Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots","authors":"Oudeyer Baranes, Adrien Baranes, Pierre-Yves Oudeyer","year":"2013","venue":"Robotics and Autonomous Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bengio et al., 2009] Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum Learn- ing. In ICML, 2009.","title":"Curriculum Learning","authors":"Yoshua Bengio, Jérôme Louradour, Ronan Collobert, Jason Weston","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chaiklin, 2003] Seth Chaiklin. The Zone of Proximal De- velopment in Vygotsky's Analysis of Learning and In- struction. Vygotsky's Educational Theory in Cultural Con- text, page 39, 2003.","title":"Seth Chaiklin. The Zone of Proximal Development in Vygotsky's Analysis of Learning and Instruction","authors":"Chaiklin","year":"2003","venue":"Vygotsky's Educational Theory in Cultural Context","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chevalier-Boisvert et al., 2023] Maxime Chevalier- Boisvert et al. Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks. CoRR, abs/2306.13831, 2023.","title":"Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks","authors":"Chevalier-Boisvert","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dendorfer et al., 2020] Patrick Dendorfer, Aljosa Osep, and Laura Leal-Taixé. Goal-GAN: Multimodal Trajectory Pre- diction based on Goal Position Estimation. In ACCV, 2020.","title":"Goal-GAN: Multimodal Trajectory Prediction based on Goal Position Estimation","authors":"Patrick Dendorfer, Aljosa Osep, Laura Leal-Taixé","year":"2020","venue":"ACCV","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dennis et al., 2020] Michael Dennis, Natasha Jaques, Eu- gene Vinitsky, Alexandre Bayen, Stuart Russell, Andrew Critch, and Sergey Levine. Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design. In NeurIPS, 2020.","title":"Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design","authors":"Michael Dennis, Natasha Jaques, Eugene Vinitsky, Alexandre Bayen, Stuart Russell, Andrew Critch, Sergey Levine","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Eimer et al., 2021] Theresa Eimer, André Biedenkapp, Frank Hutter, and Marius Lindauer. Self-Paced Context Evaluation for Contextual Reinforcement Learning. In ICML, 2021.","title":"Self-Paced Context Evaluation for Contextual Reinforcement Learning","authors":"Theresa Eimer, André Biedenkapp, Frank Hutter, Marius Lindauer","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Elman, 1993] Jeffrey L Elman. Learning and Development in Neural Networks: The Importance of Starting Small. Cognition, 48(1):71-99, 1993.","title":"Learning and Development in Neural Networks: The Importance of Starting Small","authors":"Jeffrey L Elman","year":"1993","venue":"Cognition","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Florensa et al., 2017] Carlos Florensa, David Held, Markus Wulfmeier, Michael Zhang, and Pieter Abbeel. Reverse Curriculum Generation for Reinforcement Learning. In CORL, 2017.","title":"Reverse Curriculum Generation for Reinforcement Learning","authors":"Carlos Florensa, David Held, Markus Wulfmeier, Michael Zhang, Pieter Abbeel","year":"2017","venue":"CORL","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Florensa et al., 2018] Carlos Florensa, David Held, Xinyang Geng, and Pieter Abbeel. Automatic Goal Generation for Reinforcement Learning Agents. In ICML, 2018.","title":"Automatic Goal Generation for Reinforcement Learning Agents","authors":"Carlos Florensa, David Held, Xinyang Geng, Pieter Abbeel","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Graves et al., 2017] Alex Graves, Marc G Bellemare, Ja- cob Menick, Rémi Munos, and Koray Kavukcuoglu. Au- tomated Curriculum Learning for Neural Networks. In ICML, 2017.","title":"Automated Curriculum Learning for Neural Networks","authors":"Alex Graves, Jacob Marc G Bellemare, Rémi Menick, Koray Munos, Kavukcuoglu","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hallak et al., 2015] Assaf Hallak, Dotan Di Castro, and Shie Mannor. Contextual Markov Decision Processes. CoRR, abs/1502.02259, 2015.","title":"Contextual Markov Decision Processes","authors":"Assaf Hallak, Dotan Di Castro, Shie Mannor","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Huang et al., 2022] Peide Huang, Mengdi Xu, Jiacheng Zhu, Laixi Shi, Fei Fang, and Ding Zhao. Curriculum Re- inforcement Learning using Optimal Transport via Grad- ual Domain Adaptation. In NeurIPS, 2022. Jiang et al., 2015]","title":"Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptation","authors":"Huang","year":"2015","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"Lu Jiang, Deyu Meng, Qian Zhao, Shiguang Shan, and Alexander G Hauptmann. Self-Paced Curriculum Learning. In AAAI, 2015.","title":"Self-Paced Curriculum Learning","authors":"Lu Jiang, Deyu Meng, Qian Zhao, Shiguang Shan, Alexander G Hauptmann","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Jiang et al., 2021a] Minqi Jiang, Michael Dennis, Jack Parker-Holder, Jakob Foerster, Edward Grefenstette, and Tim Rocktäschel. Replay-Guided Adversarial Environ- ment Design. In NeurIPS, 2021.","title":"Replay-Guided Adversarial Environment Design","authors":"Jiang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Jiang et al., 2021b] Minqi Jiang, Edward Grefenstette, and Tim Rocktäschel. Prioritized Level Replay. In ICML, 2021.","title":"Prioritized Level Replay","authors":"Jiang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kamalaruban et al., 2019] Parameswaran Kamalaruban, Rati Devidze, Volkan Cevher, and Adish Singla. Inter- active Teaching Algorithms for Inverse Reinforcement Learning. In IJCAI, 2019.","title":"Interactive Teaching Algorithms for Inverse Reinforcement Learning","authors":"Parameswaran Kamalaruban, Rati Devidze, Volkan Cevher, Adish Singla","year":"2019","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/374"},{"position":19,"raw":"[Kirk et al., 2021] Robert Kirk, Amy Zhang, Edward Grefenstette, and Tim Rocktäschel. A Survey of Gen- eralisation in Deep Reinforcement Learning. CoRR, abs/2111.09794, 2021.","title":"A Survey of Generalisation in Deep Reinforcement Learning","authors":"Robert Kirk, Amy Zhang, Edward Grefenstette, Tim Rocktäschel","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Klink et al., 2020] Pascal Klink, Carlo D'Eramo, Jan R Pe- ters, and Joni Pajarinen. Self-Paced Deep Reinforcement Learning. In NeurIPS, 2020.","title":"Self-Paced Deep Reinforcement Learning","authors":"Pascal Klink, D' Carlo, Jan R Eramo, Joni Peters, Pajarinen","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Klink et al., 2021] Pascal Klink, Hany Abdulsamad, Boris Belousov, Carlo D'Eramo, Jan Peters, and Joni Pajarinen. A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning. JMLR, 22:182- 1, 2021.","title":"A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning","authors":"Pascal Klink, Hany Abdulsamad, Boris Belousov, Carlo D' Eramo, Jan Peters, Joni Pajarinen","year":"2021","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Klink et al., 2022] Pascal Klink, Haoyi Yang, Carlo D'Eramo, Jan Peters, and Joni Pajarinen. Curriculum Re- inforcement Learning via Constrained Optimal Transport. In ICML, 2022.","title":"Curriculum Reinforcement Learning via Constrained Optimal Transport","authors":"Pascal Klink, Haoyi Yang, Carlo D' Eramo, Jan Peters, Joni Pajarinen","year":"2022","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Kumar et al., 2010] M Pawan Kumar, Benjamin Packer, and Daphne Koller. Self-Paced Learning for Latent Variable Models. In NeurIPS, 2010.","title":"Self-Paced Learning for Latent Variable Models","authors":"Benjamin M Pawan Kumar, Daphne Packer, Koller","year":"2010","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Levine et al., 2016] Sergey Levine, Chelsea Finn, Trevor Darrell, and Pieter Abbeel. End-to-end Training of Deep Visuomotor Policies. JMLR, 17(1):1334-1373, 2016.","title":"End-to-end Training of Deep Visuomotor Policies","authors":"Sergey Levine, Chelsea Finn, Trevor Darrell, Pieter Abbeel","year":"2016","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Lillicrap et al., 2015] Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous Control with Deep Reinforcement Learning. CoRR, abs/1509.02971, 2015.","title":"Continuous Control with Deep Reinforcement Learning","authors":"Jonathan J Timothy P Lillicrap, Alexander Hunt, Nicolas Pritzel, Tom Heess, Yuval Erez, David Tassa, Daan Silver, Wierstra","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Liu et al., 2017] Weiyang Liu, Bo Dai, Ahmad Humayun, Charlene Tay, Chen Yu, Linda B Smith, James M Rehg, and Le Song. Iterative Machine Teaching. In ICML, 2017.","title":"Iterative Machine Teaching","authors":"Weiyang Liu, Bo Dai, Ahmad Humayun, Charlene Tay, Chen Yu, Linda B Smith, James M Rehg, Le Song","year":"2017","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Matiisen et al., 2019] Tambet Matiisen, Avital Oliver, Taco Cohen, and John Schulman. Teacher-Student Curriculum Learning. IEEE Transactions on Neural Networks and Learning Systems, 31(9):3732-3740, 2019.","title":"Teacher-Student Curriculum Learning","authors":"Matiisen","year":"2019","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Mnih et al., 2015] Volodymyr Mnih et al. Human-Level Control Through Deep Reinforcement Learning. Nature, 518(7540):529-533, 2015.","title":"Volodymyr Mnih et al. Human-Level Control Through Deep Reinforcement Learning","authors":"Mnih","year":"2015","venue":"Nature","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Modi et al., 2018] Aditya Modi, Nan Jiang, Satinder Singh, and Ambuj Tewari. Markov Decision Processes with Con- tinuous Side Information. In ALT, 2018.","title":"Markov Decision Processes with Continuous Side Information","authors":"Aditya Modi, Nan Jiang, Satinder Singh, Ambuj Tewari","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Narvekar and Stone, 2019] Sanmit Narvekar and Peter Stone. Learning Curriculum Policies for Reinforcement Learning. In AAMAS, 2019.","title":"Sanmit Narvekar and Peter Stone. Learning Curriculum Policies for Reinforcement Learning","authors":"Stone Narvekar","year":"2019","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Narvekar et al., 2020] Sanmit Narvekar, Bei Peng, Matteo Leonetti, Jivko Sinapov, Matthew E Taylor, and Peter Stone. Curriculum Learning for Reinforcement Learn- ing Domains: A Framework and Survey. JMLR, 21:1-50, 2020.","title":"Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey","authors":"Narvekar","year":"2020","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Oudeyer et al., 2007] Pierre-Yves Oudeyer, Frdric Kaplan, and Verena V Hafner. Intrinsic Motivation Systems for Autonomous Mental Development. IEEE Transactions on Evolutionary Computation, 11(2):265-286, 2007.","title":"Intrinsic Motivation Systems for Autonomous Mental Development","authors":"Oudeyer","year":"2007","venue":"IEEE Transactions on Evolutionary Computation","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Parker-Holder et al., 2022] Jack Parker-Holder, Minqi Jiang, Michael Dennis, Mikayel Samvelyan, Jakob Foer- ster, Edward Grefenstette, and Tim Rocktäschel. Evolving Curricula with Regret-Based Environment Design. CoRR, abs/2203.01302, 2022.","title":"Evolving Curricula with Regret-Based Environment Design","authors":"Parker-Holder","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Portelas et al., 2019] Rémy Portelas, Cédric Colas, Katja Hofmann, and Pierre-Yves Oudeyer. Teacher algorithms for curriculum learning of Deep RL in continuously pa- rameterized environments. In CoRL, 2019.","title":"Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments","authors":"Rémy Portelas, Cédric Colas, Katja Hofmann, Pierre-Yves Oudeyer","year":"2019","venue":"CoRL","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Portelas et al., 2021] Rémy Portelas, Cédric Colas, Lilian Weng, Katja Hofmann, and Pierre-Yves Oudeyer. Auto- matic Curriculum Learning for Deep RL: A Short Survey. In IJCAI, 2021.","title":"Automatic Curriculum Learning for Deep RL: A Short Survey","authors":"Rémy Portelas, Cédric Colas, Lilian Weng, Katja Hofmann, Pierre-Yves Oudeyer","year":"2021","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2020/671"},{"position":36,"raw":"[Racanière et al., 2020] Sébastien Racanière, Andrew K Lampinen, Adam Santoro, David P Reichert, Vlad Firoiu, and Timothy P Lillicrap. Automated Curricula Through Setter-Solver Interactions. In ICLR, 2020.","title":"Automated Curricula Through Setter-Solver Interactions","authors":"Sébastien Racanière, Adam Andrew K Lampinen, David P Santoro, Vlad Reichert, Timothy P Firoiu, Lillicrap","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Raffin et al., 2021] Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable-Baselines3: Reliable Reinforcement Learning Implementations. JMLR, 22(268):1-8, 2021.","title":"Stable-Baselines3: Reliable Reinforcement Learning Implementations","authors":"Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, Noah Dormann","year":"2021","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Riedmiller et al., 2018] Martin A Riedmiller et al. Learning by Playing Solving Sparse Reward Tasks from Scratch. In ICML, 2018.","title":"Learning by Playing Solving Sparse Reward Tasks from Scratch","authors":"A Martin, Riedmiller","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Romac et al., 2021] Clément Romac, Rémy Portelas, Katja Hofmann, and Pierre-Yves Oudeyer. Teachmyagent: a Benchmark for Automatic Curriculum Learning in Deep RL. In ICML, 2021.","title":"Teachmyagent: a Benchmark for Automatic Curriculum Learning in Deep RL","authors":"Clément Romac, Rémy Portelas, Katja Hofmann, Pierre-Yves Oudeyer","year":"2021","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Salimans and Chen, 2018] Tim Salimans and Richard Chen. Learning Montezuma's Revenge from a Single Demonstration. CoRR, abs/1812.03381, 2018.","title":"Learning Montezuma's Revenge from a Single Demonstration","authors":"Chen ; Salimans, Tim Salimans, Richard Chen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Schmidhuber, 2013] Jürgen Schmidhuber. Powerplay: Training an Increasingly General Problem Solver by Con- tinually Searching for the Simplest Still Unsolvable Prob- lem. Frontiers in Psychology, 4:313, 2013.","title":"Powerplay: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem","authors":"Jürgen Schmidhuber","year":"2013","venue":"Frontiers in Psychology","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms. CoRR, abs/1707.06347, 2017.","title":"Proximal Policy Optimization Algorithms","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Silver et al., 2017] David Silver et al. Mastering the Game of Go Without Human Knowledge. Nature, 550(7676):354-359, 2017. Sukhbaatar et al., 2018] Sainbayar Sukhbaatar, Zeming Lin, Ilya Kostrikov, Gabriel Synnaeve, Arthur Szlam, and Robert Fergus. Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play. In ICLR, 2018.","title":"Mastering the Game of Go Without Human Knowledge","authors":"David Silver","year":"2017","venue":"Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Sutton et al., 1999] Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy Gradient Methods for Reinforcement Learning with Function Ap- proximation. In NeurIPS, 1999.","title":"Policy Gradient Methods for Reinforcement Learning with Function Approximation","authors":"David Richard S Sutton, Satinder Mcallester, Yishay Singh, Mansour","year":"1999","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Turchetta et al., 2020] Matteo Turchetta, Andrey Kolobov, Shital Shah, Andreas Krause, and Alekh Agarwal. Safe Reinforcement Learning via Curriculum Induction. In NeurIPS, 2020.","title":"Safe Reinforcement Learning via Curriculum Induction","authors":"Matteo Turchetta, Andrey Kolobov, Shital Shah, Andreas Krause, Alekh Agarwal","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Tzannetos et al., 2023] Georgios Tzannetos, Bar- bara Gomes Ribeiro, Parameswaran Kamalaruban, and Adish Singla. Proximal Curriculum for Reinforcement Learning Agents. TMLR, 2023.","title":"Proximal Curriculum for Reinforcement Learning Agents","authors":"Georgios Tzannetos, Barbara Gomes Ribeiro, Parameswaran Kamalaruban, Adish Singla","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Vygotsky and Cole, 1978] Lev Semenovich Vygotsky and Michael Cole. Mind in Society: Development of Higher Psychological Processes. Harvard University Press, 1978.","title":"Mind in Society: Development of Higher Psychological Processes","authors":"Cole Vygotsky","year":"1978","venue":null,"doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Weinshall et al., 2018] Daphna Weinshall, Gad Cohen, and Dan Amir. Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networks. In ICML, 2018.","title":"Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networks","authors":"Daphna Weinshall, Gad Cohen, Dan Amir","year":"2018","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Wöhlke et al., 2020] Jan Wöhlke, Felix Schmitt, and Herke van Hoof. A Performance-Based Start State Curriculum Framework for Reinforcement Learning. In AAMAS, 2020.","title":"A Performance-Based Start State Curriculum Framework for Reinforcement Learning","authors":"Wöhlke","year":"2020","venue":"AAMAS","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Wu and Tian, 2016] Yuxin Wu and Yuandong Tian. Train- ing Agent for First-Person Shooter Game with Actor- Critic Curriculum Learning. In ICLR, 2016.","title":"Training Agent for First-Person Shooter Game with Actor-Critic Curriculum Learning","authors":"Tian ; ] Yuxin Wu, Yuandong Wu, Tian","year":"2016","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Yengera et al., 2021] Gaurav Raju Yengera, Rati Devidze, Parameswaran Kamalaruban, and Adish Singla. Curricu- lum Design for Teaching via Demonstrations: Theory and Applications. In NeurIPS, 2021.","title":"Curriculum Design for Teaching via Demonstrations: Theory and Applications","authors":"Rati Gaurav Raju Yengera, Parameswaran Devidze, Adish Kamalaruban, Singla","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Zhang et al., 2020] Yunzhi Zhang, Pieter Abbeel, and Ler- rel Pinto. Automatic Curriculum Learning Through Value Disagreement. In NeurIPS, 2020.","title":"Automatic Curriculum Learning Through Value Disagreement","authors":"Zhang","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Zhou and Bilmes, 2018] Tianyi Zhou and Jeff Bilmes. Min- imax Curriculum Learning: Machine Teaching with Desir- able Difficulties and Scheduled Diversity. In ICLR, 2018.","title":"Minimax Curriculum Learning: Machine Teaching with Desirable Difficulties and Scheduled Diversity","authors":"Bilmes Zhou, Tianyi Zhou, Jeff Bilmes","year":"2018","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Zhu et al., 2018] Xiaojin Zhu, Adish Singla, Sandra Zilles, and Anna N. Rafferty. An Overview of Machine Teaching. CoRR, abs/1801.05927, 2018.","title":"An Overview of Machine Teaching","authors":"Xiaojin Zhu, Adish Singla, Sandra Zilles, Anna N Rafferty","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Zou et al., 2019] Xiaotian Zou, Wei Ma, Zhenjun Ma, and Ryan S Baker. Towards Helping Teachers Select Optimal Content for Students. In AIED, 2019.","title":"Towards Helping Teachers Select Optimal Content for Students","authors":"Xiaotian Zou, Wei Ma, Zhenjun Ma, Ryan S Baker","year":"2019","venue":"AIED","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}