{"type":"ScholarlyArticle","title":"Safe and Sample-Efficient Reinforcement Learning Algorithms for Factored Environments","authors":[{"name":"Thiago D. Simão","slug":"thiago-d-simao","given":"Thiago D.","family":"Simão","affiliation":"Delft University of Technology , The Netherlands"}],"abstract":"Reinforcement Learning (RL) deals with problems that can be modeled as a Markov Decision Process (MDP) where the transition function is unknown. In situations where an arbitrary policy pi is already in execution and the experiences with the environment were recorded in a batch D, an RL algorithm can use D to compute a new policy pi'. However, the policy computed by traditional RL algorithms might have worse performance compared to pi. Our goal is to develop safe RL algorithms, where the agent has a high confidence that the performance of pi' is better than the performance of pi given D. To develop sample-efficient and safe RL algorithms we combine ideas from exploration strategies in RL with a safe policy improvement method.","doi":"10.24963/ijcai.2019/919","url":"https://ijcai.org/proceedings/2019/919","pdf":"https://ijcai.org/proceedings/2019/0919.pdf","markdown":"https://ijcai.org/proceedings/2019/919.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/919","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Doctoral Consortium","track":"Doctoral Consortium","number":919,"pages":{"first":6460,"last":6461},"keywords":["Planning and Scheduling: Planning under Uncertainty","Machine Learning: Probabilistic Machine Learning","Machine Learning: Reinforcement Learning","Machine Learning: Learning Graphical Models"],"videos":[],"references":[{"position":1,"raw":"[Brafman and Tennenholtz, 2002] Ronen I. Brafman and Moshe Tennenholtz. R-MAX -A General Polynomial Time Algorithm for Near-Optimal Reinforcement Learning. JMLR, 3:213-231, 2002.","title":"R-MAX -A General Polynomial Time Algorithm for Near-Optimal Reinforcement Learning","authors":"Ronen I Brafman, Moshe Tennenholtz","year":"2002","venue":"JMLR","doi":null,"url":null,"internal_url":"/proceedings/2001/130"},{"position":2,"raw":"[Diuk et al., 2009] Carlos Diuk, Lihong Li, and Bethany R. Lef- fler. The Adaptive k-meteorologists Problem and Its Applica- tion to Structure Learning and Feature Selection in Reinforce- ment Learning. In ICML, pages 249-256. ACM, 2009.","title":"The Adaptive k-meteorologists Problem and Its Application to Structure Learning and Feature Selection in Reinforcement Learning","authors":"Carlos Diuk, Lihong Li, Bethany R Leffler","year":"2009","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Guestrin et al., 2003] Carlos Guestrin, Daphne Koller, Ronald Parr, and Shobha Venkataraman. Efficient Solution Algorithms for Factored MDPs. JAIR, 19:399-468, 2003.","title":"Efficient Solution Algorithms for Factored MDPs","authors":"Carlos Guestrin, Daphne Koller, Ronald Parr, Shobha Venkataraman","year":"2003","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Laroche et al., 2019] Romain Laroche, Paul Trichelair, and Rémi Tachet des Combes. Safe policy improvement with base- line bootstrapping. In ICML, pages 3652-3661. PMLR, 2019.","title":"Safe policy improvement with baseline bootstrapping","authors":"Romain Laroche, Paul Trichelair, Rémi Tachet Des Combes","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Petrik et al., 2016] Marek Petrik, Mohammad Ghavamzadeh, and Yinlam Chow. Safe Policy Improvement by Minimizing Robust Baseline Regret. In NeurIPS, pages 2298-2306.","title":"Safe Policy Improvement by Minimizing Robust Baseline Regret","authors":"Marek Petrik, Mohammad Ghavamzadeh, Yinlam Chow","year":null,"venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Simão and Spaan, 2019] Thiago D. Simão and Matthijs T. J. Spaan. Safe Policy Improvement with Baseline Bootstrapping in Factored Environments. In AAAI. AAAI Press, 2019.","title":"Safe Policy Improvement with Baseline Bootstrapping in Factored Environments","authors":"D Thiago, Simão, T J Matthijs, Spaan","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Strehl et al., 2007] Alexander L. Strehl, Carlos Diuk, and Michael L. Littman. Efficient Structure Learning in Factored- State MDPs. In AAAI, pages 645-650. AAAI Press, 2007.","title":"Efficient Structure Learning in Factored-State MDPs","authors":"Alexander L Strehl, Carlos Diuk, Michael L Littman","year":"2007","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Strehl, 2007] Alexander L Strehl. Model-Based Reinforcement Learning in Factored-State MDPs. In 2007 IEEE International Symposium on Approximate Dynamic Programming and Rein- forcement Learning, pages 103-110. IEEE, 2007.","title":"Model-Based Reinforcement Learning in Factored-State MDPs","authors":"L Alexander, Strehl","year":"2007","venue":"IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}