{"type":"ScholarlyArticle","title":"Safe Reinforcement Learning via Probabilistic Logic Shields","authors":[{"name":"Wen-Chi Yang","slug":"wen-chi-yang","given":"Wen-Chi","family":"Yang","affiliation":"Leuven AI , KU Leuven , Belgium"},{"name":"Giuseppe Marra","slug":"giuseppe-marra","given":"Giuseppe","family":"Marra","affiliation":"Leuven AI , KU Leuven , Belgium"},{"name":"Gavin Rens","slug":"gavin-rens","given":"Gavin","family":"Rens","affiliation":"Stellenbosch University , South Africa"},{"name":"Luc De Raedt","slug":"luc-de-raedt","given":"Luc","family":"de Raedt","affiliation":"Leuven AI , KU Leuven , Belgium; Centre for Applied Autonomous Sensor Systems , Örebro University , Sweden"}],"abstract":"Safe Reinforcement learning (Safe RL) aims at learning optimal policies while staying safe. A popular solution to Safe RL is shielding, which uses a logical safety specification to prevent an RL agent from taking unsafe actions. However, traditional shielding techniques are difficult to integrate with continuous, end-to-end deep RL methods. To this end, we introduce Probabilistic Logic Policy Gradient (PLPG). PLPG is a model-based Safe RL technique that uses probabilistic logic programming to model logical safety constraints as differentiable functions. Therefore, PLPG can be seamlessly applied to any policy gradient algorithm while still providing the same convergence guarantees. In our experiments, we show that PLPG learns safer and more rewarding policies compared to other state-of-the-art shielding techniques.","doi":"10.24963/ijcai.2023/637","url":"https://ijcai.org/proceedings/2023/637","pdf":"https://ijcai.org/proceedings/2023/0637.pdf","markdown":"https://ijcai.org/proceedings/2023/637.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/637","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Uncertainty in AI","track":"Main Track","number":637,"pages":{"first":5739,"last":5749},"keywords":["Machine Learning: ML: Reinforcement learning","Knowledge Representation and Reasoning: KRR: Learning and reasoning","Uncertainty in AI: UAI: Statistical relational AI"],"videos":[],"references":[{"position":1,"raw":"[Achiam et al., 2017] Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. Constrained policy optimiza- tion. In Proceedings of the 34th International Conference on Machine Learning -Volume 70, ICML'17, page 22-31. JMLR.org, 2017.","title":"Constrained policy optimization","authors":"Achiam","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ahmed et al., 2022] Kareem Ahmed, Stefano Teso, Kai- Wei Chang, Guy Van den Broeck, and Antonio Vergari. Semantic probabilistic layers for neuro-symbolic learning. In Advances in Neural Information Processing Systems, 2022.","title":"Semantic probabilistic layers for neuro-symbolic learning","authors":"Kareem Ahmed, Stefano Teso, Kai-Wei Chang, Guy Van Den, Antonio Broeck, Vergari","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Alshiekh et al., 2018] Mohammed Alshiekh, Roderick Bloem, Rüdiger Ehlers, Bettina Könighofer, Scott Niekum, and Ufuk Topcu. Safe reinforcement learning via shielding. AAAI'18/IAAI'18/EAAI'18. AAAI Press, 2018.","title":"Safe reinforcement learning via shielding","authors":"Mohammed Alshiekh, Roderick Bloem, Rüdiger Ehlers, Bettina Könighofer, Scott Niekum, Ufuk Topcu","year":"2018","venue":"AAAI'18/IAAI'18/EAAI'18","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Anderson et al., 2020] Greg Anderson, Abhinav Verma, Isil Dillig, and Swarat Chaudhuri. Neurosymbolic reinforce- ment learning with formally verified exploration. In Pro- ceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20, 2020.","title":"Neurosymbolic reinforcement learning with formally verified exploration","authors":"Greg Anderson, Abhinav Verma, Isil Dillig, Swarat Chaudhuri","year":"2020","venue":"Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bastani et al., 2018] Osbert Bastani, Yewen Pu, and Ar- mando Solar-Lezama. Verifiable reinforcement learning via policy extraction. In Proceedings of the 32nd Interna- tional Conference on Neural Information Processing Sys- tems, NIPS'18, page 2499-2509.","title":"Osbert Bastani, Yewen Pu, and Armando Solar-Lezama. Verifiable reinforcement learning via policy extraction","authors":"Bastani","year":"2018","venue":"Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS'18","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Brockman et al., 2016] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym, 2016.","title":"Openai gym","authors":"Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, Wojciech Zaremba","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Camacho et al., 2019] Alberto Camacho, Rodrigo Toro Icarte, Toryn Q. Klassen, Richard Valenzano, and Sheila A. McIlraith. Ltl and beyond: Formal languages for reward function specification in reinforce- ment learning. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, 7 2019.","title":"Ltl and beyond: Formal languages for reward function specification in reinforcement learning","authors":"Alberto Camacho, Rodrigo Toro Icarte, Toryn Q Klassen, Richard Valenzano, Sheila A Mcilraith","year":"2019","venue":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19","doi":null,"url":null,"internal_url":"/proceedings/2019/840"},{"position":8,"raw":"[Carr et al., 2022] Steven Carr, Nils Jansen, Sebastian Junges, and Ufuk Topcu. Safe reinforcement learning via shielding for pomdps. CoRR, abs/2204.00755, 2022.","title":"Safe reinforcement learning via shielding for pomdps","authors":"Steven Carr, Nils Jansen, Sebastian Junges, Ufuk Topcu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cheng et al., 2019] Richard Cheng, Gábor Orosz, Richard M. Murray, and Joel W. Burdick. End- to-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks. AAAI'19/IAAI'19/EAAI'19. AAAI Press, 2019.","title":"Endto-end safe reinforcement learning through barrier functions for safety-critical continuous control tasks","authors":"Richard Cheng, Gábor Orosz, Richard M Murray, Joel W Burdick","year":"2019","venue":"AAAI'19/IAAI'19/EAAI'19","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dalal et al., 2018] Gal Dalal, Krishnamurthy Dvijotham, Matej Vecerík, Todd Hester, Cosmin Paduraru, and Yu- val Tassa. Safe exploration in continuous action spaces. CoRR, abs/1801.08757, 2018.","title":"Safe exploration in continuous action spaces","authors":"Dalal","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Darwiche, 2003] Adnan Darwiche. A differential ap- proach to inference in bayesian networks. J. ACM, 50(3):280-305, may 2003.","title":"A differential approach to inference in bayesian networks","authors":"Adnan Darwiche","year":"2003","venue":"J. ACM","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[De Giacomo et al., 2021] Giuseppe De Giacomo, Luca Ioc- chi, Marco Favorito, and Fabio Patrizi. Foundations for restraining bolts: Reinforcement learning with ltlf/ldlf restraining specifications. Proceedings of the Interna- tional Conference on Automated Planning and Scheduling, 29(1):128-136, 2021.","title":"Foundations for restraining bolts: Reinforcement learning with ltlf/ldlf restraining specifications","authors":"Giuseppe De, Giacomo, Luca Iocchi, Marco Favorito, Fabio Patrizi","year":"2021","venue":"Proceedings of the International Conference on Automated Planning and Scheduling","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[De Raedt and Kimmig, 2015] Luc De Raedt and Angelika Kimmig. Probabilistic (logic) programming concepts. Mach. Learn., 100(1):5-47, 2015.","title":"Probabilistic (logic) programming concepts","authors":"De Raedt, ; Kimmig, De Luc, Angelika Raedt, Kimmig","year":"2015","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[De Raedt et al., 2016] Luc De Raedt, Kristian Kersting, Sri- raam Natarajan, and David Poole. Statistical Relational Artificial Intelligence: Logic, Probability, and Computa- tion, volume 32 of Synthesis Lectures on Artificial Intelli- gence and Machine Learning. Morgan & Claypool, 2016.","title":"Statistical Relational Artificial Intelligence: Logic, Probability, and Computation","authors":"De Raedt","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Den Hengst et al., 2022] Floris Den Hengst, Vincent Franc ¸ois-Lavet, Mark Hoogendoorn, and Frank van Harmelen. Planning for potential: Efficient safe rein- forcement learning. Mach. Learn., 111(6):2255-2274, 2022.","title":"Planning for potential: Efficient safe reinforcement learning","authors":"Den Hengst","year":"2022","venue":"Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Fernández and Veloso, 2006] Fernando Fernández and Manuela Veloso. Probabilistic policy reuse in a rein- forcement learning agent. AAMAS '06. Association for Computing Machinery, 2006.","title":"Probabilistic policy reuse in a reinforcement learning agent","authors":"Fernando Fernández, Manuela Veloso","year":"2006","venue":"AAMAS '06","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Fierens et al., 2015] Daan Fierens, Guy Van den Broeck, Joris Renkens, Dimitar Shterionov, Bernd Gutmann, Ingo Thon, Gerda Janssens, and Luc De Raedt. Inference and learning in probabilistic logic programs using weighted Boolean formulas. Theory and Practice of Logic Program- ming, 15:358-401, 5 2015.","title":"Inference and learning in probabilistic logic programs using weighted Boolean formulas","authors":"Daan Fierens, Guy Van Den, Joris Broeck, Dimitar Renkens, Bernd Shterionov, Ingo Gutmann, Gerda Thon, Luc Janssens, Raedt De","year":"2015","venue":"Theory and Practice of Logic Programming","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[García and Fernández, 2015] Javier García and Fernando Fernández. A comprehensive survey on safe reinforce- ment learning. Journal of Machine Learning Research, 16:1437-1480, 2015.","title":"Javier García and Fernando Fernández. A comprehensive survey on safe reinforcement learning","authors":"Fernández García","year":"2015","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[García and Fernández, 2019] Javier García and Fernando Fernández. Probabilistic policy reuse for safe reinforce- ment learning. ACM Trans. Auton. Adapt. Syst., 13(3), 2019.","title":"Javier García and Fernando Fernández. Probabilistic policy reuse for safe reinforcement learning","authors":"Fernández García","year":"2019","venue":"ACM Trans. Auton. Adapt. Syst","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Giacobbe et al., 2021] Mirco Giacobbe, Mohammadhosein Hasanbeig, Daniel Kroening, and Hjalmar Wijk. Shielding atari games with bounded prescience. In Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS '21, 2021.","title":"Shielding atari games with bounded prescience","authors":"Mirco Giacobbe, Mohammadhosein Hasanbeig, Daniel Kroening, Hjalmar Wijk","year":"2021","venue":"Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS '21","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Harris and Schaub, 2020] Andrew Harris and Hanspeter Schaub. Spacecraft command and control with safety guar- antees using shielded deep reinforcement learning. AIAA Scitech 2020 Forum, 1 PartF, 2020.","title":"Spacecraft command and control with safety guarantees using shielded deep reinforcement learning","authors":"Andrew Harris, Hanspeter Schaub","year":"2020","venue":"AIAA Scitech 2020 Forum","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Hasanbeig et al., 2019] M. Hasanbeig, Y. Kantaros, A. Abate, D. Kroening, G. J. Pappas, and I. Lee. Rein- forcement learning for temporal logic control synthesis with probabilistic satisfaction guarantees. In 2019 IEEE 58th Conference on Decision and Control (CDC), 2019.","title":"Reinforcement learning for temporal logic control synthesis with probabilistic satisfaction guarantees","authors":"M Hasanbeig, Y Kantaros, A Abate, D Kroening, G J Pappas, I Lee","year":"2019","venue":"IEEE 58th Conference on Decision and Control (CDC)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Hill et al., 2018] Ashley Hill, Antonin Raffin, Maximilian Ernestus, Adam Gleave, Anssi Kanervisto, Rene Traore, Prafulla Dhariwal, Christopher Hesse, Oleg Klimov, Alex Nichol, Matthias Plappert, Alec Radford, John Schulman, Szymon Sidor, and Yuhuai Wu. Stable baselines. https:// github.com/hill-a/stable-baselines, 2018. Accessed: 2023- 01-18.","title":"Stable baselines","authors":"Ashley Hill, Antonin Raffin, Maximilian Ernestus, Adam Gleave, Anssi Kanervisto, Rene Traore, Prafulla Dhariwal, Christopher Hesse, Oleg Klimov, Alex Nichol, Matthias Plappert, Alec Radford, John Schulman, Szymon Sidor, Yuhuai Wu","year":"2018","venue":null,"doi":null,"url":"https://github.com/hill-a/stable-baselines","internal_url":null},{"position":24,"raw":"[Hunt et al., 2021] Nathan Hunt, Nathan Fulton, Sara Magli- acane, Trong Nghia Hoang, Subhro Das, and Armando Solar-Lezama. Verifiably safe exploration for end-to-end reinforcement learning. In Proceedings of the 24th Inter- national Conference on Hybrid Systems: Computation and Control, HSCC '21, 2021.","title":"Verifiably safe exploration for end-to-end reinforcement learning","authors":"Nathan Hunt, Nathan Fulton, Sara Magliacane, Trong Nghia Hoang, Subhro Das, Armando Solar-Lezama","year":"2021","venue":"Proceedings of the 24th International Conference on Hybrid Systems: Computation and Control, HSCC '21","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Jansen et al., 2020] Nils Jansen, Bettina Könighofer, Sebas- tian Junges, Alex Serban, and Roderick Bloem. Safe re- inforcement learning using probabilistic shields. In 31st International Conference on Concurrency Theory, CON- CUR 2020, 2020.","title":"Safe reinforcement learning using probabilistic shields","authors":"Nils Jansen, Bettina Könighofer, Sebastian Junges, Alex Serban, Roderick Bloem","year":"2020","venue":"31st International Conference on Concurrency Theory","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Jiang et al., 2021] Yuqian Jiang, Suda Bharadwaj, Bo Wu, Rishi Shah, Ufuk Topcu, and Peter Stone. Temporal-logic- based reward shaping for continuing reinforcement learn- ing tasks. Proceedings of the AAAI Conference on Artifi- cial Intelligence, 35(9):7995-8003, 2021.","title":"Temporal-logic-based reward shaping for continuing reinforcement learning tasks","authors":"Jiang","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Junges et al., 2021] Sebastian Junges, Nils Jansen, and San- jit A. Seshia. Enforcing almost-sure reachability in pomdps. In Computer Aided Verification. Springer Inter- national Publishing, 2021.","title":"Enforcing almost-sure reachability in pomdps","authors":"Sebastian Junges, Nils Jansen, Sanjit A Seshia","year":"2021","venue":"Computer Aided Verification","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Kalweit et al., 2020] Gabriel Kalweit, Maria Huegle, Moritz Werling, and Joschka Boedecker. Deep inverse q-learning with constraints. In Advances in Neural Information Processing Systems, volume 33, pages 14291-14302.","title":"Deep inverse q-learning with constraints","authors":"Gabriel Kalweit, Maria Huegle, Moritz Werling, Joschka Boedecker","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Curran Associates, Inc., 2020. Karkus et al., 2017] Peter Karkus, David Hsu, and Wee Sun Lee. Qmdp-net: Deep learning for planning under par- tial observability. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17.","title":"Qmdp-net: Deep learning for planning under partial observability","authors":"Peter Karkus, David Hsu, Wee Sun, Lee","year":null,"venue":"Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Kimura et al., 2021] Daiki Kimura, Subhajit Chaudhury, Akifumi Wachi, Ryosuke Kohita, Asim Munawar, Michi- aki Tatsubori, and Alexander Gray. Reinforcement Learn- ing with External Knowledge by using Logical Neural Networks. 2021.","title":"Reinforcement Learning with External Knowledge by using Logical Neural Networks","authors":"Daiki Kimura, Subhajit Chaudhury, Akifumi Wachi, Ryosuke Kohita, Asim Munawar, Michiaki Tatsubori, Alexander Gray","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Li and Bastani, 2020] Shuo Li and Osbert Bastani. Robust Model Predictive Shielding for Safe Reinforcement Learn- ing with Stochastic Dynamics. Proceedings -IEEE Inter- national Conference on Robotics and Automation, 2020.","title":"Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamics","authors":"Bastani ; ] Shuo Li, Osbert Li, Bastani","year":"2020","venue":"Proceedings -IEEE International Conference on Robotics and Automation","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Memarian et al., 2021] Farzan Memarian, Wonjoon Goo, Rudolf Lioutikov, Scott Niekum, and Ufuk Topcu. Self- supervised online reward shaping in sparse-reward envi- ronments. In IEEE/RSJ International Conference on Intel- ligent Robots and Systems, IROS, pages 2369-2375. IEEE, 2021.","title":"Selfsupervised online reward shaping in sparse-reward environments","authors":"Farzan Memarian, Wonjoon Goo, Rudolf Lioutikov, Scott Niekum, Ufuk Topcu","year":"2021","venue":"IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Mnih et al., 2016] Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asyn- chronous methods for deep reinforcement learning. In Proceedings of the 33nd International Conference on Ma- chine Learning, ICML, volume 48 of JMLR Workshop and Conference Proceedings, pages 1928-1937. JMLR.org, 2016.","title":"Asynchronous methods for deep reinforcement learning","authors":"Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy P Lillicrap, Tim Harley, David Silver, Koray Kavukcuoglu","year":"2016","venue":"Proceedings of the 33nd International Conference on Machine Learning, ICML","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Moldovan and Abbeel, 2012] Teodor Mihai Moldovan and Pieter Abbeel. Safe exploration in markov decision pro- cesses. In Proceedings of the 29th International Cofer- ence on International Conference on Machine Learning, ICML'12. Omnipress, 2012.","title":"Safe exploration in markov decision processes","authors":"Abbeel Moldovan, Teodor Mihai Moldovan, Pieter Abbeel","year":"2012","venue":"Proceedings of the 29th International Coference on International Conference on Machine Learning, ICML'12","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Nitti et al., 2016] Davide Nitti, Tinne De Laet, and Luc De Raedt. Probabilistic logic programming for hybrid re- lational domains. Machine Learning, 103:407-449, 2016.","title":"Probabilistic logic programming for hybrid relational domains","authors":"Davide Nitti, Tinne De Laet, Luc De, Raedt","year":"2016","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Pham et al., 2018] Tu-Hoa Pham, Giovanni De Magistris, and Ryuki Tachibana. Optlayer -practical constrained optimization for deep reinforcement learning in the real world. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 6236-6243, 2018.","title":"Optlayer -practical constrained optimization for deep reinforcement learning in the real world","authors":"Tu-Hoa Pham, Giovanni De Magistris, Ryuki Tachibana","year":"2018","venue":"IEEE International Conference on Robotics and Automation (ICRA)","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Pishro-Nik, 2014] H. Pishro-Nik. Introduction to Probabil- ity, Statistics, and Random Processes. Kappa Research, LLC, 2014.","title":"Introduction to Probability, Statistics, and Random Processes","authors":"H Pishro-Nik","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Ray et al., 2019] Alex Ray, Joshua Achiam, and Dario Amodei. Benchmarking Safe Exploration in Deep Rein- forcement Learning. arXiv preprint, pages S. 1-6, 2019.","title":"Benchmarking Safe Exploration in Deep Reinforcement Learning","authors":"Alex Ray, Joshua Achiam, Dario Amodei","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Robert et al., 1999] Christian P Robert, George Casella, and George Casella. Monte Carlo statistical methods, vol- ume 2. Springer, 1999.","title":"Monte Carlo statistical methods","authors":"Robert","year":"1999","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Schulman et al., 2015] John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In Proceedings of the 32nd International Conference on Machine Learning, Proceed- ings of Machine Learning Research. PMLR, 2015.","title":"Trust region policy optimization","authors":"John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, Philipp Moritz","year":"2015","venue":"Proceedings of the 32nd International Conference on Machine Learning, Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Schulman et al., 2016] John Schulman, Philipp Moritz, Sergey Levine, Michael I. Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation. In 4th International Conference on Learning Representations, ICLR, 2016.","title":"High-dimensional continuous control using generalized advantage estimation","authors":"John Schulman, Philipp Moritz, Sergey Levine, Michael I Jordan, Pieter Abbeel","year":"2016","venue":"4th International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Schulman et al., 2017] John Schulman, Filip Wolski, Pra- fulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. CoRR, abs/1707.06347, 2017.","title":"Proximal policy optimization algorithms","authors":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Sutton and Barto, 2018] Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. A Brad- ford Book, 2018.","title":"Reinforcement Learning: An Introduction","authors":"Richard S Sutton, Andrew G Barto","year":"2018","venue":"A Bradford Book","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Sutton et al., 2000] Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approx- imation. In Advances in Neural Information Processing Systems, volume 12. MIT Press, 2000.","title":"Policy gradient methods for reinforcement learning with function approximation","authors":"David Richard S Sutton, Satinder Mcallester, Yishay Singh, Mansour","year":"2000","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Tessler et al., 2019] Chen Tessler, Daniel J. Mankowitz, and Shie Mannor. Reward constrained policy optimization. In 7th International Conference on Learning Representa- tions, ICLR. OpenReview.net, 2019.","title":"Reward constrained policy optimization","authors":"Chen Tessler, Daniel J Mankowitz, Shie Mannor","year":"2019","venue":"7th International Conference on Learning Representations, ICLR. OpenReview.net","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Verma et al., 2019] Abhinav Verma, Hoang M. Le, Yisong Yue, and Swarat Chaudhuri. Imitation-projected program- matic reinforcement learning. In Proceedings of the 33rd International Conference on Neural Information Process- ing Systems, 2019.","title":"Imitation-projected programmatic reinforcement learning","authors":"Abhinav Verma, M Hoang, Yisong Le, Swarat Yue, Chaudhuri","year":"2019","venue":"Proceedings of the 33rd International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Xu et al., 2018] Jingyi Xu, Zilu Zhang, Tal Friedman, Yitao Liang, and Guy Van den Broeck. A semantic loss func- tion for deep learning with symbolic knowledge. In Pro- ceedings of the 35th International Conference on Machine Learning, ICML'18. PMLR, 2018.","title":"A semantic loss function for deep learning with symbolic knowledge","authors":"Jingyi Xu, Zilu Zhang, Tal Friedman, Yitao Liang, Guy Van Den Broeck","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning, ICML'18","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}