{"type":"ScholarlyArticle","title":"Persistent Safety Set Guided Offline Safe Reinforcement Learning","authors":[{"name":"Ayan Choudhury","slug":"ayan-choudhury","given":"Ayan","family":"Choudhury","affiliation":""},{"name":"Janaka Brahmanage","slug":"janaka-brahmanage","given":"Janaka","family":"Brahmanage","affiliation":""},{"name":"Akshat Kumar","slug":"akshat-kumar","given":"Akshat","family":"Kumar","affiliation":""},{"name":"Praveen Paruchuri","slug":"praveen-paruchuri","given":"Praveen","family":"Paruchuri","affiliation":""}],"abstract":"Offline safe reinforcement learning learns high-return policies that satisfy hard safety constraints using only a pre-collected dataset. This setting is challenging due to the inability to explore, and the risk of propagating value errors through unsafe state-space regions. To address this, first, we characterize the safe state region by developing a framework for learning control barrier functions (CBFs) using a novel generalized Bellman operator, yielding a persistent safety set, from which the agent can remain safe indefinitely. Second, we show that several existing safety set estimation methods (e.g., reachability-constrained RL) can be formulated within our CBF learning framework, highlighting its generality. We further propose a new CBF that ensures safety under environment dynamics uncertainty, unlike standard CBFs designed for deterministic settings. Third, we propose a new reward maximization algorithm that effectively exploits our learned persistent safety set for reward critic estimation. Empirical results on standard benchmarks show that our approach achieves state-of-the-art safety with fewer constraint violations while maintaining competitive returns.","doi":"10.24963/ijcai.2026/470","url":"https://ijcai.org/proceedings/2026/470","pdf":"https://ijcai.org/proceedings/2026/0470.pdf","markdown":"https://ijcai.org/proceedings/2026/470.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/470","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Machine Learning","track":"Main Track","number":470,"pages":{"first":4224,"last":4232},"keywords":["Planning and Scheduling: Markov decisions processes","Machine Learning: Offline reinforcement learning","Uncertainty in AI: Sequential decision making"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}