---
title: "Persistent Safety Set Guided Offline Safe Reinforcement Learning"
authors:
  - "Ayan Choudhury"
  - "Janaka Brahmanage"
  - "Akshat Kumar"
  - "Praveen Paruchuri"
doi: "10.24963/ijcai.2026/470"
url: "https://ijcai.org/proceedings/2026/470"
pdf: "https://ijcai.org/proceedings/2026/0470.pdf"
venue: "Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence"
year: 2026
pages: "4224-4232"
track: "Main Track"
keywords:
  - "Planning and Scheduling: Markov decisions processes"
  - "Machine Learning: Offline reinforcement learning"
  - "Uncertainty in AI: Sequential decision making"
---

# Persistent Safety Set Guided Offline Safe Reinforcement Learning

Ayan Choudhury, Janaka Brahmanage, Akshat Kumar, Praveen Paruchuri

*Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence*, pages 4224-4232. DOI: [10.24963/ijcai.2026/470](https://doi.org/10.24963/ijcai.2026/470)

## Abstract

Offline safe reinforcement learning learns high-return policies that satisfy hard safety constraints using only a pre-collected dataset. This setting is challenging due to the inability to explore, and the risk of propagating value errors through unsafe state-space regions. To address this, first, we characterize the safe state region by developing a framework for learning control barrier functions (CBFs) using a novel generalized Bellman operator, yielding a persistent safety set, from which the agent can remain safe indefinitely. Second, we show that several existing safety set estimation methods (e.g., reachability-constrained RL) can be formulated within our CBF learning framework, highlighting its generality. We further propose a new CBF that ensures safety under environment dynamics uncertainty, unlike standard CBFs designed for deterministic settings. Third, we propose a new reward maximization algorithm that effectively exploits our learned persistent safety set for reward critic estimation. Empirical results on standard benchmarks show that our approach achieves state-of-the-art safety with fewer constraint violations while maintaining competitive returns.