{"type":"ScholarlyArticle","title":"Incentivizing Safer Actions in Policy Optimization for Constrained Reinforcement Learning","authors":[{"name":"Somnath Hazra","slug":"somnath-hazra","given":"Somnath","family":"Hazra","affiliation":""},{"name":"Pallab Dasgupta","slug":"pallab-dasgupta","given":"Pallab","family":"Dasgupta","affiliation":""},{"name":"Soumyajit Dey","slug":"soumyajit-dey","given":"Soumyajit","family":"Dey","affiliation":""}],"abstract":"Constrained Reinforcement Learning (RL) aims to maximize the return while adhering to predefined constraint limits, which represent domain-specific safety requirements. In continuous control settings, where learning agents govern system actions, balancing the trade-off between reward maximization and constraint satisfaction remains a significant challenge. Policy optimization methods often exhibit instability near constraint boundaries, resulting in suboptimal training performance. To address this issue, we introduce a novel approach that integrates an adaptive incentive mechanism in addition to the reward structure to stay within the constraint bound before approaching the constraint boundary. Building on this insight, we propose Incrementally Penalized Proximal Policy Optimization (IP3O), a practical algorithm that enforces a progressively increasing penalty to stabilize training dynamics. Through empirical evaluation on benchmark environments, we demonstrate the efficacy of IP3O compared to the performance of state-of-the-art Safe RL algorithms. Furthermore, we provide theoretical guarantees by deriving a bound on the worst-case error of the optimality achieved by our algorithm.","doi":"10.24963/ijcai.2025/592","url":"https://ijcai.org/proceedings/2025/592","pdf":"https://ijcai.org/proceedings/2025/0592.pdf","markdown":"https://ijcai.org/proceedings/2025/592.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/592","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Machine Learning","track":"Main Track","number":592,"pages":{"first":5318,"last":5326},"keywords":["Agent-based and Multi-agent Systems: MAS: Agent theories and models","Machine Learning: ML: Optimization","Machine Learning: ML: Reinforcement learning","Constraint Satisfaction and Optimization: CSO: Constraint optimization problems"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}