On Adaptivity and Safety in Sequential Decision Making

On Adaptivity and Safety in Sequential Decision Making

Sapana Chaudhary

Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence
Doctoral Consortium. Pages 7077-7078. https://doi.org/10.24963/ijcai.2023/813

Sequential decision making is an important field in machine learning, encompassing techniques such as online optimization, structured bandits, and reinforcement learning, which have numerous applications such as recommendation systems, online advertising, conversational agents, and robot learning. However, two key challenges face real-world sequential decision making: the need for adaptable models and the need for safety during both learning and execution. Adaptability refers to the ability of a model to quickly adapt to new and diverse environments, which is especially challenging in environments where feedback is sparse. To address this challenge, we propose using meta reinforcement learning with sub-optimal demonstration data. Safety is also critical in real-world sequential decision making. A model that adheres to safety requirements can avoid dangerous outcomes and ensure the safety of humans and other agents in the environment. We propose an approach based on online convex optimization that ensures safety at every time step. Addressing these challenges can lead to the development of more robust, safe, and adaptable AI systems that can perform a wide range of tasks and operate in a variety of environments.
Keywords:
Machine Learning: ML: Reinforcement learning
Machine Learning: ML: Optimization
AI Ethics, Trust, Fairness: ETF: Safety and robustness