Proceedings

Counterfactual Strategies for Markov Decision Processes

Paul Kobialka, Lina Gerlach, Francesco Leofante, Erika Ábrahám, Silvia Lizeth Tapia Tarifa, Einar Broch Johnsen

Download BibTeX Cite

Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 412-420.

https://doi.org/10.24963/ijcai.2025/47

Keywords

  • Planning and Scheduling: PS: Markov decisions processes
  • AI Ethics, Trust, Fairness: ETF: Explainability and interpretability

Abstract

Counterfactuals are widely used in AI to explain how minimal changes to a model’s input can lead to a different output.

However, established methods for computing counterfactuals typically focus on one-step decision-making, and are not directly applicable to sequential decision-making tasks.

This paper fills this gap by introducing counterfactual strategies for Markov Decision Processes (MDPs).

During MDP execution, a strategy decides which of the enabled actions (with known probabilistic effects) to execute next.

Given an initial strategy that reaches an undesired outcome with a probability above some limit, we identify minimal changes to the initial strategy to reduce that probability below the limit.

We encode such counterfactual strategies as solutions to non-linear optimization problems, and further extend our encoding to synthesize diverse counterfactual strategies.

We evaluate our approach on four real-world datasets and demonstrate its practical viability in sophisticated sequential decision-making tasks.

Paul Kobialka, Lina Gerlach, Francesco Leofante, Erika Ábráham, Silvia Lizeth Tapia Tarifa, and Einar Broch Johnsen. 2025. Counterfactual Strategies for Markov Decision Processes. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-25. IJCAI, 412–420. https://doi.org/10.24963/ijcai.2025/47
Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards