---
title: "Counterfactual Strategies for Markov Decision Processes"
authors:
  - "Paul Kobialka"
  - "Lina Gerlach"
  - "Francesco Leofante"
  - "Erika Ábrahám"
  - "Silvia Lizeth Tapia Tarifa"
  - "Einar Broch Johnsen"
doi: "10.24963/ijcai.2025/47"
url: "https://ijcai.org/proceedings/2025/47"
pdf: "https://ijcai.org/proceedings/2025/0047.pdf"
venue: "Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence"
year: 2025
pages: "412-420"
track: "Main Track"
keywords:
  - "Planning and Scheduling: PS: Markov decisions processes"
  - "AI Ethics, Trust, Fairness: ETF: Explainability and interpretability"
---

# Counterfactual Strategies for Markov Decision Processes

Paul Kobialka, Lina Gerlach, Francesco Leofante, Erika Ábrahám, Silvia Lizeth Tapia Tarifa, Einar Broch Johnsen

*Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence*, pages 412-420. DOI: [10.24963/ijcai.2025/47](https://doi.org/10.24963/ijcai.2025/47)

## Abstract

Counterfactuals are widely used in AI to explain how minimal changes to a model’s input can lead to a different output. 
However, established methods for computing counterfactuals typically focus on one-step decision-making, and are not directly applicable to sequential decision-making tasks. 
This paper fills this gap by introducing counterfactual strategies for Markov Decision Processes (MDPs). 
During MDP execution, a strategy decides which of the enabled actions (with known probabilistic effects) to execute next. 
Given an initial strategy that reaches an undesired outcome with a probability above some limit, we identify minimal changes to the initial strategy to reduce that probability below the limit. 
We encode such counterfactual strategies as solutions to non-linear optimization problems, and further extend our encoding to synthesize diverse counterfactual strategies. 
We evaluate our approach on four real-world datasets and demonstrate its practical viability in sophisticated sequential decision-making tasks.