{"type":"ScholarlyArticle","title":"Online Planning in MDPs with Stochastic Durative Actions","authors":[{"name":"Tal Berman","slug":"tal-berman","given":"Tal","family":"Berman","affiliation":""},{"name":"Ronen I. Brafman","slug":"ronen-i-brafman","given":"Ronen I.","family":"Brafman","affiliation":""},{"name":"Erez Karpas","slug":"erez-karpas","given":"Erez","family":"Karpas","affiliation":""}],"abstract":"Stochastic planning problems are typically modeled as Markov Decision Processes, in which actions are assumed to be instantaneous and applied sequentially. Yet, real-world actions often have durations and are applied concurrently. This paper presents an online planning approach that can deal with durative actions with stochastic outcomes. Our approach relies on Monte Carlo Tree Search with a new backpropagation procedure and temporal reasoning techniques that address the need to not only choose which action to execute, but also when to execute it. We also introduce a novel heuristic that combines reasoning about time and probabilities. Overall, we present the first online planner for stochastic temporal planning, solving a richer problem representation than previous work while achieving state-of-the-art empirical results.","doi":"10.24963/ijcai.2025/941","url":"https://ijcai.org/proceedings/2025/941","pdf":"https://ijcai.org/proceedings/2025/0941.pdf","markdown":"https://ijcai.org/proceedings/2025/941.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/941","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Planning and Scheduling","track":"Main Track","number":941,"pages":{"first":8465,"last":8473},"keywords":["Planning and Scheduling: PS: Markov decisions processes","Planning and Scheduling: PS: Planning algorithms","Planning and Scheduling: PS: Planning under uncertainty"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}