{"type":"ScholarlyArticle","title":"Missingness-MDPs: Bridging the Theory of Missing Data and POMDPs","authors":[{"name":"Joshua Wendland","slug":"joshua-wendland","given":"Joshua","family":"Wendland","affiliation":""},{"name":"Markel Zubia","slug":"markel-zubia","given":"Markel","family":"Zubia","affiliation":""},{"name":"Roman Andriushchenko","slug":"roman-andriushchenko","given":"Roman","family":"Andriushchenko","affiliation":""},{"name":"Maris F. L. Galesloot","slug":"maris-f-l-galesloot","given":"Maris F. L.","family":"Galesloot","affiliation":""},{"name":"Milan Češka","slug":"milan-ceska","given":"Milan","family":"Ceska","affiliation":""},{"name":"Henrik von Kleist","slug":"henrik-von-kleist","given":"Henrik","family":"von Kleist","affiliation":""},{"name":"Thiago D. Simão","slug":"thiago-d-simao","given":"Thiago D.","family":"Simão","affiliation":""},{"name":"Maximilian Weininger","slug":"maximilian-weininger","given":"Maximilian","family":"Weininger","affiliation":""},{"name":"Nils Jansen","slug":"nils-jansen","given":"Nils","family":"Jansen","affiliation":""}],"abstract":"We introduce missingness-MDPs (miss-MDPs), a novel subclass of partially observable Markov decision processes (POMDPs) that incorporates the theory of missing data.\r\nA miss-MDP is a POMDP whose observation function is a missingness function, specifying the probability that individual state features are missing (i.e., unobserved) at a time step.\r\nThe literature distinguishes three canonical missingness types: (1) missing completely at random (MCAR), (2) missing at random (MAR), and (3) missing not at random (MNAR).\r\nThe problem is to compute near-optimal policies for a miss-MDP with an unknown missingness function, given a dataset of action-observation histories.\r\nAchieving such optimality guarantees for policies requires learning the missingness function from data, which is infeasible for general POMDPs.\r\nTo overcome this challenge, we exploit the structural properties of different missingness types to derive probably approximately correct (PAC) algorithms for learning the missingness function.\r\nThese algorithms yield an approximate but fully specified miss-MDP that we solve using off-the-shelf planning methods.\r\nWe prove that, with high probability, the resulting policies are ε-optimal in the true miss-MDP.\r\nEmpirical results confirm the theory and demonstrate superior performance of our approach over two model-free methods.","doi":"10.24963/ijcai.2026/692","url":"https://ijcai.org/proceedings/2026/692","pdf":"https://ijcai.org/proceedings/2026/0692.pdf","markdown":"https://ijcai.org/proceedings/2026/692.md","bibtex":"https://ijcai.org/proceedings/2026/bibtex/692","volume":{"slug":"2026","title":"Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2026","url":"https://ijcai.org/proceedings/2026/","year":2026,"location":"Bremen, Germany","dates":"15-21 August 2026","editors":"Diego Calvanese","isbn":"978-1-956792-09-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2026"},"section":"Planning and Scheduling","track":"Main Track","number":692,"pages":{"first":6217,"last":6226},"keywords":["Planning and Scheduling: POMDPs","Planning and Scheduling: Learning in planning and scheduling","Machine Learning: Partially observable reinforcement learning and POMDPs","Knowledge Representation and Reasoning: Causality"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}