{"type":"ScholarlyArticle","title":"Exploiting Structure in Policy Construction","authors":[{"name":"Craig Boutilier","slug":"craig-boutilier","given":"Craig","family":"Boutilier","affiliation":""},{"name":"Richard Dearden","slug":"richard-dearden","given":"Richard","family":"Dearden","affiliation":""},{"name":"Moisés Goldszmidt","slug":"moises-goldszmidt","given":"Moisés","family":"Goldszmidt","affiliation":""}],"abstract":"Markov decision processes (MDPs) have recently been applied to the problem of modeling decisiontheoretic planning. While traditional methods for solving MDPs are often practical for small states spaces, their effectiveness for large AI planning problems is questionable. We present an algorithm, called structured policy iteration (SPI), that constructs optimal policies without explicit enumeration of the state space. The algorithm retains the fundamental computational steps of the commonly used modified policy iteration algorithm, but exploits the variable and propositional independencies reflected in a temporal Bayesian network representation of MDPs. The principles behind SPI can be applied to any structured representation of stochastic actions, policies and value functions, and the algorithm itself can be used in conjunction with recent approximation methods.","doi":null,"url":"https://ijcai.org/proceedings/1995/143","pdf":"https://ijcai.org/proceedings/1995/0143.pdf","markdown":"https://ijcai.org/proceedings/1995/143.md","bibtex":"https://ijcai.org/proceedings/1995/bibtex/143","volume":{"slug":"1995","title":"Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 1995","url":"https://ijcai.org/proceedings/1995/","year":1995,"location":"Montréal, Québec, Canada","dates":"","editors":"Chris S. Mellish","isbn":"","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":"Learning","track":"Learning","number":143,"pages":{"first":1104,"last":1112},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}