{"type":"ScholarlyArticle","title":"DetH*: Approximate Hierarchical Solution of Large Markov Decision Processes","authors":[{"name":"Jennifer L. Barry","slug":"jennifer-l-barry","given":"Jennifer L.","family":"Barry","affiliation":""},{"name":"Leslie Pack Kaelbling","slug":"leslie-pack-kaelbling","given":"Leslie Pack","family":"Kaelbling","affiliation":""},{"name":"Tomás Lozano-Pérez","slug":"tomas-lozano-perez","given":"Tomas","family":"Lozano-Perez","affiliation":""}],"abstract":"This paper presents an algorithm for finding approximately optimal policies in very large Markov decision processes by constructing a hierarchical model and then solving it approximately. It exploits factored representations to achieve compactness and efficiency and to discover connectivity properties of the domain. We provide a bound on the quality of the solutions and give asymptotic analysis of the runtimes; in addition we demonstrate performance on a collection of very large domains. Results show that the quality of resulting policies is very good and the total running times, for both creating and solving the hierarchy, are significantly less than for an optimal factored MDP solver.","doi":"10.5591/978-1-57735-516-8/IJCAI11-323","url":"https://ijcai.org/proceedings/2011/323","pdf":"https://ijcai.org/proceedings/2011/0323.pdf","markdown":"https://ijcai.org/proceedings/2011/323.md","bibtex":"https://ijcai.org/proceedings/2011/bibtex/323","volume":{"slug":"2011","title":"Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2011","url":"https://ijcai.org/proceedings/2011/","year":2011,"location":"Barcelona, Catalonia, Spain","dates":"16–22 July 2011","editors":"Toby Walsh","isbn":"978-1-57735-512-0; 978-1-57735-513-7; 978-1-57735-514-4; 978-1-57735-515-1; 978-1-57735-516-8","publisher":"AAAI Press/International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":323,"pages":{"first":1928,"last":1934},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}