{"type":"ScholarlyArticle","title":"Multi-Value-Functions: Efficient Automatic Action Hierarchies for Multiple Goal MDPs","authors":[{"name":"Andrew W. Moore","slug":"andrew-w-moore","given":"Andrew W.","family":"Moore","affiliation":""},{"name":"Leemon C. Baird III","slug":"leemon-c-baird-iii","given":"Leemon C. Baird","family":"III","affiliation":""},{"name":"Leslie Pack Kaelbling","slug":"leslie-pack-kaelbling","given":"Leslie Pack","family":"Kaelbling","affiliation":""}],"abstract":"If you have planned to achieve one particular goal in a stochastic delayed rewards problem and then someone asks about a different goal what should you do? What if you need to be ready to quickly supply an answer for any possible goal? This paper shows that by using a new kind of automatically generated abstract action hierarchy that with N states, preparing for all of N possible goals can be much much cheaper than N times the work of preparing for one goal. In goal-based Markov Decision Problems, it is usual to generate a policy rr( x), mapping states to actions, and a value function J ( x), mapping states to an estimate of minimum expected cost-to-goal, starting at x. In this paper we will use the terminology that a multi-policy rr*(x, y) (for all state-pairs (x, y)) maps a state x to the first action it should take in order to reach y with expected minimum cost and a multi-valuefunction J* ( x, y) is a definition of this minimum cost. Building these objects quickly and with little memory is the main purpose of this paper, but a secondary result is a natural, automatic, way to create a set of parsimonious yet powerful abstract actions for MDPs. The paper concludes with a set of empirical results on increasingly large MDPs.","doi":null,"url":"https://ijcai.org/proceedings/1999/187","pdf":"https://ijcai.org/proceedings/1999/0187.pdf","markdown":"https://ijcai.org/proceedings/1999/187.md","bibtex":"https://ijcai.org/proceedings/1999/bibtex/187","volume":{"slug":"1999","title":"Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 1999","url":"https://ijcai.org/proceedings/1999/","year":1999,"location":"Stockholm, Sweden","dates":"","editors":"Thomas Dean","isbn":"","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":"Uncertainty and Probabilistic Reasoning","track":"Uncertainty and Probabilistic Reasoning","number":187,"pages":{"first":1316,"last":1322},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}