{"type":"ScholarlyArticle","title":"Stationary Deterministic Policies for Constrained MDPs with Multiple Rewards, Costs, and Discount Factors","authors":[{"name":"Dmitri A. Dolgov","slug":"dmitri-a-dolgov","given":"Dmitri A.","family":"Dolgov","affiliation":""},{"name":"Edmund H. Durfee","slug":"edmund-h-durfee","given":"Edmund H.","family":"Durfee","affiliation":""}],"abstract":"We consider the problem of policy optimization for a resource-limited agent with multiple timedependent objectives, represented as an MDP with multiple discount factors in the objective function and constraints. We show that limiting search to stationary deterministic policies, coupled with a novel problem reduction to mixed integer programming, yields an algorithm for finding such policies that is computationally feasible, where no such algorithm has heretofore been identified. In the simpler case where the constrained MDP has a single discount factor, our technique provides a new way for finding an optimal deterministic policy, where previous methods could only find randomized policies. We analyze the properties of our approach and describe implementation results.","doi":null,"url":"https://ijcai.org/proceedings/2005/236","pdf":"https://ijcai.org/proceedings/2005/0236.pdf","markdown":"https://ijcai.org/proceedings/2005/236.md","bibtex":"https://ijcai.org/proceedings/2005/bibtex/236","volume":{"slug":"2005","title":"Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2005","url":"https://ijcai.org/proceedings/2005/","year":2005,"location":"Edinburgh, Scotland, UK","dates":"","editors":"","isbn":"","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":236,"pages":{"first":1326,"last":1330},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}