Stationary Deterministic Policies for Constrained MDPs with Multiple Rewards, Costs, and Discount Factors
Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence. Pages 1326-1330.
Abstract
We consider the problem of policy optimization for a resource-limited agent with multiple timedependent objectives, represented as an MDP with multiple discount factors in the objective function and constraints. We show that limiting search to stationary deterministic policies, coupled with a novel problem reduction to mixed integer programming, yields an algorithm for finding such policies that is computationally feasible, where no such algorithm has heretofore been identified. In the simpler case where the constrained MDP has a single discount factor, our technique provides a new way for finding an optimal deterministic policy, where previous methods could only find randomized policies. We analyze the properties of our approach and describe implementation results.
@inproceedings{ijcai2005p236,
title = {Stationary Deterministic Policies for Constrained MDPs with Multiple Rewards, Costs, and Discount Factors},
author = {Dolgov, Dmitri A. and Durfee, Edmund H.},
booktitle = {Proceedings of the Nineteenth International Joint Conference on
Artificial Intelligence, {IJCAI-05}},
publisher = {International Joint Conferences on Artificial Intelligence},
pages = {1326--1330},
year = {2005},
url = {https://ijcai.org/proceedings/2005/236},
}
TY - CPAPER TI - Stationary Deterministic Policies for Constrained MDPs with Multiple Rewards, Costs, and Discount Factors AU - Dolgov, Dmitri A. AU - Durfee, Edmund H. T2 - Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, IJCAI-05 PB - International Joint Conferences on Artificial Intelligence PY - 2005 SP - 1326 EP - 1330 UR - https://ijcai.org/proceedings/2005/236 ER -Copied
Machine-readable versions of this paper, for tools and AI agents:
