Online Symbolic Gradient-Based Optimization for Factored Action MDPs
Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. Pages 3075-3080.
Abstract
This paper investigates online stochastic planning for problems with large factored state and action spaces. We introduce a novel algorithm that builds a symbolic representation capturing an approximation of the action-value Q-function in terms of action variables, and then performs gradient based search to select an action for the current state. The algorithm can be seen as a symbolic extension of Monte-Carlo search, induced by independence assumptions on state and action variables, and augmented with gradients to speed up the search. This avoids the space explosion typically faced by symbolic methods, and the dearth of samples faced by Monte-Carlo methods when the action space is large. An experimental evaluation on benchmark problems shows that the algorithm is competitive with state of the art across problem sizes and that it provides significant improvements for large factored action spaces.
@inproceedings{ijcai2016p436,
title = {Online Symbolic Gradient-Based Optimization for Factored Action MDPs},
author = {Cui, Hao and Khardon, Roni},
booktitle = {Proceedings of the Twenty-Fifth International Joint Conference on
Artificial Intelligence, {IJCAI-16}},
publisher = {AAAI Press / International Joint Conferences on Artificial Intelligence},
editor = {Subbarao Kambhampati},
pages = {3075--3080},
year = {2016},
month = {7},
url = {https://ijcai.org/proceedings/2016/436},
}
TY - CPAPER TI - Online Symbolic Gradient-Based Optimization for Factored Action MDPs AU - Cui, Hao AU - Khardon, Roni T2 - Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-16 PB - AAAI Press / International Joint Conferences on Artificial Intelligence PY - 2016 SP - 3075 EP - 3080 UR - https://ijcai.org/proceedings/2016/436 ER -Copied
Machine-readable versions of this paper, for tools and AI agents:
