Online Expectation Maximization for Reinforcement Learning in POMDPs
Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence. Pages 1501-1506.
Abstract
We present online nested expectation maximization for model-free reinforcement learning in a POMDP. The algorithm evaluates the policy only in the current learning episode, discarding the episode after the evaluation and memorizing the sufficient statistic, from which the policy is computed in closed-form. As a result, the online algorithm has a time complexity O(n) and a memory complexity O(1), compared to O(n2) and O(n) for the corresponding batch-mode algorithm, where $n$ is the number of learning episodes. The online algorithm, which has a provable convergence, is demonstrated on five benchmark POMDP problems.
@inproceedings{ijcai2013p224,
title = {Online Expectation Maximization for Reinforcement Learning in POMDPs},
author = {Liu, Miao and Liao, Xuejun and Carin, Lawrence},
booktitle = {Proceedings of the Twenty-Third International Joint Conference on
Artificial Intelligence, {IJCAI-13}},
publisher = {AAAI Press / International Joint Conferences on Artificial Intelligence},
editor = {Francesca Rossi},
pages = {1501--1506},
year = {2013},
month = {8},
url = {https://ijcai.org/proceedings/2013/224},
}
TY - CPAPER TI - Online Expectation Maximization for Reinforcement Learning in POMDPs AU - Liu, Miao AU - Liao, Xuejun AU - Carin, Lawrence T2 - Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI-13 PB - AAAI Press / International Joint Conferences on Artificial Intelligence PY - 2013 SP - 1501 EP - 1506 UR - https://ijcai.org/proceedings/2013/224 ER -Copied
Machine-readable versions of this paper, for tools and AI agents:
