{"type":"ScholarlyArticle","title":"Online Expectation Maximization for Reinforcement Learning in POMDPs","authors":[{"name":"Miao Liu","slug":"miao-liu","given":"Miao","family":"Liu","affiliation":""},{"name":"Xuejun Liao","slug":"xuejun-liao","given":"Xuejun","family":"Liao","affiliation":""},{"name":"Lawrence Carin","slug":"lawrence-carin","given":"Lawrence","family":"Carin","affiliation":""}],"abstract":"We present online nested expectation maximization for model-free reinforcement learning in a POMDP. The algorithm evaluates the policy only in the current learning episode, discarding the episode after the evaluation and memorizing the sufficient statistic, from which the policy is computed in closed-form. As a result, the online algorithm has a time complexity O(n) and a memory complexity O(1), compared to O(n2) and O(n) for the corresponding batch-mode algorithm, where $n$ is the number of learning episodes. The online algorithm, which has a provable convergence, is demonstrated on five benchmark POMDP problems.","doi":null,"url":"https://ijcai.org/proceedings/2013/224","pdf":"https://ijcai.org/proceedings/2013/0224.pdf","markdown":"https://ijcai.org/proceedings/2013/224.md","bibtex":"https://ijcai.org/proceedings/2013/bibtex/224","volume":{"slug":"2013","title":"Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2013","url":"https://ijcai.org/proceedings/2013/","year":2013,"location":"Beijing, China","dates":"3–9 August 2013","editors":"Francesca Rossi","isbn":"978-1-57735-633-2","publisher":"AAAI Press / International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":224,"pages":{"first":1501,"last":1506},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}