{"type":"ScholarlyArticle","title":"Policy Search in Reproducing Kernel Hilbert Space","authors":[{"name":"Ngo Anh Vien","slug":"ngo-anh-vien","given":"Ngo Anh","family":"Vien","affiliation":""},{"name":"Peter Englert","slug":"peter-englert","given":"Peter","family":"Englert","affiliation":""},{"name":"Marc Toussaint","slug":"marc-toussaint","given":"Marc","family":"Toussaint","affiliation":""}],"abstract":"Modeling policies in reproducing kernel Hilbert space (RKHS) renders policy gradient reinforcement learning algorithms non-parametric. As a result, the policies become very flexible and have a rich representational potential without a pre-defined set of features. However, their performances might be either non-covariant under re-parameterization of the chosen kernel, or very sensitive to step-size selection. In this paper, we propose to use a general framework to derive a new RKHS policy search technique. The new derivation leads to both a natural RKHS actor-critic algorithm and a RKHS expectation maximization (EM) policy search algorithm. Further,we show that kernelization enables us to learn in partially observable (POMDP) tasks which is considered daunting for parametric approaches. Via sparsification, a small set of support vectors representing the history is shown to be effectively discovered.For evaluations, we use three simulated (PO)MDP reinforcement learning tasks, and a simulated PR2's robotic manipulation task. The results demonstrate the effectiveness of the new RKHS policy search framework in comparison to plain RKHS actor-critic, episodic natural actor-critic, plain actor-critic, and PoWER approaches.","doi":null,"url":"https://ijcai.org/proceedings/2016/298","pdf":"https://ijcai.org/proceedings/2016/0298.pdf","markdown":"https://ijcai.org/proceedings/2016/298.md","bibtex":"https://ijcai.org/proceedings/2016/bibtex/298","volume":{"slug":"2016","title":"Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2016","url":"https://ijcai.org/proceedings/2016/","year":2016,"location":"New York, NY, USA","dates":"9–15 July 2016","editors":"Subbarao Kambhampati","isbn":"978-1-57735-770-4; 978-1-57735-771-1","publisher":"AAAI Press / International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":298,"pages":{"first":2089,"last":2095},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}