---
title: "Policy Search in Reproducing Kernel Hilbert Space"
authors:
  - "Ngo Anh Vien"
  - "Peter Englert"
  - "Marc Toussaint"
doi: ""
url: "https://ijcai.org/proceedings/2016/298"
pdf: "https://ijcai.org/proceedings/2016/0298.pdf"
venue: "Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence"
year: 2016
pages: "2089-2095"
track: ""
keywords:
---

# Policy Search in Reproducing Kernel Hilbert Space

Ngo Anh Vien, Peter Englert, Marc Toussaint

*Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence*, pages 2089-2095.

## Abstract

Modeling policies in reproducing kernel Hilbert space (RKHS) renders policy gradient reinforcement learning algorithms non-parametric. As a result, the policies become very flexible and have a rich representational potential without a pre-defined set of features. However, their performances might be either non-covariant under re-parameterization of the chosen kernel, or very sensitive to step-size selection. In this paper, we propose to use a general framework to derive a new RKHS policy search technique. The new derivation leads to both a natural RKHS actor-critic algorithm and a RKHS expectation maximization (EM) policy search algorithm. Further,we show that kernelization enables us to learn in partially observable (POMDP) tasks which is considered daunting for parametric approaches. Via sparsification, a small set of support vectors representing the history is shown to be effectively discovered.For evaluations, we use three simulated (PO)MDP reinforcement learning tasks, and a simulated PR2's robotic manipulation task. The results demonstrate the effectiveness of the new RKHS policy search framework in comparison to plain RKHS actor-critic, episodic natural actor-critic, plain actor-critic, and PoWER approaches.