---
title: "Covariant Policy Search"
authors:
  - "J. Andrew Bagnell"
  - "Jeff G. Schneider"
doi: ""
url: "https://ijcai.org/proceedings/2003/145"
pdf: "https://ijcai.org/proceedings/2003/0145.pdf"
venue: "Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence"
year: 2003
pages: "1019-1023"
track: ""
keywords:
---

# Covariant Policy Search

J. Andrew Bagnell, Jeff G. Schneider

*Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence*, pages 1019-1023.

## Abstract

We investigate the problem of non-covariant behavior of policy gradient reinforcement learning algorithms. The policy gradient approach is amenable to analysis by information geometric methods. This leads us to propose a natural metric on controller parameterization that results from considering the manifold of probability distributions over paths induced by a stochastic controller. Investigation of this approach leads to a covariant gradient ascent rule. Interesting properties of this rule are discussed, including its relation with actor-critic style reinforcement learning algorithms. The algorithms discussed here are computationally quite efficient and on some interesting problems lead to dramatic performance improvement over noncovariant rules.