Covariant Policy Search

Bagnell, J. Andrew; Schneider, Jeff G.

Covariant Policy Search

Download BibTeX Cite

Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence. Pages 1019-1023.

Abstract

We investigate the problem of non-covariant behavior of policy gradient reinforcement learning algorithms. The policy gradient approach is amenable to analysis by information geometric methods. This leads us to propose a natural metric on controller parameterization that results from considering the manifold of probability distributions over paths induced by a stochastic controller. Investigation of this approach leads to a covariant gradient ascent rule. Interesting properties of this rule are discussed, including its relation with actor-critic style reinforcement learning algorithms. The algorithms discussed here are computationally quite efficient and on some interesting problems lead to dramatic performance improvement over noncovariant rules.

J. Andrew Bagnell, and Jeff G. Schneider. 2003. Covariant Policy Search. In Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, IJCAI-03. IJCAI, 1019–1023. https://ijcai.org/proceedings/2003/145

@inproceedings{ijcai2003p145,
  title     = {Covariant Policy Search},
  author    = {Bagnell, J. Andrew and Schneider, Jeff G.},
  booktitle = {Proceedings of the Eighteenth International Joint Conference on
               Artificial Intelligence, {IJCAI-03}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Georg Gottlob},
  pages     = {1019--1023},
  year      = {2003},
  url       = {https://ijcai.org/proceedings/2003/145},
}

TY  - CPAPER
TI  - Covariant Policy Search
AU  - Bagnell, J. Andrew
AU  - Schneider, Jeff G.
T2  - Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, IJCAI-03
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2003
SP  - 1019
EP  - 1023
UR  - https://ijcai.org/proceedings/2003/145
ER  -