{"type":"ScholarlyArticle","title":"Covariant Policy Search","authors":[{"name":"J. Andrew Bagnell","slug":"j-andrew-bagnell","given":"J. Andrew","family":"Bagnell","affiliation":""},{"name":"Jeff G. Schneider","slug":"jeff-g-schneider","given":"Jeff G.","family":"Schneider","affiliation":""}],"abstract":"We investigate the problem of non-covariant behavior of policy gradient reinforcement learning algorithms. The policy gradient approach is amenable to analysis by information geometric methods. This leads us to propose a natural metric on controller parameterization that results from considering the manifold of probability distributions over paths induced by a stochastic controller. Investigation of this approach leads to a covariant gradient ascent rule. Interesting properties of this rule are discussed, including its relation with actor-critic style reinforcement learning algorithms. The algorithms discussed here are computationally quite efficient and on some interesting problems lead to dramatic performance improvement over noncovariant rules.","doi":null,"url":"https://ijcai.org/proceedings/2003/145","pdf":"https://ijcai.org/proceedings/2003/0145.pdf","markdown":"https://ijcai.org/proceedings/2003/145.md","bibtex":"https://ijcai.org/proceedings/2003/bibtex/145","volume":{"slug":"2003","title":"Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2003","url":"https://ijcai.org/proceedings/2003/","year":2003,"location":"Acapulco, Mexico","dates":"","editors":"Georg Gottlob","isbn":"","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":145,"pages":{"first":1019,"last":1023},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}