Inverse Reinforcement Learning in Relational Domains

Munzer, Thibaut; Piot, Bilal; Geist, Matthieu; Pietquin, Olivier; Lopes, Manuel

Inverse Reinforcement Learning in Relational Domains

Thibaut Munzer, Bilal Piot, Matthieu Geist, Olivier Pietquin, Manuel Lopes

Download BibTeX Cite

Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. Special Track on Machine Learning. Pages 3735-3740.

Abstract

In this work, we introduce the first approach to the Inverse Reinforcement Learning (IRL) problem in relational domains. IRL has been used to recover a more compact representation of the expert policy leading to better generalization performances among different contexts. On the other hand, relational learning allows representing problems with a varying number of objects (potentially infinite), thus provides more generalizable representations of problems and skills. We show how these different formalisms allow one to create a new IRL algorithm for relational domains that can recover with great efficiency rewards from expert data that have strong generalization and transfer properties. We evaluate our algorithm in representative tasks and study the impact of diverse experimental conditions such as : the number of demonstrations, knowledge about the dynamics, transfer among varying dimensions of a problem, and changing dynamics.

Thibaut Munzer, Bilal Piot, Matthieu Geist, Olivier Pietquin, and Manuel Lopes. 2015. Inverse Reinforcement Learning in Relational Domains. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15. IJCAI, 3735–3740. https://ijcai.org/proceedings/2015/525

@inproceedings{ijcai2015p525,
  title     = {Inverse Reinforcement Learning in Relational Domains},
  author    = {Munzer, Thibaut and Piot, Bilal and Geist, Matthieu and Pietquin, Olivier and Lopes, Manuel},
  booktitle = {Proceedings of the Twenty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-15}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Qiang Yang and Michael Wooldridge},
  pages     = {3735--3740},
  year      = {2015},
  month     = {7},
  note      = {Special Track on Machine Learning},
  url       = {https://ijcai.org/proceedings/2015/525},
}

TY  - CPAPER
TI  - Inverse Reinforcement Learning in Relational Domains
AU  - Munzer, Thibaut
AU  - Piot, Bilal
AU  - Geist, Matthieu
AU  - Pietquin, Olivier
AU  - Lopes, Manuel
T2  - Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2015
SP  - 3735
EP  - 3740
UR  - https://ijcai.org/proceedings/2015/525
ER  -