Maximum Entropy Semi-Supervised Inverse Reinforcement Learning

Audiffren, Julien; Valko, Michal; Lazaric, Alessandro; Ghavamzadeh, Mohammad

Maximum Entropy Semi-Supervised Inverse Reinforcement Learning

Julien Audiffren, Michal Valko, Alessandro Lazaric, Mohammad Ghavamzadeh

Download BibTeX Cite

Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. Special Track on Machine Learning. Pages 3315-3320.

Abstract

A popular approach to apprenticeship learning (AL) is to formulate it as an inverse reinforcement learning (IRL) problem. The MaxEnt-IRL algorithm successfully integrates the maximum entropy principle into IRL and unlike its predecessors, it resolves the ambiguity arising from the fact that a possibly large number of policies could match the expert's behavior. In this paper, we study an AL setting in which in addition to the expert's trajectories,a number of unsupervised trajectories is available. We introduce MESSI,a novel algorithm that combines MaxEnt-IRL with principles coming from semisupervised learning. In particular, MESSI integrates the unsupervised data into the MaxEnt-IRL framework using a pairwise penalty on trajectories. Empirical results in a highway driving and grid-world problems indicate that MESSI is able to take advantage of the unsupervised trajectories and improve the performance of MaxEnt-IRL.

Julien Audiffren, Michal Valko, Alessandro Lazaric, and Mohammad Ghavamzadeh. 2015. Maximum Entropy Semi-Supervised Inverse Reinforcement Learning. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15. IJCAI, 3315–3320. https://ijcai.org/proceedings/2015/467

@inproceedings{ijcai2015p467,
  title     = {Maximum Entropy Semi-Supervised Inverse Reinforcement Learning},
  author    = {Audiffren, Julien and Valko, Michal and Lazaric, Alessandro and Ghavamzadeh, Mohammad},
  booktitle = {Proceedings of the Twenty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-15}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Qiang Yang and Michael Wooldridge},
  pages     = {3315--3320},
  year      = {2015},
  month     = {7},
  note      = {Special Track on Machine Learning},
  url       = {https://ijcai.org/proceedings/2015/467},
}

TY  - CPAPER
TI  - Maximum Entropy Semi-Supervised Inverse Reinforcement Learning
AU  - Audiffren, Julien
AU  - Valko, Michal
AU  - Lazaric, Alessandro
AU  - Ghavamzadeh, Mohammad
T2  - Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2015
SP  - 3315
EP  - 3320
UR  - https://ijcai.org/proceedings/2015/467
ER  -