{"type":"ScholarlyArticle","title":"Maximum Entropy Semi-Supervised Inverse Reinforcement Learning","authors":[{"name":"Julien Audiffren","slug":"julien-audiffren","given":"Julien","family":"Audiffren","affiliation":""},{"name":"Michal Valko","slug":"michal-valko","given":"Michal","family":"Valko","affiliation":""},{"name":"Alessandro Lazaric","slug":"alessandro-lazaric","given":"Alessandro","family":"Lazaric","affiliation":""},{"name":"Mohammad Ghavamzadeh","slug":"mohammad-ghavamzadeh","given":"Mohammad","family":"Ghavamzadeh","affiliation":""}],"abstract":"A popular approach to apprenticeship learning (AL) is to formulate it as an inverse reinforcement learning (IRL) problem. The MaxEnt-IRL algorithm successfully integrates the maximum entropy principle into IRL and unlike its predecessors, it resolves the ambiguity arising from the fact that a possibly large number of policies could match the expert's behavior. In this paper, we study an AL setting in which in addition to the expert's trajectories,a number of unsupervised trajectories is available. We introduce MESSI,a novel algorithm that combines MaxEnt-IRL with principles coming from semisupervised learning. In particular, MESSI integrates the unsupervised data into the MaxEnt-IRL framework using a pairwise penalty on trajectories. Empirical results in a highway driving and grid-world problems indicate that MESSI is able to take advantage of the unsupervised trajectories and improve the performance of MaxEnt-IRL.","doi":null,"url":"https://ijcai.org/proceedings/2015/467","pdf":"https://ijcai.org/proceedings/2015/0467.pdf","markdown":"https://ijcai.org/proceedings/2015/467.md","bibtex":"https://ijcai.org/proceedings/2015/bibtex/467","volume":{"slug":"2015","title":"Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2015","url":"https://ijcai.org/proceedings/2015/","year":2015,"location":"Buenos Aires, Argentina","dates":"25–31 July 2015","editors":"Qiang Yang and Michael Wooldridge","isbn":"978-1-57735-738-4","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":"Special Track on Machine Learning","track":"Special Track on Machine Learning","number":467,"pages":{"first":3315,"last":3320},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}