{"type":"ScholarlyArticle","title":"Active Imitation Learning of Hierarchical Policies","authors":[{"name":"Mandana Hamidi","slug":"mandana-hamidi","given":"Mandana","family":"Hamidi","affiliation":""},{"name":"Prasad Tadepalli","slug":"prasad-tadepalli","given":"Prasad","family":"Tadepalli","affiliation":""},{"name":"Robby Goetschalckx","slug":"robby-goetschalckx","given":"Robby","family":"Goetschalckx","affiliation":""},{"name":"Alan Fern","slug":"alan-fern","given":"Alan","family":"Fern","affiliation":""}],"abstract":"In this paper, we study the problem of imitation learning of hierarchical policies from demonstrations. The main difficulty in learning hierarchical policies by imitation is that the high level intention structure of the policy, which is often critical for understanding the demonstration, is unobserved. We formulate this problem as active learning of Probabilistic State-Dependent Grammars (PSDGs) from demonstrations. Given a set of expert demonstrations, our approach learns a hierarchical policy by actively selecting demonstrations and using queries to explicate their intentional structure at selected points. Our contributions include a new algorithm for imitation learning of hierarchical policies and principled heuristics for the selection of demonstrations and queries. Experimental results in five different domains exhibit successful learning using fewer queries than a variety of alternatives.","doi":null,"url":"https://ijcai.org/proceedings/2015/500","pdf":"https://ijcai.org/proceedings/2015/0500.pdf","markdown":"https://ijcai.org/proceedings/2015/500.md","bibtex":"https://ijcai.org/proceedings/2015/bibtex/500","volume":{"slug":"2015","title":"Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2015","url":"https://ijcai.org/proceedings/2015/","year":2015,"location":"Buenos Aires, Argentina","dates":"25–31 July 2015","editors":"Qiang Yang and Michael Wooldridge","isbn":"978-1-57735-738-4","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":"Special Track on Machine Learning","track":"Special Track on Machine Learning","number":500,"pages":{"first":3554,"last":3559},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}