Active Imitation Learning of Hierarchical Policies

Hamidi, Mandana; Tadepalli, Prasad; Goetschalckx, Robby; Fern, Alan

Active Imitation Learning of Hierarchical Policies

Mandana Hamidi, Prasad Tadepalli, Robby Goetschalckx, Alan Fern

Download BibTeX Cite

Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. Special Track on Machine Learning. Pages 3554-3559.

Abstract

In this paper, we study the problem of imitation learning of hierarchical policies from demonstrations. The main difficulty in learning hierarchical policies by imitation is that the high level intention structure of the policy, which is often critical for understanding the demonstration, is unobserved. We formulate this problem as active learning of Probabilistic State-Dependent Grammars (PSDGs) from demonstrations. Given a set of expert demonstrations, our approach learns a hierarchical policy by actively selecting demonstrations and using queries to explicate their intentional structure at selected points. Our contributions include a new algorithm for imitation learning of hierarchical policies and principled heuristics for the selection of demonstrations and queries. Experimental results in five different domains exhibit successful learning using fewer queries than a variety of alternatives.

Mandana Hamidi, Prasad Tadepalli, Robby Goetschalckx, and Alan Fern. 2015. Active Imitation Learning of Hierarchical Policies. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15. IJCAI, 3554–3559. https://ijcai.org/proceedings/2015/500

@inproceedings{ijcai2015p500,
  title     = {Active Imitation Learning of Hierarchical Policies},
  author    = {Hamidi, Mandana and Tadepalli, Prasad and Goetschalckx, Robby and Fern, Alan},
  booktitle = {Proceedings of the Twenty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-15}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Qiang Yang and Michael Wooldridge},
  pages     = {3554--3559},
  year      = {2015},
  month     = {7},
  note      = {Special Track on Machine Learning},
  url       = {https://ijcai.org/proceedings/2015/500},
}

TY  - CPAPER
TI  - Active Imitation Learning of Hierarchical Policies
AU  - Hamidi, Mandana
AU  - Tadepalli, Prasad
AU  - Goetschalckx, Robby
AU  - Fern, Alan
T2  - Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2015
SP  - 3554
EP  - 3559
UR  - https://ijcai.org/proceedings/2015/500
ER  -