Portable Option Discovery for Automated Learning Transfer in Object-Oriented Markov Decision Processes

Topin, Nicholay; Haltmeyer, Nicholas; Squire, Shawn; Winder, John; desJardins, Marie; MacGlashan, James

Portable Option Discovery for Automated Learning Transfer in Object-Oriented Markov Decision Processes

Nicholay Topin, Nicholas Haltmeyer, Shawn Squire, John Winder, Marie desJardins, James MacGlashan

Download BibTeX Cite

Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. Special Track on Machine Learning. Pages 3856-3863.

Abstract

We introduce a novel framework for option discovery and learning transfer in complex domains that are represented as object-oriented Markov decision processes (OO-MDPs) [Diuk et al., 2008]. Our framework, Portable Option Discovery (POD), extends existing option discovery methods, and enables transfer across related but different domains by providing an unsupervised method for finding a mapping between object-oriented domains with different state spaces. The framework also includes heuristic approaches for increasing the efficiency of the mapping process. We present the results of applying POD to Pickett and Barto's [2002] PolicyBlocks and MacGlashan's [2013] Option-Based Policy Transfer in two application domains. We show that our approach can discover options effectively, transfer options among different domains, and improve learning performance with low computational overhead.

Nicholay Topin, Nicholas Haltmeyer, Shawn Squire, John Winder, Marie desJardins, and James MacGlashan. 2015. Portable Option Discovery for Automated Learning Transfer in Object-Oriented Markov Decision Processes. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15. IJCAI, 3856–3863. https://ijcai.org/proceedings/2015/542

@inproceedings{ijcai2015p542,
  title     = {Portable Option Discovery for Automated Learning Transfer in Object-Oriented Markov Decision Processes},
  author    = {Topin, Nicholay and Haltmeyer, Nicholas and Squire, Shawn and Winder, John and desJardins, Marie and MacGlashan, James},
  booktitle = {Proceedings of the Twenty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-15}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Qiang Yang and Michael Wooldridge},
  pages     = {3856--3863},
  year      = {2015},
  month     = {7},
  note      = {Special Track on Machine Learning},
  url       = {https://ijcai.org/proceedings/2015/542},
}

TY  - CPAPER
TI  - Portable Option Discovery for Automated Learning Transfer in Object-Oriented Markov Decision Processes
AU  - Topin, Nicholay
AU  - Haltmeyer, Nicholas
AU  - Squire, Shawn
AU  - Winder, John
AU  - desJardins, Marie
AU  - MacGlashan, James
T2  - Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2015
SP  - 3856
EP  - 3863
UR  - https://ijcai.org/proceedings/2015/542
ER  -