Proceedings

Efficient Skill Learning Using Abstraction Selection

Download BibTeX Cite

Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence. .

© 2009 International Joint Conferences on Artificial Intelligence

Abstract

We present an algorithm for selecting an appropriate abstraction when learning a new skill. We show empirically that it can consistently select an appropriate abstraction using very little sample data, and that it significantly improves skill learning performance in a reasonably large real-valued reinforcement learning domain.

  1. [Barto and Mahadevan, 2003] A.G. Barto and S. Mahade- van. Recent advances in hierarchical reinforcement learn- ing. Discrete Event Systems, 13:41-77, 2003. Special Is- sue on Reinforcement Learning.
  2. [Bishop, 2006] C.M. Bishop. Pattern Recognition and Ma- chine Learning. Springer, New York NY, 2006.
  3. [Boyan, 1999] J.A. Boyan. Least squares temporal differ- ence learning. In Proceedings of the 16th International Conference on Machine Learning, pages 49-56, 1999.
  4. [Dietterich, 2000] T.G. Dietterich. Hierarchical reinforce- ment learning with the MAXQ value function decompo- sition. Journal of Artificial Intelligence Research, 13:227- 303, 2000.
  5. [Hengst, 2002] B. Hengst. Discovering hierarchy in rein- forcement learning with HEXQ. In Proceedings of the Nineteenth International Conference on Machine Learn- ing, pages 243-250, 2002.
  6. [Jonsson and Barto, 2001] A. Jonsson and A.G. Barto. Au- tomated state abstraction for options using the U-Tree al- gorithm. In Advances in Neural Information Processing Systems 13, pages 1054-1060, 2001.
  7. [Konidaris and Barto, 2008] G.D. Konidaris and A.G. Barto. Skill discovery in continuous reinforcement learning do- mains. Technical Report UM-CS-2008-24, Department of Computer Science, University of Massachusetts Amherst, July 2008.
  8. [Konidaris and Osentoski, 2008] G.D. Konidaris and S. Os- entoski. Value function approximation in reinforcement learning using the Fourier basis. Technical Report UM- CS-2008-19, Department of Computer Science, University of Massachusetts, Amherst, June 2008.
  9. [Li et al., 2006] L. Li, T.J. Walsh, and M.L. Littman. To- wards a unified theory of state abstraction for MDPs. In Proceedings of the Ninth International Symposium on Ar- tificial Intelligence and Mathematics, 2006.
  10. [McCallum, 1996] A. McCallum. Learning to use selective attention and short-term memory in sequential tasks. In From Animals to Animats 4: Proceedings of the Fourth In- ternational Conference on Simulation of Adaptive Behav- ior, 1996.
  11. [Schwarz, 1978] G. Schwarz. Estimating the dimension of a model. Annals of Statistics, 6:461-464, 1978.
  12. [S ¸ims ¸ek et al., 2005] Ö. S ¸ims ¸ek, A.P. Wolfe, and A.G. Barto. Identifying useful subgoals in reinforcement learning by local graph partitioning. In Proceedings of the Twenty- Second International Conference on Machine Learning, 2005.
  13. [Singh et al., 2004] S. Singh, A.G. Barto, and N. Chentanez. Intrinsically motivated reinforcement learning. In Pro- ceedings of the 18th Annual Conference on Neural Infor- mation Processing Systems, 2004.
  14. [Sutton and Barto, 1998] R.S. Sutton and A.G. Barto. Rein- forcement Learning: An Introduction. MIT Press, Cam- bridge, MA, 1998.
  15. [Sutton et al., 1998] R.S. Sutton, D. Precup, and S.P. Singh. Intra-option learning about temporally abstract actions. In Proceedings of the Fifteenth International Conference on Machine Learning, pages 556-564, 1998.
  16. [Sutton et al., 1999] R.S. Sutton, D. Precup, and S.P. Singh. Between MDPs and semi-MDPs: A framework for tempo- ral abstraction in reinforcement learning. Artificial Intelli- gence, 112(1-2):181-211, 1999.
  17. [van Seijen et al., 2007] H.H. van Seijen, B. Bakker, and L.J.H.M. Kester. Reinforcement learning with multiple, qualitatively different state representations. In Proceed- ings of NIPS 2007 Workshop on Hierarchical Organiza- tion of Behavior, 2007.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards