Proceedings

Predictive Projections

Download BibTeX Cite

Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence. Pages 1223-1228.

© 2009 International Joint Conferences on Artificial Intelligence

Abstract

This paper addresses the problem of learning control policies in very high dimensional state spaces. We propose a linear dimensionality reduction algorithm that discovers predictive projections: projections in which accurate predictions of future states can be made using simple nearest neighbor style learning. The goal of this work is to extend the reach of existing reinforcement learning algorithms to domains where they would otherwise be inapplicable without extensive engineering of features. The approach is demonstrated on a synthetic pendulum balancing domain, as well as on a robot domain requiring visually guidedNathan Sprague

  1. [Biggs et al., 2008] Michael Biggs, Ali Ghodsi, Dana Wilkinson, and Michael Bowling. Scalable action re- specting embedding. In Proceedings of the Tenth Inter- national Symposium on Artificial Intelligence and Mathe- matics (ISAIM), 2008.
  2. [Bowling et al., 2005] Michael Bowling, Ali Ghodsi, and Dana Wilkinson. Action respecting embedding. In Pro- ceedings of the 22nd International Conference on Machine Learning, Bonn, Germany, 2005.
  3. [Goldberger et al., 2004] Jacob Goldberger, Sam Roweis, Geoff Hinton, and Ruslan Salakhutdinov. Neighbourhood component analysis. In Neural Information Processing Systems, 2004.
  4. [Keller et al., 2006] Philipp W. Keller, Shie Mannor, and Doina Precup. Automatic basis function construction for approximate dynamic programming and reinforcement learning. In Proceedings of the 23rd international confer- ence on Machine learning, pages 449 -456, Pittsburgh, PA, 2006.
  5. [Lagoudakis and Parr, 2003] Michail G. Lagoudakis and Ronald Parr. Least-squares policy iteration. Journal of Machine Learning Research, 4:1107-1149, 2003.
  6. [Mahadevan and Maggioni, 2007] Sridhar Mahadevan and Mauro Maggioni. Proto-value functions: A Lapla- cian framework for learning representation and control in markov decision processes. Journal of Machine Learning Research, 8:2169-2231, 2007.
  7. [Parr et al., 2008] Ronald Parr, Lihong Li, Gavin Taylor, Christopher Painter-Wakefield, and Michael L. Littman. An analysis of linear models, linear value-function ap- proximation, and feature selection for reinforcement learn- ing. In International Conference on Machine Learning (ICML), 2008.
  8. [Sprague, 2007] Nathan Sprague. Basis iteration for reward based dimensionality reduction. In Proceedings of the 6th IEEE International Conference on Development and Learning (ICDL), London, 2007.
  9. [Weinberger and Tesauro, 2007] Kilian Weinberger and Ger- ald Tesauro. Metric learning for kernel regression. In Eleventh International Conference on Artificial Intelli- gence and Statistics. Puerto Rico, 2007.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards