Proceedings

An Experts Algorithm for Transfer Learning

Download BibTeX Cite

Proceedings of the Twentieth International Joint Conference on Artificial Intelligence. Pages 1065-1069.

© 2007 International Joint Conferences on Artificial Intelligence

Abstract

A long-lived agent continually faces new tasks in its environment. Such an agent may be able to use knowledge learned in solving earlier tasks to produce candidate policies for its current task. There may, however, be multiple reasonable policies suggested by prior experience, and the agent must choose between them potentially without any a priori knowledge about their applicability to its current situation. We present an experts algorithm for efficiently choosing amongst candidate policies in solving an unknown Markov decision process task. We conclude with the results of experiments on two domains in which we generate candidate policies from solutions to related tasks and use our experts algorithm to choose amongst them.

  1. [Auer et al., 2002] Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund, and Robert E. Schapire. The non-stochastic multi- armed bandit problem. SIAM Journal on Computing, 32(1):48-77, 2002.
  2. [Cesa-Bianchi et al., 1997] Nicolò Cesa-Bianchi, Yoav Fre- und, David Haussler, David P. Helmbold, and Robert E. Schapire. How to use expert advice. Journal of the Asso- ciation for Computing Machinery, 44(3):427-485, 1997.
  3. [de Farias and Megiddo, 2004] Daniela Pucci de Farias and Nimrod Megiddo. Exploration-exploitation tradeoffs for experts algorithms in reactive environments. In Advances in Neural Information Processing Systems 17, pages 409- 416, 2004.
  4. [Herbster and Warmuth, 1998] Mark Herbster and Manfred Warmuth. Tracking the best expert. Machine Learning, 32(2):151-78, 1998.
  5. [Jordan and Xu, 1995] Michael I. Jordan and Lei Xu. Cover- gence results for the EM approach to mixtures of experts architectures. Neural Networks, 8:1409-1431, 1995.
  6. [Kearns and Singh, 2002] Michael Kearns and Satinder Singh. Near-optimal reinforcement learning in polyno- mial time. Machine Learning, 49:209-232, 2002.
  7. [Lai and Robbins, 1985] T. L. Lai and Herbert Robbins. Asymptotically efficient allocation rules. Advances in Ap- plied Mathematics, 6:4-22, 1985.
  8. [Ravindran and Barto, 2003] Balaraman Ravindran and An- drew Barto. SMDP homomorphisms: An algebraic ap- proach to abstraction in semi markov decision processes. In Proceedings of the Eighteenth International Joint Con- verence on Artificial Intelligence (IJCAI 03), pages 1011- 1016, 2003. IJCAI
  9. [Robbins, 1952] Herbert Robbins. Some aspects of the se- quential design of experiments. Bulletins of the American Mathematical Society, 58:527-535, 1952.
  10. [Stone et al., 2006] Peter Stone, Gregory Huhlmann, Matthew E. Taylor, and Yaxin Liu. Keepaway soccer: From machine learning testbed to benchmark. In Itsuki Noda, Adam Jacoff, Ansgar Bredenfeld, and Yasutake Takahashi, editors, RoboCup-2005: Robot Soccer World Cup IX. Springer Verlag, Berlin, 2006. To appear.
  11. [Tadepalli and Ok, 1998] Prasad Tadepalli and DoKyeong Ok. Model-based average reward reinforcement learning. Artificial Intelligence, 100:177-224, 1998.
  12. [Taylor and Stone, 2005] Matthew E. Taylor and Peter Stone. Behavior transfer for value-function-based rein- forcement learning. In The Fourth International Joint Con- ference on Autonomous Agents and Multiagent Systems, pages 53-59, 2005.
  13. [Vermorel and Mohri, 2005] Joannès Vermorel and Mehryar Mohri. Multi-armed bandit algorithms and empirical eval- uation. In Proceedings of the 16th European Conference on Machine Learning, pages 437-448, 2005.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards