Proceedings

Kernel Carpentry for Online Regression Using Randomly Varying Coefficient Model

Download BibTeX Cite

Proceedings of the Twentieth International Joint Conference on Artificial Intelligence. Pages 762-766.

© 2007 International Joint Conferences on Artificial Intelligence

Abstract

We present a Bayesian formulation of locally weighted learning (LWL) using the novel concept of a randomly varying coefficient model. Based on this, we propose a mechanism for multivariate non-linear regression using spatially localised linear models that learns completely independent of each other, uses only local information and adapts the local model complexity in a data driven fashion. We derive online updates for the model parameters based on variational Bayesian EM. The evaluation of the proposed algorithm against other state-of-the-art methods reveal the excellent, robust generalization performance beside surprisingly efficient time and space complexity properties. This paper, for the first time, brings together the computational efficiency and the adaptability of ‘non-competitive’ locally weighted learning schemes and the modelling guarantees of the Bayesian formulation.

  1. Beal, 2003
  2. M.J. Beal. Variational algorithms for approximate Bayesian inference. PhD thesis, Gatsby Computational Neuro- science Unit, University College London, 2003.
  3. [Bruntz et al., 1974] S.M. Bruntz, W.S. Cleveland, B. Kleiner, and J.L. Warner. The dependence of ambient ozone on solar radiation, temperature and mixing height. In Symp. Atmospheric Diffusion and Air pollution. MIT Press, 1974.
  4. [Hinton, 1999] G.E. Hinton. Product of experts. In Ninth Interna- tional Conference on Artificial Neural Networks, 1999.
  5. [Longford, 1993] N.T. Longford. Random coefficient models. Clarendon Press, 1993.
  6. [Rasmussen and Gharamani, 2000] C.E. Rasmussen and Z. Ghara- mani. Infinite mixtures of Gaussian process experts. In Advances in Neural Information Processing Systems 14. MIT Press, 2000.
  7. [Schaal and Atkeson, 1998] S. Schaal and C.G. Atkeson. Construc- tive incremental learning from only local information. Neural Computation, 10:2047-2084, 1998.
  8. [Tresp, 2000] V. Tresp. A Bayesian committee machine. Neural Computation, 12(11):2719-2741, 2000.
  9. [Vijayakumar et al., 2002] S. Vijayakumar, A. D'Souza, T. Shibata, J. Conradt, and S. Schaal. Statistical learning for humanoid robots. Autonomous Robot, 12(1):55-69, 2002.
  10. [Vijayakumar et al., 2005] S. Vijayakumar, A. D'Souza, and S. Schaal. Incremental online learning in high dimensions. Neu- ral Computation, 17(12), 2005.
  11. [Williams, 1998] C.K.I. Williams. Prediction with Gaussian pro- cesses: from linear regression to linear prediction and beyond. In Michael I. Jordan, editor, Learning in Graphical Models, pages 599-621. Kluwer, 1998.

References were extracted automatically from the PDF and may contain errors.

Welcome Future Conferences AI Journal Division IJCAI Proceedings About IJCAI IJCAI Awards