Linear Bayesian Reinforcement Learning

Tziortziotis, Nikolaos; Dimitrakakis, Christos; Blekas, Konstantinos

Linear Bayesian Reinforcement Learning

Nikolaos Tziortziotis, Christos Dimitrakakis, Konstantinos Blekas

Download BibTeX Cite

Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence. Pages 1721-1727.

Abstract

This paper proposes a simple linear Bayesian approach to reinforcement learning. We show that with an appropriate basis, a Bayesian linear Gaussian model is sufficient for accurately estimating the system dynamics, and in particular when we allow for correlated noise. Policies are estimated by first sampling a transition model from the current posterior, and then performing approximate dynamic programming on the sampled model. This form of approximate Thompson sampling results in good exploration in unknown environments. The approach can also be seen as a Bayesian generalisation of least-squares policy iteration, where the empirical transition matrix is replaced with a sample from the posterior.

Nikolaos Tziortziotis, Christos Dimitrakakis, and Konstantinos Blekas. 2013. Linear Bayesian Reinforcement Learning. In Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI-13. IJCAI, 1721–1727. https://ijcai.org/proceedings/2013/255

@inproceedings{ijcai2013p255,
  title     = {Linear Bayesian Reinforcement Learning},
  author    = {Tziortziotis, Nikolaos and Dimitrakakis, Christos and Blekas, Konstantinos},
  booktitle = {Proceedings of the Twenty-Third International Joint Conference on
               Artificial Intelligence, {IJCAI-13}},
  publisher = {AAAI Press / International Joint Conferences on Artificial Intelligence},
  editor    = {Francesca Rossi},
  pages     = {1721--1727},
  year      = {2013},
  month     = {8},
  url       = {https://ijcai.org/proceedings/2013/255},
}

TY  - CPAPER
TI  - Linear Bayesian Reinforcement Learning
AU  - Tziortziotis, Nikolaos
AU  - Dimitrakakis, Christos
AU  - Blekas, Konstantinos
T2  - Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI-13
PB  - AAAI Press / International Joint Conferences on Artificial Intelligence
PY  - 2013
SP  - 1721
EP  - 1727
UR  - https://ijcai.org/proceedings/2013/255
ER  -