On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference (Extended Abstract)

Rawlik, Konrad; Toussaint, Marc; Vijayakumar, Sethu

On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference (Extended Abstract)

Konrad Rawlik, Marc Toussaint, Sethu Vijayakumar

Download BibTeX Cite

Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence. Pages 3052-3055.

Abstract

We present a reformulation of the stochastic optimal control problem in terms of KL divergence minimisation, not only providing a unifying perspective of previous approaches in this area, but also demonstrating that the formalism leads to novel practical approaches to the control problem. Specifically, a natural relaxation of the dual formulation gives rise to exact iterative solutions to the finite and infinite horizon stochastic optimal control problem, while direct application of Bayesian inference methods yields instances of risk sensitive control.

Konrad Rawlik, Marc Toussaint, and Sethu Vijayakumar. 2013. On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference (Extended Abstract). In Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI-13. IJCAI, 3052–3055. https://ijcai.org/proceedings/2013/455

@inproceedings{ijcai2013p455,
  title     = {On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference (Extended Abstract)},
  author    = {Rawlik, Konrad and Toussaint, Marc and Vijayakumar, Sethu},
  booktitle = {Proceedings of the Twenty-Third International Joint Conference on
               Artificial Intelligence, {IJCAI-13}},
  publisher = {AAAI Press / International Joint Conferences on Artificial Intelligence},
  editor    = {Francesca Rossi},
  pages     = {3052--3055},
  year      = {2013},
  month     = {8},
  url       = {https://ijcai.org/proceedings/2013/455},
}

TY  - CPAPER
TI  - On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference (Extended Abstract)
AU  - Rawlik, Konrad
AU  - Toussaint, Marc
AU  - Vijayakumar, Sethu
T2  - Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, IJCAI-13
PB  - AAAI Press / International Joint Conferences on Artificial Intelligence
PY  - 2013
SP  - 3052
EP  - 3055
UR  - https://ijcai.org/proceedings/2013/455
ER  -