Stick-Breaking Policy Learning in Dec-POMDPs

Liu, Miao; Amato, Christopher; Liao, Xuejun; Carin, Lawrence; How, Jonathan P.

Stick-Breaking Policy Learning in Dec-POMDPs

Miao Liu, Christopher Amato, Xuejun Liao, Lawrence Carin, Jonathan P. How

Download BibTeX Cite

Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. Main Track. Pages 2011-2017.

Abstract

Expectation maximization (EM) has recently been shown to be an efficient algorithm for learning finite-state controllers (FSCs) in large decentralized POMDPs (Dec-POMDPs). However, current methods use fixed-size FSCs and often converge to maxima that are far from the optimal value. This paper considers a variable-size FSC to represent the local policy of each agent. These variable-size FSCs are constructed using a stick-breaking prior, leading to a new framework called decentralized stick-breaking policy representation (Dec-SBPR). This approach learns the controller parameters with a variational Bayesian algorithm without having to assume that the Dec-POMDP model is available. The performance of Dec-SBPR is demonstrated on several benchmark problems, showing that the algorithm scales to large problems while outperforming other state-of-the-art methods.

Miao Liu, Christopher Amato, Xuejun Liao, Lawrence Carin, and Jonathan P. How. 2015. Stick-Breaking Policy Learning in Dec-POMDPs. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15. IJCAI, 2011–2017. https://ijcai.org/proceedings/2015/285

@inproceedings{ijcai2015p285,
  title     = {Stick-Breaking Policy Learning in Dec-POMDPs},
  author    = {Liu, Miao and Amato, Christopher and Liao, Xuejun and Carin, Lawrence and How, Jonathan P.},
  booktitle = {Proceedings of the Twenty-Fourth International Joint Conference on
               Artificial Intelligence, {IJCAI-15}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Qiang Yang and Michael Wooldridge},
  pages     = {2011--2017},
  year      = {2015},
  month     = {7},
  note      = {Main Track},
  url       = {https://ijcai.org/proceedings/2015/285},
}

TY  - CPAPER
TI  - Stick-Breaking Policy Learning in Dec-POMDPs
AU  - Liu, Miao
AU  - Amato, Christopher
AU  - Liao, Xuejun
AU  - Carin, Lawrence
AU  - How, Jonathan P.
T2  - Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, IJCAI-15
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2015
SP  - 2011
EP  - 2017
UR  - https://ijcai.org/proceedings/2015/285
ER  -