{"type":"ScholarlyArticle","title":"Stick-Breaking Policy Learning in Dec-POMDPs","authors":[{"name":"Miao Liu","slug":"miao-liu","given":"Miao","family":"Liu","affiliation":""},{"name":"Christopher Amato","slug":"christopher-amato","given":"Christopher","family":"Amato","affiliation":""},{"name":"Xuejun Liao","slug":"xuejun-liao","given":"Xuejun","family":"Liao","affiliation":""},{"name":"Lawrence Carin","slug":"lawrence-carin","given":"Lawrence","family":"Carin","affiliation":""},{"name":"Jonathan P. How","slug":"jonathan-p-how","given":"Jonathan P.","family":"How","affiliation":""}],"abstract":"Expectation maximization (EM) has recently been shown to be an efficient algorithm for learning finite-state controllers (FSCs) in large decentralized POMDPs (Dec-POMDPs). However, current methods use fixed-size FSCs and often converge to maxima that are far from the optimal value. This paper considers a variable-size FSC to represent the local policy of each agent. These variable-size FSCs are constructed using a stick-breaking prior, leading to a new framework called decentralized stick-breaking policy representation (Dec-SBPR). This approach learns the controller parameters with a variational Bayesian algorithm without having to assume that the Dec-POMDP model is available. The performance of Dec-SBPR is demonstrated on several benchmark problems, showing that the algorithm scales to large problems while outperforming other state-of-the-art methods.","doi":null,"url":"https://ijcai.org/proceedings/2015/285","pdf":"https://ijcai.org/proceedings/2015/0285.pdf","markdown":"https://ijcai.org/proceedings/2015/285.md","bibtex":"https://ijcai.org/proceedings/2015/bibtex/285","volume":{"slug":"2015","title":"Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2015","url":"https://ijcai.org/proceedings/2015/","year":2015,"location":"Buenos Aires, Argentina","dates":"25–31 July 2015","editors":"Qiang Yang and Michael Wooldridge","isbn":"978-1-57735-738-4","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":"Sequential Decision Making","track":"Main Track","number":285,"pages":{"first":2011,"last":2017},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}