{"type":"ScholarlyArticle","title":"Online Symbolic Gradient-Based Optimization for Factored Action MDPs","authors":[{"name":"Hao Cui","slug":"hao-cui","given":"Hao","family":"Cui","affiliation":""},{"name":"Roni Khardon","slug":"roni-khardon","given":"Roni","family":"Khardon","affiliation":""}],"abstract":"This paper investigates online stochastic planning for problems with large factored state and action spaces. We introduce a novel algorithm that builds a symbolic representation capturing an approximation of the action-value Q-function in terms of action variables, and then performs gradient based search to select an action for the current state. The algorithm can be seen as a symbolic extension of Monte-Carlo search, induced by independence assumptions on state and action variables, and augmented with gradients to speed up the search. This avoids the space explosion typically faced by symbolic methods, and the dearth of samples faced by Monte-Carlo methods when the action space is large. An experimental evaluation on benchmark problems shows that the algorithm is competitive with state of the art across problem sizes and that it provides significant improvements for large factored action spaces.","doi":null,"url":"https://ijcai.org/proceedings/2016/436","pdf":"https://ijcai.org/proceedings/2016/0436.pdf","markdown":"https://ijcai.org/proceedings/2016/436.md","bibtex":"https://ijcai.org/proceedings/2016/bibtex/436","volume":{"slug":"2016","title":"Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2016","url":"https://ijcai.org/proceedings/2016/","year":2016,"location":"New York, NY, USA","dates":"9–15 July 2016","editors":"Subbarao Kambhampati","isbn":"978-1-57735-770-4; 978-1-57735-771-1","publisher":"AAAI Press / International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":436,"pages":{"first":3075,"last":3080},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}