{"type":"ScholarlyArticle","title":"Bayesian Real-Time Dynamic Programming","authors":[{"name":"Scott Sanner","slug":"scott-sanner","given":"Scott","family":"Sanner","affiliation":""},{"name":"Robby Goetschalckx","slug":"robby-goetschalckx","given":"Robby","family":"Goetschalckx","affiliation":""},{"name":"Kurt Driessens","slug":"kurt-driessens","given":"Kurt","family":"Driessens","affiliation":""},{"name":"Guy Shani","slug":"guy-shani","given":"Guy","family":"Shani","affiliation":""}],"abstract":"Real-time dynamic programming (RTDP) solves Markov decision processes (MDPs) when the initial state is restricted, by focusing dynamic programming on the envelope of states reachable from an initial state set. RTDP often provides performance guarantees without visiting the entire state space. Building on RTDP, recent work has sought to improve its efficiency through various optimizations, including maintaining upper and lower bounds to both govern trial termination and prioritize state exploration. In this work, we take a Bayesian perspective on these upper and lower bounds and use a value of perfect information (VPI) analysis to govern trial termination and exploration in a novel algorithm we call VPI-RTDP. VPI-RTDP leads to an improvement over state-of-the-art RTDP methods, empirically yielding up to a three-fold reduction in the amount of time and number of visited states required to achieve comparable policy performance.Scott Sanner, Robby Goetschalckx, Kurt Driessens, Guy Shani","doi":null,"url":"https://ijcai.org/proceedings/2009/297","pdf":"https://ijcai.org/proceedings/2009/0297.pdf","markdown":"https://ijcai.org/proceedings/2009/297.md","bibtex":"https://ijcai.org/proceedings/2009/bibtex/297","volume":{"slug":"2009","title":"Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2009","url":"https://ijcai.org/proceedings/2009/","year":2009,"location":"Pasadena, California, USA","dates":"","editors":"Hiroaki Kitano","isbn":"","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":297,"pages":{"first":1784,"last":1788},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}