{"type":"ScholarlyArticle","title":"Interactive Value Iteration for Markov Decision Processes with Unknown Rewards","authors":[{"name":"Paul Weng","slug":"paul-weng","given":"Paul","family":"Weng","affiliation":""},{"name":"Bruno Zanuttini","slug":"bruno-zanuttini","given":"Bruno","family":"Zanuttini","affiliation":""}],"abstract":"To tackle the potentially hard task of defining the reward function in a Markov Decision Process, we propose a new approach, based on Value Iteration, which interweaves the elicitation and optimization phases. We assume that rewards whose numeric values are unknown can only be ordered, and that a tutor is present to help comparing sequences of re- wards. We first show how the set of possible reward functions for a given preference relation can be rep- resented as a polytope. Then our algorithm, called Interactive Value Iteration, searches for an optimal policy while refining its knowledge about the pos- sible reward functions, by querying a tutor when necessary. We prove that the number of queries needed before finding an optimal policy is upper- bounded by a polynomial in the size of the problem, and we present experimental results which demon- strate that our approach is efficient in practice.","doi":null,"url":"https://ijcai.org/proceedings/2013/355","pdf":"https://ijcai.org/proceedings/2013/0355.pdf","markdown":"https://ijcai.org/proceedings/2013/355.md","bibtex":"https://ijcai.org/proceedings/2013/bibtex/355","volume":{"slug":"2013","title":"Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2013","url":"https://ijcai.org/proceedings/2013/","year":2013,"location":"Beijing, China","dates":"3–9 August 2013","editors":"Francesca Rossi","isbn":"978-1-57735-633-2","publisher":"AAAI Press / International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":355,"pages":{"first":2415,"last":2420},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}