{"type":"ScholarlyArticle","title":"Utile Distinctions for Relational Reinforcement Learning","authors":[{"name":"William Dabney","slug":"william-dabney","given":"William","family":"Dabney","affiliation":""},{"name":"Amy McGovern","slug":"amy-mcgovern","given":"Amy","family":"McGovern","affiliation":""}],"abstract":"We introduce an approach to autonomously creating state space abstractions for an online reinforcement learning agent using a relational representation. Our approach uses a tree-based function approximation derived from McCallum's [1995] UTree algorithm. We have extended this approach to use a relational representation where relational observations are represented by attributed graphs [McGovern et al., 2003]. We address the challenges introduced by a relational representation by using stochastic sampling to manage the search space [Srinivasan, 1999] and temporal sampling to manage autocorrelation [Jensen and Neville, 2002]. Relational UTree incorporates Iterative Tree Induction [Utgoff et al., 1997] to allow it to adapt to changing environments. We empirically demonstrate that Relational UTree performs better than similar relational learning methods [Finney et al., 2002; Driessens et al., 2001] in a blocks world domain. We also demonstrate that Relational UTree can learn to play a sub-task of the game of Go called Tsume-Go [Ramon et al., 2001].","doi":null,"url":"https://ijcai.org/proceedings/2007/118","pdf":"https://ijcai.org/proceedings/2007/0118.pdf","markdown":"https://ijcai.org/proceedings/2007/118.md","bibtex":"https://ijcai.org/proceedings/2007/bibtex/118","volume":{"slug":"2007","title":"Proceedings of the Twentieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2007","url":"https://ijcai.org/proceedings/2007/","year":2007,"location":"Hyderabad, India","dates":"","editors":"Ramon López de Mántaras","isbn":"","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":118,"pages":{"first":738,"last":742},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}