{"type":"ScholarlyArticle","title":"Temporal-Difference Networks with History","authors":[{"name":"Brian Tanner","slug":"brian-tanner","given":"Brian","family":"Tanner","affiliation":""},{"name":"Richard S. Sutton","slug":"richard-s-sutton","given":"Richard S.","family":"Sutton","affiliation":""}],"abstract":"emporal-difference (TD) networks are a formalism for expressing and learning grounded world knowledge in a predictive form [Sutton and Tanner, 2005]. However, not all partially observable Markov decision processes can be efficiently learned with TD networks. In this paper, we extend TD networks by allowing the network-update process (answer network) to depend on the recent history of previous actions and observations rather than only on the most recent action and observation. We show that this extension enables the solution of a larger class of problems than can be solved by the original TD networks or by historybased methods alone. In addition, we apply TD networks to a problem that, while still simple, is significantly larger than has previously been considered. We show that history-extended TD networks can learn much of the common-sense knowledge of an egocentric gridworld domain with a single bit of perception.","doi":null,"url":"https://ijcai.org/proceedings/2005/225","pdf":"https://ijcai.org/proceedings/2005/0225.pdf","markdown":"https://ijcai.org/proceedings/2005/225.md","bibtex":"https://ijcai.org/proceedings/2005/bibtex/225","volume":{"slug":"2005","title":"Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2005","url":"https://ijcai.org/proceedings/2005/","year":2005,"location":"Edinburgh, Scotland, UK","dates":"","editors":"","isbn":"","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":225,"pages":{"first":865,"last":869},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}