---
title: "Temporal-Difference Networks with History"
authors:
  - "Brian Tanner"
  - "Richard S. Sutton"
doi: ""
url: "https://ijcai.org/proceedings/2005/225"
pdf: "https://ijcai.org/proceedings/2005/0225.pdf"
venue: "Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence"
year: 2005
pages: "865-869"
track: ""
keywords:
---

# Temporal-Difference Networks with History

Brian Tanner, Richard S. Sutton

*Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence*, pages 865-869.

## Abstract

emporal-difference (TD) networks are a formalism for expressing and learning grounded world knowledge in a predictive form [Sutton and Tanner, 2005]. However, not all partially observable Markov decision processes can be efficiently learned with TD networks. In this paper, we extend TD networks by allowing the network-update process (answer network) to depend on the recent history of previous actions and observations rather than only on the most recent action and observation. We show that this extension enables the solution of a larger class of problems than can be solved by the original TD networks or by historybased methods alone. In addition, we apply TD networks to a problem that, while still simple, is significantly larger than has previously been considered. We show that history-extended TD networks can learn much of the common-sense knowledge of an egocentric gridworld domain with a single bit of perception.