---
title: "Dynamics of Temporal Difference Learning"
authors:
  - "Andreas Wendemuth"
doi: ""
url: "https://ijcai.org/proceedings/2007/179"
pdf: "https://ijcai.org/proceedings/2007/0179.pdf"
venue: "Proceedings of the Twentieth International Joint Conference on Artificial Intelligence"
year: 2007
pages: "1107-1111"
track: ""
keywords:
---

# Dynamics of Temporal Difference Learning

Andreas Wendemuth

*Proceedings of the Twentieth International Joint Conference on Artificial Intelligence*, pages 1107-1111.

## Abstract

In behavioural sciences, the problem that a sequence of stimuli is followed by a sequence of rewards r(t) is considered. The subject is to learn the full sequence of rewards from the stimuli, where the prediction is modelled by the Sutton-Barto rule. In a sequence of n trials, this prediction rule is learned iteratively by temporal difference learning. We present a closed formula of the prediction of rewards at trial time t within trial n. From that formula, we show directly that for n to infinity, the predictions converge to the real rewards. In this approach, a new quality of correlation type Toeplitz matrices is proven. We give learning rates which optimally speed up the learning process.