Dynamics of Temporal Difference Learning

Wendemuth, Andreas

Dynamics of Temporal Difference Learning

Download BibTeX Cite

Proceedings of the Twentieth International Joint Conference on Artificial Intelligence. Pages 1107-1111.

Abstract

In behavioural sciences, the problem that a sequence of stimuli is followed by a sequence of rewards r(t) is considered. The subject is to learn the full sequence of rewards from the stimuli, where the prediction is modelled by the Sutton-Barto rule. In a sequence of n trials, this prediction rule is learned iteratively by temporal difference learning. We present a closed formula of the prediction of rewards at trial time t within trial n. From that formula, we show directly that for n to infinity, the predictions converge to the real rewards. In this approach, a new quality of correlation type Toeplitz matrices is proven. We give learning rates which optimally speed up the learning process.

Andreas Wendemuth. 2007. Dynamics of Temporal Difference Learning. In Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, IJCAI-07. IJCAI, 1107–1111. https://ijcai.org/proceedings/2007/179

@inproceedings{ijcai2007p179,
  title     = {Dynamics of Temporal Difference Learning},
  author    = {Wendemuth, Andreas},
  booktitle = {Proceedings of the Twentieth International Joint Conference on
               Artificial Intelligence, {IJCAI-07}},
  publisher = {International Joint Conferences on Artificial Intelligence},
  editor    = {Ramon López de Mántaras},
  pages     = {1107--1111},
  year      = {2007},
  url       = {https://ijcai.org/proceedings/2007/179},
}

TY  - CPAPER
TI  - Dynamics of Temporal Difference Learning
AU  - Wendemuth, Andreas
T2  - Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, IJCAI-07
PB  - International Joint Conferences on Artificial Intelligence
PY  - 2007
SP  - 1107
EP  - 1111
UR  - https://ijcai.org/proceedings/2007/179
ER  -