---
title: "Temporal Coherence and Prediction Decay in TD Learning"
authors:
  - "Donald F. Beal"
  - "Martin C. Smith"
doi: ""
url: "https://ijcai.org/proceedings/1999/81"
pdf: "https://ijcai.org/proceedings/1999/0081.pdf"
venue: "Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence"
year: 1999
pages: "564-568"
track: "Computer Game Playing"
keywords:
---

# Temporal Coherence and Prediction Decay in TD Learning

Donald F. Beal, Martin C. Smith

*Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence*, pages 564-568.

## Abstract

This paper describes improvements to the temporal difference TD(A.) learning method . The standard form of the TD(A.) method has the problem that two control parameters, learning rate and temporal discount, need to be chosen appropriately. These parameters can have a major effect on performance, particularly the learning rate parameter, which af fects the stability of the process as well as the number of observations required . Our extension to the TD(A.) algorithm automatically sets and subsequently adjusts these parameters. The le arning rate adjustment is based on a new concept we call temporal coherence (TC). The experiments reported here compare the extended TD(A.) algorithm performance with human-chosen parameters and with an earlier method for learning rate adjustment, in a complex game domain . The learning task was that of learning the relative values of pieces, without any initial domain-specific knowledge, and from self-play only. The results show that the improved method leads to better learning (i.e. faster and less subject to the effects of noise), than the selection of human-chosen values for the control parameters, and a comparison method.