{"type":"ScholarlyArticle","title":"Dynamics of Temporal Difference Learning","authors":[{"name":"Andreas Wendemuth","slug":"andreas-wendemuth","given":"Andreas","family":"Wendemuth","affiliation":""}],"abstract":"In behavioural sciences, the problem that a sequence of stimuli is followed by a sequence of rewards r(t) is considered. The subject is to learn the full sequence of rewards from the stimuli, where the prediction is modelled by the Sutton-Barto rule. In a sequence of n trials, this prediction rule is learned iteratively by temporal difference learning. We present a closed formula of the prediction of rewards at trial time t within trial n. From that formula, we show directly that for n to infinity, the predictions converge to the real rewards. In this approach, a new quality of correlation type Toeplitz matrices is proven. We give learning rates which optimally speed up the learning process.","doi":null,"url":"https://ijcai.org/proceedings/2007/179","pdf":"https://ijcai.org/proceedings/2007/0179.pdf","markdown":"https://ijcai.org/proceedings/2007/179.md","bibtex":"https://ijcai.org/proceedings/2007/bibtex/179","volume":{"slug":"2007","title":"Proceedings of the Twentieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2007","url":"https://ijcai.org/proceedings/2007/","year":2007,"location":"Hyderabad, India","dates":"","editors":"Ramon López de Mántaras","isbn":"","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":179,"pages":{"first":1107,"last":1111},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}