{"type":"ScholarlyArticle","title":"A Primal-dual Perspective for Distributed TD-learning","authors":[{"name":"Han Dong Lim","slug":"han-dong-lim","given":"Han Dong","family":"Lim","affiliation":""},{"name":"Donghwan Lee","slug":"donghwan-lee","given":"Donghwan","family":"Lee","affiliation":""}],"abstract":"The goal of this paper is to investigate distributed temporal difference (TD) learning for a networked multi-agent Markov decision process. The proposed approach is based on distributed optimization algorithms, which can be interpreted as primal-dual ordinary differential equation (ODE) dynamics subject to null-space constraints. Based on the exponential convergence behavior of the primal-dual ODE dynamics subject to null-space constraints, we examine the behavior of the final iterate in various distributed TD-learning scenarios, considering both constant and diminishing step-sizes and incorporating both i.i.d. and Markovian observation models. Unlike existing methods, the proposed algorithm does not require the assumption that the underlying communication network structure is characterized by a doubly stochastic matrix.","doi":"10.24963/ijcai.2025/634","url":"https://ijcai.org/proceedings/2025/634","pdf":"https://ijcai.org/proceedings/2025/0634.pdf","markdown":"https://ijcai.org/proceedings/2025/634.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/634","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"Machine Learning","track":"Main Track","number":634,"pages":{"first":5698,"last":5706},"keywords":["Machine Learning: ML: Reinforcement learning","Agent-based and Multi-agent Systems: MAS: Multi-agent learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}