{"type":"ScholarlyArticle","title":"Finite-Time Convergence Rates of Decentralized Local Markovian Stochastic Approximation","authors":[{"name":"Pengfei Wang","slug":"pengfei-wang","given":"Pengfei","family":"Wang","affiliation":"School of Software Technology , Zhejiang University , Ningbo , China; Qiushi Academy for Advanced Studies , Zhejiang University , Hangzhou , China; College of Computer Science and Technology , Zhejiang University , Hangzhou , China"},{"name":"Nenggan Zheng","slug":"nenggan-zheng","given":"Nenggan","family":"Zheng","affiliation":"Qiushi Academy for Advanced Studies , Zhejiang University , Hangzhou , China; College of Computer Science and Technology , Zhejiang University , Hangzhou , China; State Key Lab of Brain-Machine Intelligence , Zhejiang University , Hangzhou , China; CCAI by MOE and Zhejiang Provincial Government (ZJU) , Hangzhou , China"}],"abstract":"Markovian stochastic approximation has recently aroused a great deal of interest in many fields; however, it is not well understood in decentralized settings. Decentralized Markovian stochastic approximation is far more challenging than its single-agent counterpart due to the complex coupling structure between decentralized communication and Markovian noise-corrupted local updates. In this paper, a decentralized local markovian stochastic approximation (DLMSA) algorithm has been proposed and attains a near-optimal convergence rate. Specifically, we first provide a local variant of decentralized Markovian stochastic approximation so that each agent performs multiple local updates and then periodically communicate with its neighbors. Furthermore, we propose DLMSA with compressed communication (C-DLMSA) for further reducing the communication overhead. In this way, each agent only needs to communicate compressed information (e.g., sign compression) with its neighbors. We show that C-DLMSA enjoys the same convergence rate as that of the original DLMSA. Finally, we verify our theoretical results by applying our methods to solve multi-task reinforcement learning problems over multi-agent systems.","doi":"10.24963/ijcai.2024/562","url":"https://ijcai.org/proceedings/2024/562","pdf":"https://ijcai.org/proceedings/2024/0562.pdf","markdown":"https://ijcai.org/proceedings/2024/562.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/562","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":562,"pages":{"first":5082,"last":5090},"keywords":["Machine Learning: ML: Optimization"],"videos":[],"references":[{"position":1,"raw":"[Alacaoglu and Lyu, 2023] Ahmet Alacaoglu and Hanbaek Lyu. Convergence of first-order methods for constrained nonconvex optimization with dependent data. In Inter- national Conference on Machine Learning, volume 202, pages 458-489. PMLR, 2023.","title":"Convergence of first-order methods for constrained nonconvex optimization with dependent data","authors":"Lyu Alacaoglu, Ahmet Alacaoglu, Hanbaek Lyu","year":"2023","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Benveniste et al., 2012] Albert Benveniste, Michel Métivier, and Pierre Priouret. Adaptive algorithms and stochastic approximations, volume 22. Springer Science & Business Media, 2012.","title":"Adaptive algorithms and stochastic approximations","authors":"Albert Benveniste, Michel Métivier, Pierre Priouret","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bertsekas and Tsitsiklis, 1996] Dimitri Bertsekas and John N Tsitsiklis. Neuro-dynamic programming. Athena Scientific, 1996.","title":"Neuro-dynamic programming","authors":"Dimitri Bertsekas, John N Tsitsiklis","year":"1996","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bhandari et al., 2018] Jalaj Bhandari, Daniel Russo, and Raghav Singal. A finite time analysis of temporal dif- ference learning with linear function approximation. In Conference on learning theory, pages 1691-1692. PMLR, 2018.","title":"A finite time analysis of temporal difference learning with linear function approximation","authors":"Jalaj Bhandari, Daniel Russo, Raghav Singal","year":"2018","venue":"Conference on learning theory","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Borkar et al., 2021] Vivek Borkar, Shuhang Chen, Adithya Devraj, Ioannis Kontoyiannis, and Sean Meyn. The ode method for asymptotic statistics in stochastic ap- proximation and reinforcement learning. arXiv preprint arXiv:2110.14427, 2021.","title":"The ode method for asymptotic statistics in stochastic approximation and reinforcement learning","authors":"Vivek Borkar, Shuhang Chen, Adithya Devraj, Ioannis Kontoyiannis, Sean Meyn","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Borkar, 2009] Vivek S Borkar. Stochastic approximation: a dynamical systems viewpoint, volume 48. Springer, 2009.","title":"Stochastic approximation: a dynamical systems viewpoint","authors":"S Vivek, Borkar","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Bottou et al., 2018] Léon Bottou, Frank E Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. SIAM review, 60(2):223-311, 2018.","title":"Optimization methods for large-scale machine learning","authors":"Bottou","year":"2018","venue":"SIAM review","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2018] Tianyi Chen, Kaiqing Zhang, Geor- gios B Giannakis, and Tamer Basar. Communication- efficient distributed reinforcement learning. arXiv preprint arXiv:1812.03239, 16, 2018.","title":"Communicationefficient distributed reinforcement learning","authors":"Tianyi Chen, Kaiqing Zhang, Georgios B Giannakis, Tamer Basar","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chen et al., 2020] Zaiwei Chen, Siva Theja Maguluri, San- jay Shakkottai, and Karthikeyan Shanmugam. Finite- sample analysis of contractive stochastic approximation using smooth convex envelopes. Advances in Neural Infor- mation Processing Systems, 33:8223-8234, 2020.","title":"Finitesample analysis of contractive stochastic approximation using smooth convex envelopes","authors":"Chen","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Chen et al., 2021] Zaiwei Chen, Siva Theja Maguluri, San- jay Shakkottai, and Karthikeyan Shanmugam. A lya- punov theory for finite-sample guarantees of asynchronous q-learning and td-learning variants. arXiv preprint arXiv:2102.01567, 2021.","title":"A lyapunov theory for finite-sample guarantees of asynchronous q-learning and td-learning variants","authors":"Chen","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Chen et al., 2022] Zaiwei Chen, Sheng Zhang, Thinh T Doan, John-Paul Clarke, and Siva Theja Maguluri. Finite- sample analysis of nonlinear stochastic approximation with applications in reinforcement learning. Automatica, 146:110623, 2022.","title":"Finitesample analysis of nonlinear stochastic approximation with applications in reinforcement learning","authors":"Zaiwei Chen, Sheng Zhang, John-Paul Thinh T Doan, Siva Clarke, Maguluri Theja","year":"2022","venue":"Automatica","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Dean et al., 2012] Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Marc'aurelio Ran- zato, Andrew Senior, Paul Tucker, Ke Yang, et al. Large scale distributed deep networks. Advances in neural infor- mation processing systems, 25, 2012.","title":"Large scale distributed deep networks","authors":"Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Marc'aurelio Ranzato, Andrew Senior, Paul Tucker, Ke Yang","year":"2012","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Debnath and Mikusinski, 2005] Lokenath Debnath and Piotr Mikusinski. Introduction to Hilbert spaces with applica- tions. Academic press, 2005.","title":"Lokenath Debnath and Piotr Mikusinski. Introduction to Hilbert spaces with applications","authors":"Mikusinski Debnath","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Doan, 2020] Thinh T Doan. Local stochastic approximation: A unified view of federated learning and distributed multi- task reinforcement learning algorithms. arXiv preprint arXiv:2006.13460, 2020.","title":"Local stochastic approximation: A unified view of federated learning and distributed multitask reinforcement learning algorithms","authors":"Thinh T Doan","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Doan, 2022] Thinh T Doan. Finite-time analysis of markov gradient descent. IEEE Transactions on Automatic Control, 68(4):2140-2153, 2022.","title":"Finite-time analysis of markov gradient descent","authors":"Thinh T Doan","year":"2022","venue":"IEEE Transactions on Automatic Control","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Dorfman and Levy, 2022] Ron Dorfman and Kfir Yehuda Levy. Adapting to mixing time in stochastic optimiza- tion with markovian data. In International Conference on Machine Learning, pages 5429-5446. PMLR, 2022.","title":"Adapting to mixing time in stochastic optimization with markovian data","authors":"Levy ; Dorfman, Ron Dorfman, Kfir Yehuda, Levy","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Duchi et al., 2012] John C Duchi, Alekh Agarwal, Mikael Johansson, and Michael I Jordan. Ergodic mirror descent. SIAM Journal on Optimization, 22(4):1549-1578, 2012.","title":"Ergodic mirror descent","authors":"Alekh John C Duchi, Mikael Agarwal, Michael I Johansson, Jordan","year":"2012","venue":"SIAM Journal on Optimization","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Gao et al., 2022] Hongchang Gao, Junyi Li, and Heng Huang. On the convergence of local stochastic compo- sitional gradient descent with momentum. In Interna- tional Conference on Machine Learning, pages 7017-7035.","title":"On the convergence of local stochastic compositional gradient descent with momentum","authors":"Hongchang Gao, Junyi Li, Heng Huang","year":null,"venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"PMLR, 2022.","title":null,"authors":"Pmlr","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Heredia et al., 2020] Paulo Heredia, Hasan Ghadialy, and Shaoshuai Mou. Finite-sample analysis of distributed q- learning for multi-agent networks. In 2020 American Con- trol Conference, pages 3511-3516. IEEE, 2020.","title":"Finite-sample analysis of distributed qlearning for multi-agent networks","authors":"Heredia","year":"2020","venue":"2020 American Control Conference","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Jiang and Xu, 2008] Houyuan Jiang and Huifu Xu. Stochas- tic approximation approaches to the stochastic variational inequality problem. IEEE Transactions on Automatic Con- trol, 53(6):1462-1475, 2008.","title":"Stochastic approximation approaches to the stochastic variational inequality problem","authors":"Xu ; ] Houyuan Jiang, Huifu Jiang, Xu","year":"2008","venue":"IEEE Transactions on Automatic Control","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Khaled et al., 2019] Ahmed Khaled, Konstantin Mishchenko, and Peter Richtárik. First analysis of local gd on heterogeneous data. arXiv preprint arXiv:1909.04715, 2019.","title":"First analysis of local gd on heterogeneous data","authors":"Ahmed Khaled, Konstantin Mishchenko, Peter Richtárik","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Khodadadian et al., 2022] Sajad Khodadadian, Pranay Sharma, Gauri Joshi, and Siva Theja Maguluri. Federated reinforcement learning: Linear speedup under markovian sampling. In International Conference on Machine Learning, pages 10997-11057. PMLR, 2022.","title":"Federated reinforcement learning: Linear speedup under markovian sampling","authors":"Pranay Sajad Khodadadian, Gauri Sharma, Siva Joshi, Maguluri Theja","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Kumar et al., 2023] Harshat Kumar, Alec Koppel, and Ale- jandro Ribeiro. On the sample complexity of actor-critic method for reinforcement learning with function approxi- mation. Machine Learning, pages 1-35, 2023.","title":"On the sample complexity of actor-critic method for reinforcement learning with function approximation","authors":"Kumar","year":"2023","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Kushner and Yin, 1987] Harold Joseph Kushner and George Yin. Stochastic approximation algorithms for parallel and distributed processing. Stochastics: An International Jour- nal of Probability and Stochastic Processes, 22(3-4):219- 250, 1987.","title":"Stochastic approximation algorithms for parallel and distributed processing","authors":"Yin ; Kushner, Joseph Harold, George Kushner, Yin","year":"1987","venue":"Stochastics: An International Journal of Probability and Stochastic Processes","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Lakshmanan and De Farias, 2008] Hariharan Lakshmanan and Daniela Pucci De Farias. Decentralized resource allo- cation in dynamic networks of agents. SIAM Journal on Optimization, 19(2):911-940, 2008.","title":"Decentralized resource allocation in dynamic networks of agents","authors":"Lakshmanan, De Farias ; Hariharan Lakshmanan, Daniela Pucci, De Farias","year":"2008","venue":"SIAM Journal on Optimization","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Levin and Peres, 2017] David A Levin and Yuval Peres. Markov chains and mixing times, volume 107. American Mathematical Soc., 2017.","title":"Markov chains and mixing times","authors":"A David, Yuval Levin, Peres","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Li et al., 2019] Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of fedavg on non-iid data. In International Conference on Learning Representations, 2019.","title":"On the convergence of fedavg on non-iid data","authors":"Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, Zhihua Zhang","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Li et al., 2020a] Gen Li, Yuting Wei, Yuejie Chi, Yuantao Gu, and Yuxin Chen. Sample complexity of asynchronous q-learning: Sharper analysis and variance reduction. Ad- vances in neural information processing systems, 33:7031- 7043, 2020.","title":"Sample complexity of asynchronous q-learning: Sharper analysis and variance reduction","authors":"Li","year":"2020","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Li et al., 2020b] Tian Li, Anit Kumar Sahu, Ameet Tal- walkar, and Virginia Smith. Federated learning: Challenges, methods, and future directions. IEEE signal processing magazine, 37(3):50-60, 2020.","title":"Federated learning: Challenges, methods, and future directions","authors":"Li","year":"2020","venue":"IEEE signal processing magazine","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Liu et al., 2022] Wei Liu, Li Chen, and Wenyi Zhang. De- centralized federated learning: Balancing communication and computing costs. IEEE Transactions on Signal and Information Processing over Networks, 8:131-143, 2022.","title":"Decentralized federated learning: Balancing communication and computing costs","authors":"Wei Liu, Li Chen, Wenyi Zhang","year":"2022","venue":"IEEE Transactions on Signal and Information Processing over Networks","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[McMahan et al., 2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273-1282. PMLR, 2017.","title":"Communication-efficient learning of deep networks from decentralized data","authors":"Brendan Mcmahan, Eider Moore, Daniel Ramage, Seth Hampson, Blaise Aguera Y Arcas","year":"2017","venue":"Artificial intelligence and statistics","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Mnih et al., 2016] Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asyn- chronous methods for deep reinforcement learning. In International conference on machine learning, pages 1928- 1937. PMLR, 2016.","title":"Asynchronous methods for deep reinforcement learning","authors":"Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, Koray Kavukcuoglu","year":"2016","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Nedić et al., 2018] Angelia Nedić, Alex Olshevsky, and Michael G Rabbat. Network topology and communication- computation tradeoffs in decentralized optimization. Pro- ceedings of the IEEE, 106(5):953-976, 2018.","title":"Network topology and communicationcomputation tradeoffs in decentralized optimization","authors":"Angelia Nedić, Alex Olshevsky, Michael G Rabbat","year":"2018","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Robbins and Monro, 1951] Herbert Robbins and Sutton Monro. A stochastic approximation method. The annals of mathematical statistics, pages 400-407, 1951.","title":"The annals of mathematical statistics","authors":"Monro Robbins, Herbert Robbins, Sutton Monro","year":"1951","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Sayin et al., 2021] Muhammed Sayin, Kaiqing Zhang, David Leslie, Tamer Basar, and Asuman Ozdaglar. Decen- tralized q-learning in zero-sum markov games. Advances in Neural Information Processing Systems, 34:18320-18334, 2021.","title":"Decentralized q-learning in zero-sum markov games","authors":"Kaiqing Muhammed Sayin, David Zhang, Tamer Leslie, Asuman Basar, Ozdaglar","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Srikant and Ying, 2019] Rayadurgam Srikant and Lei Ying. Finite-time error bounds for linear stochastic approximation and td learning. In Conference on Learning Theory, pages 2803-2830. PMLR, 2019.","title":"Rayadurgam Srikant and Lei Ying. Finite-time error bounds for linear stochastic approximation and td learning","authors":"Ying Srikant","year":"2019","venue":"Conference on Learning Theory","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Stich, 2018] Sebastian U Stich. Local sgd converges fast and communicates little. In International Conference on Learning Representations, 2018.","title":"Local sgd converges fast and communicates little","authors":"U Sebastian, Stich","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Sun et al., 2020] Jun Sun, Gang Wang, Georgios B Gian- nakis, Qinmin Yang, and Zaiyue Yang. Finite-time analysis of decentralized temporal-difference learning with linear function approximation. In International Conference on Ar- tificial Intelligence and Statistics, pages 4485-4495. PMLR, 2020.","title":"Finite-time analysis of decentralized temporal-difference learning with linear function approximation","authors":"Jun Sun, Gang Wang, Georgios B Giannakis, Qinmin Yang, Zaiyue Yang","year":"2020","venue":"International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Sun et al., 2022] Tao Sun, Dongsheng Li, and Bao Wang. Decentralized federated averaging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.","title":"Decentralized federated averaging","authors":"Sun","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Sutton and Barto, 2018] Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018.","title":"Reinforcement learning: An introduction","authors":"S Richard, Andrew G Sutton, Barto","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Tsitsiklis, 1994] John N Tsitsiklis. Asynchronous stochastic approximation and q-learning. Machine learning, 16:185- 202, 1994.","title":"Asynchronous stochastic approximation and q-learning","authors":"John N Tsitsiklis","year":"1994","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Wai, 2020] Hoi-To Wai. On the convergence of consensus algorithms with markovian noise and gradient bias. In 2020 59th IEEE Conference on Decision and Control, pages 4897-4902. IEEE, 2020.","title":"On the convergence of consensus algorithms with markovian noise and gradient bias","authors":"Hoi-To Wai","year":"2020","venue":"59th IEEE Conference on Decision and Control","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wang et al., 2020] Gang Wang, Songtao Lu, Georgios Gi- annakis, Gerald Tesauro, and Jian Sun. Decentralized td tracking with linear function approximation and its finite- time analysis. Advances in Neural Information Processing Systems, 33:13762-13772, 2020.","title":"Decentralized td tracking with linear function approximation and its finitetime analysis","authors":"Gang Wang, Songtao Lu, Georgios Giannakis, Gerald Tesauro, Jian Sun","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Watkins and Dayan, 1992] Christopher JCH Watkins and Pe- ter Dayan. Q-learning. Machine learning, 8:279-292, 1992.","title":"Q-learning","authors":"Jch Christopher, Peter Watkins, Dayan","year":"1992","venue":"Machine learning","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Xin et al., 2020] Ran Xin, Soummya Kar, and Usman A Khan. Decentralized stochastic optimization and machine learning: A unified variance-reduction framework for ro- bust performance and fast convergence. IEEE Signal Pro- cessing Magazine, 37(3):102-113, 2020.","title":"Decentralized stochastic optimization and machine learning: A unified variance-reduction framework for robust performance and fast convergence","authors":"Ran Xin, Soummya Kar, Usman A Khan","year":"2020","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Xu and Gu, 2020] Pan Xu and Quanquan Gu. A finite-time analysis of q-learning with neural network function approx- imation. In International Conference on Machine Learning, pages 10555-10565. PMLR, 2020.","title":"A finite-time analysis of q-learning with neural network function approximation","authors":"Gu ; ] Pan Xu, Quanquan Xu, Gu","year":"2020","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Yang et al., 2019] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):1-19, 2019.","title":"Federated machine learning: Concept and applications","authors":"Qiang Yang, Yang Liu, Tianjian Chen, Yongxin Tong","year":"2019","venue":"ACM Transactions on Intelligent Systems and Technology (TIST)","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Zeng et al., 2021a] Sihan Zeng, Malik Aqeel Anwar, Thinh T Doan, Arijit Raychowdhury, and Justin Romberg. A decentralized policy gradient approach to multi-task rein- forcement learning. In Uncertainty in Artificial Intelligence, pages 1002-1012. PMLR, 2021.","title":"A decentralized policy gradient approach to multi-task reinforcement learning","authors":"Zeng","year":"2021","venue":"Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Zeng et al., 2021b] Sihan Zeng, Thinh T Doan, and Justin Romberg. Finite-time analysis of decentralized stochastic approximation with applications in multi-agent and multi- task learning. In 2021 60th IEEE Conference on Decision and Control (CDC), pages 2641-2646. IEEE, 2021.","title":"Finite-time analysis of decentralized stochastic approximation with applications in multi-agent and multitask learning","authors":"Zeng","year":"2021","venue":"60th IEEE Conference on Decision and Control (CDC)","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Zeng et al., 2022] Sihan Zeng, Thinh T Doan, and Justin Romberg. Finite-time convergence rates of decentralized stochastic approximation with applications in multi-agent and multi-task learning. IEEE Transactions on Automatic Control, 2022.","title":"Finite-time convergence rates of decentralized stochastic approximation with applications in multi-agent and multi-task learning","authors":"Sihan Zeng, Justin Thinh T Doan, Romberg","year":"2022","venue":"IEEE Transactions on Automatic Control","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Zhang et al., 2021] Xin Zhang, Zhuqing Liu, Jia Liu, Zhengyuan Zhu, and Songtao Lu. Taming communica- tion and sample complexities in decentralized policy eval- uation for cooperative multi-agent reinforcement learn- ing. Advances in Neural Information Processing Systems, 34:18825-18838, 2021.","title":"Taming communication and sample complexities in decentralized policy evaluation for cooperative multi-agent reinforcement learning","authors":"Xin Zhang, Zhuqing Liu, Jia Liu, Zhengyuan Zhu, Songtao Lu","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}