{"type":"ScholarlyArticle","title":"Learning to Learn Gradient Aggregation by Gradient Descent","authors":[{"name":"Jinlong Ji","slug":"jinlong-ji","given":"Jinlong","family":"Ji","affiliation":"Case Western Reserve University"},{"name":"Xuhui Chen","slug":"xuhui-chen","given":"Xuhui","family":"Chen","affiliation":"Case Western Reserve University; Kent State University"},{"name":"Qianlong Wang","slug":"qianlong-wang","given":"Qianlong","family":"Wang","affiliation":"Case Western Reserve University"},{"name":"Lixing Yu","slug":"lixing-yu","given":"Lixing","family":"Yu","affiliation":"Case Western Reserve University"},{"name":"Pan Li","slug":"pan-li","given":"Pan","family":"Li","affiliation":"Case Western Reserve University"}],"abstract":"In the big data era, distributed machine learning emerges as an important learning paradigm to mine large volumes of data by taking advantage of distributed computing resources. In this work, motivated by learning to learn, we propose a meta-learning approach to coordinate the learning process in the master-slave type of distributed systems. Specifically, we utilize a recurrent neural network (RNN) in the parameter server (the master) to learn to aggregate the gradients from the workers (the slaves). We design a coordinatewise preprocessing and postprocessing method to make the neural network based aggregator more robust. Besides, to address the fault tolerance, especially the Byzantine attack, in distributed machine learning systems, we propose an RNN aggregator with additional loss information (ARNN) to improve the system resilience. We conduct extensive experiments to demonstrate the effectiveness of the RNN aggregator, and also show that it can be easily generalized and achieve remarkable performance when transferred to other distributed systems. Moreover, under majoritarian Byzantine attacks, the ARNN aggregator outperforms the Krum, the state-of-art fault tolerance aggregation method, by 43.14%. In addition, our RNN aggregator enables the server to aggregate gradients from variant local models, which significantly improve the scalability of distributed learning.","doi":"10.24963/ijcai.2019/363","url":"https://ijcai.org/proceedings/2019/363","pdf":"https://ijcai.org/proceedings/2019/0363.pdf","markdown":"https://ijcai.org/proceedings/2019/363.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/363","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":363,"pages":{"first":2614,"last":2620},"keywords":["Machine Learning: Transfer, Adaptation, Multi-task Learning","Machine Learning: Deep Learning"],"videos":[],"references":[{"position":1,"raw":"[Abadi et al., 2016] Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. Tensorflow: a system for large-scale machine learn- ing. In OSDI, volume 16, pages 265-283, 2016.","title":"Tensorflow: a system for large-scale machine learning","authors":"Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard","year":"2016","venue":"OSDI","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Andrychowicz et al., 2016] Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas. Learning to learn by gradient descent by gradient descent. In Advances in Neural Information Processing Systems, pages 3981-3989, 2016.","title":"Learning to learn by gradient descent by gradient descent","authors":"Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, Nando De Freitas","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bengio et al., 1992] Samy Bengio, Yoshua Bengio, Jocelyn Cloutier, and Jan Gecsei. On the optimization of a synap- tic learning rule. In Preprints Conf. Optimality in Artifi- cial and Biological Neural Networks, pages 6-8. Univ. of Texas, 1992.","title":"On the optimization of a synaptic learning rule","authors":"Samy Bengio, Yoshua Bengio, Jocelyn Cloutier, Jan Gecsei","year":"1992","venue":"Preprints Conf. Optimality in Artificial and Biological Neural Networks","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blanchard et al., 2017] Peva Blanchard, Rachid Guerraoui, Julien Stainer, et al. Machine learning with adversaries: Byzantine tolerant gradient descent. In Advances in Neural Information Processing Systems, pages 119-129, 2017.","title":"Machine learning with adversaries: Byzantine tolerant gradient descent","authors":"Peva Blanchard, Rachid Guerraoui, Julien Stainer","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 2016] Yutian Chen, Matthew W Hoffman, Ser- gio Gómez Colmenarejo, Misha Denil, Timothy P Lilli- crap, Matt Botvinick, and Nando de Freitas. Learning to learn without gradient descent by gradient descent. arXiv preprint arXiv:1611.03824, 2016.","title":"Learning to learn without gradient descent by gradient descent","authors":"Yutian Chen, Matthew W Hoffman, Sergio Gómez Colmenarejo, Misha Denil, Timothy P Lillicrap, Matt Botvinick, Nando De Freitas","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2017] Xuhui Chen, Jinlong Ji, Kenneth Loparo, and Pan Li. Real-time personalized cardiac arrhythmia de- tection and diagnosis: A cloud computing architecture. In 2017 IEEE EMBS International Conference on Biomed- ical & Health Informatics (BHI), pages 201-204. IEEE, 2017.","title":"Real-time personalized cardiac arrhythmia detection and diagnosis: A cloud computing architecture","authors":"Xuhui Chen, Jinlong Ji, Kenneth Loparo, Pan Li","year":"2017","venue":"IEEE EMBS International Conference on Biomedical & Health Informatics (BHI)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2018a] Tianyi Chen, Georgios B Giannakis, Tao Sun, and Wotao Yin. Lag: Lazily aggregated gradi- ent for communication-efficient distributed learning. arXiv preprint arXiv:1805.09965, 2018.","title":"Lag: Lazily aggregated gradient for communication-efficient distributed learning","authors":"Chen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2018b] Xuhui Chen, Jinlong Ji, Changqing Luo, Weixian Liao, and Pan Li. When machine learn- ing meets blockchain: A decentralized, privacy-preserving and secure design. In 2018 IEEE International Conference on Big Data (Big Data), pages 1178-1187. IEEE, 2018.","title":"When machine learning meets blockchain: A decentralized, privacy-preserving and secure design","authors":"Chen","year":"2018","venue":"2018 IEEE International Conference on Big Data (Big Data)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chung et al., 2014] Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evalua- tion of gated recurrent neural networks on sequence mod- eling. arXiv preprint arXiv:1412.3555, 2014.","title":"Empirical evaluation of gated recurrent neural networks on sequence modeling","authors":"Chung","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dean et al., 2012] Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Andrew Senior, Paul Tucker, Ke Yang, Quoc V Le, et al. Large scale distributed deep networks. In Advances in neural in- formation processing systems, pages 1223-1231, 2012.","title":"Large scale distributed deep networks","authors":"Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Andrew Senior, Paul Tucker, Ke Yang, Quoc V Le","year":"2012","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ge et al., 2017] Rong Ge, Jason D Lee, and Tengyu Ma. Learning one-hidden-layer neural networks with land- scape design. arXiv preprint arXiv:1711.00501, 2017.","title":"Learning one-hidden-layer neural networks with landscape design","authors":"Rong Ge, Jason D Lee, Tengyu Ma","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hamm et al., 2016] Jihun Hamm, Yingjun Cao, and Mikhail Belkin. Learning privately from multiparty data. In International Conference on Machine Learning, pages 555-563, 2016.","title":"Learning privately from multiparty data","authors":"Jihun Hamm, Yingjun Cao, Mikhail Belkin","year":"2016","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Houthooft et al., 2018] Rein Houthooft, Richard Y Chen, Phillip Isola, Bradly C Stadie, Filip Wolski, Jonathan Ho, and Pieter Abbeel. Evolved policy gradients. arXiv preprint arXiv:1802.04821, 2018.","title":"Evolved policy gradients","authors":"Rein Houthooft, Phillip Richard Y Chen, Bradly C Isola, Filip Stadie, Jonathan Wolski, Pieter Ho, Abbeel","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Konečnỳ et al., 2015] Jakub Konečnỳ, Brendan McMahan, and Daniel Ramage. Federated optimization: Distributed optimization beyond the datacenter. arXiv preprint arXiv:1511.03575, 2015.","title":"Federated optimization: Distributed optimization beyond the datacenter","authors":"Konečnỳ","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Konečnỳ et al., 2016] Jakub Konečnỳ, H Brendan McMa- han, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492, 2016.","title":"Federated learning: Strategies for improving communication efficiency","authors":"Jakub Konečnỳ, Brendan Mcmahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, Dave Bacon","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lian et al., 2015] Xiangru Lian, Yijun Huang, Yuncheng Li, and Ji Liu. Asynchronous parallel stochastic gradient for nonconvex optimization. In Advances in Neural Informa- tion Processing Systems, pages 2737-2745, 2015.","title":"Asynchronous parallel stochastic gradient for nonconvex optimization","authors":"Lian","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Maclaurin et al., 2015] Dougal Maclaurin, David Duve- naud, and Ryan Adams. Gradient-based hyperparame- ter optimization through reversible learning. In Inter- national Conference on Machine Learning, pages 2113- 2122, 2015.","title":"Gradient-based hyperparameter optimization through reversible learning","authors":"Dougal Maclaurin, David Duvenaud, Ryan Adams","year":"2015","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Nedic and Ozdaglar, 2009] Angelia Nedic and Asuman Ozdaglar. Distributed subgradient methods for multi-agent optimization. IEEE Transactions on Automatic Control, 54(1):48-61, 2009.","title":"Distributed subgradient methods for multi-agent optimization","authors":"Angelia Nedic, Asuman Ozdaglar","year":"2009","venue":"IEEE Transactions on Automatic Control","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Polyak and Juditsky, 1992] Boris T Polyak and Anatoli B Juditsky. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization, 30(4):838-855, 1992.","title":"Acceleration of stochastic approximation by averaging","authors":"T Boris, Anatoli B Polyak, Juditsky","year":"1992","venue":"SIAM Journal on Control and Optimization","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rajkumar and Agarwal, 2012] Arun Rajkumar and Shivani Agarwal. A differentially private stochastic gradient de- scent algorithm for multiparty classification. In Artificial Intelligence and Statistics, pages 933-941, 2012.","title":"A differentially private stochastic gradient descent algorithm for multiparty classification","authors":"Arun Rajkumar, Shivani Agarwal","year":"2012","venue":"Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Scardovi and Leonard, 2009] Luca Scardovi and Naomi Ehrich Leonard. Robustness of aggregation in networked dynamical systems. In Robot Communica- tion and Coordination, 2009. ROBOCOMM'09. Second International Conference on, pages 1-6. IEEE, 2009.","title":"Robustness of aggregation in networked dynamical systems","authors":"Leonard ; Scardovi, Luca Scardovi, Naomi Ehrich Leonard","year":"2009","venue":"Robot Communication and Coordination","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Thrun and Pratt, 1998] Sebastian Thrun and Lorien Pratt. Learning to learn: Introduction and overview. In Learn- ing to learn, pages 3-17. Springer, 1998.","title":"Learning to learn: Introduction and overview","authors":"Pratt Thrun","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Tsitsiklis et al., 1986] John Tsitsiklis, Dimitri Bertsekas, and Michael Athans. Distributed asynchronous deter- ministic and stochastic gradient optimization algorithms. IEEE transactions on automatic control, 31(9):803-812, 1986.","title":"Distributed asynchronous deterministic and stochastic gradient optimization algorithms","authors":"John Tsitsiklis, Dimitri Bertsekas, Michael Athans","year":"1986","venue":"IEEE transactions on automatic control","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Ward, 1937] Lewis B Ward. Reminiscence and rote learn- ing. Psychological Monographs, 49(4):i, 1937.","title":"Reminiscence and rote learning","authors":"B Lewis, Ward","year":"1937","venue":"Psychological Monographs","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xie et al., 2018] Cong Xie, Oluwasanmi Koyejo, and In- dranil Gupta. Zeno: Byzantine-suspicious stochastic gra- dient descent. arXiv preprint arXiv:1805.10032, 2018.","title":"Zeno: Byzantine-suspicious stochastic gradient descent","authors":"Cong Xie, Oluwasanmi Koyejo, Indranil Gupta","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2015] Sixin Zhang, Anna E Choromanska, and Yann LeCun. Deep learning with elastic averaging sgd. In Advances in Neural Information Processing Systems, pages 685-693, 2015.","title":"Deep learning with elastic averaging sgd","authors":"Zhang","year":"2015","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}