{"type":"ScholarlyArticle","title":"Efficient Protocol for Collaborative Dictionary Learning in Decentralized Networks","authors":[{"name":"Tsuyoshi Idé","slug":"tsuyoshi-ide","given":"Tsuyoshi","family":"Idé","affiliation":"IBM Research , Thomas J. Watson Research Center"},{"name":"Rudy Raymond","slug":"rudy-raymond","given":"Rudy","family":"Raymond","affiliation":"IBM Research -Tokyo; Quantum Computing Center , Keio University"},{"name":"Dzung T. Phan","slug":"dzung-t-phan","given":"Dzung T.","family":"Phan","affiliation":"IBM Research , Thomas J. Watson Research Center"}],"abstract":"This paper is concerned with the task of collaborative density estimation in the distributed multi-task setting. Major application scenarios include collaborative anomaly detection among distributed industrial assets owned by different companies competing with each other. Of critical importance here is to achieve two conflicting goals at once: data privacy and collaboration.\r\n\r\nTo this end, we propose a new framework for collaborative dictionary learning. By using a mixture of the exponential family, we show that collaborative learning can be nicely separated into three steps: local updates, global consensus, and optimization. For the critical step of consensus building, we propose a new algorithm that does not rely on expensive encryption-based multi-party computation. Our theoretical and experimental analysis shows that our method is several orders of magnitude faster than the alternative.","doi":"10.24963/ijcai.2019/359","url":"https://ijcai.org/proceedings/2019/359","pdf":"https://ijcai.org/proceedings/2019/0359.pdf","markdown":"https://ijcai.org/proceedings/2019/359.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/359","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning A-L","track":"Main track","number":359,"pages":{"first":2585,"last":2591},"keywords":["Multidisciplinary Topics and Applications: Security and Privacy","Robotics: Sensor Networks","Planning and Scheduling: Distributed;Multi-agent Planning","Machine Learning Applications: Applications of Unsupervised Learning","Machine Learning: Transfer, Adaptation, Multi-task Learning"],"videos":[],"references":[{"position":1,"raw":"[Ahram et al., 2017] Tareq Ahram, Arman Sargolzaei, Saman Sargolzaei, Jeff Daniels, and Ben Amaba. Blockchain technology innovations. In 2017 IEEE Technology & Engineering Management Conference (TEMSCON), pages 137-141. IEEE, 2017.","title":"Blockchain technology innovations","authors":"Tareq Ahram, Arman Sargolzaei, Saman Sargolzaei, Jeff Daniels, Ben Amaba","year":"2017","venue":"2017 IEEE Technology & Engineering Management Conference (TEMSCON)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bonawitz et al., 2017] Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy-preserving ma- chine learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 1175-1191. ACM, 2017.","title":"Practical secure aggregation for privacy-preserving machine learning","authors":"Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, Brendan Mcmahan, Sarvar Patel, Daniel Ramage, Aaron Segal, Karn Seth","year":"2017","venue":"Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cachin and Vukolić, 2017] Christian Cachin and Marko Vukolić. Blockchains consensus protocols in the wild. arXiv preprint arXiv:1707.01873, 2017.","title":"Christian Cachin and Marko Vukolić. Blockchains consensus protocols in the wild","authors":"Vukolić Cachin","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chiquet et al., 2011] Julien Chiquet, Yves Grandvalet, and Christophe Ambroise. Inferring multiple graphical struc- tures. Statistics and Computing, 21(4):537-553, 2011.","title":"Inferring multiple graphical structures","authors":"Julien Chiquet, Yves Grandvalet, Christophe Ambroise","year":"2011","venue":"Statistics and Computing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Corduneanu and Bishop, 2001] Adrian Corduneanu and Christopher M Bishop. Variational Bayesian model se- lection for mixture distributions. In Artificial intelligence and Statistics, volume 2001, pages 27-34, 2001.","title":"Variational Bayesian model selection for mixture distributions","authors":"Adrian Corduneanu, Christopher M Bishop","year":"2001","venue":"Artificial intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Danezis et al., 2013] George Danezis, Cédric Fournet, Markulf Kohlweiss, and Santiago Zanella-Béguelin. Smart meter aggregation via secret-sharing. In Proceed- ings of the first ACM workshop on Smart energy grid se- curity, pages 75-80. ACM, 2013.","title":"Smart meter aggregation via secret-sharing","authors":"George Danezis, Cédric Fournet, Markulf Kohlweiss, Santiago Zanella-Béguelin","year":"2013","venue":"Proceedings of the first ACM workshop on Smart energy grid security","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Feldman et al., 1988] P. Feldman, J. Friedman, and N. Pip- penger. Wide-sense nonblocking networks. SIAM Journal on Discrete Mathematics, 1(2):158-173, 1988.","title":"Wide-sense nonblocking networks","authors":"P Feldman, J Friedman, N","year":"1988","venue":"SIAM Journal on Discrete Mathematics","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Goryczka et al., 2013] Slawomir Goryczka, Li Xiong, and Vaidy Sunderam. Secure multiparty aggregation with dif- ferential privacy: A comparative study. In Proceedings of the Joint EDBT/ICDT 2013 Workshops, pages 155-163. ACM, 2013.","title":"Secure multiparty aggregation with differential privacy: A comparative study","authors":"Slawomir Goryczka, Li Xiong, Vaidy Sunderam","year":"2013","venue":"Proceedings of the Joint EDBT/ICDT 2013 Workshops","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Honorio and Samaras, 2010] Jean Honorio and Dimitris Samaras. Multi-task learning of Gaussian graphical mod- els. In Proceedings of the 27th International Conference on Machine Learning (ICML), pages 447-454, 2010.","title":"Multi-task learning of Gaussian graphical models","authors":"Samaras Honorio, Jean Honorio, Dimitris Samaras","year":"2010","venue":"Proceedings of the 27th International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Idé et al., 2017] Tsuyoshi Idé, Dzung T. Phan, and Jayant Kalagnanam. Multi-task multi-modal models for collec- tive anomaly detection. In Proc. of the 17th IEEE Intl. Conf. on Data Mining (ICDM), pages 177-186, 2017.","title":"Multi-task multi-modal models for collective anomaly detection","authors":"Tsuyoshi Idé, T Dzung, Jayant Phan, Kalagnanam","year":"2017","venue":"Proc. of the 17th IEEE Intl. Conf. on Data Mining (ICDM)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Lindell, 2005] Yehida Lindell. Secure multiparty computa- tion for privacy preserving data mining. In Encyclopedia of Data Warehousing and Mining, pages 1005-1009. IGI Global, 2005.","title":"Secure multiparty computation for privacy preserving data mining","authors":"Yehida Lindell, Lindell","year":"2005","venue":"Encyclopedia of Data Warehousing and Mining","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Mahimkar and Rappaport, 2004] Ajay Mahimkar and Theodore S Rappaport. Securedav: A secure data aggregation and verification protocol for sensor net- works. In IEEE Global Telecommunications Conference (GLOBECOM), volume 4, pages 2175-2179. IEEE, 2004.","title":"Securedav: A secure data aggregation and verification protocol for sensor networks","authors":"Rappaport Mahimkar, Ajay Mahimkar, Theodore S Rappaport","year":"2004","venue":"IEEE Global Telecommunications Conference (GLOBECOM)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Mohassel and Zhang, 2017] Payman Mohassel and Yupeng Zhang. Secureml: A system for scalable privacy- preserving machine learning. In 2017 IEEE Symposium on Security and Privacy (SP), pages 19-38. IEEE, 2017.","title":"Secureml: A system for scalable privacypreserving machine learning","authors":"Zhang Mohassel, Payman Mohassel, Yupeng Zhang","year":"2017","venue":"IEEE Symposium on Security and Privacy (SP)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Münsing et al., 2017] Eric Münsing, Jonathan Mather, and Scott Moura. Blockchains for decentralized optimization of energy resources in microgrid networks. In Control Technology and Applications (CCTA), 2017 IEEE Confer- ence on, pages 2164-2171. IEEE, 2017.","title":"Blockchains for decentralized optimization of energy resources in microgrid networks","authors":"Eric Münsing, Jonathan Mather, Scott Moura","year":"2017","venue":"Control Technology and Applications (CCTA), 2017 IEEE Conference on","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nakamoto, 2008] Satoshi Nakamoto. Bitcoin: A peer-to- peer electronic cash system. 2008.","title":"Bitcoin: A peer-topeer electronic cash system","authors":"Satoshi Nakamoto","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Narasimhan, 2019] Balasubramanian Narasimhan. homo- morpheR. In CRAN. 2019.","title":"homo-morpheR","authors":"Balasubramanian Narasimhan","year":"2019","venue":"CRAN","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Olfati-Saber et al., 2007] Reza Olfati-Saber, J Alex Fax, and Richard M Murray. Consensus and cooperation in networked multi-agent systems. Proceedings of the IEEE, 95(1):215-233, 2007.","title":"Consensus and cooperation in networked multi-agent systems","authors":"Reza Olfati-Saber, Alex Fax, Richard M Murray","year":"2007","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ren et al., 2005] Wei Ren, Randal W Beard, and Ella M Atkins. A survey of consensus problems in multi-agent coordination. In Proceedings of the 2005, American Con- trol Conference, pages 1859-1864. IEEE, 2005.","title":"A survey of consensus problems in multi-agent coordination","authors":"Wei Ren, Randal W Beard, Ella M Atkins","year":"2005","venue":"Proceedings of the 2005, American Control Conference","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ruan et al., 2017] Minghao Ruan, Muaz Ahmad, and Yongqiang Wang. Secure and privacy-preserving aver- age consensus. In Proceedings of the 2017 Workshop on Cyber-Physical Systems Security and Privacy, pages 123- 129. ACM, 2017.","title":"Secure and privacy-preserving average consensus","authors":"Muaz Minghao Ruan, Yongqiang Ahmad, Wang","year":"2017","venue":"Proceedings of the 2017 Workshop on Cyber-Physical Systems Security and Privacy","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Ruan et al., 2019] Minghao Ruan, Huan Gao, and Yongqiang Wang. Secure and privacy-preserving consensus. IEEE Transactions on Automatic Control, page (to appear), 2019.","title":"Secure and privacy-preserving consensus","authors":"Huan Minghao Ruan, Yongqiang Gao, Wang","year":"2019","venue":"IEEE Transactions on Automatic Control","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Sankar et al., 2017] Lakshmi Siva Sankar, M Sindhu, and M Sethumadhavan. Survey of consensus protocols on blockchain applications. In Proc. of the 4th International Conference on Advanced Computing and Communication Systems (ICACCS), pages 1-5, 2017.","title":"Survey of consensus protocols on blockchain applications","authors":"Lakshmi Siva Sankar, M Sindhu, M Sethumadhavan","year":"2017","venue":"Proc. of the 4th International Conference on Advanced Computing and Communication Systems (ICACCS)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Strang, 1976] Gilbert Strang. Linear Algebra and its Appli- cations. Academic Press, 1976.","title":"Linear Algebra and its Applications","authors":"Gilbert Strang","year":"1976","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vadhan and others, 2012] Salil P Vadhan et al. Pseudoran- domness. Foundations and Trends in Theoretical Com- puter Science, 7(1-3):1-336, 2012.","title":"Pseudorandomness","authors":"P Salil, Vadhan","year":"2012","venue":"Foundations and Trends in Theoretical Computer Science","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wu, 2005] Chai Wah Wu. Privacy preserving data mining with unidirectional interaction. In 2005 IEEE Interna- tional Symposium on Circuits and Systems, pages 5521- 5524. IEEE, 2005.","title":"Privacy preserving data mining with unidirectional interaction","authors":"Wah Chai, Wu","year":"2005","venue":"IEEE International Symposium on Circuits and Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xie et al., 2017] Liyang Xie, Inci M Baytas, Kaixiang Lin, and Jiayu Zhou. Privacy-preserving distributed multi- task learning with asynchronous updates. In Proceed- ings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1195- 1204. ACM, 2017.","title":"Privacy-preserving distributed multitask learning with asynchronous updates","authors":"Xie","year":"2017","venue":"Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang et al., 2019] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 10(2):12, 2019.","title":"Federated machine learning: Concept and applications","authors":"Qiang Yang, Yang Liu, Tianjian Chen, Yongxin Tong","year":"2019","venue":"ACM Transactions on Intelligent Systems and Technology (TIST)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}