{"type":"ScholarlyArticle","title":"Globally Consistent Federated Graph Autoencoder for Non-IID Graphs","authors":[{"name":"Kun Guo","slug":"kun-guo","given":"Kun","family":"Guo","affiliation":"College of Computer and Data Science , Fuzhou University , Fuzhou 350108 , China"},{"name":"Yutong Fang","slug":"yutong-fang","given":"Yutong","family":"Fang","affiliation":"College of Computer and Data Science , Fuzhou University , Fuzhou 350108 , China"},{"name":"Qingqing Huang","slug":"qingqing-huang","given":"Qingqing","family":"Huang","affiliation":"College of Computer and Data Science , Fuzhou University , Fuzhou 350108 , China"},{"name":"Yuting Liang","slug":"yuting-liang","given":"Yuting","family":"Liang","affiliation":"College of Computer and Data Science , Fuzhou University , Fuzhou 350108 , China"},{"name":"Ziyao Zhang","slug":"ziyao-zhang","given":"Ziyao","family":"Zhang","affiliation":"College of Computer and Data Science , Fuzhou University , Fuzhou 350108 , China"},{"name":"Wenyu He","slug":"wenyu-he","given":"Wenyu","family":"He","affiliation":"College of Computer and Data Science , Fuzhou University , Fuzhou 350108 , China"},{"name":"Liu Yang","slug":"liu-yang","given":"Liu","family":"Yang","affiliation":"Department of Computer Science and Engineering , The Hong Kong University of Science and Technology , Hong Kong 999077 , China"},{"name":"Kai Chen","slug":"kai-chen","given":"Kai","family":"Chen","affiliation":"Department of Computer Science and Engineering , The Hong Kong University of Science and Technology , Hong Kong 999077 , China"},{"name":"Ximeng Liu","slug":"ximeng-liu","given":"Ximeng","family":"Liu","affiliation":"College of Computer and Data Science , Fuzhou University , Fuzhou 350108 , China"},{"name":"Wenzhong Guo","slug":"wenzhong-guo","given":"Wenzhong","family":"Guo","affiliation":"College of Computer and Data Science , Fuzhou University , Fuzhou 350108 , China"}],"abstract":"Graph neural networks (GNNs) have been applied successfully in many machine learning tasks due to their advantages in utilizing neighboring information. Recently, with the global enactment of privacy protection regulations, federated GNNs have gained increasing attention in academia and industry. However, the graphs owned by different participants could be non-independently-and-identically distributed (non-IID), leading to the deterioration of federated GNNs' accuracy. In this paper, we propose a globally consistent federated graph autoencoder (GCFGAE) to overcome the non-IID problem in unsupervised federated graph learning via three innovations. First, by integrating federated learning with split learning, we train a unique global model instead of FedAvg-styled global and local models, yielding results consistent with that of the centralized GAE. Second, we design a collaborative computation mechanism considering overlapping vertices to reduce communication overhead during forward propagation. Third, we develop a layer-wise and block-wise gradient computation strategy to reduce the space and communication complexity during backward propagation. Experiments on real-world datasets demonstrate that GCFGAE achieves not only higher accuracy but also around 500 times lower communication overhead and 1000 times smaller space overhead than existing federated GNN models.","doi":"10.24963/ijcai.2023/419","url":"https://ijcai.org/proceedings/2023/419","pdf":"https://ijcai.org/proceedings/2023/0419.pdf","markdown":"https://ijcai.org/proceedings/2023/419.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/419","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":419,"pages":{"first":3768,"last":3776},"keywords":["Data Mining: DM: Mining graphs","Machine Learning: ML: Federated learning"],"videos":[],"references":[{"position":1,"raw":"[Abadal et al., 2021] Sergi Abadal, Akshay Jain, Robert Guirado, Jorge López-Alonso, and Eduard Alarcón. Com- puting graph neural networks: A survey from algo- rithms to accelerators. ACM Computing Surveys (CSUR), 54(9):1-38, 2021.","title":"Computing graph neural networks: A survey from algorithms to accelerators","authors":"Akshay Sergi Abadal, Robert Jain, Jorge Guirado, Eduard López-Alonso, Alarcón","year":"2021","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Baek et al., 2022] Jinheon Baek, Wonyong Jeong, Jiong- dao Jin, Jaehong Yoon, and Sung Ju Hwang. Per- sonalized subgraph federated learning. arXiv preprint arXiv:2206.10206, 2022.","title":"Personalized subgraph federated learning","authors":"Baek","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Baik, 2020] Jeeyun Sophia Baik. Data privacy against inno- vation or against discrimination?: The case of the califor- nia consumer privacy act (ccpa). Telematics and Informat- ics, 52, 2020.","title":"Jeeyun Sophia Baik. Data privacy against innovation or against discrimination?: The case of the california consumer privacy act (ccpa)","authors":"Baik","year":"2020","venue":"Telematics and Informatics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brickell and Shmatikov, 2005] Justin Brickell and Vitaly Shmatikov. Privacy-preserving graph algorithms in the semi-honest model. In Advances in Cryptology -ASI- ACRYPT 2005, 11th International Conference on the The- ory and Application of Cryptology and Information Secu- rity, Chennai, India, December 4-8, 2005, Proceedings, 2005.","title":"Privacy-preserving graph algorithms in the semi-honest model","authors":"Shmatikov Brickell, Justin Brickell, Vitaly Shmatikov","year":"2005","venue":"Advances in Cryptology -ASI-ACRYPT 2005, 11th International Conference on the Theory and Application of Cryptology and Information Security","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Caldarola et al., 2021] Debora Caldarola, Massimiliano Mancini, Fabio Galasso, Marco Ciccone, Emanuele Rodolà, and Barbara Caputo. Cluster-driven graph federated learning over multiple domains. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2749-2758, 2021.","title":"Cluster-driven graph federated learning over multiple domains","authors":"Debora Caldarola, Massimiliano Mancini, Fabio Galasso, Marco Ciccone, Emanuele Rodolà, Barbara Caputo","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chakraborty et al., 2017] Tanmoy Chakraborty, Ayushi Dalmia, Animesh Mukherjee, and Niloy Ganguly. Metrics for community analysis: A survey. ACM Computing Surveys (CSUR), 50(4):1-37, 2017.","title":"Metrics for community analysis: A survey","authors":"Tanmoy Chakraborty, Ayushi Dalmia, Animesh Mukherjee, Niloy Ganguly","year":"2017","venue":"ACM Computing Surveys (CSUR)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Du and Wu, 2022] Bingqian Du and Chuan Wu. Federated graph learning with periodic neighbour sampling. In 2022 IEEE/ACM 30th International Symposium on Quality of Service (IWQoS), pages 1-10. IEEE, 2022.","title":"Bingqian Du and Chuan Wu. Federated graph learning with periodic neighbour sampling","authors":"Wu Du","year":"2022","venue":"IEEE/ACM 30th International Symposium on Quality of Service (IWQoS)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fu et al., 2022] Xingbo Fu, Binchi Zhang, Yushun Dong, Chen Chen, and Jundong Li. Federated graph machine learning: A survey of concepts, techniques, and applica- tions. ACM SIGKDD Explorations Newsletter, 24(2):32- 47, 2022.","title":"Federated graph machine learning: A survey of concepts, techniques, and applications","authors":"Fu","year":"2022","venue":"ACM SIGKDD Explorations Newsletter","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gonzalez et al., 2012] Joseph E Gonzalez, Yucheng Low, Haijie Gu, Danny Bickson, and Carlos Guestrin. {PowerGraph}: Distributed {Graph-Parallel} computa- tion on natural graphs. In 10th USENIX symposium on operating systems design and implementation (OSDI 12), pages 17-30, 2012.","title":"{PowerGraph}: Distributed {Graph-Parallel} computation on natural graphs","authors":"Yucheng Joseph E Gonzalez, Haijie Low, Danny Gu, Carlos Bickson, Guestrin","year":"2012","venue":"10th USENIX symposium on operating systems design and implementation (OSDI 12)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hamilton et al., 2017] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing sys- tems, 30, 2017.","title":"Inductive representation learning on large graphs","authors":"Hamilton","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jia et al., 2018] Jian Jia, Ginger Zhe Jin, and Liad Wagman. The short-run effects of gdpr on technology venture in- vestment. Technical report, National Bureau of Economic Research, 2018.","title":"The short-run effects of gdpr on technology venture investment","authors":"Jian Jia, Ginger Zhe, Jin, Liad Wagman","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kipf and Welling, 2016] Thomas N Kipf and Max Welling. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308, 2016.","title":"Variational graph auto-encoders","authors":"N Thomas, Max Kipf, Welling","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Liu and Yu, 2022] Rui Liu and Han Yu. Federated graph neural networks: Overview, techniques and challenges. arXiv preprint arXiv:2202.07256, 2022.","title":"Federated graph neural networks: Overview, techniques and challenges","authors":"Rui Liu, Han Yu","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Majeed and Lee, 2020] Abdul Majeed and Sungchang Lee. Anonymization techniques for privacy preserving data publishing: A comprehensive survey. IEEE access, 9:8512-8545, 2020.","title":"Anonymization techniques for privacy preserving data publishing: A comprehensive survey","authors":"Lee ; Majeed, Abdul Majeed, Sungchang Lee","year":"2020","venue":"IEEE access","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[McMahan et al., 2016] H Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, et al. Communication- efficient learning of deep networks from decentralized data. arXiv preprint arXiv:1602.05629, 2016.","title":"Communicationefficient learning of deep networks from decentralized data","authors":"Eider H Brendan Mcmahan, Daniel Moore, Seth Ramage, Hampson","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Peng et al., 2022] Liang Peng, Nan Wang, Nicha Dvornek, Xiaofeng Zhu, and Xiaoxiao Li. Fedni: Federated graph learning with network inpainting for population-based dis- ease prediction. IEEE Transactions on Medical Imaging, 2022.","title":"Fedni: Federated graph learning with network inpainting for population-based disease prediction","authors":"Liang Peng, Nan Wang, Nicha Dvornek, Xiaofeng Zhu, Xiaoxiao Li","year":"2022","venue":"IEEE Transactions on Medical Imaging","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shamir, 1979] Adi Shamir. How to share a secret. Commu- nications of the ACM, 22(11):612-613, nov 1979.","title":"Adi Shamir. How to share a secret","authors":"Shamir","year":"1979","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Shan et al., 2021] Chuanqiang Shan, Huiyun Jiao, and Jie Fu. Towards representation identical privacy-preserving graph neural network via split learning. arXiv preprint arXiv:2107.05917, 2021.","title":"Towards representation identical privacy-preserving graph neural network via split learning","authors":"Shan","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Shi et al., 2020] Han Shi, Haozheng Fan, and James T Kwok. Effective decoding in graph auto-encoder using triadic closure. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 906-913, 2020.","title":"Effective decoding in graph auto-encoder using triadic closure","authors":"Han Shi, Haozheng Fan, James T Kwok","year":"2020","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sooksatra et al., 2022] Korn Sooksatra, Rokin Maharjan, and Tomas Cerny. Monolith to microservices: Vae-based gnn approach with duplication consideration. In 2022 IEEE International Conference on Service-Oriented Sys- tem Engineering (SOSE), pages 1-10. IEEE, 2022.","title":"Monolith to microservices: Vae-based gnn approach with duplication consideration","authors":"Korn Sooksatra, Rokin Maharjan, Tomas Cerny","year":"2022","venue":"IEEE International Conference on Service-Oriented System Engineering (SOSE)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Thapa et al., 2022] Chandra Thapa, Pathum Chamikara Mahawaga Arachchige, Seyit Camtepe, and Lichao Sun. Splitfed: When federated learning meets split learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8485-8493, 2022.","title":"Splitfed: When federated learning meets split learning","authors":"Chandra Thapa, Pathum Chamikara Mahawaga Arachchige, Seyit Camtepe, Lichao Sun","year":"2022","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Tian et al., 2022] Muchen Tian, Zhengming Zhang, Qinzhen Xu, and Luxi Yang. A privacy-preserved split learning solution for deep learning-based mmwave beam selection. IEEE Communications Letters, 2022.","title":"A privacy-preserved split learning solution for deep learning-based mmwave beam selection","authors":"Tian","year":"2022","venue":"IEEE Communications Letters","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wu et al., 2021] Yulei Wu, Hong-Ning Dai, and Haina Tang. Graph neural networks for anomaly detection in in- dustrial internet of things. IEEE Internet of Things Jour- nal, 2021.","title":"Graph neural networks for anomaly detection in industrial internet of things","authors":"Wu","year":"2021","venue":"IEEE Internet of Things Journal","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wu et al., 2022] Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, and Bin Cui. Graph neural networks in rec- ommender systems: a survey. ACM Computing Surveys, 55(5):1-37, 2022.","title":"Graph neural networks in recommender systems: a survey","authors":"Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, Bin Cui","year":"2022","venue":"ACM Computing Surveys","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Xia et al., 2022] Tengxi Xia, Yongheng Deng, Sheng Yue, Junyi He, Ju Ren, and Yaoxue Zhang. Hsfl: An efficient split federated learning framework via hierarchical orga- nization. In 2022 18th International Conference on Net- work and Service Management (CNSM), pages 1-9. IEEE, 2022.","title":"Hsfl: An efficient split federated learning framework via hierarchical organization","authors":"Tengxi Xia, Yongheng Deng, Sheng Yue, Junyi He, Ju Ren, Yaoxue Zhang","year":"2022","venue":"18th International Conference on Network and Service Management (CNSM)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Xie et al., 2021] Han Xie, Jing Ma, Li Xiong, and Carl Yang. Federated graph classification over non-iid graphs. Advances in Neural Information Processing Systems, 34:18839-18852, 2021.","title":"Federated graph classification over non-iid graphs","authors":"Han Xie, Jing Ma, Li Xiong, Carl Yang","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yang et al., 2022] Ziyuan Yang, Yingyu Chen, Huijie Huangfu, Maosong Ran, Hui Wang, Xiaoxiao Li, and Yi Zhang. Robust split federated learning for u-shaped medical image networks. arXiv preprint arXiv:2212.06378, 2022.","title":"Robust split federated learning for u-shaped medical image networks","authors":"Ziyuan Yang, Yingyu Chen, Huijie Huangfu, Maosong Ran, Hui Wang, Xiaoxiao Li, Yi Zhang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2021] Ke Zhang, Carl Yang, Xiaoxiao Li, Lichao Sun, and Siu Ming Yiu. Subgraph federated learn- ing with missing neighbor generation. Advances in Neural Information Processing Systems, 34:6671-6682, 2021.","title":"Subgraph federated learning with missing neighbor generation","authors":"Ke Zhang, Carl Yang, Xiaoxiao Li, Lichao Sun, Siu Ming, Yiu","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhang et al., 2022a] Miao Zhang, Liangqiong Qu, Praveer Singh, Jayashree Kalpathy-Cramer, and Daniel L Rubin. Splitavg: A heterogeneity-aware federated deep learning method for medical imaging. IEEE Journal of Biomedical and Health Informatics, 26(9):4635-4644, 2022.","title":"Splitavg: A heterogeneity-aware federated deep learning method for medical imaging","authors":"Zhang","year":"2022","venue":"IEEE Journal of Biomedical and Health Informatics","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2022b] Taolin Zhang, Chuan Chen, Yaomin Chang, Lin Shu, and Zibin Zheng. Fedego: Privacy- preserving personalized federated graph learning with ego-graphs. arXiv preprint arXiv:2208.13685, 2022.","title":"Fedego: Privacypreserving personalized federated graph learning with ego-graphs","authors":"Zhang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}