{"type":"ScholarlyArticle","title":"LEAP: Optimization Hierarchical Federated Learning on Non-IID Data with Coalition Formation Game","authors":[{"name":"Jianfeng Lu","slug":"jianfeng-lu","given":"Jianfeng","family":"Lu","affiliation":"School of Computer Science and Technology , Wuhan University of Science and Technology , China; Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System , Wuhan University of Science and Technology , China"},{"name":"Yue Chen","slug":"yue-chen","given":"Yue","family":"Chen","affiliation":"School of Computer Science and Technology , Wuhan University of Science and Technology , China"},{"name":"Shuqin Cao","slug":"shuqin-cao","given":"Shuqin","family":"Cao","affiliation":"School of Computer Science and Technology , Wuhan University of Science and Technology , China; Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System , Wuhan University of Science and Technology , China"},{"name":"Longbiao Chen","slug":"longbiao-chen","given":"Longbiao","family":"Chen","affiliation":"School of Informatics , Xiamen University , China"},{"name":"Wei Wang","slug":"wei-wang","given":"Wei","family":"Wang","affiliation":"School of Computer Science and Technology , Wuhan University of Science and Technology , China; Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System , Wuhan University of Science and Technology , China"},{"name":"Yun Xin","slug":"yun-xin","given":"Yun","family":"Xin","affiliation":"School of Computer Science and Technology , Wuhan University of Science and Technology , China"}],"abstract":"Although Hierarchical Federated Learning (HFL) utilizes edge servers (ESs) to alleviate communication burdens, its model performance will be degraded by non-IID data and limited communication resources. Current works often assume that data is uniformly distributed, which however contradicts the heterogeneity of IoT. Solutions involving additional model training to check the data distribution inevitably increase computational costs and the risk of privacy leakage. The challenges in solving these issues are how to reduce the impact of non-IID data without involving raw data, and how to rationalize the communication resource allocation for addressing straggler problem. To tackle these challenges, we propose a novel optimization method based on coaLition formation gamE and grAdient Projection, called LEAP. Specifically, we combine edge data distribution with coalition formation game innovatively to adjust the correlations between clients and ESs dynamically, ensuring optimal correlations. We further capture the client heterogeneity to achieve the rational bandwidth allocation from coalition perception and determine the optimal transmission power within specified delay constraints at the client level. Experimental results on four real datasets show that LEAP is able to achieve 20.62% improvement in model accuracy compared to the state-of-the-art baselines. Moreover, LEAP effectively reduces transmission energy consumption by at least about 2.24 times.","doi":"10.24963/ijcai.2024/515","url":"https://ijcai.org/proceedings/2024/515","pdf":"https://ijcai.org/proceedings/2024/0515.pdf","markdown":"https://ijcai.org/proceedings/2024/515.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/515","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":515,"pages":{"first":4660,"last":4668},"keywords":["Agent-based and Multi-agent Systems: MAS: Resource allocation","Machine Learning: ML: Evaluation","Game Theory and Economic Paradigms: GTEP: Mechanism design","Machine Learning: ML: Game Theory"],"videos":[],"references":[{"position":1,"raw":"[McMahan et al., 2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Ar- cas. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th Inter- national Conference on Artificial Intelligence and Statis- tics, pages 1273-1282, Fort Lauderdale, USA, April 2017. Proceedings of Machine Learning Research.","title":"Communication-efficient learning of deep networks from decentralized data","authors":"Brendan Mcmahan, Eider Moore, Daniel Ramage, Seth Hampson, Blaise Agüera Y Arcas","year":"2017","venue":"Proceedings of the 20th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Rahman et al., 2023] Anichur Rahman, Kamrul Hasan, Di- panjali Kundu, Md. Jahidul Islam, Tanoy Debnath, Shahab S. Band, and Neeraj Kumar. On the ICN-IoT with fed- erated learning integration of communication: Concepts, security-privacy issues, applications, and future perspec- tives. Future Generation Computer Systems, 138:61-88, 2023.","title":"On the ICN-IoT with federated learning integration of communication: Concepts, security-privacy issues, applications, and future perspectives","authors":"Rahman","year":"2023","venue":"Future Generation Computer Systems","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Pandya et al., 2023] Sharnil Pandya, Gautam Srivastava, Rutvij Jhaveri, M. Rajasekhara Babu, Sweta Bhattacharya, Praveen Kumar Reddy Maddikunta, Spyridon Mastorakis, Md. Jalil Piran, and Thippa Reddy Gadekallu. Fed- erated learning for smart cities: A comprehensive sur- vey. Sustainable Energy Technologies and Assessments, 55:102987, 2023.","title":"Federated learning for smart cities: A comprehensive survey","authors":"Sharnil Pandya, Gautam Srivastava, Rutvij Jhaveri, M Rajasekhara Babu, Sweta Bhattacharya, Praveen Kumar Reddy Maddikunta, Spyridon Mastorakis, Md Jalil Piran, Thippa Reddy Gadekallu","year":"2023","venue":"Sustainable Energy Technologies and Assessments","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Zhang et al., 2023] Li Zhang, Jianbo Xu, Pandi Vijayaku- mar, Pradip Kumar Sharma, and Uttam Ghosh. Homomor- phic encryption-based privacy-preserving federated learn- ing in iot-enabled healthcare system. IEEE Transactions on Network Science and Engineering, 10(5):2864-2880, 2023.","title":"Homomorphic encryption-based privacy-preserving federated learning in iot-enabled healthcare system","authors":"Li Zhang, Jianbo Xu, Pandi Vijayakumar, Pradip Kumar Sharma, Uttam Ghosh","year":"2023","venue":"IEEE Transactions on Network Science and Engineering","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Wang et al., 2023] Zhilin Wang, Qin Hu, Ruinian Li, Minghui Xu, and Zehui Xiong. Incentive mechanism de- sign for joint resource allocation in blockchain-based fed- erated learning. IEEE Transactions on Parallel and Dis- tributed Systems, 34(5):1536-1547, 2023.","title":"Incentive mechanism design for joint resource allocation in blockchain-based federated learning","authors":"Zhilin Wang, Qin Hu, Ruinian Li, Minghui Xu, Zehui Xiong","year":"2023","venue":"IEEE Transactions on Parallel and Distributed Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Xu et al., 2018] Wenchao Xu, Haibo Zhou, Nan Cheng, Feng Lyu, Weisen Shi, Jiayin Chen, and Xuemin Shen. Internet of vehicles in big data era. IEEE/CAA Journal of Automatica Sinica, 5(1):19-35, 2018.","title":"Internet of vehicles in big data era","authors":"Wenchao Xu, Haibo Zhou, Nan Cheng, Feng Lyu, Weisen Shi, Jiayin Chen, Xuemin Shen","year":"2018","venue":"IEEE/CAA Journal of Automatica Sinica","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Xia et al., 2020] Wenchao Xia, Tony Q. S. Quek, Kun Guo, Wanli Wen, Howard H. Yang, and Hongbo Zhu. Multi- armed bandit-based client scheduling for federated learn- ing. IEEE Transactions on Wireless Communications, 19(11):7108-7123, 2020.","title":"Multiarmed bandit-based client scheduling for federated learning","authors":"Wenchao Xia, Q S Tony, Kun Quek, Wanli Guo, Howard H Wen, Hongbo Yang, Zhu","year":"2020","venue":"IEEE Transactions on Wireless Communications","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2023] Haokun Chen, Ahmed Frikha, Denis Krompass, Jindong Gu, and Volker Tresp. Fraug: Tack- ling federated learning with non-iid features via represen- tation augmentation. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 4849- 4859, Paris, France, October, 2023. Institute of Electrical and Electronics Engineers. Zhu et al., 2023] Xi Zhu, Junbo Wang, Wuhui Chen, and Kento Sato. Model compression and privacy preserv- ing framework for federated learning. Future Generation Computer Systems, 140:376-389, 2023.","title":"Fraug: Tackling federated learning with non-iid features via representation augmentation","authors":"Chen","year":"2023","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lin et al., 2023] Xiaohan Lin, Yuan Liu, Fangjiong Chen, Xiaohu Ge, and Yang Huang. Joint gradient sparsifica- tion and device scheduling for federated learning. IEEE Transactions on Green Communications and Networking, 7(3):1407-1419, 2023.","title":"Joint gradient sparsification and device scheduling for federated learning","authors":"Xiaohan Lin, Yuan Liu, Fangjiong Chen, Xiaohu Ge, Yang Huang","year":"2023","venue":"IEEE Transactions on Green Communications and Networking","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[An et al., 2023] Qiaochu An, Yong Zhou, Zhibin Wang, Hangguan Shan, Yuanming Shi, and Mehdi Bennis. On- line optimization for over-the-air federated learning with energy harvesting. IEEE Transactions on Wireless Com- munications, 1-1, 2023.","title":"Online optimization for over-the-air federated learning with energy harvesting","authors":"Qiaochu An, Yong Zhou, Zhibin Wang, Hangguan Shan, Yuanming Shi, Mehdi Bennis","year":"2023","venue":"IEEE Transactions on Wireless Communications","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2020] Lumin Liu, Jun Zhang, Shenghui Song, and Khaled B. Letaief. Client-edge-cloud hierarchical fed- erated learning. ICC 2020 -2020 IEEE International Con- ference on Communications, pages 1-6, Dublin, Ireland, June 2020. Institute of Electrical and Electronics Engi- neers.","title":"Client-edge-cloud hierarchical federated learning","authors":"Lumin Liu, Jun Zhang, Shenghui Song, Khaled B Letaief","year":"2020","venue":"ICC 2020 -2020 IEEE International Conference on Communications","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Yao et al., 2019] Xin Yao, Tianchi Huang, Chenglei Wu, Ruixiao Zhang, and Lifeng Sun. Federated learning with additional mechanisms on clients to reduce communica- tion costs. ArXiv Preprint, arXiv:1908.05891, 2019.","title":"Federated learning with additional mechanisms on clients to reduce communication costs","authors":"Xin Yao, Tianchi Huang, Chenglei Wu, Ruixiao Zhang, Lifeng Sun","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Arisdakessian et al., 2023] Sarhad Arisdakessian, Omar Abdel Wahab, Azzam Mourad, and Hadi Otrok. Coali- tional federated learning: Improving communication and training on non-iid data with selfish clients. IEEE Transactions on Services Computing, 16(4):2462-2476, 2023.","title":"Coalitional federated learning: Improving communication and training on non-iid data with selfish clients","authors":"Omar Abdel Sarhad Arisdakessian, Azzam Wahab, Hadi Mourad, Otrok","year":"2023","venue":"IEEE Transactions on Services Computing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lu et al., 2023] Renhao Lu, Weizhe Zhang, Yan Wang, Qiong Li, Xiaoxiong Zhong, Hongwei Yang, and Desheng Wang. Auction-based cluster federated learning in mobile edge computing systems. IEEE Transactions on Parallel and Distributed Systems, 34(4):1145-1158, 2023.","title":"Auction-based cluster federated learning in mobile edge computing systems","authors":"Lu","year":"2023","venue":"IEEE Transactions on Parallel and Distributed Systems","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Shin et al., 2020] Myungjae Shin, Chihoon Hwang, Joongheon Kim, Jihong Park, Mehdi Bennis, and Seong-Lyun Kim. Xor mixup: Privacy-preserving data augmentation for one-shot federated learning. ArXiv Preprint, arXiv:2006.05148, 2020.","title":"Xor mixup: Privacy-preserving data augmentation for one-shot federated learning","authors":"Shin","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Mills et al., 2019] Jed Mills, Jia Hu, and Geyong Min. Communication-efficient federated learning for wireless edge intelligence in iot. IEEE Internet of Things Journal, 7(7):5986-5994, 2019.","title":"Communication-efficient federated learning for wireless edge intelligence in iot","authors":"Jed Mills, Jia Hu, Geyong Min","year":"2019","venue":"IEEE Internet of Things Journal","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2021] Shengli Liu, Guanding Yu, Rui Yin, and Jiantao Yuan. Adaptive network pruning for wireless fed- erated learning. IEEE Wireless Communications Letters, 10(7):1572-1576, 2021.","title":"Adaptive network pruning for wireless federated learning","authors":"Shengli Liu, Guanding Yu, Rui Yin, Jiantao Yuan","year":"2021","venue":"IEEE Wireless Communications Letters","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Ng et al., 2022] Jer Shyuan Ng, Wei Yang Bryan Lim, Ze- hui Xiong, Xianbin Cao, Jiangming Jin, Dusit Niyato, Cyril Leung, and Chunyan Miao. Reputation-aware he- donic coalition formation for efficient serverless hierarchi- cal federated learning. IEEE Transactions on Parallel and Distributed Systems, 33(11):2675-2686, 2022.","title":"Reputation-aware hedonic coalition formation for efficient serverless hierarchical federated learning","authors":"Ng","year":"2022","venue":"IEEE Transactions on Parallel and Distributed Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[van Erven and Harremos, 2014] Tim van Erven, and Peter Harremoës. Rényi Divergence and Kullback-Leibler Di- vergence. IEEE Transactions on Information Theory, 60(7):3797-3820, 2014.","title":"Rényi Divergence and Kullback-Leibler Divergence","authors":"Harremos ; Tim Van Erven, Peter Van Erven, Harremoës","year":"2014","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Menéndez et al., 1997] María Luisa Menéndez, Julio An- gel Pardo, Leandro Pardo, and María Del Carmen Pardo. The jensen-shannon divergence. Journal of The Franklin Institute-engineering and Applied Mathematics, 334(2):307-318, 1997.","title":"María Luisa Menéndez, Julio Angel Pardo, Leandro Pardo, and María Del Carmen Pardo. The jensen-shannon divergence","authors":"Menéndez","year":"1997","venue":"Journal of The Franklin Institute-engineering and Applied Mathematics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhang et al., 2018] Yuli Zhang, Yuhua Xu, Qihui Wu, Yun- peng Luo, Yitao Xu, Xueqiang Chen, Alagan Anpalagan, and Daoqiang Zhang. Context awareness group buying in d2d networks: A coalition formation game-theoretic approach. IEEE Transactions on Vehicular Technology, 67(12):12259-12272, 2018.","title":"Context awareness group buying in d2d networks: A coalition formation game-theoretic approach","authors":"Yuli Zhang, Yuhua Xu, Qihui Wu, Yunpeng Luo, Yitao Xu, Xueqiang Chen, Alagan Anpalagan, Daoqiang Zhang","year":"2018","venue":"IEEE Transactions on Vehicular Technology","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Chu et al., 2023] Shunfeng Chu, Jun Li, Kang Wei, Yuwen Qian, Kunlun Wang, Feng Shu, and Wen Chen. Design of two-level incentive mechanisms for hierarchical federated learning. ArXiv Preprint, arXiv:2304.04162, 2023.","title":"Design of two-level incentive mechanisms for hierarchical federated learning","authors":"Chu","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Lecun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Ben- gio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Krizhevsky, 2009] Alex Krizhevsky. Learning multiple lay- ers of features from tiny images. Handbook of Systemic Autoimmune Diseases, 1(4), 2009.","title":"Learning multiple layers of features from tiny images","authors":"Alex Krizhevsky","year":"2009","venue":"Handbook of Systemic Autoimmune Diseases","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Netzer et al., 2011] Yuval Netzer, Tao Wang, Adam Coates, A. Bissacco, Bo Wu, and A. Ng. Reading digits in natural images with unsupervised feature learning. Nips Work- shop on Deep Learning & Unsupervised Feature Learn- ing, 2011.","title":"Reading digits in natural images with unsupervised feature learning. Nips Workshop on Deep Learning & Unsupervised Feature Learning","authors":"Yuval Netzer, Tao Wang, Adam Coates, A Bissacco, Bo Wu, A Ng","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Darlow et al., 2018] Luke Nicholas Darlow, Elliot J. Crow- ley, Antreas Antoniou, and Amos J. Storkey. Cinic- 10 is not imagenet or cifar-10. ArXiv Preprint, arXiv:1810.03505, 2018.","title":"Cinic-10 is not imagenet or cifar-10","authors":"Darlow","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Lim et al., 2022] Wei Yang Bryan Lim, Jer Shyuan Ng, Ze- hui Xiong, Jiangming Jin, Yang Zhang, Dusit Niyato, Cyril Leung, and Chunyan Miao. Decentralized edge in- telligence: A dynamic resource allocation framework for hierarchical federated learning. IEEE Transactions on Par- allel and Distributed Systems, 33(3):536-550, 2022.","title":"Decentralized edge intelligence: A dynamic resource allocation framework for hierarchical federated learning","authors":"Wei Yang, Bryan Lim, Jer Shyuan Ng, Zehui Xiong, Jiangming Jin, Yang Zhang, Dusit Niyato, Cyril Leung, Chunyan Miao","year":"2022","venue":"IEEE Transactions on Parallel and Distributed Systems","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhang et al., 2021] Jingwen Zhang, Yuezhou Wu, and Rong Pan. Incentive mechanism for horizontal federated learn- ing based on reputation and reverse auction. In Proceed- ings of the Web Conference 2021, pages 947-956, New York, United States, April 2021. Association for Comput- ing Machinery.","title":"Incentive mechanism for horizontal federated learning based on reputation and reverse auction","authors":"Jingwen Zhang, Yuezhou Wu, Rong Pan","year":"2021","venue":"Proceedings of the Web Conference 2021","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Shi et al., 2022] Zhuan Shi, Lan Zhang, Zhenyu Yao, Lingjuan Lyu, Cen Chen, Li Wang, Junhao Wang, and Xiang-Yang Li. Fedfaim: A model performance-based fair incentive mechanism for federated learning. IEEE Trans- actions on Big Data, 1-13, 2022.","title":"Fedfaim: A model performance-based fair incentive mechanism for federated learning","authors":"Zhuan Shi, Lan Zhang, Zhenyu Yao, Lingjuan Lyu, Cen Chen, Li Wang, Junhao Wang, Xiang-Yang Li","year":"2022","venue":"IEEE Transactions on Big Data","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}