{"type":"ScholarlyArticle","title":"A Survey of Federated Evaluation in Federated Learning","authors":[{"name":"Behnaz Soltani","slug":"behnaz-soltani","given":"Behnaz","family":"Soltani","affiliation":"School of Computing , Macquarie University , Sydney , Australia"},{"name":"Yipeng Zhou","slug":"yipeng-zhou","given":"Yipeng","family":"Zhou","affiliation":"School of Computing , Macquarie University , Sydney , Australia"},{"name":"Venus Haghighi","slug":"venus-haghighi","given":"Venus","family":"Haghighi","affiliation":"School of Computing , Macquarie University , Sydney , Australia"},{"name":"John C. S. Lui","slug":"john-c-s-lui","given":"John C.S.","family":"Lui","affiliation":"Department of Computer Science and Engineering , The Chinese University of Hong Kong , HKSAR"}],"abstract":"In traditional machine learning, it is trivial to conduct model evaluation since all data samples are managed centrally by a server. However, model evaluation becomes a challenging problem in federated learning (FL), which is called federated evaluation in this work. This is because clients do not expose their original data to preserve data privacy. Federated evaluation plays a vital role in client selection, incentive mechanism design, malicious attack detection, etc. In this paper, we provide the first comprehensive survey of existing federated evaluation methods. Moreover, we explore various applications of federated evaluation for enhancing FL performance and finally present future research directions by envisioning some challenges.","doi":"10.24963/ijcai.2023/758","url":"https://ijcai.org/proceedings/2023/758","pdf":"https://ijcai.org/proceedings/2023/0758.pdf","markdown":"https://ijcai.org/proceedings/2023/758.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/758","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Survey Track","track":"Survey Track","number":758,"pages":{"first":6769,"last":6777},"keywords":["Survey: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[Agrawal et al., 2003] Rakesh Agrawal, Alexandre Evfimievski, and Ramakrishnan Srikant. Information sharing across private databases. In Proceedings of the 2003 ACM SIGMOD interna- tional conference on Management of data, pages 86-97, 2003.","title":"Information sharing across private databases","authors":"Alexandre Rakesh Agrawal, Ramakrishnan Evfimievski, Srikant","year":"2003","venue":"Proceedings of the 2003 ACM SIGMOD international conference on Management of data","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Anand et al., 1993] Rangachari Anand, Kishan G Mehrotra, Chilukuri K Mohan, and Sanjay Ranka. An improved algo- rithm for neural network classification of imbalanced training sets. IEEE Transactions on Neural Networks, 4(6):962-969, 1993.","title":"An improved algorithm for neural network classification of imbalanced training sets","authors":"Rangachari Anand, G Kishan, Mehrotra, Sanjay Chilukuri K Mohan, Ranka","year":"1993","venue":"IEEE Transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Biswas et al., 2019] Chandan Biswas, Debasis Ganguly, Dwai- payan Roy, and Ujjwal Bhattacharya. Privacy preserving approx- imate k-means clustering. In Proceedings of the 28th ACM In- ternational Conference on Information and Knowledge Manage- ment, pages 1321-1330, 2019.","title":"Privacy preserving approximate k-means clustering","authors":"Chandan Biswas, Debasis Ganguly, Dwaipayan Roy, Ujjwal Bhattacharya","year":"2019","venue":"Proceedings of the 28th ACM International Conference on Information and Knowledge Management","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chai et al., 2020] Zheng Chai, Ahsan Ali, Syed Zawad, Stacey Truex, Ali Anwar, Nathalie Baracaldo, Yi Zhou, Heiko Ludwig, Feng Yan, and Yue Cheng. Tifl: A tier-based federated learn- ing system. In Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing, pages 125-136, 2020.","title":"Tifl: A tier-based federated learning system","authors":"Zheng Chai, Ahsan Ali, Syed Zawad, Stacey Truex, Ali Anwar, Nathalie Baracaldo, Yi Zhou, Heiko Ludwig, Feng Yan, Yue Cheng","year":"2020","venue":"Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Che et al., 2022] Chunjiang Che, Xiaoli Li, Chuan Chen, Xi- aoyu He, and Zibin Zheng. A decentralized federated learning framework via committee mechanism with convergence guaran- tee. IEEE Transactions on Parallel and Distributed Systems, 33(12):4783-4800, 2022.","title":"A decentralized federated learning framework via committee mechanism with convergence guarantee","authors":"Chunjiang Che, Xiaoli Li, Chuan Chen, Xiaoyu He, Zibin Zheng","year":"2022","venue":"IEEE Transactions on Parallel and Distributed Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2013] Wei Chen, Yajun Wang, and Yang Yuan. Com- binatorial multi-armed bandit: General framework and applica- tions. In International conference on machine learning, pages 151-159. PMLR, 2013.","title":"Combinatorial multi-armed bandit: General framework and applications","authors":"Wei Chen, Yajun Wang, Yang Yuan","year":"2013","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2020a] Yiqiang Chen, Xiaodong Yang, Xin Qin, Han Yu, Biao Chen, and Zhiqi Shen. Focus: Dealing with label quality disparity in federated learning. arXiv preprint arXiv:2001.11359, 2020.","title":"Focus: Dealing with label quality disparity in federated learning","authors":"Chen","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2020b] Yujing Chen, Yue Ning, Martin Slawski, and Huzefa Rangwala. Asynchronous online federated learning for edge devices with non-iid data. In 2020 IEEE International Con- ference on Big Data (Big Data), pages 15-24. IEEE, 2020.","title":"Asynchronous online federated learning for edge devices with non-iid data","authors":"Chen","year":"2020","venue":"IEEE International Conference on Big Data (Big Data)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Deng et al., 2021] Yongheng Deng, Feng Lyu, Ju Ren, Yi-Chao Chen, Peng Yang, Yuezhi Zhou, and Yaoxue Zhang. Fair: Quality-aware federated learning with precise user incentive and model aggregation. In IEEE INFOCOM 2021-IEEE Conference on Computer Communications, pages 1-10. IEEE, 2021.","title":"Fair: Quality-aware federated learning with precise user incentive and model aggregation","authors":"Yongheng Deng, Feng Lyu, Ju Ren, Yi-Chao Chen, Peng Yang, Yuezhi Zhou, Yaoxue Zhang","year":"2021","venue":"IEEE INFOCOM 2021-IEEE Conference on Computer Communications","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fan et al., 2022] Zhenan Fan, Huang Fang, Zirui Zhou, Jian Pei, Michael P Friedlander, Changxin Liu, and Yong Zhang. Improv- ing fairness for data valuation in horizontal federated learning. In 2022 IEEE 38th International Conference on Data Engineering (ICDE), pages 2440-2453. IEEE, 2022.","title":"Improving fairness for data valuation in horizontal federated learning","authors":"Zhenan Fan, Huang Fang, Zirui Zhou, Jian Pei, Changxin Michael P Friedlander, Yong Liu, Zhang","year":"2022","venue":"IEEE 38th International Conference on Data Engineering (ICDE)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Feng et al., 2019] Shaohan Feng, Dusit Niyato, Ping Wang, Dong In Kim, and Ying-Chang Liang. Joint service pricing and cooperative relay communication for federated learning. In 2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), pages 815-820. IEEE, 2019.","title":"Joint service pricing and cooperative relay communication for federated learning","authors":"Feng","year":"2019","venue":"2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Fung et al., 2018] Clement Fung, Chris JM Yoon, and Ivan Beschastnikh. Mitigating sybils in federated learning poisoning. arXiv preprint arXiv:1808.04866, 2018.","title":"Mitigating sybils in federated learning poisoning","authors":"Clement Fung, Chris Jm Yoon, Ivan Beschastnikh","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gao et al., 2021] Liang Gao, Li Li, Yingwen Chen, Wenli Zheng, ChengZhong Xu, and Ming Xu. Fifl: A fair incentive mecha- nism for federated learning. In 50th International Conference on Parallel Processing, pages 1-10, 2021.","title":"Fifl: A fair incentive mechanism for federated learning","authors":"Liang Gao, Li Li, Yingwen Chen, Wenli Zheng, Chengzhong Xu, Ming Xu","year":"2021","venue":"50th International Conference on Parallel Processing","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Ghorbani and Zou, 2019] Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In In- ternational Conference on Machine Learning, pages 2242-2251. PMLR, 2019.","title":"Data shapley: Equitable valuation of data for machine learning","authors":"Zou Ghorbani, Amirata Ghorbani, James Zou","year":"2019","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Ha et al., 2016] David Ha, Andrew Dai, and Quoc V Le. Hyper- networks. arXiv preprint arXiv:1609.09106, 2016.","title":"Hypernetworks","authors":"David Ha, Andrew Dai, Quoc V Le","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hu et al., 2022a] Miao Hu, Di Wu, Yipeng Zhou, Xu Chen, and Min Chen. Incentive-aware autonomous client participation in federated learning. IEEE Transactions on Parallel and Dis- tributed Systems, 33(10):2612-2627, 2022.","title":"Incentive-aware autonomous client participation in federated learning","authors":"Hu","year":"2022","venue":"IEEE Transactions on Parallel and Distributed Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hu et al., 2022b] Miao Hu, Wenzhuo Yang, Zhenxiao Luo, Xuezheng Liu, Yipeng Zhou, Xu Chen, and Di Wu. Autofl: A bayesian game approach for autonomous client participation in federated edge learning. IEEE Transactions on Mobile Comput- ing, 2022.","title":"Autofl: A bayesian game approach for autonomous client participation in federated edge learning","authors":"Hu","year":"2022","venue":"IEEE Transactions on Mobile Computing","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Huang et al., 2021] Yutao Huang, Lingyang Chu, Zirui Zhou, Lan- jun Wang, Jiangchuan Liu, Jian Pei, and Yong Zhang. Personal- ized cross-silo federated learning on non-iid data. In AAAI, pages 7865-7873, 2021.","title":"Personalized cross-silo federated learning on non-iid data","authors":"Yutao Huang, Lingyang Chu, Zirui Zhou, Lanjun Wang, Jiangchuan Liu, Jian Pei, Yong Zhang","year":"2021","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Jia et al., 2019] Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gürel, Bo Li, Ce Zhang, Dawn Song, and Costas J Spanos. Towards efficient data val- uation based on the shapley value. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 1167- 1176. PMLR, 2019.","title":"Towards efficient data valuation based on the shapley value","authors":"Ruoxi Jia, David Dao, Boxin Wang, Frances, Ann Hubis, Nick Hynes, Nezihe Merve Gürel, Bo Li, Ce Zhang, Dawn Song, Costas, Spanos","year":"2019","venue":"The 22nd International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Kang et al., 2019] Jiawen Kang, Zehui Xiong, Dusit Niyato, Shengli Xie, and Junshan Zhang. Incentive mechanism for re- liable federated learning: A joint optimization approach to com- bining reputation and contract theory. IEEE Internet of Things Journal, 6(6):10700-10714, 2019.","title":"Incentive mechanism for reliable federated learning: A joint optimization approach to combining reputation and contract theory","authors":"Jiawen Kang, Zehui Xiong, Dusit Niyato, Shengli Xie, Junshan Zhang","year":"2019","venue":"IEEE Internet of Things Journal","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kang et al., 2020a] Jiawen Kang, Zehui Xiong, Chunxiao Jiang, Yi Liu, Song Guo, Yang Zhang, Dusit Niyato, Cyril Leung, and Chunyan Miao. Scalable and communication-efficient decentral- ized federated edge learning with multi-blockchain framework. In International Conference on Blockchain and Trustworthy Sys- tems, pages 152-165. Springer, 2020.","title":"Scalable and communication-efficient decentralized federated edge learning with multi-blockchain framework","authors":"Kang","year":"2020","venue":"International Conference on Blockchain and Trustworthy Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Kang et al., 2020b] Jiawen Kang, Zehui Xiong, Dusit Niyato, Yuze Zou, Yang Zhang, and Mohsen Guizani. Reliable federated learning for mobile networks. IEEE Wireless Communications, 27(2):72-80, 2020.","title":"Reliable federated learning for mobile networks","authors":"Kang","year":"2020","venue":"IEEE Wireless Communications","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Kim et al., 2019] Hyesung Kim, Jihong Park, Mehdi Bennis, and Seong-Lyun Kim. Blockchained on-device federated learning. IEEE Communications Letters, 24(6):1279-1283, 2019.","title":"Blockchained on-device federated learning","authors":"Hyesung Kim, Jihong Park, Mehdi Bennis, Seong-Lyun Kim","year":"2019","venue":"IEEE Communications Letters","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Lai et al., 2021] Fan Lai, Xiangfeng Zhu, Harsha V Madhyastha, and Mosharaf Chowdhury. Oort: Efficient federated learning via guided participant selection. In 15th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 21), pages 19-35, 2021.","title":"Oort: Efficient federated learning via guided participant selection","authors":"Fan Lai, Xiangfeng Zhu, Mosharaf Harsha V Madhyastha, Chowdhury","year":"2021","venue":"15th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 21)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Li et al., 2020] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated opti- mization in heterogeneous networks. Proceedings of Machine Learning and Systems, 2:429-450, 2020.","title":"Federated optimization in heterogeneous networks","authors":"Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, Virginia Smith","year":"2020","venue":"Proceedings of Machine Learning and Systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Li et al., 2021a] Anran Li, Lan Zhang, Juntao Tan, Yaxuan Qin, Junhao Wang, and Xiang-Yang Li. Sample-level data selection for federated learning. In IEEE INFOCOM 2021-IEEE Confer- ence on Computer Communications, pages 1-10. IEEE, 2021.","title":"Sample-level data selection for federated learning","authors":"Li","year":"2021","venue":"IEEE INFOCOM 2021-IEEE Conference on Computer Communications","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Li et al., 2021b] Li Li, Moming Duan, Duo Liu, Yu Zhang, Ao Ren, Xianzhang Chen, Yujuan Tan, and Chengliang Wang. Fedsae: A novel self-adaptive federated learning framework in heterogeneous systems. In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1-10. IEEE, 2021.","title":"Fedsae: A novel self-adaptive federated learning framework in heterogeneous systems","authors":"Li","year":"2021","venue":"2021 International Joint Conference on Neural Networks (IJCNN)","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Li et al., 2021c] Qinbin Li, Bingsheng He, and Dawn Song. Model-contrastive federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pages 10713-10722, 2021.","title":"Bingsheng He, and Dawn Song. Model-contrastive federated learning","authors":"Li","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Li et al., 2022] Shuangtong Li, Tianyi Zhou, Xinmei Tian, and Dacheng Tao. Learning to collaborate in decentralized learn- ing of personalized models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9766-9775, 2022.","title":"Learning to collaborate in decentralized learning of personalized models","authors":"Li","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Liu et al., 2011] Yining Liu, Keqiu Li, Yingwei Jin, Yong Zhang, and Wenyu Qu. A novel reputation computation model based on subjective logic for mobile ad hoc networks. Future Generation Computer Systems, 27(5):547-554, 2011.","title":"A novel reputation computation model based on subjective logic for mobile ad hoc networks","authors":"Yining Liu, Keqiu Li, Yingwei Jin, Yong Zhang, Wenyu Qu","year":"2011","venue":"Future Generation Computer Systems","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Ma et al., 2021a] Jiahua Ma, Xinghua Sun, Wenchao Xia, Xijun Wang, Xiang Chen, and Hongbo Zhu. Client selection based on label quantity information for federated learning. In 2021 IEEE 32nd Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), pages 1-6. IEEE, 2021.","title":"Client selection based on label quantity information for federated learning","authors":"Ma","year":"2021","venue":"IEEE 32nd Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC)","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Ma et al., 2021b] Shuaicheng Ma, Yang Cao, and Li Xiong. Trans- parent contribution evaluation for secure federated learning on blockchain. In 2021 IEEE 37th International Conference on Data Engineering Workshops (ICDEW), pages 88-91. IEEE, 2021.","title":"Transparent contribution evaluation for secure federated learning on blockchain","authors":"Ma","year":"2021","venue":"IEEE 37th International Conference on Data Engineering Workshops (ICDEW)","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Ma et al., 2022] Xiaosong Ma, Jie Zhang, Song Guo, and Wen- chao Xu. Layer-wised model aggregation for personalized fed- erated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10092-10101, 2022.","title":"Layer-wised model aggregation for personalized federated learning","authors":"Xiaosong Ma, Jie Zhang, Song Guo, Wenchao Xu","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Marfoq et al., 2022] Othmane Marfoq, Giovanni Neglia, Richard Vidal, and Laetitia Kameni. Personalized federated learning through local memorization. In International Conference on Ma- chine Learning, pages 15070-15092. PMLR, 2022.","title":"Personalized federated learning through local memorization","authors":"Othmane Marfoq, Giovanni Neglia, Richard Vidal, Laetitia Kameni","year":"2022","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[McMahan et al., 2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decen- tralized data. In Artificial intelligence and statistics, pages 1273- 1282. PMLR, 2017.","title":"Communication-efficient learning of deep networks from decentralized data","authors":"Brendan Mcmahan, Eider Moore, Daniel Ramage, Seth Hampson, Blaise Aguera Y Arcas","year":"2017","venue":"Artificial intelligence and statistics","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Qu et al., 2020] Youyang Qu, Longxiang Gao, Tom H Luan, Yong Xiang, Shui Yu, Bai Li, and Gavin Zheng. Decentralized privacy using blockchain-enabled federated learning in fog computing. IEEE Internet of Things Journal, 7(6):5171-5183, 2020.","title":"Decentralized privacy using blockchain-enabled federated learning in fog computing","authors":"Qu","year":"2020","venue":"IEEE Internet of Things Journal","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Sannara et al., 2021] EK Sannara, Franc ¸ois Portet, Philippe La- landa, and VEGA German. A federated learning aggregation algorithm for pervasive computing: Evaluation and comparison. In 2021 IEEE International Conference on Pervasive Computing and Communications (PerCom), pages 1-10. IEEE, 2021.","title":"VEGA German. A federated learning aggregation algorithm for pervasive computing: Evaluation and comparison","authors":"Sannara","year":"2021","venue":"2021 IEEE International Conference on Pervasive Computing and Communications (PerCom)","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Shayan et al., 2018] Muhammad Shayan, Clement Fung, Chris JM Yoon, and Ivan Beschastnikh. Biscotti: A ledger for pri- vate and secure peer-to-peer machine learning. arXiv preprint arXiv:1811.09904, 2018.","title":"Biscotti: A ledger for private and secure peer-to-peer machine learning","authors":"Shayan","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Soltani et al., 2022] Behnaz Soltani, Venus Haghighi, Adnan Mahmood, Quan Z Sheng, and Lina Yao. A survey on partici- pant selection for federated learning in mobile networks. In Pro- ceedings of the 17th ACM Workshop on Mobility in the Evolving Internet Architecture, pages 19-24, 2022.","title":"A survey on participant selection for federated learning in mobile networks","authors":"Behnaz Soltani, Venus Haghighi, Adnan Mahmood, Lina Quan Z Sheng, Yao","year":"2022","venue":"Proceedings of the 17th ACM Workshop on Mobility in the Evolving Internet Architecture","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Su et al., 2022] Dongyuan Su, Yipeng Zhou, and Laizhong Cui. Boost decentralized federated learning in vehicular networks by diversifying data sources. In 2022 IEEE 30th International Con- ference on Network Protocols (ICNP), pages 1-11. IEEE, 2022.","title":"Boost decentralized federated learning in vehicular networks by diversifying data sources","authors":"Su","year":"2022","venue":"IEEE 30th International Conference on Network Protocols (ICNP)","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Tan et al., 2022] Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang. Towards personalized federated learning. IEEE Transactions on Neural Networks and Learning Systems, 2022.","title":"Towards personalized federated learning","authors":"Alysa Ziying Tan, Han Yu, Lizhen Cui, Qiang Yang","year":"2022","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Uddin et al., 2020] Md Palash Uddin, Yong Xiang, Xuequan Lu, John Yearwood, and Longxiang Gao. Mutual information driven federated learning. IEEE Transactions on Parallel and Dis- tributed Systems, 32(7):1526-1538, 2020.","title":"Mutual information driven federated learning","authors":"Md Palash Uddin, Yong Xiang, Xuequan Lu, John Yearwood, Longxiang Gao","year":"2020","venue":"IEEE Transactions on Parallel and Distributed Systems","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Wang et al., 2019] Guan Wang, Charlie Xiaoqian Dang, and Ziye Zhou. Measure contribution of participants in federated learning. In 2019 IEEE International Conference on Big Data (Big Data), pages 2597-2604. IEEE, 2019.","title":"Measure contribution of participants in federated learning","authors":"Guan Wang, Charlie Xiaoqian Dang, Ziye Zhou","year":"2019","venue":"2019 IEEE International Conference on Big Data (Big Data)","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wang et al., 2020] Tianhao Wang, Johannes Rausch, Ce Zhang, Ruoxi Jia, and Dawn Song. A principled approach to data valu- ation for federated learning. In Federated Learning, pages 153- 167. Springer, 2020.","title":"A principled approach to data valuation for federated learning","authors":"Tianhao Wang, Johannes Rausch, Ce Zhang, Ruoxi Jia, Dawn Song","year":"2020","venue":"Federated Learning","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Wu and Wang, 2022] Hongda Wu and Ping Wang. Node selection toward faster convergence for federated learning on non-iid data. IEEE Transactions on Network Science and Engineering, 2022.","title":"Node selection toward faster convergence for federated learning on non-iid data","authors":"Wang ; ] Hongda Wu, Ping Wu, Wang","year":"2022","venue":"IEEE Transactions on Network Science and Engineering","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Yang et al., 2021] Miao Yang, Ximin Wang, Hongbin Zhu, Haifeng Wang, and Hua Qian. Federated learning with class imbalance reduction. In 2021 29th European Signal Processing Conference (EUSIPCO), pages 2174-2178. IEEE, 2021.","title":"Federated learning with class imbalance reduction","authors":"Miao Yang, Ximin Wang, Hongbin Zhu, Haifeng Wang, Hua Qian","year":"2021","venue":"2021 29th European Signal Processing Conference (EUSIPCO)","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Zhang et al., 2020] Michael Zhang, Karan Sapra, Sanja Fidler, Serena Yeung, and Jose M Alvarez. Personalized federated learning with first order model optimization. arXiv preprint arXiv:2012.08565, 2020.","title":"Personalized federated learning with first order model optimization","authors":"Michael Zhang, Karan Sapra, Sanja Fidler, Serena Yeung, Jose M Alvarez","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Zhang et al., 2021a] Jingwen Zhang, Yuezhou Wu, and Rong Pan. Incentive mechanism for horizontal federated learning based on reputation and reverse auction. In Proceedings of the Web Con- ference 2021, pages 947-956, 2021.","title":"Incentive mechanism for horizontal federated learning based on reputation and reverse auction","authors":"Zhang","year":"2021","venue":"Proceedings of the Web Conference 2021","doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Zhang et al., 2021b] Wenyu Zhang, Xiumin Wang, Pan Zhou, Weiwei Wu, and Xinglin Zhang. Client selection for federated learning with non-iid data in mobile edge computing. IEEE Ac- cess, 9:24462-24474, 2021.","title":"Client selection for federated learning with non-iid data in mobile edge computing","authors":"Zhang","year":"2021","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Zhang et al., 2021c] Zhebin Zhang, Dajie Dong, Yuhang Ma, Yi- long Ying, Dawei Jiang, Ke Chen, Lidan Shou, and Gang Chen. Refiner: A reliable incentive-driven federated learning system powered by blockchain. Proceedings of the VLDB Endowment, 14(12):2659-2662, 2021.","title":"Refiner: A reliable incentive-driven federated learning system powered by blockchain","authors":"Zhang","year":"2021","venue":"Proceedings of the VLDB Endowment","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Zhao et al., 2018] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. arXiv preprint arXiv:1806.00582, 2018.","title":"Federated learning with non-iid data","authors":"Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, Vikas Chandra","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Zhao et al., 2022] Jianxin Zhao, Xinyu Chang, Yanhao Feng, Chi Harold Liu, and Ningbo Liu. Participant selection for fed- erated learning with heterogeneous data in intelligent transport system. IEEE Transactions on Intelligent Transportation Sys- tems, 2022.","title":"Participant selection for federated learning with heterogeneous data in intelligent transport system","authors":"Jianxin Zhao, Xinyu Chang, Yanhao Feng, Chi Harold Liu, Ningbo Liu","year":"2022","venue":"IEEE Transactions on Intelligent Transportation Systems","doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Zhou et al., 2019] Zhi Zhou, Xu Chen, En Li, Liekang Zeng, Ke Luo, and Junshan Zhang. Edge intelligence: Paving the last mile of artificial intelligence with edge computing. Proceedings of the IEEE, 107(8):1738-1762, 2019.","title":"Edge intelligence: Paving the last mile of artificial intelligence with edge computing","authors":"Zhi Zhou, Xu Chen, En Li, Liekang Zeng, Ke Luo, Junshan Zhang","year":"2019","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Zhou et al., 2023] Yipeng Zhou, Xuezheng Liu, Yao Fu, Di Wu, Jessie Hui Wang, and Shui Yu. Optimizing the numbers of queries and replies in convex federated learning with differential privacy. IEEE Transactions on Dependable and Secure Comput- ing, 2023.","title":"Optimizing the numbers of queries and replies in convex federated learning with differential privacy","authors":"Yipeng Zhou, Xuezheng Liu, Yao Fu, Di Wu, Jessie Hui Wang, Shui Yu","year":"2023","venue":"IEEE Transactions on Dependable and Secure Computing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}