{"type":"ScholarlyArticle","title":"Game-theoretic Mechanisms for Eliciting Accurate Information","authors":[{"name":"Boi Faltings","slug":"boi-faltings","given":"Boi","family":"Faltings","affiliation":"Artificial Intelligence Laboratory (LIA) Ecole Polytechnique Fédérale de Lausanne (EPFL) 1015 Lausanne , Switzerland"}],"abstract":"Artificial Intelligence often relies on information obtained from others through crowdsourcing, federated learning, or data markets. It is crucial to ensure that this data is accurate. Over the past 20 years, a variety of incentive mechanisms have been developed that use game theory to reward the accuracy of contributed data. These techniques are applicable to many settings where AI uses contributed data.\r\n\r\nThis survey categorizes the different techniques and their properties, and shows their limits and tradeoffs. It identifies open issues and points to possible directions to address these.","doi":"10.24963/ijcai.2023/740","url":"https://ijcai.org/proceedings/2023/740","pdf":"https://ijcai.org/proceedings/2023/0740.pdf","markdown":"https://ijcai.org/proceedings/2023/740.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/740","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Survey Track","track":"Survey Track","number":740,"pages":{"first":6601,"last":6609},"keywords":["Survey: Game Theory and Economic Paradigms","Survey: Humans and AI","Survey: Machine Learning"],"videos":[],"references":[{"position":1,"raw":"[von Ahn and Dabbish, 2004] Luis von Ahn and Laura Dab- bish. Labeling images with a computer game. Proceedings of the SIGCHI Conference on Human Factors in Comput- ing Systems, pp. 319-326, 2004.","title":"Luis von Ahn and Laura Dabbish. Labeling images with a computer game","authors":"Von Ahn, Dabbish","year":"2004","venue":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Hu et al., 2017] Nan Hu, Paul A. Pavlou and Jennifer Zhang. On Self-Selection Bias in Online Product Reviews. Management Information Systems Quarterly, 2017.","title":"On Self-Selection Bias in Online Product Reviews","authors":"Nan Hu, Paul A Pavlou, Jennifer Zhang","year":"2017","venue":"Management Information Systems Quarterly","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Faltings et al., 2014] Boi Faltings, Jason J. Li, and Radu Ju- rca. Incentive Mechanisms for Community Sensing. IEEE Transaction on Computers, Vol. 63(1), pp. 115-128, 2014.","title":"Incentive Mechanisms for Community Sensing","authors":"Boi Faltings, Jason J Li, Radu Jurca","year":"2014","venue":"IEEE Transaction on Computers","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Jurca et al., 2007] Radu Jurca, Walter Binder and Boi Falt- ings. Reliable QoS Monitoring Based on Client Feedback. Proceedings of the 16th International World Wide Web Conference, pp. 1003-1012, 2007.","title":"Reliable QoS Monitoring Based on Client Feedback","authors":"Radu Jurca, Walter Binder, Boi Faltings","year":"2007","venue":"Proceedings of the 16th International World Wide Web Conference","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Geiting and Raftery, 2007] Tilmann Gneiting, and Adrian E. Raftery. Strictly proper scoring rules, prediction, and es- timation. Journal of the American Statistical Association, 102, pp. 359-378, 2007.","title":"Strictly proper scoring rules, prediction, and estimation","authors":"Tilmann Gneiting, Adrian E Raftery","year":"2007","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hanson, 2007] Robin Hanson, Logarithmic market scoring rules for modular combinatorial information aggregation. Journal of Prediction Markets 1(1), pp. 3-15, 2007.","title":"Logarithmic market scoring rules for modular combinatorial information aggregation","authors":"Robin Hanson, Hanson","year":"2007","venue":"Journal of Prediction Markets","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Miller, Resnick and Zeckhauser, 2005] Nolan Miller, Paul Resnick, and Richard Zeckhauser. Eliciting informative feedback: the peer prediction method. Management Sci- ence, 2005.","title":"Eliciting informative feedback: the peer prediction method","authors":"Resnick Miller, Zeckhauser ; Nolan Miller, Paul Resnick, Richard Zeckhauser","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kamble et al., 2016] Vijay Kamble, Nihar B. Shah, David Marn, Abhay Parekh, and Kannan Ramachandran. Truth Serums for Massively Crowdsourced Evaluation Tasks, arXiv:1507.07045, 2016.","title":"Truth Serums for Massively Crowdsourced Evaluation Tasks","authors":"Vijay Kamble, B Nihar, David Shah, Abhay Marn, Kannan Parekh, Ramachandran","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Jurca and Faltings, 2011] Radu Jurca and Boi Faltings. In- centives for answering hypothetical questions. 1st Work- shop on on Social Computing and User Generated Content ACM Conference on Electronic Commerce, 2011.","title":"Incentives for answering hypothetical questions","authors":"Radu Jurca, Boi Faltings","year":"2011","venue":"1st Workshop on on Social Computing and User Generated Content ACM Conference on Electronic Commerce","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Radanovic et al., 2016] Goran Radanovic, Radu Jurca, and Boi Faltings. Incentives for effort in crowdsourcing using the peer truth serum. ACM Transactions on Intelligent Sys- tems and Technology (TIST), Vol. 7, Issue 4, art. No 48, 2016.","title":"Incentives for effort in crowdsourcing using the peer truth serum","authors":"Goran Radanovic, Radu Jurca, Boi Faltings","year":"2016","venue":"ACM Transactions on Intelligent Systems and Technology (TIST)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Jurca and Faltings, 2005] Radu Jurca and Boi Faltings. En- forcing Truthful Strategies in Incentive Compatible Rep- utation Mechanisms. Internet and Network Economics, Springer LNCS 3828 , pp. 268 -277, 2005.","title":"Enforcing Truthful Strategies in Incentive Compatible Reputation Mechanisms","authors":"Radu Jurca, Boi Faltings","year":"2005","venue":"Internet and Network Economics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Waggoner and Chen, 2014] Bo Waggoner and Yiling Chen. Output Agreement Mechanisms and Common Knowl- edge. Second AAAI Conference on Human Computation and Crowdsourcing, pp. 220-226, 2014.","title":"Output Agreement Mechanisms and Common Knowledge","authors":"Chen ; Waggoner, Bo Waggoner, Yiling Chen","year":"2014","venue":"Second AAAI Conference on Human Computation and Crowdsourcing","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Dasgupta and Ghosh, 2013] Anirban Dasgupta, and Arpita Ghosh. Crowdsourced judgement elicitation with endoge- nous proficiency. Proceedings of the 22nd international conference on World Wide Web (WWW'13), pp. 319-330, 2013.","title":"Crowdsourced judgement elicitation with endogenous proficiency","authors":"Anirban Dasgupta, Arpita Ghosh","year":"2013","venue":"Proceedings of the 22nd international conference on World Wide Web (WWW'13)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Shnayder et al., 2016] Victor Shnayder, Arpit Agarwal, Rafael Frongillo, and David. C. Parkes. Informed Truth- fulness in Multi-Task Peer Prediction. Proceedings of the 2016 ACM Conference on Economics and Computation, pp. 179-196, 2016.","title":"Informed Truthfulness in Multi-Task Peer Prediction","authors":"Victor Shnayder, Arpit Agarwal, Rafael Frongillo, David C Parkes","year":"2016","venue":"Proceedings of the 2016 ACM Conference on Economics and Computation","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Zhang and Schoenebeck, 2023] Yichi Zhang and Grant Schoenebeck, Multitask Peer Prediction with Task- dependent Strategies. Proceedings of the ACM Web Con- ference 2023, pp. 3436-3446,2023.","title":"Yichi Zhang and Grant Schoenebeck, Multitask Peer Prediction with Taskdependent Strategies","authors":"Schoenebeck Zhang","year":"2023","venue":"Proceedings of the ACM Web Conference 2023","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Feng at al., 2022] Shi Feng, Fang-Yi Yu, and Yiling Chen, Peer Prediction for Learning Agents,Proceedings of the Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS), 2022.","title":"Peer Prediction for Learning Agents","authors":"Fang-Yi Shi Feng, Yiling Yu, Chen","year":"2022","venue":"Proceedings of the Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Prelec, 2004] Drazen Prelec. A Bayesian truth serum for subjective data. Science, 306(5695), pp. 462-466, 2004.","title":"A Bayesian truth serum for subjective data","authors":"Drazen Prelec","year":"2004","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kong and Schoenebeck, 2019] Yuqing Kong, and Grant Schoenebeck. An information theoretic framework for de- signing information elicitation mechanisms that reward truth-telling. ACM Transactions on Economics and Com- putation (TEAC) 7, pp. 1-33, 2019.","title":"An information theoretic framework for designing information elicitation mechanisms that reward truth-telling","authors":"Schoenebeck ; ] Yuqing Kong, Grant Kong, Schoenebeck","year":"2019","venue":"ACM Transactions on Economics and Computation (TEAC)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Luo and Liu, 2022] Tianyi Luo and Yang Liu. Machine Truth Serum. Journal of Machine Learning Research 112, pp. 789-815, 2022.","title":"Machine Truth Serum","authors":"Liu ; ] Tianyi Luo, Yang Luo, Liu","year":"2022","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"Thomas Cover. Elements of Information The- ory. John Wiley & Sons, 1991.","title":"Elements of Information Theory","authors":"Thomas Cover","year":"1991","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kong and Schoenebeck, 2018] Yuqing Kong and Grant Schoenebeck. Water from two rocks: Maximizing the mu- tual information. Proceedings of the 2018 ACM Confer- ence on Economics and Computation, pp. 177-194. 2018.","title":"Water from two rocks: Maximizing the mutual information","authors":"Schoenebeck Kong, Yuqing Kong, Grant Schoenebeck","year":"2018","venue":"Proceedings of the 2018 ACM Conference on Economics and Computation","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Chen et al., 2020] Yiling Chen, Yiheng Shen, and Shuran Zheng. Truthful Data Acquisition via Peer Prediction. Pro- ceedings of the Thirty-fourth Conference on Neural Infor- mation Processing Systems (NeurIPS), pp. 18194-18204, December 2020.","title":"Truthful Data Acquisition via Peer Prediction","authors":"Yiling Chen, Yiheng Shen, Shuran Zheng","year":"2020","venue":"Proceedings of the Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Radanovic and Faltings, 2015] Goran Radanovic and Boi Faltings. Incentive Schemes for Participatory Sensing. Proceedings of the 14th International Conference on Au- tonomous Agents and Multiagent Systems, pp. 1081-1089, 2015.","title":"Incentive Schemes for Participatory Sensing","authors":"Faltings Radanovic","year":"2015","venue":"Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2021] Juntao Wang, Yang Liu and Yiling Chen. Forecast Aggregation via Peer Prediction. Proceedings of The 9th AAAI Conference on Human Computation and Crowdsourcing, HCOMP), 2021.","title":"Forecast Aggregation via Peer Prediction","authors":"Juntao Wang, Yang Liu, Yiling Chen","year":"2021","venue":"Proceedings of The 9th AAAI Conference on Human Computation and Crowdsourcing","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Witkowski and Parkes, 2012] Jens Witkowski, and David C. Parkes. A robust Bayesian truth serum for small pop- ulations. Proceedings of the 26th AAAI Conference on Ar- tificial Intelligence (AAAI'12), pp. 1492-1498, 2012.","title":"A robust Bayesian truth serum for small populations","authors":"Parkes Witkowski, Jens Witkowski, David C Parkes","year":"2012","venue":"Proceedings of the 26th AAAI Conference on Artificial Intelligence (AAAI'12)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Radanovic and Faltings, 2013] Goran Radanovic and Boi Faltings. A robust Bayesian truth serum for non-binary signals. Proceedings of the 27th AAAI Conference on Arti- ficial Intelligence (AAAI'13), pp. 833-839, 2013.","title":"A robust Bayesian truth serum for non-binary signals","authors":"Faltings Radanovic, Goran Radanovic, Boi Faltings","year":"2013","venue":"Proceedings of the 27th AAAI Conference on Artificial Intelligence (AAAI'13)","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Faltings et al., 2017] Boi Faltings, Radu Jurca and Goran Radanovic. Peer Truth Serum: Incentives for Crowdsourc- ing Measurements and Opinions. CoRR abs/1704.05269, 2017.","title":"Peer Truth Serum: Incentives for Crowdsourcing Measurements and Opinions","authors":"Faltings","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"Yuqing Kong. Dominantly truthful multi-task peer prediction with a constant number of tasks. Proceed- ings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 2398-2411, 2020.","title":"Dominantly truthful multi-task peer prediction with a constant number of tasks","authors":"Yuqing Kong","year":"2020","venue":"Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Faltings and Radanovic, 2017] Boi Faltings and Goran Radanovic. Game Theory for Data Science: Eliciting truthful information. Morgan Kauffman, 2017.","title":"Game Theory for Data Science: Eliciting truthful information","authors":"Radanovic Faltings","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Koh and Liang, 2017] Pang Wei Koh and Percy Liang. Un- derstanding Black-box Predictions via Influence Func- tions. 34th International Conference on Machine Learn- ing, pp. 1885-1894, 2017.","title":"Understanding Black-box Predictions via Influence Functions","authors":"Pang Wei, Koh, Percy Liang","year":"2017","venue":"34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Richardson et al., 2020] Adam Richardson, Aris Filos- Ratsikas, and Boi Faltings. Budget-Bounded Incentives for Federated Learning. in Q. Yang L. Fan, Yu H (Eds.): Federated Learning, pp. 176-188, Springer LNCS 12500, 2020.","title":"Budget-Bounded Incentives for Federated Learning","authors":"Adam Richardson, Aris Filos-Ratsikas, Boi Faltings","year":"2020","venue":"Federated Learning","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Jia et al., 2019] Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gürel, Bo Li, Ce Zhang, Dawn Song, and Costas J. Spanos. Towards efficient data valuation based on the shapley value. Pro- ceedings of the 22nd International Conference on Artifi- cial Intelligence and Statistics (AISTATS), 2019.","title":"Towards efficient data valuation based on the shapley value","authors":"Jia","year":"2019","venue":"Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics (AISTATS)","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Loog et al., 2019] Marco Loog, Tom Viering and Alexander Mey. Minimizers of the empirical risk and risk monotonic- ity. Proceedings of the Thirty-third Conference on Neu- ral Information Processing Systems (NeurIPS), pp. 7476- 7485, 2019.","title":"Minimizers of the empirical risk and risk monotonicity","authors":"Loog","year":"2019","venue":"Proceedings of the Thirty-third Conference on Neural Information Processing Systems (NeurIPS)","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Gao et al., 2019] Xi Alice Gao, James R Wright, Kevin Leyton-Brown. Incentivizing evaluation with peer predic- tion and limited access to ground truth. Artificial Intelli- gence 275, pp. 618-638, 2019.","title":"Incentivizing evaluation with peer prediction and limited access to ground truth","authors":"Alice Xi, James R Gao, Kevin Wright, Leyton-Brown","year":"2019","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Goel and Faltings, 2019] Naman Goel and Boi Faltings. Deep Bayesian Trust : A Dominant and Fair Incen- tive Mechanism for Crowd. Proceeedings of the 33rd AAAI Conference on Artificial Intelligence, pp. 1996- 2003, 2019.","title":"Deep Bayesian Trust : A Dominant and Fair Incentive Mechanism for Crowd","authors":"Naman Goel, Boi Faltings","year":"2019","venue":"Proceeedings of the 33rd AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Kong and Schoenebeck, 2018] Yuqing Kong and Grant Schoenebeck. Eliciting expertise without verification. Pro- ceedings of the ACM Conference on Economics and Com- putation, pp. 195-212, 2018.","title":"Eliciting expertise without verification","authors":"Schoenebeck Kong, Yuqing Kong, Grant Schoenebeck","year":"2018","venue":"Proceedings of the ACM Conference on Economics and Computation","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Durlauf and Blume, 2010] Steven Durlauf and Lawrence Blume. Revelation Principle. Game Theory, pp. 312-318, Palgrave, 2010.","title":"Revelation Principle. Game Theory","authors":"Steven Durlauf, Lawrence Blume","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Goel et al., 2020] Naman Goel, Maxime Rutagarama and Boi Faltings. Tackling Peer to Peer Discrimination in the Sharing Economy. Proceedings of the 12th ACM Web Sci- ence Conference (WebSci 2020), pp. 355-361, 2020. (Best paper award).","title":"Tackling Peer to Peer Discrimination in the Sharing Economy","authors":"Naman Goel, Maxime Rutagarama, Boi Faltings","year":"2020","venue":"Proceedings of the 12th ACM Web Science Conference","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Goel et al., 2020] Naman Goel, Cyril van Schreven, Aris Filos-Ratsikas and Boi Faltings. Infochain: A Decentral- ized, Trustless and Transparent Oracle on Blockchain. Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI 2020), pp. 4604-4610, 2020.","title":"Infochain: A Decentralized, Trustless and Transparent Oracle on Blockchain","authors":"Naman Goel, Cyril Van Schreven, Aris Filos-Ratsikas, Boi Faltings","year":"2020","venue":"Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI 2020)","doi":null,"url":null,"internal_url":"/proceedings/2020/635"},{"position":40,"raw":"[Moti et al., 2020] Moin Hussain Moti, Dimitris Chatzopou- los, Pan Hui, Boi Faltings and Sujit Gujar. Orthos: A Trustworthy AI Framework for Data Acquisition. Pro- ceedings of the International Workshop on Engineering Multi-Agent Systems, Springer LNCS 12589, pp. 100-118, 2020.","title":"Orthos: A Trustworthy AI Framework for Data Acquisition","authors":"Moin Hussain Moti, Dimitris Chatzopoulos, Pan Hui, Boi Faltings, Sujit Gujar","year":"2020","venue":"Proceedings of the International Workshop on Engineering Multi-Agent Systems","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Freeman et al., 2017] Rupert Freeman, Sébastien Lahaie, David M. Pennock. Crowdsourced Outcome Determina- tion in Prediction Markets. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp. 523-529, 2017.","title":"Crowdsourced Outcome Determination in Prediction Markets","authors":"Rupert Freeman, Sébastien Lahaie, David M Pennock","year":"2017","venue":"Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Goel et al., 2020] Naman Goel, Aris Filos-Ratsikas and Boi Faltings. Peer-Prediction in the Presence of Outcome De- pendent Lying Incentives. Proceedings of the 29th Inter- national Joint Conference on Artificial Intelligence (IJCAI 2020), pp. 124-131, 2020.","title":"Peer-Prediction in the Presence of Outcome Dependent Lying Incentives","authors":"Naman Goel, Aris Filos-Ratsikas, Boi Faltings","year":"2020","venue":"Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI 2020)","doi":null,"url":null,"internal_url":"/proceedings/2020/18"},{"position":43,"raw":"[Cai et al., 2015] Yang Cai, Constantinos Daskalakis and Christos H. Papadimitriou. Optimum statistical estimation with strategic data sources. Proceedings of The 28th Con- ference on Learning Theory, pp. 280-296, 2015.","title":"Optimum statistical estimation with strategic data sources","authors":"Yang Cai, Constantinos Daskalakis, Christos H Papadimitriou","year":"2015","venue":"Proceedings of The 28th Conference on Learning Theory","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Chen et al., 2018] Yiling Chen, Chara Podimata, Ariel Pro- caccia and Nihar Shah. Strategyproof linear regression in high dimensions. Proceedings of the 2018 ACM Confer- ence on Economics and Computation, pp.9-26, 2018.","title":"Strategyproof linear regression in high dimensions","authors":"Yiling Chen, Chara Podimata, Ariel Procaccia, Nihar Shah","year":"2018","venue":"Proceedings of the 2018 ACM Conference on Economics and Computation","doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Yang et al., 2020] Qiang Yang, Yang Liu, Yong Cheng, Yan Kang, Tianjian Chen and Han Yu. Incentive Mechanism Design for Federated Learning. In: Federated Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, pp. 95-105, 2020.","title":"Incentive Mechanism Design for Federated Learning","authors":"Qiang Yang, Yang Liu, Yong Cheng, Yan Kang, Tianjian Chen, Han Yu","year":"2020","venue":"Federated Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"Karimireddy et al., 2022) ]","title":null,"authors":"Karimireddy","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":47,"raw":"Sai Praneeth Karimireddy, Wen- shuo Guo, Michael Jordan. Mechanisms that incentivize data sharing in federated learning. NeurIPS Workshop on Federated Learning, 2022..","title":"Mechanisms that incentivize data sharing in federated learning","authors":"Praneeth Sai, Wenshuo Karimireddy, Michael Guo, Jordan","year":"2022","venue":"NeurIPS Workshop on Federated Learning","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"Rokvic et al, 2022) ]","title":null,"authors":"Rokvic","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":49,"raw":"Ljubomir Rokvic, Panayiotis Danassis and Boi Faltings. Privacy-Preserving Data Filtering in Fed- erated Learning Using Influence Approximation NeurIPS Workshop on Federated Learning, 2022.","title":"Privacy-Preserving Data Filtering in Federated Learning Using Influence Approximation NeurIPS Workshop on Federated Learning","authors":"Ljubomir Rokvic, Panayiotis Danassis, Boi Faltings","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Goel and Faltings, 2019] Naman Goel and Boi Faltings. Personalized Peer Truth Serum for Eliciting Multi- Attribute Personal Data. Proceedings of the 35th Confer- ence on Uncertainty in Artificial Intelligence, PMLR 115, pp. 18-27, 2019.","title":"Personalized Peer Truth Serum for Eliciting Multi-Attribute Personal Data","authors":"Naman Goel, Boi Faltings","year":"2019","venue":"Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Faltings et al., 2014] Boi Faltings, Rady Jurca, Pearl Pu and Bao Duy Tran. Incentives to Counter Bias in Human Com- putation. Proceedings of the 2nd AAAI Conference on Hu- man Computation and Crowdsourcing, pp. 59-66, 2014.","title":"Incentives to Counter Bias in Human Computation","authors":"Boi Faltings, Rady Jurca, Pearl Pu, Bao Duy, Tran","year":"2014","venue":"Proceedings of the 2nd AAAI Conference on Human Computation and Crowdsourcing","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Gordon et al., 2020] Michael Gordon et al. Are Replication Rates the Same across Academic Fields? Community Forecasts from the DARPA SCORE Program. Royal So- ciety Open Science, 2020.","title":"Are Replication Rates the Same across Academic Fields? Community Forecasts from the DARPA SCORE Program","authors":"Michael Gordon","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Liu et al., 2020] Yang Liu, Juntao Wang and Yiling Chen. Surrogate Scoring Rules. ACM Tramsactions on Eco- nomics and Computation, pp. 1-37, October 2022.","title":"Surrogate Scoring Rules","authors":"Yang Liu, Juntao Wang, Yiling Chen","year":"2022","venue":"ACM Tramsactions on Economics and Computation","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}