{"type":"ScholarlyArticle","title":"Peer-Prediction in the Presence of Outcome Dependent Lying Incentives","authors":[{"name":"Naman Goel","slug":"naman-goel","given":"Naman","family":"Goel","affiliation":"Swiss Federal Institute of Technology , Lausanne ("},{"name":"Aris Filos-Ratsikas","slug":"aris-filos-ratsikas","given":"Aris","family":"Filos-Ratsikas","affiliation":"University of Liverpool , UK"},{"name":"Boi Faltings","slug":"boi-faltings","given":"Boi","family":"Faltings","affiliation":"Swiss Federal Institute of Technology , Lausanne (; EPFL)"}],"abstract":"We derive conditions under which a peer-consistency mechanism can be used to elicit truthful data from non-trusted rational agents when an aggregate statistic of the collected data affects the amount of their incentives to lie. Furthermore, we discuss the relative saving that can be achieved by the mechanism, compared to the rational outcome, if no such mechanism was implemented. Our work is motivated by distributed platforms, where decentralized data oracles collect information about real-world events, based on the aggregate information provided by often self-interested participants. We compare our theoretical observations with numerical simulations on two public real datasets.","doi":"10.24963/ijcai.2020/18","url":"https://ijcai.org/proceedings/2020/18","pdf":"https://ijcai.org/proceedings/2020/0018.pdf","markdown":"https://ijcai.org/proceedings/2020/18.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/18","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Agent-based and Multi-agent Systems","track":"Main track","number":18,"pages":{"first":124,"last":131},"keywords":["Agent-based and Multi-agent Systems: Trust and Reputation","Humans and AI: Human Computation and Crowdsourcing","Trust, Fairness, Bias: General","Agent-based and Multi-agent Systems: Economic Paradigms, Auctions and Market-Based Systems"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25224/file_25224.m3u8","length_sec":253,"provider":"hls"}],"references":[{"position":1,"raw":"[Agarwal et al., 2017] Arpit Agarwal, Debmalya Mandal, David C. Parkes, and Nisarg Shah. Peer prediction with heterogeneous users. In Proceedings of the 18th ACM Conference on Economics and Computation (EC), 2017.","title":"Peer prediction with heterogeneous users","authors":"Arpit Agarwal, Debmalya Mandal, David C Parkes, Nisarg Shah","year":"2017","venue":"Proceedings of the 18th ACM Conference on Economics and Computation (EC)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chakraborty and Das, 2016] Mithun Chakraborty and San- may Das. Trading on a rigged game: Outcome manipula- tion in prediction markets. In IJCAI, 2016.","title":"Mithun Chakraborty and Sanmay Das. Trading on a rigged game: Outcome manipulation in prediction markets","authors":"Das Chakraborty","year":"2016","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chakraborty et al., 2013] Mithun Chakraborty, Sanmay Das, Allen Lavoie, Malik Magdon-Ismail, and Yonatan Naamad. Instructor rating markets. In Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence. AAAI Press, 2013.","title":"Instructor rating markets","authors":"Mithun Chakraborty, Sanmay Das, Allen Lavoie, Malik Magdon-Ismail, Yonatan Naamad","year":"2013","venue":"Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2011] Yiling Chen, Xi Alice Gao, Rick Gold- stein, and Ian A Kash. Market manipulation with outside incentives. In AAAI, 2011.","title":"Market manipulation with outside incentives","authors":"Yiling Chen, Alice Xi, Rick Gao, Ian A Goldstein, Kash","year":"2011","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dasgupta and Ghosh, 2013] Anirban Dasgupta and Arpita Ghosh. Crowdsourced judgement elicitation with endoge- nous proficiency. In Proceedings of the 22nd international conference on World Wide Web. ACM, 2013.","title":"Crowdsourced judgement elicitation with endogenous proficiency","authors":"Ghosh Dasgupta, Anirban Dasgupta, Arpita Ghosh","year":"2013","venue":"Proceedings of the 22nd international conference on World Wide Web","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Faltings and Radanovic, 2017] Boi Faltings and Goran Radanovic. Game theory for data science: Eliciting truthful information. Synthesis Lectures on Artificial Intelligence and Machine Learning, 11(2):1-151, 2017.","title":"Boi Faltings and Goran Radanovic. Game theory for data science: Eliciting truthful information","authors":"Radanovic Faltings","year":"2017","venue":"Synthesis Lectures on Artificial Intelligence and Machine Learning","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Freeman et al., 2017] Rupert Freeman, Sébastien Lahaie, and David M Pennock. Crowdsourced outcome determi- nation in prediction markets. In AAAI, 2017.","title":"Crowdsourced outcome determination in prediction markets","authors":"Rupert Freeman, Sébastien Lahaie, David M Pennock","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gao et al., 2016] Alice Gao, James R Wright, and Kevin Leyton-Brown. Incentivizing evaluation via limited ac- cess to ground truth: Peer-prediction makes things worse. In 2nd Workshop on Algorithmic Game Theory and Data Science at EC 2016., 2016.","title":"Incentivizing evaluation via limited access to ground truth: Peer-prediction makes things worse","authors":"Alice Gao, James R Wright, Kevin Leyton-Brown","year":"2016","venue":"2nd Workshop on Algorithmic Game Theory and Data Science at EC","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Goel and Faltings, 2019a] Naman Goel and Boi Faltings. Deep bayesian trust: A dominant and fair incentive mech- anism for crowd. Proceedings of the AAAI Conference on Artificial Intelligence, 2019.","title":"Deep bayesian trust: A dominant and fair incentive mechanism for crowd","authors":"Faltings Goel, Naman Goel, Boi Faltings","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goel and Faltings, 2019b] Naman Goel and Boi Faltings. Personalized peer truth serum for eliciting multi-attribute personal data. In 35th Conference on Uncertainty in Arti- ficial Intelligence, Tel Aviv (UAI 2019). AUAI, 2019.","title":"Personalized peer truth serum for eliciting multi-attribute personal data","authors":"Faltings Goel, Naman Goel, Boi Faltings","year":"2019","venue":"35th Conference on Uncertainty in Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Goel et al., 2020] Naman Goel, Cyril van Schreven, Aris Filos-Ratsikas, and Boi Faltings. Infochain: A decentral- ized, trustless and transparent oracle on blockchain. Pro- ceedings of the 29th International Joint Conference on Ar- tificial Intelligence (IJCAI), 2020.","title":"Infochain: A decentralized, trustless and transparent oracle on blockchain","authors":"Naman Goel, Cyril Van Schreven, Aris Filos-Ratsikas, Boi Faltings","year":"2020","venue":"Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2020/635"},{"position":12,"raw":"[Jurca and Faltings, 2005] Radu Jurca and Boi Faltings. En- forcing truthful strategies in incentive compatible reputa- tion mechanisms. In International Workshop on Internet and Network Economics, pages 268-277. Springer, 2005.","title":"Enforcing truthful strategies in incentive compatible reputation mechanisms","authors":"Radu Jurca, Boi Faltings","year":"2005","venue":"International Workshop on Internet and Network Economics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Jurca and Faltings, 2009] Radu Jurca and Boi Faltings. Mechanisms for making crowds truthful. Journal of Ar- tificial Intelligence Research, 34:209-253, 2009.","title":"Mechanisms for making crowds truthful","authors":"Radu Jurca, Boi Faltings","year":"2009","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Jurca and Faltings, 2011] Radu Jurca and Boi Faltings. In- centives for answering hypothetical questions. In The 1st Workshop on Social Computing and User Generated Con- tent, EC, 2011.","title":"Incentives for answering hypothetical questions","authors":"Radu Jurca, Boi Faltings","year":"2011","venue":"The 1st Workshop on Social Computing and User Generated Content","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jurca et al., 2007] Radu Jurca, Boi Faltings, and Walter Binder. Reliable qos monitoring based on client feed- back. In Proceedings of the 16th international conference on World Wide Web, pages 1003-1012. ACM, 2007.","title":"Reliable qos monitoring based on client feedback","authors":"Radu Jurca, Boi Faltings, Walter Binder","year":"2007","venue":"Proceedings of the 16th international conference on World Wide Web","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kamble et al., 2015] Vijay Kamble, Nihar Shah, David Marn, Abhay Parekh, and Kannan Ramachandran. Truth serums for massively crowdsourced evaluation tasks. arXiv preprint arXiv:1507.07045, 2015.","title":"Truth serums for massively crowdsourced evaluation tasks","authors":"Vijay Kamble, Nihar Shah, David Marn, Abhay Parekh, Kannan Ramachandran","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kong and Schoenebeck, 2018] Yuqing Kong and Grant Schoenebeck. Equilibrium selection in information elic- itation without verification via information monotonicity. Proceedings of the 9th Innovations in Theoretical Com- puter Science Conference (ITCS), 2018.","title":"Equilibrium selection in information elicitation without verification via information monotonicity","authors":"Schoenebeck Kong, Yuqing Kong, Grant Schoenebeck","year":"2018","venue":"Proceedings of the 9th Innovations in Theoretical Computer Science Conference (ITCS)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu and Chen, 2017] Yang Liu and Yiling Chen. Machine- learning aided peer prediction. In Proceedings of the 2017 ACM Conference on Economics and Computation, pages 63-80. ACM, 2017.","title":"Machinelearning aided peer prediction","authors":"Chen ; Liu, Yang Liu, Yiling Chen","year":"2017","venue":"Proceedings of the 2017 ACM Conference on Economics and Computation","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Miller et al., 2005] Nolan Miller, Paul Resnick, and Richard Zeckhauser. Eliciting informative feedback: The peer- prediction method. Management Science, 2005.","title":"Eliciting informative feedback: The peerprediction method","authors":"Nolan Miller, Paul Resnick, Richard Zeckhauser","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Prelec, 2004] Dražen Prelec. A bayesian truth serum for subjective data. science, 306(5695):462-466, 2004.","title":"Dražen Prelec. A bayesian truth serum for subjective data","authors":"Prelec","year":"2004","venue":"science","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Radanovic and Faltings, 2013] Goran Radanovic and Boi Faltings. A robust bayesian truth serum for non-binary signals. In Proceedings of the 27th AAAI Conference on Artificial Intelligence (AAAI\" 13), 2013.","title":"A robust bayesian truth serum for non-binary signals","authors":"Faltings Radanovic, Goran Radanovic, Boi Faltings","year":"2013","venue":"Proceedings of the 27th AAAI Conference on Artificial Intelligence (AAAI\" 13)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Radanovic and Faltings, 2015] Goran Radanovic and Boi Faltings. Incentive schemes for participatory sensing. In Proceedings of the 2015 International Conference on Au- tonomous Agents and Multiagent Systems, pages 1081- 1089. International Foundation for Autonomous Agents and Multiagent Systems, 2015.","title":"Incentive schemes for participatory sensing","authors":"Faltings Radanovic, Goran Radanovic, Boi Faltings","year":"2015","venue":"Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Radanovic et al., 2016] Goran Radanovic, Boi Faltings, and Radu Jurca. Incentives for effort in crowdsourcing using the peer truth serum. ACM Transactions on Intelligent Sys- tems and Technology (TIST), 7(4):48, 2016.","title":"Incentives for effort in crowdsourcing using the peer truth serum","authors":"Goran Radanovic, Boi Faltings, Radu Jurca","year":"2016","venue":"ACM Transactions on Intelligent Systems and Technology (TIST)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Rubinstein, 1998] Ariel Rubinstein. Modeling bounded ra- tionality. MIT press, 1998.","title":"Modeling bounded rationality","authors":"Ariel Rubinstein","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Shnayder et al., 2016] Victor Shnayder, Arpit Agarwal, Rafael Frongillo, and David C. Parkes. Informed truth- fulness in multi-task peer prediction. EC '16. ACM, 2016.","title":"Informed truthfulness in multi-task peer prediction","authors":"Victor Shnayder, Arpit Agarwal, Rafael Frongillo, David C Parkes","year":"2016","venue":"EC '16","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Waggoner and Chen, 2014] Bo Waggoner and Yiling Chen. Output agreement mechanisms and common knowledge. In Second AAAI Conference on Human Computation and Crowdsourcing, 2014.","title":"Output agreement mechanisms and common knowledge","authors":"Chen ; Waggoner, Bo Waggoner, Yiling Chen","year":"2014","venue":"Second AAAI Conference on Human Computation and Crowdsourcing","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Witkowski and Parkes, 2012] Jens Witkowski and David C Parkes. Peer prediction without a common prior. In Pro- ceedings of the 13th ACM Conference on Electronic Com- merce, pages 964-981. ACM, 2012.","title":"Peer prediction without a common prior","authors":"Parkes Witkowski, Jens Witkowski, David C Parkes","year":"2012","venue":"Proceedings of the 13th ACM Conference on Electronic Commerce","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zheng et al., 2014] Zibin Zheng, Yilei Zhang, and Michael R Lyu. Investigating qos of real-world web services. IEEE transactions on services computing, 2014.","title":"Investigating qos of real-world web services","authors":"Zheng","year":"2014","venue":"IEEE transactions on services computing","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}