{"type":"ScholarlyArticle","title":"Picking the Right Winner: Why Tie-Breaking in Crowdsourcing Contests Matters","authors":[{"name":"Coral Haggiag","slug":"coral-haggiag","given":"Coral","family":"Haggiag","affiliation":"Ben-Gurion University of the Negev"},{"name":"Sigal Oren","slug":"sigal-oren","given":"Sigal","family":"Oren","affiliation":"Ben-Gurion University of the Negev"},{"name":"Ella Segev","slug":"ella-segev","given":"Ella","family":"Segev","affiliation":"Ben-Gurion University of the Negev"}],"abstract":"We present a complete information game-theoretic model for crowdsourcing contests. We observe that in design contests, coding contests and other domains, separating low quality submissions from high quality ones is often easy. However, pinning down the best submission is more challenging since there is no objective measure. We model this situation by assuming that each contestant has an ability, which we interpret as its probability of submitting a high-quality submission. After the contestants decide whether or not they want to participate, the organizer of the contest needs to break ties between the high quality submissions. A common assumption in the literature is that the exact tie-breaking rule does not matter as long as ties are broken consistently. However, we show that the choice of the tie-breaking rule may have significant implications on the efficiency of the contest.\r\n\r\nOur results highlight both qualitative and quantitative differences between various deterministic tie-breaking rules. Perhaps counterintuitively, we show that in many scenarios, the utility of the organizer is maximized when ties are broken in favor of successful players with lower ability. Nevertheless, we show that the natural rule of choosing the submission of the successful player with the highest ability guarantees the organizer at least 1/3 of its utility under any tie-breaking rule. To complement these results, we provide an upper bound of  Hn ~ \\ln(n) on the price of anarchy (the ratio between the social welfare of the optimal solution and the social welfare of the Nash equilibrium). We show that this ratio is tight when ties are broken in favor of players with higher abilities.","doi":"10.24963/ijcai.2022/44","url":"https://ijcai.org/proceedings/2022/44","pdf":"https://ijcai.org/proceedings/2022/0044.pdf","markdown":"https://ijcai.org/proceedings/2022/44.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/44","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":44,"pages":{"first":307,"last":313},"keywords":["Agent-based and Multi-agent Systems: Algorithmic Game Theory"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986739","length_sec":90,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984882","length_sec":354,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Araujo, 2013] Ricardo Matsumura Araujo. 99designs: An analysis of creative competition in crowdsourced design. In First AAAI conference on Human computation and crowdsourcing, 2013.","title":"99designs: An analysis of creative competition in crowdsourced design","authors":"Ricardo Matsumura, Araujo","year":"2013","venue":"First AAAI conference on Human computation and crowdsourcing","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Archak and Sundararajan, 2009] Nikolay Archak and Arun Sundararajan. Optimal design of crowdsourcing contests. ICIS 2009 proceedings, page 200, 2009.","title":"Optimal design of crowdsourcing contests","authors":"Sundararajan Archak, Nikolay Archak, Arun Sundararajan","year":"2009","venue":"ICIS 2009 proceedings","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Archak, 2010] Nikolay Archak. Money, glory and cheap talk: analyzing strategic behavior of contestants in simul- taneous crowdsourcing contests on topcoder. com. In Pro- ceedings of the 19th international conference on World wide web, pages 21-30. ACM, 2010.","title":"Money, glory and cheap talk: analyzing strategic behavior of contestants in simultaneous crowdsourcing contests on topcoder","authors":"Nikolay Archak","year":"2010","venue":"Proceedings of the 19th international conference on World wide web","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cavallo and Jain, 2012] Ruggiero Cavallo and Shaili Jain. Efficient crowdsourcing contests. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems-Volume 2, pages 677-686. Interna- tional Foundation for Autonomous Agents and Multiagent Systems, 2012.","title":"Efficient crowdsourcing contests","authors":"Ruggiero Cavallo, Shaili Jain","year":"2012","venue":"Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chawla et al., 2015] Shuchi Chawla, Jason D Hartline, and Balasubramanian Sivan. Optimal crowdsourcing contests. Games and Economic Behavior, 2015.","title":"Optimal crowdsourcing contests","authors":"Shuchi Chawla, Jason D Hartline, Balasubramanian Sivan","year":"2015","venue":"Games and Economic Behavior","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Franke et al., 2018] Jörg Franke, Wolfgang Leininger, and Cédric Wasser. Optimal favoritism in all-pay auctions and lottery contests. European Economic Review, 104:22-37, 2018.","title":"Optimal favoritism in all-pay auctions and lottery contests","authors":"Franke","year":"2018","venue":"European Economic Review","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ghosh and Hummel, 2012] Arpita Ghosh and Patrick Hum- mel. Implementing optimal outcomes in social comput- ing: a game-theoretic approach. In Proceedings of the 21st international conference on World Wide Web, pages 539-548. ACM, 2012.","title":"Implementing optimal outcomes in social computing: a game-theoretic approach","authors":"Hummel Ghosh, Arpita Ghosh, Patrick Hummel","year":"2012","venue":"Proceedings of the 21st international conference on World Wide Web","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Ghosh and Kleinberg, 2016] Arpita Ghosh and Robert Kleinberg. Optimal contest design for simple agents. ACM Transactions on Economics and Computation (TEAC), 4(4):22, 2016.","title":"Optimal contest design for simple agents","authors":"Kleinberg Ghosh, Arpita Ghosh, Robert Kleinberg","year":"2016","venue":"ACM Transactions on Economics and Computation (TEAC)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kleinberg and Oren, 2011] Jon Kleinberg and Sigal Oren. Mechanisms for (mis) allocating scientific credit. In Pro- ceedings of the forty-third annual ACM symposium on Theory of computing, pages 529-538. ACM, 2011.","title":"Mechanisms for (mis) allocating scientific credit","authors":"Oren ; Kleinberg, Jon Kleinberg, Sigal Oren","year":"2011","venue":"Proceedings of the forty-third annual ACM symposium on Theory of computing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Konrad and Kovenock, 2009] Kai A Konrad and Dan Kovenock. Multi-battle contests. Games and Economic Behavior, 66(1):256-274, 2009.","title":"Multi-battle contests","authors":"A Kai, Dan Konrad, Kovenock","year":"2009","venue":"Games and Economic Behavior","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Konrad, 2009] Kai A Konrad. Strategy and dynamics in contests. OUP Catalogue, 2009.","title":"Strategy and dynamics in contests","authors":"Konrad Kai","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Levy et al., 2017] Priel Levy, David Sarne, and Igor Rochlin. Contest design with uncertain performance and costly participation. In Proceedings of the 26th Interna- tional Joint Conference on Artificial Intelligence, pages 302-309. AAAI Press, 2017.","title":"Contest design with uncertain performance and costly participation","authors":"Priel Levy, David Sarne, Igor Rochlin","year":"2017","venue":"Proceedings of the 26th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2017/43"},{"position":13,"raw":"[Levy et al., 2018] Priel Levy, David Sarne, and Yonatan Aumann. Tractable (simple) contests. In IJCAI, pages 361-367, 2018.","title":"Tractable (simple) contests","authors":"Priel Levy, David Sarne, Yonatan Aumann","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/50"},{"position":14,"raw":"[Liu et al., 2014] Tracy Xiao Liu, Jiang Yang, Lada A Adamic, and Yan Chen. Crowdsourcing with all-pay auc- tions: a field experiment on taskcn. Management Science, 60(8):2020-2037, 2014.","title":"Crowdsourcing with all-pay auctions: a field experiment on taskcn","authors":"Xiao Tracy, Jiang Liu, Lada A Yang, Yan Adamic, Chen","year":"2014","venue":"Management Science","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Llorente-Saguer et al., 2018] Aniol Llorente-Saguer, Ro- man M Sheremeta, and Nora Szech. Designing con- tests between heterogeneous contestants: An experimental study of tie-breaks and bid-caps in all-pay auctions. Avail- able at SSRN 2766732, 2018.","title":"Designing contests between heterogeneous contestants: An experimental study of tie-breaks and bid-caps in all-pay auctions","authors":"Aniol Llorente-Saguer, Roman M Sheremeta, Nora Szech","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Nisbett and Wilson, 1977] Richard E Nisbett and Timo- thy D Wilson. The halo effect: Evidence for unconscious alteration of judgments. Journal of personality and social psychology, 35(4):250, 1977.","title":"The halo effect: Evidence for unconscious alteration of judgments","authors":"E Richard, Timothy D Nisbett, Wilson","year":"1977","venue":"Journal of personality and social psychology","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Segev and Sela, 2014] Ella Segev and Aner Sela. Multi- stage sequential all-pay auctions. European Economic Re- view, 70:371-382, 2014.","title":"Multistage sequential all-pay auctions","authors":"Sela ; Segev, Ella Segev, Aner Sela","year":"2014","venue":"European Economic Review","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Segev, 2020] Ella Segev. Crowdsourcing contests. Euro- pean Journal of Operational Research, 281(2):241-255, 2020.","title":"Crowdsourcing contests","authors":"Segev","year":"2020","venue":"European Journal of Operational Research","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Vojonovic, 2015] M Vojonovic. Contest Theory. Cam- bridge: Cambridge University Press, 2015.","title":"Contest Theory","authors":"Vojonovic","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Zheng et al., 2011] Haichao Zheng, Dahui Li, and Wenhua Hou. Task design, motivation, and participation in crowd- sourcing contests. International Journal of Electronic Commerce, 15(4):57-88, 2011.","title":"Task design, motivation, and participation in crowdsourcing contests","authors":"Zheng","year":"2011","venue":"International Journal of Electronic Commerce","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zhu, 2021] Feng Zhu. On optimal favoritism in all-pay con- tests. Journal of Mathematical Economics, page 102472, 2021.","title":"On optimal favoritism in all-pay contests","authors":"Feng Zhu","year":"2021","venue":"Journal of Mathematical Economics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}