{"type":"ScholarlyArticle","title":"Guide to Numerical Experiments on Elections in Computational Social Choice","authors":[{"name":"Niclas Boehmer","slug":"niclas-boehmer","given":"Niclas","family":"Boehmer","affiliation":"Harvard University"},{"name":"Piotr Faliszewski","slug":"piotr-faliszewski","given":"Piotr","family":"Faliszewski","affiliation":"AGH University"},{"name":"Łukasz Janeczko","slug":"ukasz-janeczko","given":"Łukasz","family":"Janeczko","affiliation":"AGH University"},{"name":"Andrzej Kaczmarczyk","slug":"andrzej-kaczmarczyk","given":"Andrzej","family":"Kaczmarczyk","affiliation":"AGH University"},{"name":"Grzegorz Lisowski","slug":"grzegorz-lisowski","given":"Grzegorz","family":"Lisowski","affiliation":"AGH University"},{"name":"Grzegorz Pierczyński","slug":"grzegorz-pierczynski","given":"Grzegorz","family":"Pierczyński","affiliation":"AGH University"},{"name":"Simon Rey","slug":"simon-rey","given":"Simon","family":"Rey","affiliation":"ILLC , University of Amsterdam"},{"name":"Dariusz Stolicki","slug":"dariusz-stolicki","given":"Dariusz","family":"Stolicki","affiliation":"Jagiellonian University"},{"name":"Stanisław Szufa","slug":"stanis-aw-szufa","given":"Stanisław","family":"Szufa","affiliation":"AGH University; CNRS , LAMSADE , Université Paris Dauphine-PSL"},{"name":"Tomasz Wąs","slug":"tomasz-was","given":"Tomasz","family":"Wąs","affiliation":"University of Oxford"}],"abstract":"We analyze how numerical experiments regarding elections were conducted within computational social choice literature (focusing on papers published in the IJCAI, AAAI, and AAMAS conferences). We analyze the sizes of the studied elections and the methods of generating preference data, thereby making previously hidden standards and practices explicit. In particular, we survey a number of statistical cultures for generating elections and their commonly used parameters.","doi":"10.24963/ijcai.2024/881","url":"https://ijcai.org/proceedings/2024/881","pdf":"https://ijcai.org/proceedings/2024/0881.pdf","markdown":"https://ijcai.org/proceedings/2024/881.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/881","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Survey Track","track":"Survey Track","number":881,"pages":{"first":7962,"last":7970},"keywords":["Game Theory and Economic Paradigms: GTEP: Computational social choice"],"videos":[],"references":[{"position":1,"raw":"[Bachrach et al., 2016] Y. Bachrach, O. Lev, Y. Lewenberg, and Y. Zick. Misrepresentation in district voting. In Pro- ceedings of IJCAI-2016, pages 81-87, 2016.","title":"Misrepresentation in district voting","authors":"Y Bachrach, O Lev, Y Lewenberg, Y Zick","year":"2016","venue":"Proceedings of IJCAI-2016","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Berg, 1985] S. Berg. Paradox of voting under an urn model: The effect of homogeneity. Public Choice, 47(2):377-387, 1985.","title":"Paradox of voting under an urn model: The effect of homogeneity","authors":"S Berg","year":"1985","venue":"Public Choice","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Black, 1958] D. Black. The Theory of Committees and Elec- tions. Cambridge University Press, 1958.","title":"The Theory of Committees and Elections","authors":"D Black","year":"1958","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bloembergen et al., 2019] D. Bloembergen, D. Grossi, and M. Lackner. On rational delegations in liquid democracy. In Proceedings of AAAI-2019, pages 1796-1803, 2019.","title":"On rational delegations in liquid democracy","authors":"D Bloembergen, D Grossi, M Lackner","year":"2019","venue":"Proceedings of AAAI-2019","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Boehmer and Schaar, 2023] N. Boehmer and N. Schaar. Collecting, classifying, analyzing, and using real-world ranking data. In Proceedings of AAMAS-2023, pages 1706-1715, 2023.","title":"Collecting, classifying, analyzing, and using real-world ranking data","authors":"N Boehmer, N Schaar","year":"2023","venue":"Proceedings of AAMAS-2023","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Boehmer et al., 2021] N. Boehmer, R. Bredereck, P. Fal- iszewski, R. Niedermeier, and S. Szufa. Putting a compass on the map of elections. In Proceedings of IJCAI-2021, pages 59-65, 2021.","title":"Putting a compass on the map of elections","authors":"N Boehmer, R Bredereck, P Faliszewski, R Niedermeier, S Szufa","year":"2021","venue":"Proceedings of IJCAI-2021","doi":null,"url":null,"internal_url":"/proceedings/2021/9"},{"position":7,"raw":"[Boehmer et al., 2022] N. Boehmer, R. Bredereck, E. Elkind, P. Faliszewski, and S. Szufa. Expected frequency matrices of elections: Computation, geometry, and preference learning. In Proceedings of NeurIPS-2022, 2022.","title":"Expected frequency matrices of elections: Computation, geometry, and preference learning","authors":"N Boehmer, R Bredereck, E Elkind, P Faliszewski, S Szufa","year":"2022","venue":"Proceedings of NeurIPS-2022","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Boehmer et al., 2023a] N. Boehmer, J.-Y. Cai, P. Fal- iszewski, A. Z. Fan, Ł. Janeczko, A. Kaczmarczyk, and T. W ąs. Properties of position matrices and their elections. In Proceedings of AAAI-2023, pages 5507-5514, 2023.","title":"Properties of position matrices and their elections","authors":"Boehmer","year":"2023","venue":"Proceedings of AAAI-2023","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Boehmer et al., 2023b] N. Boehmer, P. Faliszewski, and S. Kraiczy. Properties of the Mallows model depending on the number of alternatives: A warning for an experimen- talist. In Proceedings of ICML-2023, pages 2689-2711.","title":"Properties of the Mallows model depending on the number of alternatives: A warning for an experimentalist","authors":"Boehmer","year":"2023","venue":"Proceedings of ICML-2023","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"PMLR, 2023.","title":null,"authors":"Pmlr","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Boehmer et al., 2024] N. Boehmer, P. Faliszewski, Ł. Janeczko, A. Kaczmarczyk, G. Lisowski, G. Pierczyński, S. Rey, D. Stolicki, S. Szufa, and T. W ąs. Guide to nu- merical experiments on elections in computational social choice. Technical Report arXiv:2402.11765, arXiv.org, 2024.","title":"Guide to numerical experiments on elections in computational social choice","authors":"N Boehmer, P Faliszewski, Ł Janeczko, A Kaczmarczyk, G Lisowski, G Pierczyński, S Rey, D Stolicki, S Szufa, T W Ąs","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Boehmer, 2023] N. Boehmer. Application-oriented collec- tive decision making: Experimental toolbox and dynamic environments. PhD thesis, Technical University of Berlin, Germany, 2023.","title":"Application-oriented collective decision making: Experimental toolbox and dynamic environments","authors":"N Boehmer","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Borodin et al., 2018] A. Borodin, O. Lev, N. Shah, and T. Strangway. Big city vs. the great outdoors: Voter distri- bution and how it affects gerrymandering. In Proceedings of IJCAI-2018, pages 98-104, 2018.","title":"Big city vs. the great outdoors: Voter distribution and how it affects gerrymandering","authors":"A Borodin, O Lev, N Shah, T Strangway","year":"2018","venue":"Proceedings of IJCAI-2018","doi":null,"url":null,"internal_url":"/proceedings/2018/14"},{"position":14,"raw":"[Brandt et al., 2016] F. Brandt, V. Conitzer, U. Endriss, J. Lang, and A. Procaccia, editors. Handbook of Computa- tional Social Choice. Cambridge University Press, 2016.","title":"Handbook of Computational Social Choice","authors":"F Brandt, V Conitzer, U","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Colley et al., 2023] R. Colley, T. Delemazure, and H. Gilbert. Measuring a priori voting power in liq- uid democracy. In Proceedings of IJCAI-2023, pages 2607-2615, 2023.","title":"Measuring a priori voting power in liquid democracy","authors":"R Colley, T Delemazure, H Gilbert","year":"2023","venue":"Proceedings of IJCAI-2023","doi":null,"url":null,"internal_url":"/proceedings/2023/290"},{"position":16,"raw":"[Conitzer, 2009] V. Conitzer. Eliciting single-peaked pref- erences using comparison queries. Journal of Artificial Intelligence Research, 35:161-191, 2009.","title":"Eliciting single-peaked preferences using comparison queries","authors":"V Conitzer","year":"2009","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Doucette and Cohen, 2017] J. Doucette and R. Cohen. A re- stricted markov tree model for inference and generation in social choice with incomplete preferences. In Proceedings of AAMAS-2017, pages 893-901, 2017.","title":"A restricted markov tree model for inference and generation in social choice with incomplete preferences","authors":"Cohen Doucette, ] J Doucette, R Cohen","year":"2017","venue":"Proceedings of AAMAS-2017","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Eggenberger and Pólya, 1923] F. Eggenberger and G. Pólya. Über die statistik verketteter vorgänge. ZAMM-Journal of Applied Mathematics and Mechan- ics/Zeitschrift für Angewandte Mathematik und Mechanik, 3(4):279-289, 1923.","title":"Über die statistik verketteter vorgänge","authors":"F Eggenberger, G Pólya","year":"1923","venue":"ZAMM-Journal of Applied Mathematics and Mechanics/Zeitschrift für Angewandte Mathematik und Mechanik","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Egecioglu and Giritligil, 2013] Ö. Egecioglu and A. Gir- itligil. The impartial, anonymous, and neutral culture model: A probability model for sampling public pref- erence structures. Journal of Mathematical Sociology, 37(4):203-222, 2013.","title":"Ö. Egecioglu and A. Giritligil. The impartial, anonymous, and neutral culture model: A probability model for sampling public preference structures","authors":"Giritligil Egecioglu","year":"2013","venue":"Journal of Mathematical Sociology","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Faliszewski et al., 2017] P. Faliszewski, P. Skowron, A. Slinko, and N. Talmon. Multiwinner voting: A new challenge for social choice theory. In U. Endriss, editor, Trends in Computational Social Choice. AI Access Foundation, 2017.","title":"Multiwinner voting: A new challenge for social choice theory","authors":"P Faliszewski, P Skowron, A Slinko, N Talmon","year":"2017","venue":"Trends in Computational Social Choice","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Faliszewski et al., 2019] P. Faliszewski, P. Skowron, A. Slinko, S. Szufa, and N. Talmon. How similar are two elections? In Proceedings of AAAI-2019, pages 1909-1916, 2019.","title":"How similar are two elections?","authors":"P Faliszewski, P Skowron, A Slinko, S Szufa, N Talmon","year":"2019","venue":"Proceedings of AAAI-2019","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Faliszewski et al., 2023a] P. Faliszewski, J. Flis, D. Peters, G. Pierczynski, P. Skowron, D. Stolicki, S. Szufa, and N. Talmon. Participatory budgeting: Data, tools and anal- ysis. In Proceedings of IJCAI-2023, pages 2667-2674, 2023.","title":"Participatory budgeting: Data, tools and analysis","authors":"Faliszewski","year":"2023","venue":"Proceedings of IJCAI-2023","doi":null,"url":null,"internal_url":"/proceedings/2023/297"},{"position":23,"raw":"[Faliszewski et al., 2023b] P. Faliszewski, A. Kaczmarczyk, K. Sornat, S. Szufa, and T. W ąs. Diversity, agreement, and polarization in elections. In Proceedings of IJCAI-2023, pages 2684-2692, 2023.","title":"Diversity, agreement, and polarization in elections","authors":"Faliszewski","year":"2023","venue":"Proceedings of IJCAI-2023","doi":null,"url":null,"internal_url":"/proceedings/2023/299"},{"position":24,"raw":"[Fishburn and Gehrlein, 1978] P. Fishburn and W. Gehrlein. Condorcet paradox and anonymous preference profiles. Public Choice, 26:1-18, 1978.","title":"Condorcet paradox and anonymous preference profiles","authors":"P Fishburn, W Gehrlein","year":"1978","venue":"Public Choice","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Guilbaud, 1952] G. Guilbaud. Les théories de l'intéret genéral et le problémelogique de l'agrégation. Economie Appliquée, 5:501-584, 1952.","title":"Les théories de l'intéret genéral et le problémelogique de l'agrégation","authors":"G Guilbaud","year":"1952","venue":"Economie Appliquée","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Inada, 1964] K. Inada. A note on the simple majority deci- sion rule. Econometrica, 32(32):525-531, 1964.","title":"A note on the simple majority decision rule","authors":"K Inada","year":"1964","venue":"Econometrica","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Inada, 1969] K. Inada. The simple majority decision rule. Econometrica, 37(3):490-506, 1969.","title":"The simple majority decision rule","authors":"K Inada","year":"1969","venue":"Econometrica","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Karpov, 2019] Alexander Karpov. On the number of group- separable preference profiles. Group Decision and Nego- tiation, 28(3):501-517, 2019.","title":"On the number of groupseparable preference profiles","authors":"Alexander Karpov","year":"2019","venue":"Group Decision and Negotiation","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Kruskal, 1964] J. Kruskal. Multidimensional scaling by op- timizing goodness of fit to a nonmetric hypothesis. Psy- chometrika, 29(1):1-27, 1964.","title":"Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis","authors":"J","year":"1964","venue":"Psychometrika","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Kuga and Nagatani, 1974] K. Kuga and H. Nagatani. Voter antagonism and the paradox of voting. Econometrica, 42(6):1045-1067, 1974.","title":"Voter antagonism and the paradox of voting","authors":"K Kuga, H Nagatani","year":"1974","venue":"Econometrica","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Lackner and Skowron, 2023] M. Lackner and P. Skowron. Multi-Winner Voting with Approval Preferences. Springer, 2023.","title":"Multi-Winner Voting with Approval Preferences","authors":"M Lackner, P Skowron","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Lang and Xia, 2016] J. Lang and L. Xia. Voting in combi- natorial domains. In F. Brandt, V. Conitzer, U. Endriss, J. Lang, and A. D. Procaccia, editors, Handbook of Com- putational Social Choice, chapter 9, pages 197-222.","title":"Voting in combinatorial domains","authors":"J Lang, L Xia","year":null,"venue":"Handbook of Computational Social Choice","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Mallows, 1957] C. Mallows. Non-null ranking models. Bio- metrica, 44:114-130, 1957.","title":"Non-null ranking models","authors":"C Mallows","year":"1957","venue":"Biometrica","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Mattei and Walsh, 2013] N. Mattei and T. Walsh. Preflib: A library for preferences. In Proceedings of ADT-2013, pages 259-270, 2013.","title":"Preflib: A library for preferences","authors":"N Mattei, T Walsh","year":"2013","venue":"Proceedings of ADT-2013","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[McCabe-Dansted and Slinko, 2006] J. McCabe-Dansted and A. Slinko. Exploratory analysis of similarities between social choice rules. Group Decision and Negotiation, 15:77-107, 2006.","title":"Exploratory analysis of similarities between social choice rules","authors":"J Mccabe-Dansted, A Slinko","year":"2006","venue":"Group Decision and Negotiation","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Mirrlees, 1971] J. Mirrlees. An exploration in the theory of optimal income taxation. Review of Economic Studies, 38:175-208, 1971.","title":"An exploration in the theory of optimal income taxation","authors":"J Mirrlees","year":"1971","venue":"Review of Economic Studies","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Peters and Lackner, 2020] D. Peters and M. Lackner. Pref- erences single-peaked on a circle. Journal of Artificial In- telligence Research, 68:463-502, 2020.","title":"Preferences single-peaked on a circle","authors":"D Peters, M Lackner","year":"2020","venue":"Journal of Artificial Intelligence Research","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Peters et al., 2021] D. Peters, G. Pierczynski, N. Shah, and P. Skowron. Market-based explanations of collective de- cisions. In Proceedings of AAAI-2021, pages 5656-5663. AAAI Press, 2021.","title":"Market-based explanations of collective decisions","authors":"D Peters, G Pierczynski, N Shah, P Skowron","year":"2021","venue":"Proceedings of AAAI-2021","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Rey and Maly, 2023] S. Rey and J. Maly. The (compu- tational) social choice take on indivisible participatory budgeting. Technical Report arXiv.2303.00621 [cs.GT], arXiv.org, 2023.","title":"The (computational) social choice take on indivisible participatory budgeting","authors":"S Rey, J Maly","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Roberts, 1977] K. Roberts. Voting over income tax sched- ules. Journal of Public Economics, 8(3):329-340, 1977.","title":"Voting over income tax schedules","authors":"K Roberts","year":"1977","venue":"Journal of Public Economics","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Szufa et al., 2020] S. Szufa, P. Faliszewski, P. Skowron, A. Slinko, and N. Talmon. Drawing a map of elections in the space of statistical cultures. In Proceedings of AAMAS- 2020, pages 1341-1349, 2020.","title":"Drawing a map of elections in the space of statistical cultures","authors":"S Szufa, P Faliszewski, P Skowron, A Slinko, N Talmon","year":"2020","venue":"Proceedings of AAMAS-2020","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Szufa et al., 2022] S. Szufa, P. Faliszewski, L. Janeczko, M. Lackner, A. Slinko, K. Sornat, and N. Talmon. How to sample approval elections? In Proceedings of IJCAI- 2022, pages 496-502, 2022.","title":"How to sample approval elections?","authors":"S Szufa, P Faliszewski, L Janeczko, M Lackner, A Slinko, K Sornat, N Talmon","year":"2022","venue":"Proceedings of IJCAI-2022","doi":null,"url":null,"internal_url":"/proceedings/2022/71"},{"position":43,"raw":"[Walsh, 2015] T. Walsh. Generating single peaked votes. Technical Report arXiv:1503.02766 [cs.GT], arXiv.org, March 2015.","title":"Generating single peaked votes","authors":"T Walsh","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Wilder and Vorobeychik, 2019] B. Wilder and Y. Vorobey- chik. Defending elections against malicious spread of mis- information. In Proceedings of AAAI-2019, pages 2213- 2220, 2019.","title":"Defending elections against malicious spread of misinformation","authors":"B Wilder, Y Vorobeychik","year":"2019","venue":"Proceedings of AAAI-2019","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}