{"type":"ScholarlyArticle","title":"Closing the Loop: Bringing Humans into Empirical Computational Social Choice and Preference Reasoning","authors":[{"name":"Nicholas Mattei","slug":"nicholas-mattei","given":"Nicholas","family":"Mattei","affiliation":"Department of Computer Science , Tulane University"}],"abstract":"Research in both computational social choice and preference reasoning uses tools and techniques from computer science, generally algorithms and complexity analysis, to examine topics in group decision making.  This has brought tremendous progress in the last decades, creating new avenues for research and results in areas including voting and resource allocation.  I argue that of equal importance to the theoretical results are impacts in research and development from the empirical part of the computer scientists toolkit: data, system building, and human interaction. I highlight work by myself and others to establish data driven, application driven research in the computational social choice and preference reasoning areas. Along the way, I highlight interesting application domains and important results from the community in driving this area to make concrete, real-world impact.","doi":"10.24963/ijcai.2020/729","url":"https://ijcai.org/proceedings/2020/729","pdf":"https://ijcai.org/proceedings/2020/0729.pdf","markdown":"https://ijcai.org/proceedings/2020/729.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/729","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Early Career","track":"Early Career","number":729,"pages":{"first":5169,"last":5173},"keywords":["Agent-based and Multi-agent Systems: Voting","AI Ethics: Moral Decision Making","Agent-based and Multi-agent Systems: Algorithmic Game Theory","Agent-based and Multi-agent Systems: Computational Social Choice"],"videos":[],"references":[{"position":1,"raw":"[Abdulkadiroglu et al., 2005] A. Abdulkadiroglu, P. A. Pathak, and A. E. Roth. The new york city high school match. American Economic Review, 95(2):364-367, 2005.","title":"The new york city high school match","authors":"A Abdulkadiroglu, P A Pathak, A E Roth","year":"2005","venue":"American Economic Review","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Allen et al., 2015] T.E. Allen, M. Chen, J. Goldsmith, N. Mattei, A. Popova, M. Regenwetter, F. Rossi, and C. Zwilling. Beyond theory and data in preference modeling: Bringing humans into the loop. In Proc. 4th ADT, 2015.","title":"Beyond theory and data in preference modeling: Bringing humans into the loop","authors":"T E Allen, M Chen, J Goldsmith, N Mattei, A Popova, M Regenwetter, F Rossi, C Zwilling","year":"2015","venue":"Proc. 4th ADT","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Aziz and Mackenzie, 2016] H. Aziz and S. Mackenzie. A discrete and bounded envy-free cake cutting protocol for any number of agents. In Proc. 57th FOCS, pages 416-427, 2016.","title":"A discrete and bounded envy-free cake cutting protocol for any number of agents","authors":"H Aziz, S Mackenzie","year":"2016","venue":"Proc. 57th FOCS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Aziz et al., 2016a] H. Aziz, C. Cahan, C. Gretton, P. Kilby, N. Mattei, and T. Walsh. A study of proxies for shapley allo- cations of transport costs. JAIR, 56:573-611, 2016.","title":"A study of proxies for shapley allocations of transport costs","authors":"Aziz","year":"2016","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Aziz et al., 2016b] H. Aziz, O. Lev, N. Mattei, J. S. Rosenschein, and T. Walsh. Strategyproof peer selection: Mechanisms, analy- ses, and experiments. In Proc. 30th AAAI, 2016.","title":"Strategyproof peer selection: Mechanisms, analyses, and experiments","authors":"Aziz","year":"2016","venue":"Proc. 30th AAAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Aziz et al., 2019] H. Aziz, O. Lev, N. Mattei, J. S. Rosenschein, and T. Walsh. Strategyproof peer selection using randomiza- tion, partitioning, and apportionment. Artif. Intell., 275:295-309, 2019.","title":"Strategyproof peer selection using randomization, partitioning, and apportionment","authors":"H Aziz, O Lev, N Mattei, J S Rosenschein, T Walsh","year":"2019","venue":"Artif. Intell","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Aziz, 2019] H. Aziz. Developments in multi-agent fair allocation. CoRR, abs/1911.09852, 2019.","title":"Developments in multi-agent fair allocation","authors":"H Aziz","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Bache and Lichman, 2013] K. Bache and M. Lichman. UCI Ma- chine Learning Repository, 2013. University of California, Irvine, School of Information and Computer Sciences.","title":"UCI Machine Learning Repository","authors":"K Bache, M Lichman","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Brams, 2008] Steven J Brams. Mathematics and democracy: De- signing better voting and fair division procedures. Mathematical and Computer Modelling, 48(9-10):1666-1670, 2008.","title":"Mathematics and democracy: Designing better voting and fair division procedures","authors":"J Steven, Brams","year":"2008","venue":"Mathematical and Computer Modelling","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Brandt et al., 2015] F. Brandt, G. Chabin, and C. Geist. Pnyx: a powerful and user-friendly tool for preference aggregation. In Proc. 14th AAMAS, pages 1915-1916, 2015.","title":"Pnyx: a powerful and user-friendly tool for preference aggregation","authors":"F Brandt, G Chabin, C Geist","year":"2015","venue":"Proc. 14th AAMAS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Brandt et al., 2016a] F. Brandt, V. Conitzer, U. Endriss, J. Lang, and A. D. Procaccia, editors. Handbook of Computational Social Choice. Cambridge University Press, 2016.","title":"Handbook of Computational Social Choice","authors":"Brandt","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Brandt et al., 2016b] F. Brandt, C. Geist, and M. Strobel. Analyz- ing the practical relevance of voting paradoxes via ehrhart theory, computer simulations, and empirical data. In Proc. 16th AAMAS, pages 385-393, 2016.","title":"Analyzing the practical relevance of voting paradoxes via ehrhart theory, computer simulations, and empirical data","authors":"Brandt","year":"2016","venue":"Proc. 16th AAMAS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Bredereck et al., 2019] R. Bredereck, P. Faliszewski, A. Kaczmar- czyk, and R. Niedermeier. An experimental view on committees providing justified representation. In Proc. 28th IJCAI, pages 109-115. AAAI Press, 2019.","title":"An experimental view on committees providing justified representation","authors":"R Bredereck, P Faliszewski, A Kaczmarczyk, R Niedermeier","year":"2019","venue":"Proc. 28th IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2019/16"},{"position":14,"raw":"M. Brill. Interactive democracy. In Proc. 17th AA- MAS, pages 1183-1187, 2018.","title":"Interactive democracy","authors":"M Brill","year":"2018","venue":"Proc. 17th AA-MAS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Burton et al., 2018] E. Burton, J. Goldsmith, and N. Mattei. How to teach computer ethics through science fiction. Commun. ACM, 61(8):54-64, 2018.","title":"How to teach computer ethics through science fiction","authors":"E Burton, J Goldsmith, N Mattei","year":"2018","venue":"Commun. ACM","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Chen et al., 2020] Y. Chen, J. Qian, J. Wang, L. Xia, and G. Za- havi. OPRA: An open-source online preference reporting and aggregation system, 2020.","title":"OPRA: An open-source online preference reporting and aggregation system","authors":"Y Chen, J Qian, J Wang, L Xia, G Zahavi","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Conitzer et al., 2007] V. Conitzer, T. Sandholm, and J. Lang. When are elections with few candidates hard to manipulate? JACM, 54(3):14, 2007.","title":"When are elections with few candidates hard to manipulate?","authors":"V Conitzer, T Sandholm, J Lang","year":"2007","venue":"JACM","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Corbett-Davies et al., 2017] S. Corbett-Davies, E. Pierson, A. Feller, S. Goel, and A. Huq. Algorithmic decision making and the cost of fairness. In Proc. 23rd SIGKDD, pages 797-806. ACM, 2017.","title":"Algorithmic decision making and the cost of fairness","authors":"S Corbett-Davies, E Pierson, A Feller, S Goel, A Huq","year":"2017","venue":"Proc. 23rd SIGKDD","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Dickerson et al., 2012] J. P. Dickerson, A. D. Procaccia, and T. Sandholm. Optimizing kidney exchange with transplant chains: Theory and reality. In Proc. 11th AAMAS, pages 711- 718, 2012.","title":"Optimizing kidney exchange with transplant chains: Theory and reality","authors":"J P Dickerson, A D Procaccia, T Sandholm","year":"2012","venue":"Proc. 11th AAMAS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Dickerson et al., 2014] J. P. Dickerson, A. D. Procaccia, and T. Sandholm. Price of fairness in kidney exchange. In Proc. 13th AAMAS, pages 1013-1020, 2014.","title":"Price of fairness in kidney exchange","authors":"J P Dickerson, A D Procaccia, T Sandholm","year":"2014","venue":"Proc. 13th AAMAS","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Domshlak et al., 2011] C. Domshlak, E. Hüllermeier, S. Kaci, and H. Prade. Preferences in AI: An overview. AI, 175(7):1037-1052, 2011.","title":"Preferences in AI: An overview","authors":"C Domshlak, E Hüllermeier, S Kaci, H Prade","year":"2011","venue":"AI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Fairstein et al., 2019] R. Fairstein, A. Lauz, R. Meir, and K. Gal. Modeling people's voting behavior with poll information. In Proc. 18th AAMAS, 2019.","title":"Modeling people's voting behavior with poll information","authors":"R Fairstein, A Lauz, R Meir, K Gal","year":"2019","venue":"Proc. 18th AAMAS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Faliszewski et al., 2017] P. Faliszewski, P. Skowron, A. Slinko, and N. Talmon. Multiwinner voting: A new challenge for social choice theory. In U. Endriss, editor, Trends in Computational Social Choice, chapter 2. 2017.","title":"Multiwinner voting: A new challenge for social choice theory","authors":"P Faliszewski, P Skowron, A Slinko, N Talmon","year":"2017","venue":"Trends in Computational Social Choice","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Freedman et al., 2020] R. Freedman, J. S. Borg, W. Sinnott- Armstrong, J. P. Dickerson, and V. Conitzer. Adapting a kidney exchange algorithm to align with human values. AI, page 103261, 2020.","title":"Adapting a kidney exchange algorithm to align with human values","authors":"R Freedman, J S Borg, W Sinnott-Armstrong, J P Dickerson, V Conitzer","year":"2020","venue":"AI","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Goldman and Procaccia, 2014] J. Goldman and A. D. Procaccia. Spliddit: Unleashing fair division algorithms. 13(2):41-46, 2014.","title":"Spliddit: Unleashing fair division algorithms","authors":"J Goldman, A D Procaccia","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Kagel and Roth, 1995] J. H. Kagel and A. E Roth. The Handbook of Experimental Economics. Princeton University, 1995.","title":"The Handbook of Experimental Economics","authors":"J H Kagel, A E Roth","year":"1995","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Kamishima, 2003] T. Kamishima. Nantonac collaborative filter- ing: recommendation based on order responses. In Proc. 9th SIGKDD, pages 583-588, 2003.","title":"Nantonac collaborative filtering: recommendation based on order responses","authors":"T Kamishima","year":"2003","venue":"Proc. 9th SIGKDD","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Kleinberg et al., 1999] Jon Kleinberg, Yuval Rabani, and Éva Tar- dos. Fairness in routing and load balancing. In 40th Annual Symposium on Foundations of Computer Science (FOCS), pages 568-578. IEEE, 1999.","title":"Fairness in routing and load balancing","authors":"Jon Kleinberg, Yuval Rabani, Éva Tardos","year":"1999","venue":"40th Annual Symposium on Foundations of Computer Science (FOCS)","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Kurokawa et al., 2015] D. Kurokawa, O. Lev, J. Morgenstern, and A. D. Procaccia. Impartial peer review. In Proc. 23rd IJCAI, 2015.","title":"Impartial peer review","authors":"D Kurokawa, O Lev, J Morgenstern, A D Procaccia","year":"2015","venue":"Proc. 23rd IJCAI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Leyton-Brown et al., 2017] K. Leyton-Brown, P. Milgrom, and I. Segal. Economics and computer science of a radio spectrum reallocation. Proc. Natl. Acad. Sci. USA, 114(28):7202-7209, 2017.","title":"Economics and computer science of a radio spectrum reallocation","authors":"K Leyton-Brown, P Milgrom, I Segal","year":"2017","venue":"Proc. Natl. Acad. Sci","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Lian et al., 2018] J. W. Lian, N. Mattei, R. Noble, and T. Walsh. The conference paper assignment problem: Using order weighted averages to assign indivisible goods. In Proc. 33rd AAAI, 2018.","title":"The conference paper assignment problem: Using order weighted averages to assign indivisible goods","authors":"J W Lian, N Mattei, R Noble, T Walsh","year":"2018","venue":"Proc. 33rd AAAI","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Loreggia et al., 2018] A. Loreggia, N. Mattei, F. Rossi, and K. B. Venable. Preferences and ethical principles in decision making. In Proc. 1st AIES, 2018.","title":"Preferences and ethical principles in decision making","authors":"A Loreggia, N Mattei, F Rossi, K B Venable","year":"2018","venue":"Proc. 1st AIES","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Mattei and Walsh, 2013] N. Mattei and T. Walsh. Preflib: A library for preferences, HTTP://WWW.PREFLIB.ORG. In Proc. 3rd ADT, 2013.","title":"Preflib: A library for preferences, HTTP://WWW","authors":"N Mattei, T Walsh","year":"2013","venue":"PREFLIB.ORG. In Proc. 3rd ADT","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Mattei and Walsh, 2017] N. Mattei and T. Walsh. A PRE- FLIB.ORG Retrospective: Lessons Learned and New Directions.","title":"A PRE-FLIB","authors":"N Mattei, T Walsh","year":null,"venue":"ORG Retrospective: Lessons Learned and New Directions","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"In U. Endriss, editor, Trends in Computational Social Choice, chapter 15, pages 289-309.","title":"Trends in Computational Social Choice","authors":null,"year":null,"venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Mattei et al., 2012] N. Mattei, J. Forshee, and J. Goldsmith. An empirical study of voting rules and manipulation with large datasets. In Proc. 4th COMSOC, 2012.","title":"An empirical study of voting rules and manipulation with large datasets","authors":"N Mattei, J Forshee, J Goldsmith","year":"2012","venue":"Proc. 4th COMSOC","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Mattei et al., 2017] N. Mattei, A. Saffidine, and T. Walsh. Mecha- nisms for online organ matching. In Proc. 26th IJCAI, 2017.","title":"Mechanisms for online organ matching","authors":"N Mattei, A Saffidine, T Walsh","year":"2017","venue":"Proc. 26th IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/49"},{"position":38,"raw":"[Mattei et al., 2018] N. Mattei, A. Saffidine, and T. Walsh. Fairness in deceased organ matching. In Proc. 1st AIES, 2018.","title":"Fairness in deceased organ matching","authors":"N Mattei, A Saffidine, T Walsh","year":"2018","venue":"Proc. 1st AIES","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Mattei et al., 2020] N. Mattei, P. Turrini, and S. Zhydkov. Peer- Nomination: Relaxing exactness for increased accuracy in peer selection. CoRR, abs/2004.14939, 2020.","title":"Peer-Nomination: Relaxing exactness for increased accuracy in peer selection","authors":"N Mattei, P Turrini, S Zhydkov","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Mattei, 2011] N. Mattei. Empirical evaluation of voting rules with strictly ordered preference data. In Proc. 2nd ADT, 2011.","title":"Empirical evaluation of voting rules with strictly ordered preference data","authors":"N Mattei","year":"2011","venue":"Proc. 2nd ADT","doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Meir et al., 2020a] R. Meir, J. Lang, J. Lesca, N. Kaminski, and N. Mattei. A market-inspired bidding scheme for peer review pa- per assignment. In Proc. 2nd GAIW Workshop at AAMAS, 2020.","title":"A market-inspired bidding scheme for peer review paper assignment","authors":"Meir","year":"2020","venue":"Proc. 2nd GAIW Workshop at AAMAS","doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Meir et al., 2020b] Reshef Meir, Kobi Gal, and Maor Tal. Strategic voting in the lab: compromise and leader bias behavior. Auton. Agents Multi Agent Syst., 34(1):31, 2020.","title":"Strategic voting in the lab: compromise and leader bias behavior","authors":"Meir","year":"2020","venue":"Auton. Agents Multi Agent Syst","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Mennle et al., 2015] T. Mennle, M. Weiss, B. Philipp, and S. Seuken. The power of local manipulation strategies in assign- ment mechanisms. In Proc. 24th IJCAI, 2015.","title":"The power of local manipulation strategies in assignment mechanisms","authors":"T Mennle, M Weiss, B Philipp, S Seuken","year":"2015","venue":"Proc. 24th IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/19"},{"position":44,"raw":"[Moulin, 2004] Hervé Moulin. Fair division and collective welfare. MIT press, 2004.","title":"Fair division and collective welfare","authors":"Hervé Moulin","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Noothigattu et al., 2017] R. Noothigattu, S. Gaikwad, E. Awad, S. Dsouza, I. Rahwan, P. Ravikumar, and A. D. Procaccia. A voting-based system for ethical decision making. In Proc. 32nd AAAI, 2017.","title":"A voting-based system for ethical decision making","authors":"R Noothigattu, S Gaikwad, E Awad, S Dsouza, I Rahwan, P Ravikumar, A D Procaccia","year":"2017","venue":"Proc. 32nd AAAI","doi":null,"url":null,"internal_url":null},{"position":46,"raw":"[Plassmann and Tideman, 2014] F. Plassmann and T. N. Tideman. How frequently do different voting rules encounter voting para- doxes in three-candidate elections? Social Choice and Welfare, 42(1):31-75, 2014.","title":"How frequently do different voting rules encounter voting paradoxes in three-candidate elections?","authors":"Tideman Plassmann, F Plassmann, T N Tideman","year":"2014","venue":"Social Choice and Welfare","doi":null,"url":null,"internal_url":null},{"position":47,"raw":"[Popova et al., 2013] A. Popova, M. Regenwetter, and N. Mattei. A behavioral perspective on social choice. AMAI, 68(1-3):135- 160, 2013.","title":"A behavioral perspective on social choice","authors":"A Popova, M Regenwetter, N Mattei","year":"2013","venue":"AMAI","doi":null,"url":null,"internal_url":null},{"position":48,"raw":"[Rawls, 1971] J. Rawls. A Theory of Justice. Harvard University Press, 1971.","title":"A Theory of Justice","authors":"J Rawls","year":"1971","venue":null,"doi":null,"url":null,"internal_url":null},{"position":49,"raw":"[Regenwetter et al., 2007] M. Regenwetter, M.-H. R. Ho, and I. Tsetlin. Sophisticated approval voting, ignorance priors, and plurality heuristics: A behavioral social choice analysis in a Thurstonian framework. Psychological Review, 114(4):994- 1014, 2007.","title":"Sophisticated approval voting, ignorance priors, and plurality heuristics: A behavioral social choice analysis in a Thurstonian framework","authors":"M Regenwetter, M.-H R Ho, I Tsetlin","year":"2007","venue":"Psychological Review","doi":null,"url":null,"internal_url":null},{"position":50,"raw":"[Rossi and Mattei, 2019] F. Rossi and N. Mattei. Building ethically bounded AI. In Proc. 33rd AAAI, 2019.","title":"Building ethically bounded AI","authors":"F Rossi, N Mattei","year":"2019","venue":"Proc. 33rd AAAI","doi":null,"url":null,"internal_url":null},{"position":51,"raw":"[Roth et al., 2005] A. E. Roth, T. Sönmez, and M. U. Ünver. Pair- wise kidney exchange. Journal of Economic theory, 125(2):151- 188, 2005.","title":"Pairwise kidney exchange","authors":"A E Roth, T Sönmez, M U Ünver","year":"2005","venue":"Journal of Economic theory","doi":null,"url":null,"internal_url":null},{"position":52,"raw":"[Scheuerman et al., 2019] J. Scheuerman, J. L. Harman, N. Mattei, and K. B. Venable. Heuristic strategies in uncertain approval vot- ing environments. arXiv preprint arXiv:1912.00011, 2019.","title":"Heuristic strategies in uncertain approval voting environments","authors":"J Scheuerman, J L Harman, N Mattei, K B Venable","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":53,"raw":"[Sen, 2018] Amartya Sen. Collective Choice and Social Welfare. Harvard University Press, 2018.","title":"Collective Choice and Social Welfare","authors":"Amartya Sen","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":54,"raw":"[Shah et al., 2018] N. B. Shah, B. Tabibian, K. Muandet, I. Guyon, and U. Von Luxburg. Design and analysis of the NIPS 2016 re- view process. JMLR, 19(1):1913-1946, 2018.","title":"Design and analysis of the NIPS 2016 review process","authors":"N B Shah, B Tabibian, K Muandet, I Guyon, U Von Luxburg","year":"2018","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":55,"raw":"[Shoham and Leyton-Brown, 2008] Y. Shoham and K. Leyton- Brown. Multiagent Systems: Algorithmic, Game-theoretic, and Logical Foundations. Cambridge University Press, 2008.","title":"Multiagent Systems: Algorithmic, Game-theoretic, and Logical Foundations","authors":"Y Shoham, K Leyton-Brown","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":56,"raw":"[Skowron et al., 2015] P. Skowron, P. Faliszewski, and A. Slinko. Achieving fully proportional representation: Approximability re- sults. AI, 222:67-103, 2015.","title":"Achieving fully proportional representation: Approximability results","authors":"P Skowron, P Faliszewski, A Slinko","year":"2015","venue":"AI","doi":null,"url":null,"internal_url":null},{"position":57,"raw":"[Szufa et al., 2020] S. Szufa, P. Faliszewski, P. Skowron, A. Slinko, and N. Talmon. Drawing a map of elections in the space of sta- tistical cultures. In Proc. 19th AAMAS, pages 1341-1349, 2020.","title":"Drawing a map of elections in the space of statistical cultures","authors":"S Szufa, P Faliszewski, P Skowron, A Slinko, N Talmon","year":"2020","venue":"Proc. 19th AAMAS","doi":null,"url":null,"internal_url":null},{"position":58,"raw":"[Tambe, 2012] M. Tambe. Security and Game Theory -Algo- rithms, Deployed Systems, Lessons Learned. Cambridge Uni- versity Press, 2012.","title":"Security and Game Theory -Algorithms, Deployed Systems, Lessons Learned","authors":"M Tambe","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":59,"raw":"W. Thomson. Fair allocation rules. In Handbook of Social Choice and Welfare, volume 2, pages 393-506. Elsevier, 2011.","title":"Fair allocation rules","authors":"W Thomson","year":"2011","venue":"Handbook of Social Choice and Welfare","doi":null,"url":null,"internal_url":null},{"position":60,"raw":"[Walsh, 2011] T. Walsh. Where are the hard manipulation prob- lems? JAIR, 42:1-29, 2011.","title":"Where are the hard manipulation problems?","authors":"T Walsh","year":"2011","venue":"JAIR","doi":null,"url":null,"internal_url":null},{"position":61,"raw":"[Wang and Shah, 2019] J. Wang and N. B. Shah. Your 2 is my 1, your 3 is my 9: Handling arbitrary miscalibrations in ratings. In Proc. 18th AAMAS, pages 864-872, 2019.","title":"Your 2 is my 1, your 3 is my 9: Handling arbitrary miscalibrations in ratings","authors":"Wang Shah, ] J Wang, N B Shah","year":"2019","venue":"Proc. 18th AAMAS","doi":null,"url":null,"internal_url":null},{"position":62,"raw":"[Wooldridge, 2009] M. J. Wooldridge. An Introduction to MultiA- gent Systems, Second Edition. Wiley, 2009.","title":"An Introduction to MultiAgent Systems","authors":"M J Wooldridge","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":63,"raw":"[Xu et al., 2018] Y. Xu, H. Zhao, X. Shi, and N. B. Shah. On strategyproof conference peer review. arXiv preprint arXiv:1806.06266, 2018.","title":"On strategyproof conference peer review","authors":"Y Xu, H Zhao, X Shi, N B Shah","year":"2018","venue":null,"doi":null,"url":null,"internal_url":"/proceedings/2019/87"},{"position":64,"raw":"[Zhao et al., 2018] Z. Zhao, H. Li, J. Wang, J. O. Kephart, N. Mat- tei, H. Su, and L. Xia. A cost-effective framework for preference elicitation and aggregation. In Proc. 34th UAI, pages 446-456, 2018.","title":"A cost-effective framework for preference elicitation and aggregation","authors":"Z Zhao, H Li, J Wang, J O Kephart, N Mattei, H Su, L Xia","year":"2018","venue":"Proc. 34th UAI","doi":null,"url":null,"internal_url":null},{"position":65,"raw":"[Zou et al., 2015] J. Zou, R. Meir, and D. Parkes. Strategic voting behavior in Doodle polls. In Proc. 18th CSCW, pages 464-472, 2015.","title":"Strategic voting behavior in Doodle polls","authors":"J Zou, R Meir, D Parkes","year":"2015","venue":"Proc. 18th CSCW","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}