{"type":"ScholarlyArticle","title":"I Will Have Order! Optimizing Orders for Fair Reviewer Assignment","authors":[{"name":"Justin Payan","slug":"justin-payan","given":"Justin","family":"Payan","affiliation":"University of Massachusetts Amherst"},{"name":"Yair Zick","slug":"yair-zick","given":"Yair","family":"Zick","affiliation":"University of Massachusetts Amherst"}],"abstract":"We study mechanisms that allocate reviewers to papers in a fair and efficient manner. We model reviewer assignment as an instance of a fair allocation problem, presenting an extension of the classic round-robin mechanism, called Reviewer Round Robin (RRR). Round-robin mechanisms are a standard tool to ensure envy-free up to one item (EF1) allocations. However, fairness often comes at the cost of decreased efficiency. To overcome this challenge, we carefully select an approximately optimal round-robin order. Applying a relaxation of submodularity, γ-weak submodularity, we show that greedily inserting papers into an order yields a (1+γ²)-approximation to the maximum welfare attainable by our round-robin mechanism under any order. Our Greedy Reviewer Round Robin (GRRR) approach outputs highly efficient EF1 allocations for three real conference datasets, offering comparable performance to state-of-the-art paper assignment methods in fairness, efficiency, and runtime, while providing the only EF1 guarantee.","doi":"10.24963/ijcai.2022/63","url":"https://ijcai.org/proceedings/2022/63","pdf":"https://ijcai.org/proceedings/2022/0063.pdf","markdown":"https://ijcai.org/proceedings/2022/63.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/63","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":63,"pages":{"first":440,"last":446},"keywords":["Agent-based and Multi-agent Systems: Applications","Search: Combinatorial Search and Optimisation","Agent-based and Multi-agent Systems: Computational Social Choice","Agent-based and Multi-agent Systems: Resource Allocation"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987118","length_sec":84,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985261","length_sec":716,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Aziz et al., 2015] Haris Aziz, Toby Walsh, and Lirong Xia. Possible and necessary allocations via sequential mecha- nisms. In Proceedings of the 24th International Joint Con- ference on Artificial Intelligence (IJCAI), 2015.","title":"Possible and necessary allocations via sequential mechanisms","authors":"Haris Aziz, Toby Walsh, Lirong Xia","year":"2015","venue":"Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2015/72"},{"position":2,"raw":"[Aziz et al., 2016] Haris Aziz, Thomas Kalinowski, Toby Walsh, and Lirong Xia. Welfare of sequential alloca- tion mechanisms for indivisible goods. In Proceedings of the 22nd European Conference on Artificial Intelligence (ECAI), pages 787-794, 2016.","title":"Welfare of sequential allocation mechanisms for indivisible goods","authors":"Haris Aziz, Thomas Kalinowski, Toby Walsh, Lirong Xia","year":"2016","venue":"Proceedings of the 22nd European Conference on Artificial Intelligence (ECAI)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Aziz et al., 2019] Haris Aziz, Xin Huang, Nicholas Mattei, and Erel Segal-Halevi. The constrained round robin al- gorithm for fair and efficient allocation. arXiv preprint arXiv:1908.00161, 2019.","title":"The constrained round robin algorithm for fair and efficient allocation","authors":"Haris Aziz, Xin Huang, Nicholas Mattei, Erel Segal-Halevi","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Babaioff et al., 2021] Moshe Babaioff, Tomer Ezra, and Uriel Feige. Fair and truthful mechanisms for dichoto- mous valuations. In Proceedings of the 35th AAAI Confer- ence on Artificial Intelligence (AAAI), pages 5119-5126, 2021.","title":"Fair and truthful mechanisms for dichotomous valuations","authors":"Babaioff","year":"2021","venue":"Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Barman et al., 2019] Siddharth Barman, Ganesh Ghalme, Shweta Jain, Pooja Kulkarni, and Shivika Narang. Fair division of indivisible goods among strategic agents. In Proceedings of the 18th International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), pages 1811-1813, 2019.","title":"Fair division of indivisible goods among strategic agents","authors":"Siddharth Barman, Ganesh Ghalme, Shweta Jain, Pooja Kulkarni, Shivika Narang","year":"2019","venue":"Proceedings of the 18th International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Benabbou et al., 2020] Nawal Benabbou, Mithun Chakraborty, Ayumi Igarashi, and Yair Zick. Find- ing fair and efficient allocations when valuations don't add up. In Proceedings of the 13th International Sympo- sium on Algorithmic Game Theory (SAGT), pages 32-46, 2020.","title":"Finding fair and efficient allocations when valuations don't add up","authors":"Nawal Benabbou, Mithun Chakraborty, Ayumi Igarashi, Yair Zick","year":"2020","venue":"Proceedings of the 13th International Symposium on Algorithmic Game Theory (SAGT)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Biswas and Barman, 2018] Arpita Biswas and Siddharth Barman. Fair division under cardinality constraints. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI), pages 91-97, 2018.","title":"Fair division under cardinality constraints","authors":"Arpita Biswas, Siddharth Barman","year":"2018","venue":"Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2018/13"},{"position":8,"raw":"[Bouveret and Lang, 2011] Sylvain Bouveret and Jérôme Lang. A general elicitation-free protocol for allocating in- divisible goods. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI), pages 73-78, 2011.","title":"A general elicitation-free protocol for allocating indivisible goods","authors":"Lang ; Bouveret, Sylvain Bouveret, Jérôme Lang","year":"2011","venue":"Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2011/24"},{"position":9,"raw":"[Bouveret et al., 2016] Sylvain Bouveret, Yann Chevaleyre, and Nicolas Maudet. Fair allocation of indivisible goods. In Felix Brandt, Vincent Conitzer, Ulle Endriss, Jérôme Lang, and Ariel D Procaccia, editors, Handbook of com- putational social choice, chapter 12, pages 284-309. Cam- bridge University Press, 2016.","title":"Fair allocation of indivisible goods","authors":"Sylvain Bouveret, Yann Chevaleyre, Nicolas Maudet","year":"2016","venue":"Handbook of computational social choice","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Buchbinder et al., 2012] Niv Buchbinder, Moran Feldman, Joseph Naor, and Roy Schwartz. A tight linear time (1/2)- approximation for unconstrained submodular maximiza- tion. In Proceedings of the 53rd Symposium on Founda- tions of Computer Science (FOCS), pages 649-658, 2012.","title":"A tight linear time (1/2)approximation for unconstrained submodular maximization","authors":"Buchbinder","year":"2012","venue":"Proceedings of the 53rd Symposium on Foundations of Computer Science (FOCS)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Budish, 2011] Eric Budish. The combinatorial assign- ment problem: Approximate competitive equilibrium from equal incomes. Journal of Political Economy, 119:1061- 1061, 12 2011.","title":"The combinatorial assignment problem: Approximate competitive equilibrium from equal incomes","authors":"Eric Budish","year":"2011","venue":"Journal of Political Economy","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Caragiannis et al., 2019] Ioannis Caragiannis, David Kurokawa, Hervé Moulin, Ariel D Procaccia, Nisarg Shah, and Junxing Wang. The unreasonable fairness of maximum Nash welfare. ACM Transactions on Economics and Computation (TEAC), 7(3):1-32, 2019.","title":"The unreasonable fairness of maximum Nash welfare","authors":"Ioannis Caragiannis, David Kurokawa, Hervé Moulin, Ariel D Procaccia, Nisarg Shah, Junxing Wang","year":"2019","venue":"ACM Transactions on Economics and Computation (TEAC)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Charlin and Zemel, 2013] Laurent Charlin and Richard Zemel. The toronto paper matching system: an automated paper-reviewer assignment system. In Proceedings of the 2013 ICML Workshop on Peer Reviewing and Publishing Models, 2013.","title":"The toronto paper matching system: an automated paper-reviewer assignment system","authors":"Zemel Charlin, Laurent Charlin, Richard Zemel","year":"2013","venue":"Proceedings of the 2013 ICML Workshop on Peer Reviewing and Publishing Models","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Das and Kempe, 2011] Abhimanyu Das and David Kempe. Submodular meets spectral: greedy algorithms for subset selection, sparse approximation and dictionary selection. In Proceedings of the 28th International Conference on Machine Learning (ICML), pages 1057-1064, 2011.","title":"Abhimanyu Das and David Kempe. Submodular meets spectral: greedy algorithms for subset selection, sparse approximation and dictionary selection","authors":"Kempe Das","year":"2011","venue":"Proceedings of the 28th International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Dror et al., 2021] Amitay Dror, Michal Feldman, and Erel Segal-Halevi. On fair division under heterogeneous ma- troid constraints. In Proceedings of the 35th AAAI Confer- ence on Artificial Intelligence (AAAI), pages 5312-5320, 2021.","title":"On fair division under heterogeneous matroid constraints","authors":"Amitay Dror, Michal Feldman, Erel Segal-Halevi","year":"2021","venue":"Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Fisher et al., 1978] Marshall Fisher, George Nemhauser, and Laurence Wolsey. An analysis of approximations for maximizing submodular set functions-II. In M. L. Balin- ski and A. J. Hoffman, editors, Polyhedral Combinatorics, pages 73-87. Springer Berlin Heidelberg, Berlin, Heidel- berg, 1978.","title":"An analysis of approximations for maximizing submodular set functions-II","authors":"Fisher","year":"1978","venue":"Polyhedral Combinatorics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Gölz and Procaccia, 2019] Paul Gölz and Ariel D Procac- cia. Migration as submodular optimization. In Proceed- ings of the 33rd AAAI Conference on Artificial Intelligence (AAAI), pages 549-556, 2019.","title":"Paul Gölz and Ariel D Procaccia. Migration as submodular optimization","authors":"Procaccia Gölz","year":"2019","venue":"Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kalinowski et al., 2013] Thomas Kalinowski, Nina Naro- dytska, and Toby Walsh. A social welfare optimal sequen- tial allocation procedure. In Proceedings of the 23rd In- ternational Joint Conference on Artificial Intelligence (IJ- CAI), pages 227-233, 2013.","title":"A social welfare optimal sequential allocation procedure","authors":"Thomas Kalinowski, Nina Narodytska, Toby Walsh","year":"2013","venue":"Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJ-CAI)","doi":null,"url":null,"internal_url":"/proceedings/2013/43"},{"position":19,"raw":"[Kobren et al., 2019] Ari Kobren, Barna Saha, and Andrew McCallum. Paper matching with local fairness constraints. In Proceedings of the 25th International Conference on Knowledge Discovery and Data Mining (KDD), pages 1247-1257, 2019.","title":"Paper matching with local fairness constraints","authors":"Kobren","year":"2019","venue":"Proceedings of the 25th International Conference on Knowledge Discovery and Data Mining (KDD)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lipton et al., 2004] Richard J Lipton, Evangelos Markakis, Elchanan Mossel, and Amin Saberi. On approximately fair allocations of indivisible goods. In Proceedings of the 5th ACM Conference on Electronic Commerce (EC), pages 125-131, 2004.","title":"On approximately fair allocations of indivisible goods","authors":"Evangelos Richard J Lipton, Elchanan Markakis, Amin Mossel, Saberi","year":"2004","venue":"Proceedings of the 5th ACM Conference on Electronic Commerce (EC)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mulligan et al., 2013] Adrian Mulligan, Louise Hall, and Ellen Raphael. Peer review in a changing world: An in- ternational study measuring the attitudes of researchers. Journal of the American Society for Information Science and Technology, 64(1):132-161, 2013.","title":"Peer review in a changing world: An international study measuring the attitudes of researchers","authors":"Adrian Mulligan, Louise Hall, Ellen Raphael","year":"2013","venue":"Journal of the American Society for Information Science and Technology","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Oxley, 2011] James Oxley. Matroid Theory. Oxford Grad- uate Texts in Mathematics. Oxford Univerity Press, 2nd edition, 2011.","title":"Matroid Theory","authors":"James Oxley","year":"2011","venue":"Oxford Graduate Texts in Mathematics","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Payan and Zick, 2021] Justin Payan and Yair Zick. I will have order! Optimizing orders for fair reviewer assign- ment. arXiv:2108.02126, 2021.","title":"I will have order! Optimizing orders for fair reviewer assignment","authors":"Zick Payan, Justin Payan, Yair Zick","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Shah, 2019] Nihar B Shah. Principled methods to improve peer review. Technical report, Carnegie Mellon University, 2019.","title":"Principled methods to improve peer review","authors":"B Nihar, Shah","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Stelmakh et al., 2019] Ivan Stelmakh, Nihar B Shah, and Aarti Singh. PeerReview4All: Fair and accurate reviewer assignment in peer review. In Proceedings of the 30th In- ternational Conference on Algorithmic Learning Theory (ALT), pages 828-856, 2019.","title":"PeerReview4All: Fair and accurate reviewer assignment in peer review","authors":"Ivan Stelmakh, B Nihar, Aarti Shah, Singh","year":"2019","venue":"Proceedings of the 30th International Conference on Algorithmic Learning Theory (ALT)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Stelmakh, 2021] Ivan Stelmakh. Towards fair, equitable, and efficient peer review. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI), pages 15736-15737, 2021.","title":"Towards fair, equitable, and efficient peer review","authors":"Ivan Stelmakh","year":"2021","venue":"Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}