{"type":"ScholarlyArticle","title":"The Price of Local Fairness in Multistage Selection","authors":[{"name":"Vitalii Emelianov","slug":"vitalii-emelianov","given":"Vitalii","family":"Emelianov","affiliation":"Univ. Grenoble Alpes , Inria , CNRS , Grenoble INP , LIG"},{"name":"George Arvanitakis","slug":"george-arvanitakis","given":"George","family":"Arvanitakis","affiliation":"Univ. Grenoble Alpes , Inria , CNRS , Grenoble INP , LIG"},{"name":"Nicolas Gast","slug":"nicolas-gast","given":"Nicolas","family":"Gast","affiliation":"Univ. Grenoble Alpes , Inria , CNRS , Grenoble INP , LIG"},{"name":"Krishna Gummadi","slug":"krishna-gummadi","given":"Krishna","family":"Gummadi","affiliation":"Max Planck Institute for Software Systems"},{"name":"Patrick Loiseau","slug":"patrick-loiseau","given":"Patrick","family":"Loiseau","affiliation":"Univ. Grenoble Alpes , Inria , CNRS , Grenoble INP , LIG; Max Planck Institute for Software Systems"}],"abstract":"The rise of algorithmic decision making led to active researches on how to define and guarantee fairness, mostly focusing on one-shot decision making. In several important applications such as hiring, however, decisions are made in multiple stage with additional information at each stage. In such cases, fairness issues remain poorly understood. In this paper we study fairness in k-stage selection problems where additional features are observed at every stage. We first introduce two fairness notions, local (per stage) and global (final stage) fairness, that extend the classical fairness notions to the k-stage setting. We propose a simple model based on a probabilistic formulation and show that the locally and globally fair selections that maximize precision can be computed via a linear program. We then define the price of local fairness to measure the loss of precision induced by local constraints; and investigate theoretically and empirically this quantity. In particular, our experiments show that the price of local fairness is generally smaller when the sensitive attribute is observed at the first stage; but globally fair selections are more locally fair when the sensitive attribute is observed at the second stage – hence in both cases it is often possible to have a selection that has a small price of local fairness and is close to locally fair.","doi":"10.24963/ijcai.2019/809","url":"https://ijcai.org/proceedings/2019/809","pdf":"https://ijcai.org/proceedings/2019/0809.pdf","markdown":"https://ijcai.org/proceedings/2019/809.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/809","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"AI for Improving Human Well-being","track":"AI for Improving Human Well-being","number":809,"pages":{"first":5836,"last":5842},"keywords":["Special Track on AI for Improving Human-Well Being: AI ethics (Special Track on AI and Human Wellbeing)"],"videos":[],"references":[{"position":1,"raw":"[Bower et al., 2017] Amanda Bower, Sarah N. Kitchen, Laura Niss, Martin J. Strauss, Alex Vargo, and Suresh Venkatasubramanian. Fair pipelines. In Proceedings of the 4th Workshop on Fairness, Accountability, and Trans- parency in Machine Learning (FAT-ML), 2017.","title":"Fair pipelines","authors":"Amanda Bower, Sarah N Kitchen, Laura Niss, Martin J Strauss, Alex Vargo, Suresh Venkatasubramanian","year":"2017","venue":"Proceedings of the 4th Workshop on Fairness, Accountability, and Transparency in Machine Learning (FAT-ML)","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chouldechova, 2017] Alexandra Chouldechova. Fair pre- diction with disparate impact: A study of bias in recidi- vism prediction instruments. Big Data, 5(2):153-163, 2017.","title":"Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments","authors":"Chouldechova","year":"2017","venue":"Big Data","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Corbett-Davies et al., 2017] Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorith- mic decision making and the cost of fairness. In Proceed- ings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 797-806, 2017.","title":"Algorithmic decision making and the cost of fairness","authors":"Corbett-Davies","year":"2017","venue":"Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Dua and Graff, 2017] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.","title":"Dheeru Dua and Casey Graff. UCI machine learning repository","authors":"Graff Dua","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Dwork and Ilvento, 2019] Cynthia Dwork and Christina Il- vento. Fairness under composition. In Proceedings of the 10th conference on Innovations in Theoretical Computer Science (ITCS), pages 33:1-33:20, 2019.","title":"Fairness under composition","authors":"Cynthia Dwork, Christina Ilvento","year":"2019","venue":"Proceedings of the 10th conference on Innovations in Theoretical Computer Science (ITCS)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dwork et al., 2012] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd conference on Innovations in Theoretical Computer Science Confer- ence (ITCS), pages 214-226, 2012.","title":"Fairness through awareness","authors":"Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, Richard Zemel","year":"2012","venue":"Proceedings of the 3rd conference on Innovations in Theoretical Computer Science Conference (ITCS)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hardt et al., 2016] Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity in supervised learning. In Proceedings of the 30th International Conference on Neu- ral Information Processing Systems (NIPS), pages 3323- 3331, 2016.","title":"Equality of opportunity in supervised learning","authors":"Moritz Hardt, Eric Price, Nathan Srebro","year":"2016","venue":"Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Heidari and Krause, 2018] Hoda Heidari and Andreas Krause. Preventing disparate treatment in sequential decision making. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI), pages 2248-2254, 2018.","title":"Preventing disparate treatment in sequential decision making","authors":"Krause ; ] Hoda Heidari, Andreas Heidari, Krause","year":"2018","venue":"Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2018/311"},{"position":9,"raw":"[Jabbari et al., 2017] Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, and Aaron Roth. Fairness in reinforcement learning. In Proceedings of the 34th International Conference on Machine Learning (ICML), pages 1617-1626, 2017.","title":"Fairness in reinforcement learning","authors":"Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, Aaron Roth","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning (ICML)","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Joseph et al., 2016] Matthew Joseph, Michael Kearns, Jamie Morgenstern, and Aaron Roth. Fairness in learning: Classic and contextual bandits. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS), pages 325-333, 2016.","title":"Fairness in learning: Classic and contextual bandits","authors":"Matthew Joseph, Michael Kearns, Jamie Morgenstern, Aaron Roth","year":"2016","venue":"Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kilbertus et al., 2017] Niki Kilbertus, Mateo Rojas Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janz- ing, and Bernhard Schölkopf. Avoiding discrimination through causal reasoning. In Proceedings of the 31st In- ternational Conference on Neural Information Processing Systems (NIPS), pages 656-666, 2017.","title":"Avoiding discrimination through causal reasoning","authors":"Niki Kilbertus, Mateo Rojas Carulla, Giambattista Parascandolo, Moritz Hardt, Dominik Janzing, Bernhard Schölkopf","year":"2017","venue":"Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kleinberg and Raghavan, 2018] Jon Kleinberg and Manish Raghavan. Selection problems in the presence of implicit bias. In Proceedings of the 9th conference on Innova- tions in Theoretical Computer Science (ITCS), pages 33:1- 33:17, 2018.","title":"Selection problems in the presence of implicit bias","authors":"Raghavan Kleinberg, Jon Kleinberg, Manish Raghavan","year":"2018","venue":"Proceedings of the 9th conference on Innovations in Theoretical Computer Science (ITCS)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kleinberg et al., 2017] Jon Kleinberg, Sendhil Mul- lainathan, and Manish Raghavan. Inherent trade-offs in the fair determination of risk scores. In Proceedings of the 8th conference on Innovations in Theoretical Computer Science (ITCS), pages 43:1-43:23, 2017.","title":"Inherent trade-offs in the fair determination of risk scores","authors":"Kleinberg","year":"2017","venue":"Proceedings of the 8th conference on Innovations in Theoretical Computer Science (ITCS)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lambrecht and Tucker, 2018] Anja Lambrecht and E. Tucker, Catherine. Algorithmic bias? an empiri- cal study into apparent gender-based discrimination in the display of stem career ads, March 2018. Available at SSRN: https://ssrn.com/abstract=2852260.","title":"Algorithmic bias? an empirical study into apparent gender-based discrimination in the display of stem career ads","authors":"Anja Lambrecht, E Tucker, Catherine","year":"2018","venue":null,"doi":null,"url":"https://ssrn.com/abstract=2852260","internal_url":null},{"position":15,"raw":"[Larson et al., 2016] Jeff Larson, Surya Mattu, Lau- ren Kirchner, and Julia Angwin. How We Ana- lyzed the COMPAS Recidivism Algorithm, 2016.","title":"How We Analyzed the COMPAS Recidivism Algorithm","authors":"Jeff Larson, Surya Mattu, Lauren Kirchner, Julia Angwin","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lipton et al., 2018] Zachary Lipton, Julian McAuley, and Alexandra Chouldechova. Does mitigating ml's impact disparity require treatment disparity? In Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS), pages 8125-8135, 2018.","title":"Does mitigating ml's impact disparity require treatment disparity?","authors":"Zachary Lipton, Julian Mcauley, Alexandra Chouldechova","year":"2018","venue":"Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pedreshi et al., 2008] Dino Pedreshi, Salvatore Ruggieri, and Franco Turini. Discrimination-aware data mining. In Proceedings of the 14th ACM SIGKDD International Con- ference on Knowledge Discovery and Data Mining (KDD), pages 560-568, 2008.","title":"Discrimination-aware data mining","authors":"Dino Pedreshi, Salvatore Ruggieri, Franco Turini","year":"2008","venue":"Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Perry et al., 2013] Walter L. Perry, Brian McInnis, Carter C. Price, Susan C. Smith, and John S. Hollywood. Predictive Policing: The Role of Crime Forecasting in Law Enforce- ment Operations. Rand Corporation, 2013.","title":"Predictive Policing: The Role of Crime Forecasting in Law Enforcement Operations","authors":"L Walter, Brian Perry, Carter C Mcinnis, Susan C Price, John S Smith, Hollywood","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Schumann et al., 2019] Candice Schumann, Samsara N. Counts, Jeffrey S. Foster, and John P. Dickerson. The di- verse cohort selection problem. In Proceedings of the In- ternational Conference on Autonomous Agents and Multi- Agent Systems (AAMAS), pages 601-609, 2019.","title":"The diverse cohort selection problem","authors":"Candice Schumann, N Samsara, Jeffrey S Counts, John P Foster, Dickerson","year":"2019","venue":"Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Senator, 2005] Ted E. Senator. Multi-stage classification. In Proceedings of the Fifth IEEE International Conference on Data Mining (ICDM), pages 386-393, 2005.","title":"Multi-stage classification","authors":"Ted E Senator","year":"2005","venue":"Proceedings of the Fifth IEEE International Conference on Data Mining (ICDM)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Trapeznikov et al., 2012] Kirill Trapeznikov, Venkatesh Saligrama, and David Castañón. Multi-stage classifier design. In Proceedings of the Asian Conference on Machine Learning, pages 459-474, 2012.","title":"Multi-stage classifier design","authors":"Kirill Trapeznikov, Venkatesh Saligrama, David Castañón","year":"2012","venue":"Proceedings of the Asian Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Valera et al., 2018] Isabel Valera, Adish Singla, and Manuel Gomez Rodriguez. Enhancing the accuracy and fairness of human decision making. In Proceedings of the 32nd In- ternational Conference on Neural Information Processing Systems (NIPS), pages 1774-1783, 2018.","title":"Enhancing the accuracy and fairness of human decision making","authors":"Isabel Valera, Adish Singla, Manuel Gomez Rodriguez","year":"2018","venue":"Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zafar et al., 2017] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th International Conference on World Wide Web (WWW), pages 1171-1180, 2017.","title":"Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment","authors":"Zafar","year":"2017","venue":"Proceedings of the 26th International Conference on World Wide Web (WWW)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}