{"type":"ScholarlyArticle","title":"mdfa: Multi-Differential Fairness Auditor for Black Box Classifiers","authors":[{"name":"Xavier Gitiaux","slug":"xavier-gitiaux","given":"Xavier","family":"Gitiaux","affiliation":"University"},{"name":"Huzefa Rangwala","slug":"huzefa-rangwala","given":"Huzefa","family":"Rangwala","affiliation":"University"}],"abstract":"Machine learning algorithms are increasingly involved in sensitive decision-making processes with adversarial implications on individuals. This paper presents a new tool,  mdfa that identifies the characteristics of the victims of a classifier's discrimination. We measure discrimination as a violation of multi-differential fairness.  Multi-differential fairness is a guarantee that a black box classifier's outcomes do not leak information on the sensitive attributes of a small group of individuals. We reduce the problem of identifying worst-case violations to matching distributions and predicting where sensitive attributes and classifier's outcomes coincide. We apply mdfa to a recidivism risk assessment classifier widely used in the United States and demonstrate that for individuals with little criminal history, identified African-Americans  are three-times more likely to be considered at high risk of violent recidivism than similar non-African-Americans.","doi":"10.24963/ijcai.2019/814","url":"https://ijcai.org/proceedings/2019/814","pdf":"https://ijcai.org/proceedings/2019/0814.pdf","markdown":"https://ijcai.org/proceedings/2019/814.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/814","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"AI for Improving Human Well-being","track":"AI for Improving Human Well-being","number":814,"pages":{"first":5871,"last":5879},"keywords":["Special Track on AI for Improving Human-Well Being: AI ethics (Special Track on AI and Human Wellbeing)","Special Track on AI for Improving Human-Well Being: Societal applications (Special Track on AI and Human Wellbeing)","Special Track on AI for Improving Human-Well Being: Well being metrics (Special Track on AI and Human Wellbeing)"],"videos":[],"references":[{"position":1,"raw":"[Agarwal et al., 2018] Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. A reductions approach to fair classification. arXiv preprint arXiv:1803.02453, 2018.","title":"A reductions approach to fair classification","authors":"Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, Hanna Wallach","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Atlantic, 2016] The Atlantic. How algorithms can bring down minorities credit scores? The Atlantic, 2016. https://www.theatlantic.com/technology/archive/2016/12/ how-algorithms-can-bring-down-minorities-credit-scores/ 509333/.","title":"The Atlantic. How algorithms can bring down minorities credit scores? The Atlantic","authors":null,"year":"2016","venue":null,"doi":null,"url":"https://www.theatlantic.com/technology/archive/2016/12/how-algorithms-can-bring-down-minorities-credit-scores/509333/","internal_url":null},{"position":3,"raw":"[Calsamiglia, 2009] Caterina Calsamiglia. Decentralizing equality of opportunity. International Economic Review, 50(1):273-290, 2009.","title":"Decentralizing equality of opportunity","authors":"Caterina Calsamiglia","year":"2009","venue":"International Economic Review","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chouldechova and Roth, 2018] Alexandra Chouldechova and Aaron Roth. The frontiers of fairness in machine learning. arXiv preprint arXiv:1810.08810, 2018.","title":"The frontiers of fairness in machine learning","authors":"Roth Chouldechova","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cortes et al., 2008] Corinna Cortes, Mehryar Mohri, Michael Riley, and Afshin Rostamizadeh. Sample selec- tion bias correction theory. In International Conference on Algorithmic Learning Theory, pages 38-53. Springer, 2008.","title":"Sample selection bias correction theory","authors":"Corinna Cortes, Mehryar Mohri, Michael Riley, Afshin Rostamizadeh","year":"2008","venue":"International Conference on Algorithmic Learning Theory","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cortes et al., 2010] Corinna Cortes, Yishay Mansour, and Mehryar Mohri. Learning bounds for importance weight- ing. In Advances in neural information processing sys- tems, pages 442-450, 2010.","title":"Learning bounds for importance weighting","authors":"Corinna Cortes, Yishay Mansour, Mehryar Mohri","year":"2010","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dwork et al., 2012] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214- 226. ACM, 2012.","title":"Fairness through awareness","authors":"Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, Richard Zemel","year":"2012","venue":"Proceedings of the 3rd innovations in theoretical computer science conference","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dwork et al., 2014] Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Founda- tions and Trends R in Theoretical Computer Science, 9(3- 4):211-407, 2014.","title":"The algorithmic foundations of differential privacy","authors":"Cynthia Dwork, Aaron Roth","year":"2014","venue":"Foundations and Trends R in Theoretical Computer Science","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Feldman et al., 2012] Vitaly Feldman, Venkatesan Gu- ruswami, Prasad Raghavendra, and Yi Wu. Agnostic learn- ing of monomials by halfspaces is hard. SIAM Journal on Computing, 41(6):1558-1590, 2012.","title":"Agnostic learning of monomials by halfspaces is hard","authors":"Venkatesan Vitaly Feldman, Prasad Guruswami, Yi Raghavendra, Wu","year":"2012","venue":"SIAM Journal on Computing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Feldman et al., 2015] Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkata- subramanian. Certifying and removing disparate impact. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 259-268. ACM, 2015.","title":"Certifying and removing disparate impact","authors":"Michael Feldman, A Sorelle, John Friedler, Carlos Moeller, Suresh Scheidegger, Venkatasubramanian","year":"2015","venue":"Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Foulds and Pan, 2018] James Foulds and Shimei Pan. An intersectional definition of fairness. arXiv preprint arXiv:1807.08362, 2018.","title":"An intersectional definition of fairness","authors":"Pan Foulds, James Foulds, Shimei Pan","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Friedler et al., 2018] Sorelle A Friedler, Carlos Scheideg- ger, Suresh Venkatasubramanian, Sonam Choudhary, Evan P Hamilton, and Derek Roth. A comparative study of fairness-enhancing interventions in machine learning. arXiv preprint arXiv:1802.04422, 2018.","title":"A comparative study of fairness-enhancing interventions in machine learning","authors":"A Sorelle, Carlos Friedler, Suresh Scheidegger, Sonam Venkatasubramanian, Evan P Choudhary, Derek Hamilton, Roth","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Gretton et al., 2009] Arthur Gretton, Alex J Smola, Jiayuan Huang, Marcel Schmittfull, Karsten M Borgwardt, and Bernhard Schöllkopf. Covariate shift by kernel mean matching. In Dataset shift in machine learning, pages 131-160. MIT Press, 2009.","title":"Covariate shift by kernel mean matching","authors":"Arthur Gretton, Alex J Smola, Jiayuan Huang, Marcel Schmittfull, Karsten M Borgwardt, Bernhard Schöllkopf","year":"2009","venue":"Dataset shift in machine learning","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hébert-Johnson et al., 2017] Ursula Hébert-Johnson, Michael P Kim, Omer Reingold, and Guy N Rothblum. Calibration for the (computationally-identifiable) masses. arXiv preprint arXiv:1711.08513, 2017.","title":"Calibration for the (computationally-identifiable) masses","authors":"Ursula Hébert-Johnson, Michael P Kim, Omer Reingold, Guy N Rothblum","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jagielski et al., 2018] Matthew Jagielski, Michael Kearns, Jieming Mao, Alina Oprea, Aaron Roth, Saeed Sharifi- Malvajerdi, and Jonathan Ullman. Differentially private fair learning. arXiv preprint arXiv:1812.02696, 2018.","title":"Differentially private fair learning","authors":"Matthew Jagielski, Michael Kearns, Jieming Mao, Alina Oprea, Aaron Roth, Saeed Sharifi-Malvajerdi, Jonathan Ullman","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Johansson et al., 2016] Fredrik Johansson, Uri Shalit, and David Sontag. Learning representations for counterfactual inference. In International conference on machine learn- ing, pages 3020-3029, 2016.","title":"Learning representations for counterfactual inference","authors":"Johansson","year":"2016","venue":"International conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kearns et al., 2017] Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. Preventing fairness gerry- mandering: Auditing and learning for subgroup fairness. arXiv preprint arXiv:1711.05144, 2017.","title":"Preventing fairness gerrymandering: Auditing and learning for subgroup fairness","authors":"Michael Kearns, Seth Neel, Aaron Roth, Zhiwei Steven Wu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kearns et al., 2018] Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. An empirical study of rich subgroup fairness for machine learning. arXiv preprint arXiv:1808.08166, 2018.","title":"An empirical study of rich subgroup fairness for machine learning","authors":"Michael Kearns, Seth Neel, Aaron Roth, Zhiwei Steven Wu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Kim et al., 2018] Michael P Kim, Omer Reingold, and Guy N Rothblum. Fairness through computationally- bounded awareness. arXiv preprint arXiv:1803.03239, 2018.","title":"Fairness through computationallybounded awareness","authors":"Omer Michael P Kim, Guy N Reingold, Rothblum","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Loftus et al., 2018] Joshua R Loftus, Chris Russell, Matt J Kusner, and Ricardo Silva. Causal reasoning for algorith- mic fairness. arXiv preprint arXiv:1805.05859, 2018.","title":"Causal reasoning for algorithmic fairness","authors":"Chris Joshua R Loftus, Matt J Russell, Ricardo Kusner, Silva","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mansour et al., 2009] Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Domain adaptation: Learning bounds and algorithms. arXiv preprint arXiv:0902.3430, 2009.","title":"Domain adaptation: Learning bounds and algorithms","authors":"Yishay Mansour, Mehryar Mohri, Afshin Rostamizadeh","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[ProPublica, 2016] ProPublica. How we analyzed the com- pas recidivism algorithm. ProPublica, 2016.","title":"ProPublica. How we analyzed the compas recidivism algorithm","authors":"Propublica","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rosenbaum and Rubin, 1983] Paul R Rosenbaum and Don- ald B Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41-55, 1983.","title":"The central role of the propensity score in observational studies for causal effects","authors":"R Paul, Donald B Rosenbaum, Rubin","year":"1983","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zafar et al., 2017] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th International Conference on World Wide Web, pages 1171-1180. International World Wide Web Conferences Steering Committee, 2017.","title":"Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment","authors":"Zafar","year":"2017","venue":"Proceedings of the 26th International Conference on World Wide Web","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}