{"type":"ScholarlyArticle","title":"Delayed Impact of Fair Machine Learning","authors":[{"name":"Lydia T. Liu","slug":"lydia-t-liu","given":"Lydia T.","family":"Liu","affiliation":"Department of Electrical Engineering and Computer Sciences , University of California at Berkeley"},{"name":"Sarah Dean","slug":"sarah-dean","given":"Sarah","family":"Dean","affiliation":"Department of Electrical Engineering and Computer Sciences , University of California at Berkeley"},{"name":"Esther Rolf","slug":"esther-rolf","given":"Esther","family":"Rolf","affiliation":"Department of Electrical Engineering and Computer Sciences , University of California at Berkeley"},{"name":"Max Simchowitz","slug":"max-simchowitz","given":"Max","family":"Simchowitz","affiliation":"Department of Electrical Engineering and Computer Sciences , University of California at Berkeley"},{"name":"Moritz Hardt","slug":"moritz-hardt","given":"Moritz","family":"Hardt","affiliation":"Department of Electrical Engineering and Computer Sciences , University of California at Berkeley"}],"abstract":"Static classification has been the predominant focus of the study of fairness in machine learning. While most models do not consider how decisions change populations over time, it is conventional wisdom that fairness criteria promote the long-term well-being of groups they aim to protect. This work studies the interaction of static fairness criteria with temporal indicators of well-being. We show a simple one-step feedback model in which common criteria do not generally promote improvement over time, and may in fact cause harm. Our results highlight the importance of temporal modeling in the evaluation of fairness criteria, suggesting a range of new challenges and trade-offs.","doi":"10.24963/ijcai.2019/862","url":"https://ijcai.org/proceedings/2019/862","pdf":"https://ijcai.org/proceedings/2019/0862.pdf","markdown":"https://ijcai.org/proceedings/2019/862.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/862","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Best Sister Conferences","track":"Best Sister Conferences","number":862,"pages":{"first":6196,"last":6200},"keywords":["Machine Learning: Learning Theory","Machine Learning: Interpretability","Humans and AI: Ethical Issues in AI","Machine Learning: Classification"],"videos":[],"references":[{"position":1,"raw":"[Barocas and Selbst, 2016] Solon Barocas and Andrew D. Selbst. Big data's disparate impact. California Law Re- view, 104, 2016.","title":"Big data's disparate impact","authors":"Selbst Barocas, Solon Barocas, Andrew D Selbst","year":"2016","venue":"California Law Review","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Calders et al., 2009] Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy. Building classifiers with indepen- dency constraints. In Proc. IEEE ICDMW, ICDMW '09, pages 13-18, 2009.","title":"Building classifiers with independency constraints","authors":"Calders","year":"2009","venue":"Proc. IEEE ICDMW, ICDMW '09","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chouldechova, 2016] Alexandra Chouldechova. Fair pre- diction with disparate impact: A study of bias in recidi- vism prediction instruments. FATML, 2016.","title":"Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments","authors":"Chouldechova","year":"2016","venue":"FATML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Coate and Loury, 1993] Stephen Coate and Glenn Loury. Will affirmative-action policies eliminate negative stereo- types? 83:1220-40, 02 1993.","title":"Will affirmative-action policies eliminate negative stereotypes","authors":"Loury Coate, Stephen Coate, Glenn Loury","year":"1993","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ensign et al., 2017] Danielle Ensign, Sorelle A Friedler, Scott Neville, Carlos Scheidegger, and Suresh Venkata- subramanian. Runaway feedback loops in predictive polic- ing. arXiv preprint arXiv:1706.09847, 2017.","title":"Runaway feedback loops in predictive policing","authors":"Danielle Ensign, A Sorelle, Scott Friedler, Carlos Neville, Suresh Scheidegger, Venkatasubramanian","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Executive Office of the President, 2016] Executive Office of the President. Big data: A report on algorithmic systems, opportunity, and civil rights. Technical report, White House, May 2016.","title":"Executive Office of the President. Big data: A report on algorithmic systems, opportunity, and civil rights","authors":null,"year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Foster and Vohra, 1992] Dean P Foster and Rakesh V Vohra. An economic argument for affirmative action. Ra- tionality and Society, 4(2):176-188, 1992.","title":"An economic argument for affirmative action","authors":"P Dean, Foster, V Rakesh, Vohra","year":"1992","venue":"Rationality and Society","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fuster et al., 2017] Andreas Fuster, Paul Goldsmith- Pinkham, Tarun Ramadorai, and Ansgar Walther. Predictably unequal? the effects of machine learning on credit markets. SSRN, 2017.","title":"Predictably unequal? the effects of machine learning on credit markets","authors":"Andreas Fuster, Paul Goldsmith-Pinkham, Tarun Ramadorai, Ansgar Walther","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Green and Hu, 2018] Ben Green and Lily Hu. The myth in the methodology: Towards a recontextualization of fair- ness in machine learning. Stockholm, Sweden, 2018.","title":"The myth in the methodology: Towards a recontextualization of fairness in machine learning","authors":"Hu Green, Ben Green, Lily Hu","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Hardt et al., 2016] Moritz Hardt, Eric Price, and Nati Sre- bro. Equality of opportunity in supervised learning. In Proc. 30th NIPS, 2016.","title":"Equality of opportunity in supervised learning","authors":"Moritz Hardt, Eric Price, Nati Srebro","year":"2016","venue":"Proc. 30th NIPS","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hu and Chen, 2018] Lily Hu and Yiling Chen. A short-term intervention for long-term fairness in the labor market. In Proc. 27th WWW, 2018.","title":"A short-term intervention for long-term fairness in the labor market","authors":"Chen ; Hu, Lily Hu, Yiling Chen","year":"2018","venue":"Proc. 27th WWW","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Joseph et al., 2016] Matthew Joseph, Michael Kearns, Jamie H Morgenstern, and Aaron Roth. Fairness in learning: Classic and contextual bandits. In Proc. 30th NIPS, pages 325-333, 2016.","title":"Fairness in learning: Classic and contextual bandits","authors":"Matthew Joseph, Michael Kearns, Jamie H Morgenstern, Aaron Roth","year":"2016","venue":"Proc. 30th NIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kalev et al., 2006] Alexandra Kalev, Frank Dobbin, and Erin Kelly. Best Practices or Best Guesses? Assessing the Efficacy of Corporate Affirmative Action and Diversity Policies. American Sociological Review, 71(4):589-617, 2006.","title":"Best Practices or Best Guesses? Assessing the Efficacy of Corporate Affirmative Action and Diversity Policies","authors":"Alexandra Kalev, Frank Dobbin, Erin Kelly","year":"2006","venue":"American Sociological Review","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Keith et al., 1985] Stephen N. Keith, Robert M. Bell, Au- gust G. Swanson, and Albert P. Williams. Effects of affir- mative action in medical schools. New England Journal of Medicine, 313(24):1519-1525, 1985.","title":"Effects of affirmative action in medical schools","authors":"Stephen N Keith, Robert M Bell, August G Swanson, Albert P Williams","year":"1985","venue":"New England Journal of Medicine","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kleinberg et al., 2017] Jon M. Kleinberg, Sendhil Mul- lainathan, and Manish Raghavan. Inherent trade-offs in the fair determination of risk scores. Proc. 8th ITCS, 2017.","title":"Inherent trade-offs in the fair determination of risk scores","authors":"Jon M Kleinberg, Sendhil Mullainathan, Manish Raghavan","year":"2017","venue":"Proc. 8th ITCS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Knowles et al., 2001] John Knowles, Nicola Persico, and Petra Todd. Racial bias in motor vehicle searches: Theory and evidence. Journal of Political Economy, 109(1):203- 229, 2001.","title":"Racial bias in motor vehicle searches: Theory and evidence","authors":"John Knowles, Nicola Persico, Petra Todd","year":"2001","venue":"Journal of Political Economy","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Liu et al., 2018] Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz, and Moritz Hardt. Delayed impact of fair machine learning. In International Conference on Ma- chine Learning, pages 3156-3164, 2018.","title":"Delayed impact of fair machine learning","authors":"Lydia T Liu, Sarah Dean, Esther Rolf, Max Simchowitz, Moritz Hardt","year":"2018","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pleiss et al., 2017] Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. On fair- ness and calibration. In Advances in Neural Information Processing Systems 30, pages 5684-5693, 2017.","title":"On fairness and calibration","authors":"Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, Kilian Q Weinberger","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ross and Yinger, 2006] Stephen Ross and John Yinger. The Color of Credit: Mortgage Discrimination, Research Methodology, and Fair-Lending Enforcement. MIT Press, Cambridge, 2006.","title":"The Color of Credit: Mortgage Discrimination, Research Methodology, and Fair-Lending Enforcement","authors":"Ross, Yinger, Stephen Ross, John Yinger","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[US Federal Reserve, 2007] US Federal Reserve. Report to the congress on credit scoring and its effects on the avail- ability and affordability of credit, 2007.","title":"US Federal Reserve. Report to the congress on credit scoring and its effects on the availability and affordability of credit","authors":null,"year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Zafar et al., 2017] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P. Gummadi. Fair- ness Constraints: Mechanisms for Fair Classification. In Proc. 20th AISTATS, pages 962-970. PMLR, 2017.","title":"Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P. Gummadi. Fairness Constraints: Mechanisms for Fair Classification","authors":"Zafar","year":"2017","venue":"Proc. 20th AISTATS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}