{"type":"ScholarlyArticle","title":"Preventing Disparate Treatment in Sequential Decision Making","authors":[{"name":"Hoda Heidari","slug":"hoda-heidari","given":"Hoda","family":"Heidari","affiliation":"ETH Zürich"},{"name":"Andreas Krause","slug":"andreas-krause","given":"Andreas","family":"Krause","affiliation":"ETH Zürich"}],"abstract":"We study fairness in sequential decision making environments, where at each time step a learning algorithm receives data corresponding to a new individual (e.g. a new job application) and must make an irrevocable decision about him/her (e.g. whether to hire the applicant) based on observations made so far. In order to prevent cases of disparate treatment, our time-dependent notion of fairness requires algorithmic decisions to be consistent: if two individuals are similar in the feature space and arrive during the same time epoch, the algorithm must assign them to similar outcomes. We propose a general framework for post-processing predictions made by a black-box learning model, that guarantees the resulting sequence of outcomes is consistent. We show theoretically that imposing consistency will not significantly slow down learning. Our experiments on two real-world data sets illustrate and confirm this finding in practice.","doi":"10.24963/ijcai.2018/311","url":"https://ijcai.org/proceedings/2018/311","pdf":"https://ijcai.org/proceedings/2018/0311.pdf","markdown":"https://ijcai.org/proceedings/2018/311.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/311","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":311,"pages":{"first":2248,"last":2254},"keywords":["Machine Learning: Online Learning","Machine Learning: Machine Learning","Multidisciplinary Topics and Applications: Philosophical and Ethical Issues","Humans and AI: Ethical Issues in AI"],"videos":[],"references":[{"position":1,"raw":"[Angwin et al., 2016] Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine bias. Propublica, 2016.","title":"Machine bias","authors":"Julia Angwin, Jeff Larson, Surya Mattu, Lauren Kirchner","year":"2016","venue":"Propublica","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barocas and Selbst, 2016] Solon Barocas and Andrew D. Selbst. Big data's disparate impact. California Law Re- view, 2016.","title":"Big data's disparate impact","authors":"Solon Barocas, Andrew D Selbst","year":"2016","venue":"California Law Review","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Barry-Jester et al., 2015] Anna Barry-Jester, Ben Cassel- man, and Dana Goldstein. The new science of sentencing. The Marshall Project, August 2015. Retrieved 4/28/2016.","title":"The new science of sentencing. The Marshall Project","authors":"Anna Barry-Jester, Ben Casselman, Dana Goldstein","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Berk et al., 2017] Richard Berk, Hoda Heidari, Shahin Jab- bari, Michael Kearns, and Aaron Roth. Fairness in crim- inal justice risk assessments: The state of the art. arXiv preprint arXiv:1703.09207, 2017.","title":"Fairness in criminal justice risk assessments: The state of the art","authors":"Richard Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, Aaron Roth","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Corbett-Davies et al., 2017] Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algo- rithmic decision making and the cost of fairness. arXiv preprint arXiv:1701.08230, 2017.","title":"Algorithmic decision making and the cost of fairness","authors":"Corbett-Davies","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Dheeru and Karra Taniskidou, 2017] Dua Dheeru and Efi Karra Taniskidou. UCI machine learning repository, 2017.","title":"Dua Dheeru and Efi Karra Taniskidou. UCI machine learning repository","authors":"Karra Dheeru, Taniskidou","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dwork et al., 2012] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the Innovations in Theoretical Computer Science Conference, pages 214- 226. ACM, 2012.","title":"Fairness through awareness","authors":"Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, Richard Zemel","year":"2012","venue":"Proceedings of the Innovations in Theoretical Computer Science Conference","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Feldman et al., 2015] Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkata- subramanian. Certifying and removing disparate impact. In Proceedings of the International Conference on Knowl- edge Discovery and Data Mining, pages 259-268. ACM, 2015.","title":"Certifying and removing disparate impact","authors":"Michael Feldman, A Sorelle, John Friedler, Carlos Moeller, Suresh Scheidegger, Venkatasubramanian","year":"2015","venue":"Proceedings of the International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hardt et al., 2016] Moritz Hardt, Eric Price, and Nati Sre- bro. Equality of opportunity in supervised learning. In Proceedings of Advances in Neural Information Process- ing Systems, pages 3315-3323, 2016.","title":"Equality of opportunity in supervised learning","authors":"Moritz Hardt, Eric Price, Nati Srebro","year":"2016","venue":"Proceedings of Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Jabbari et al., 2017] Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, and Aaron Roth. Fairness in reinforcement learning. In Proceedings of the International Conference on Machine Learning, pages 1617-1626, 2017.","title":"Fairness in reinforcement learning","authors":"Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, Aaron Roth","year":"2017","venue":"Proceedings of the International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Joseph et al., 2016] Matthew Joseph, Michael Kearns, Jamie Morgenstern, and Aaron Roth. Fairness in learning: Classic and contextual bandits. In Proceedings of Ad- vances in Neural Information Processing Systems, pages 325-333, 2016.","title":"Fairness in learning: Classic and contextual bandits","authors":"Matthew Joseph, Michael Kearns, Jamie Morgenstern, Aaron Roth","year":"2016","venue":"Proceedings of Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Joseph et al., 2017] Matthew Joseph, Michael Kearns, Jamie Morgenstern, Seth Neel, and Aaron Roth. Fair algorithms for infinite contextual bandits. arXiv preprint arXiv:1610.09559, 2017.","title":"Fair algorithms for infinite contextual bandits","authors":"Matthew Joseph, Michael Kearns, Jamie Morgenstern, Seth Neel, Aaron Roth","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kamishima et al., 2012] Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. Fairness-aware classifier with prejudice remover regularizer. Machine Learning and Knowledge Discovery in Databases, pages 35-50, 2012.","title":"Fairness-aware classifier with prejudice remover regularizer","authors":"Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, Jun Sakuma","year":"2012","venue":"Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kleinberg et al., 2016] Jon Kleinberg, Sendhil Mul- lainathan, and Manish Raghavan. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807, 2016.","title":"Inherent trade-offs in the fair determination of risk scores","authors":"Kleinberg","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Levin, 2016] Sam Levin. A beauty contest was judged by AI and the robots didn't like dark skin. The Guardian, 2016.","title":"A beauty contest was judged by AI and the robots didn't like dark skin","authors":"Sam Levin","year":"2016","venue":"The Guardian","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lipton et al., 2017] Zachary C. Lipton, Alexandra Choulde- chova, and Julian McAuley. Does mitigating ml's dis- parate impact require disparate treatment? arXiv preprint arXiv:1711.07076, 2017.","title":"Does mitigating ml's disparate impact require disparate treatment","authors":"Zachary C Lipton, Alexandra Chouldechova, Julian Mcauley","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Mann and O'Neil, 2016] Gideon Mann and Cathy O'Neil. Hiring algorithms are not neutral. Harvard Business Re- view, 2016.","title":"Hiring algorithms are not neutral","authors":"O'neil ; Mann, Gideon Mann, Cathy O' Neil","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Miller, 2015] Clair Miller. Can an algorithm hire better than a human? The New York Times, June 25 2015. Retrieved 4/28/2016.","title":"Can an algorithm hire better than a human? The New York Times","authors":"Clair Miller","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pedreschi et al., 2008] Dino Pedreschi, Salvatore Ruggieri, and Franco Turini. Discrimination-aware data mining. In Proceedings of the International Conference on Knowl- edge Discovery and Data Mining, pages 560-568. ACM, 2008.","title":"Discrimination-aware data mining","authors":"Dino Pedreschi, Salvatore Ruggieri, Franco Turini","year":"2008","venue":"Proceedings of the International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Petrasic et al., 2017] Kevin Petrasic, Benjamin Saul, James Greig, and Matthew Bornfreund. Algorithms and bias: What lenders need to know. White & Case, 2017.","title":"Algorithms and bias: What lenders need to know","authors":"Kevin Petrasic, Benjamin Saul, James Greig, Matthew Bornfreund","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Podesta et al., 2014] John Podesta, Penny Pritzker, Ernest Moniz, John Holdren, and Jeffrey Zients. Big Data: Seiz- ing Opportunities, Preserving Values. Executive Office of the President. The White House., 2014.","title":"Big Data: Seizing Opportunities, Preserving Values","authors":"John Podesta, Penny Pritzker, Ernest Moniz, John Holdren, Jeffrey Zients","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Speicher et al., 2018] Till Speicher, Hoda Heidari, Nina Grgic-Hlaca, Krishna P. Gummadi, Adish Singla, Adrian Weller, and Muhammad Bilal Zafar. A unified approach to quantifying algorithmic unfairness: Measuring individual and group unfairness via inequality indices. In Proceed- ings of the International Conference on Knowledge Dis- covery and Data Mining, 2018.","title":"A unified approach to quantifying algorithmic unfairness: Measuring individual and group unfairness via inequality indices","authors":"Speicher","year":"2018","venue":"Proceedings of the International Conference on Knowledge Discovery and Data Mining","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Sweeney, 2013] Latanya Sweeney. Discrimination in online ad delivery. Queue, 11(3):10, 2013.","title":"Discrimination in online ad delivery","authors":"Latanya Sweeney, Sweeney","year":"2013","venue":"Queue","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zafar et al., 2017] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. Fair- ness constraints: Mechanisms for fair classification. arXiv preprint arXiv:1507.05259, 2017.","title":"Fairness constraints: Mechanisms for fair classification","authors":"Zafar","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}