{"type":"ScholarlyArticle","title":"Achieving Non-Discrimination in Prediction","authors":[{"name":"Lu Zhang","slug":"lu-zhang","given":"Lu","family":"Zhang","affiliation":"University of Arkansas"},{"name":"Yongkai Wu","slug":"yongkai-wu","given":"Yongkai","family":"Wu","affiliation":"University of Arkansas"},{"name":"Xintao Wu","slug":"xintao-wu","given":"Xintao","family":"Wu","affiliation":"University of Arkansas"}],"abstract":"In discrimination-aware classification, the pre-process methods for constructing a discrimination-free classifier first remove discrimination from the training data, and then learn the classifier from the cleaned data. However, they lack a theoretical guarantee for the potential discrimination when the classifier is deployed for prediction. In this paper, we fill this gap by mathematically bounding the discrimination in prediction. We adopt the causal model for modeling the data generation mechanism, and formally defining discrimination in population, in a dataset, and in prediction. We obtain two important theoretical results: (1) the discrimination in prediction can still exist even if the discrimination in the training data is completely removed; and (2) not all pre-process methods can ensure non-discrimination in prediction even though they can achieve non-discrimination in the modified training data. Based on the results, we develop a two-phase framework for constructing a discrimination-free classifier with a theoretical guarantee. The experiments demonstrate the theoretical results and show the effectiveness of our two-phase framework.","doi":"10.24963/ijcai.2018/430","url":"https://ijcai.org/proceedings/2018/430","pdf":"https://ijcai.org/proceedings/2018/0430.pdf","markdown":"https://ijcai.org/proceedings/2018/430.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/430","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Machine Learning","track":"Main track","number":430,"pages":{"first":3097,"last":3103},"keywords":["Machine Learning: Classification","Multidisciplinary Topics and Applications: Philosophical and Ethical Issues","Humans and AI: Ethical Issues in AI"],"videos":[],"references":[{"position":1,"raw":"[Adler et al., 2016] Philip Adler, Casey Falk, Sorelle A Friedler, Gabriel Rybeck, Carlos Scheidegger, Brandon Smith, and Suresh Venkatasubramanian. Auditing black- box models for indirect influence. In Proceedings of ICDM 2016, 2016.","title":"Auditing blackbox models for indirect influence","authors":"Philip Adler, Casey Falk, Sorelle A Friedler, Gabriel Rybeck, Carlos Scheidegger, Brandon Smith, Suresh Venkatasubramanian","year":"2016","venue":"Proceedings of ICDM 2016","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bonchi et al., 2017] Francesco Bonchi, Sara Hajian, Bud Mishra, and Daniele Ramazzotti. Exposing the probabilis- tic causal structure of discrimination. International Jour- nal of Data Science and Analytics, 3(1):1-21, 2017.","title":"Exposing the probabilistic causal structure of discrimination","authors":"Francesco Bonchi, Sara Hajian, Bud Mishra, Daniele Ramazzotti","year":"2017","venue":"International Journal of Data Science and Analytics","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Calders and Verwer, 2010] Toon Calders and Sicco Ver- wer. Three naive bayes approaches for discrimination-free classification. Data Mining and Knowledge Discovery, 21(2):277-292, 2010.","title":"Toon Calders and Sicco Verwer. Three naive bayes approaches for discrimination-free classification","authors":"Verwer Calders","year":"2010","venue":"Data Mining and Knowledge Discovery","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Calders et al., 2009] Toon Calders, Faisal Kamiran, and Mykola Pechenizkiy. Building classifiers with indepen- dency constraints. In Data mining workshops, 2009. ICDMW'09. IEEE international conference on, pages 13- 18. IEEE, 2009.","title":"Building classifiers with independency constraints","authors":"Calders","year":"2009","venue":"ICDMW'09. IEEE international conference on","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Calmon et al., 2017] Flavio Calmon, Dennis Wei, Bhanuki- ran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R Varshney. Optimized pre-processing for discrimi- nation prevention. In Advances in Neural Information Pro- cessing Systems, pages 3995-4004, 2017.","title":"Optimized pre-processing for discrimination prevention","authors":"Flavio Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, Kush R Varshney","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Feldman et al., 2015] Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkata- subramanian. Certifying and removing disparate impact. In KDD, pages 259-268. ACM, 2015.","title":"Certifying and removing disparate impact","authors":"Michael Feldman, A Sorelle, John Friedler, Carlos Moeller, Suresh Scheidegger, Venkatasubramanian","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Glymour and others, 2004] Clark Glymour et al. The TETRAD project. http://www.phil.cmu.edu/ tetrad, 2004.","title":"The TETRAD project","authors":"Clark Glymour","year":null,"venue":null,"doi":null,"url":"http://www.phil.cmu.edu/tetrad,2004","internal_url":null},{"position":8,"raw":"[Hardt et al., 2016] Moritz Hardt, Eric Price, Nati Srebro, et al. Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems, pages 3315-3323, 2016.","title":"Equality of opportunity in supervised learning","authors":"Hardt","year":"2016","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kamiran and Calders, 2009a] Faisal Kamiran and Toon Calders. Classifying without discriminating. In Computer, Control and Communication, 2009. IC4 2009. 2nd Inter- national Conference on, pages 1-6. IEEE, 2009.","title":"Classifying without discriminating","authors":"Calders Kamiran, Faisal Kamiran, Toon Calders","year":"2009","venue":"Computer, Control and Communication, 2009. IC4 2009. 2nd International Conference on","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kamiran and Calders, 2009b] Faisal Kamiran and Toon Calders. Discrimination-aware classification. In 21st Benelux Conference on Artificial Intelligence (BNAIC), pages 333-334, 2009.","title":"Discrimination-aware classification","authors":"Calders Kamiran, Faisal Kamiran, Toon Calders","year":"2009","venue":"21st Benelux Conference on Artificial Intelligence (BNAIC)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kamiran and Calders, 2012] Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrimination. KAIS, 33(1):1-33, 2012.","title":"Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrimination","authors":"Calders Kamiran","year":"2012","venue":"KAIS","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kamiran et al., 2010] Faisal Kamiran, Toon Calders, and Mykola Pechenizkiy. Discrimination aware decision tree learning. In ICDM, pages 869-874. IEEE, 2010.","title":"Discrimination aware decision tree learning","authors":"Faisal Kamiran, Toon Calders, Mykola Pechenizkiy","year":"2010","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kamishima et al., 2011] Toshihiro Kamishima, Shotaro Akaho, and Jun Sakuma. Fairness-aware learning through regularization approach. In ICDMW, pages 643-650. IEEE, 2011.","title":"Fairness-aware learning through regularization approach","authors":"Toshihiro Kamishima, Shotaro Akaho, Jun Sakuma","year":"2011","venue":"ICDMW","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kamishima et al., 2012] Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. Fairness-aware classifier with prejudice remover regularizer. In Joint Eu- ropean Conference on Machine Learning and Knowledge Discovery in Databases, pages 35-50. Springer, 2012.","title":"Fairness-aware classifier with prejudice remover regularizer","authors":"Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, Jun Sakuma","year":"2012","venue":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lichman, 2013] M. Lichman. UCI machine learning repos- itory. http://archive.ics.uci.edu/ml, 2013.","title":"UCI machine learning repository","authors":"M Lichman","year":null,"venue":null,"doi":null,"url":"http://archive.ics.uci.edu/ml,2013","internal_url":null},{"position":16,"raw":"[Murphy, 2012] Kevin P Murphy. Machine learning: a prob- abilistic perspective. MIT press, 2012.","title":"Machine learning: a probabilistic perspective","authors":"Kevin P Murphy","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nabi and Shpitser, 2017] Razieh Nabi and Ilya Sh- pitser. Fair inference on outcomes. arXiv preprint arXiv:1705.10378, 2017.","title":"Razieh Nabi and Ilya Shpitser","authors":"Shpitser Nabi","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pearl, 2009] Judea Pearl. Causality: models, reasoning and inference. Cambridge university press, 2009.","title":"Judea Pearl. Causality: models, reasoning and inference","authors":"Pearl","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Romei and Ruggieri, 2014] Andrea Romei and Salvatore Ruggieri. A multidisciplinary survey on discrimina- tion analysis. The Knowledge Engineering Review, 29(05):582-638, 2014.","title":"A multidisciplinary survey on discrimination analysis","authors":"Andrea Romei, Salvatore Ruggieri","year":"2014","venue":"The Knowledge Engineering Review","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Vapnik, 1998] Vlamimir Vapnik. Statistical learning the- ory, volume 1. Wiley New York, 1998.","title":"Vlamimir Vapnik. Statistical learning theory","authors":"Vapnik","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Žliobaitė et al., 2011] Indre Žliobaitė, Faisal Kamiran, and Toon Calders. Handling conditional discrimination. In ICDM, pages 992-1001. IEEE, 2011.","title":"Handling conditional discrimination","authors":"Žliobaitė","year":"2011","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Zafar et al., 2017] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. Fair- ness constraints: Mechanisms for fair classification. In Ar- tificial Intelligence and Statistics, pages 962-970, 2017.","title":"Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi","authors":"Zafar","year":"2017","venue":"Fairness constraints: Mechanisms for fair classification","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2017] Lu Zhang, Yongkai Wu, and Xintao Wu. A causal framework for discovering and removing direct and indirect discrimination. In IJCAI, 2017.","title":"A causal framework for discovering and removing direct and indirect discrimination","authors":"Lu Zhang, Yongkai Wu, Xintao Wu","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/549"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}