{"type":"ScholarlyArticle","title":"A Causal Framework for Discovering and Removing Direct and Indirect Discrimination","authors":[{"name":"Lu Zhang","slug":"lu-zhang","given":"Lu","family":"Zhang","affiliation":"University of Arkansas"},{"name":"Yongkai Wu","slug":"yongkai-wu","given":"Yongkai","family":"Wu","affiliation":"University of Arkansas"},{"name":"Xintao Wu","slug":"xintao-wu","given":"Xintao","family":"Wu","affiliation":"University of Arkansas"}],"abstract":"In this paper, we investigate the problem of discovering both direct and indirect discrimination from the historical data, and removing the discriminatory effects before the data is used for predictive analysis (e.g., building classifiers). The main drawback of existing methods is that they cannot distinguish the part of influence that is really caused by discrimination from all correlated influences. In our approach, we make use of the causal network to capture the causal structure of the data. Then we model direct and indirect discrimination as the path-specific effects, which accurately identify the two types of discrimination as the causal effects transmitted along different paths in the network. Based on that, we propose an effective algorithm for discovering direct and indirect discrimination, as well as an algorithm for precisely removing both types of discrimination while retaining good data utility. Experiments using the real dataset show the effectiveness of our approaches.","doi":"10.24963/ijcai.2017/549","url":"https://ijcai.org/proceedings/2017/549","pdf":"https://ijcai.org/proceedings/2017/0549.pdf","markdown":"https://ijcai.org/proceedings/2017/549.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/549","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Multidisciplinary Topics and Applications","track":"Main track","number":549,"pages":{"first":3929,"last":3935},"keywords":["Multidisciplinary Topics and Applications: AI&Security and Privacy","Multidisciplinary Topics and Applications: AI and Social Sciences"],"videos":[],"references":[{"position":1,"raw":"[Adler et al., 2016] Philip Adler, Casey Falk, Sorelle A Friedler, et al. Auditing black-box models for indirect in- fluence. In Proceedings of ICDM 2016, 2016.","title":"Auditing black-box models for indirect influence","authors":"Philip Adler, Casey Falk, Sorelle A Friedler","year":"2016","venue":"Proceedings of ICDM 2016","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Aliferis and others, 2010] Constantin F Aliferis et al. Local causal and markov blanket induction for causal discovery and feature selection for classification part i: algorithms and empirical evaluation. Journal of Machine Learning Research, 11(Jan):171-234, 2010.","title":"Local causal and markov blanket induction for causal discovery and feature selection for classification part i: algorithms and empirical evaluation","authors":"Constantin F Aliferis","year":"2010","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Avin et al., 2005] Chen Avin, Ilya Shpitser, and Judea Pearl. Identifiability of path-specific effects. In IJCAI'05, pages 357-363, 2005.","title":"Identifiability of path-specific effects","authors":"Chen Avin, Ilya Shpitser, Judea Pearl","year":"2005","venue":"IJCAI'05","doi":null,"url":null,"internal_url":"/proceedings/2005/100"},{"position":4,"raw":"[Bonchi et al., 2017] Francesco Bonchi, Sara Hajian, Bud Mishra, and Daniele Ramazzotti. Exposing the probabilis- tic causal structure of discrimination. International Jour- nal of Data Science and Analytics, 3(1):1-21, 2017.","title":"Exposing the probabilistic causal structure of discrimination","authors":"Francesco Bonchi, Sara Hajian, Bud Mishra, Daniele Ramazzotti","year":"2017","venue":"International Journal of Data Science and Analytics","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Calders and Verwer, 2010] Toon Calders and Sicco Ver- wer. Three naive bayes approaches for discrimination-free classification. Data Mining and Knowledge Discovery, 21(2):277-292, 2010.","title":"Toon Calders and Sicco Verwer. Three naive bayes approaches for discrimination-free classification","authors":"Verwer Calders","year":"2010","venue":"Data Mining and Knowledge Discovery","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Colombo and Maathuis, 2014] Diego Colombo and Mar- loes H Maathuis. Order-independent constraint-based causal structure learning. JMLR, 15(1):3741-3782, 2014.","title":"Diego Colombo and Marloes H Maathuis. Order-independent constraint-based causal structure learning","authors":"Maathuis Colombo","year":"2014","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dahl and others, 2006] Joachin Dahl et al. Cvxopt: a python package for convex optimization. In Proc. eur. conf. op. res, 2006.","title":"Cvxopt: a python package for convex optimization","authors":"Others ; Joachin Dahl, Dahl","year":"2006","venue":"Proc. eur. conf. op. res","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Feldman et al., 2015] Michael Feldman, Sorelle A Friedler, et al. Certifying and removing disparate impact. In KDD, pages 259-268. ACM, 2015.","title":"Certifying and removing disparate impact","authors":"Michael Feldman, Sorelle A Friedler","year":"2015","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Glymour and others, 2004] Clark Glymour et al. The TETRAD project. http://www.phil.cmu.edu/ tetrad, 2004.","title":"The TETRAD project","authors":"Clark Glymour","year":null,"venue":null,"doi":null,"url":"http://www.phil.cmu.edu/tetrad,2004","internal_url":null},{"position":10,"raw":"[Hajian and Domingo-Ferrer, 2013] Sara Hajian and Josep Domingo-Ferrer. A methodology for direct and indi- rect discrimination prevention in data mining. JKDE, 25(7):1445-1459, 2013.","title":"A methodology for direct and indirect discrimination prevention in data mining","authors":"Sara Hajian, Josep Domingo-Ferrer","year":"2013","venue":"JKDE","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Heckerman and Breese, 1994] David Heckerman and John S Breese. A new look at causal independence. In Proceedings of UAI 1994, pages 286-292, 1994.","title":"A new look at causal independence","authors":"Breese ; ] Heckerman, John S David Heckerman, Breese","year":"1994","venue":"Proceedings of UAI 1994","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Heckerman and Breese, 1996] David Heckerman and John S Breese. Causal independence for probability assessment and inference using bayesian networks. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 26(6):826-831, 1996.","title":"Causal independence for probability assessment and inference using bayesian networks","authors":"Breese ; ] Heckerman, John S David Heckerman, Breese","year":"1996","venue":"IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kalisch and Bühlmann, 2007] Markus Kalisch and Peter Bühlmann. Estimating high-dimensional directed acyclic graphs with the pc-algorithm. The Journal of Machine Learning Research, 8:613-636, 2007.","title":"Estimating high-dimensional directed acyclic graphs with the pc-algorithm","authors":"Bühlmann Kalisch, Markus Kalisch, Peter Bühlmann","year":"2007","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kamiran and Calders, 2012] Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrimination. KAIS, 33(1):1-33, 2012.","title":"Faisal Kamiran and Toon Calders. Data preprocessing techniques for classification without discrimination","authors":"Calders Kamiran","year":"2012","venue":"KAIS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kamiran et al., 2010] Faisal Kamiran, Toon Calders, and Mykola Pechenizkiy. Discrimination aware decision tree learning. In ICDM, pages 869-874. IEEE, 2010.","title":"Discrimination aware decision tree learning","authors":"Faisal Kamiran, Toon Calders, Mykola Pechenizkiy","year":"2010","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kamishima et al., 2011] Toshihiro Kamishima, Shotaro Akaho, and Jun Sakuma. Fairness-aware learning through regularization approach. In ICDMW 2011, 2011.","title":"Fairness-aware learning through regularization approach","authors":"Kamishima","year":"2011","venue":"ICDMW 2011","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Koller and Friedman, 2009] Daphne Koller and Nir Fried- man. Probabilistic graphical models: principles and tech- niques. MIT press, 2009.","title":"Probabilistic graphical models: principles and techniques","authors":"Friedman Koller","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kozlov et al., 1980] Mikhail K Kozlov, Sergei P Tarasov, et al. The polynomial solvability of convex quadratic pro- gramming. USSR Computational Mathematics and Math- ematical Physics, 20(5):223-228, 1980.","title":"The polynomial solvability of convex quadratic programming","authors":"Sergei P Mikhail K Kozlov, Tarasov","year":"1980","venue":"USSR Computational Mathematics and Mathematical Physics","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Lichman, 2013] M. Lichman. UCI machine learning repos- itory. http://archive.ics.uci.edu/ml, 2013.","title":"UCI machine learning repository","authors":"M Lichman","year":null,"venue":null,"doi":null,"url":"http://archive.ics.uci.edu/ml,2013","internal_url":null},{"position":20,"raw":"[Luong et al., 2011] Binh Thanh Luong, Salvatore Ruggieri, and Franco Turini. k-nn as an implementation of situa- tion testing for discrimination discovery and prevention. In KDD, pages 502-510. ACM, 2011.","title":"k-nn as an implementation of situation testing for discrimination discovery and prevention","authors":"Thanh Binh, Salvatore Luong, Franco Ruggieri, Turini","year":"2011","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Mancuhan and Clifton, 2014] Koray Mancuhan and Chris Clifton. Combating discrimination using bayesian net- works. AI & L, 22(2):211-238, 2014.","title":"Combating discrimination using bayesian networks","authors":"Clifton ; Mancuhan, Koray Mancuhan, Chris Clifton","year":"2014","venue":"AI & L","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Neapolitan and others, 2004] Richard E Neapolitan et al. Learning bayesian networks, volume 38. Prentice Hall Upper Saddle River, 2004.","title":"Learning bayesian networks","authors":"Richard E Neapolitan","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Pearl, 2009] Judea Pearl. Causality: models, reasoning and inference. Cambridge university press, 2009.","title":"Judea Pearl. Causality: models, reasoning and inference","authors":"Pearl","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Pedreshi et al., 2008] Dino Pedreshi, Salvatore Ruggieri, and Franco Turini. Discrimination-aware data mining. In KDD, pages 560-568. ACM, 2008.","title":"Discrimination-aware data mining","authors":"Dino Pedreshi, Salvatore Ruggieri, Franco Turini","year":"2008","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Romei and Ruggieri, 2014] Andrea Romei and Salvatore Ruggieri. A multidisciplinary survey on discrimination analysis. Knowl Eng Rev, 29(05):582-638, 2014.","title":"A multidisciplinary survey on discrimination analysis","authors":"Andrea Romei, Salvatore Ruggieri","year":"2014","venue":"Knowl Eng Rev","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Ruggieri et al., 2010] Salvatore Ruggieri, Dino Pedreschi, and Franco Turini. Data mining for discrimination dis- covery. TKDD, 4(2):9, 2010.","title":"Data mining for discrimination discovery","authors":"Salvatore Ruggieri, Dino Pedreschi, Franco Turini","year":"2010","venue":"TKDD","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Shpitser, 2013] Ilya Shpitser. Counterfactual graphical models for longitudinal mediation analysis with unob- served confounding. Cognitive science, 37(6):1011-1035, 2013.","title":"Ilya Shpitser. Counterfactual graphical models for longitudinal mediation analysis with unobserved confounding","authors":"Shpitser","year":"2013","venue":"Cognitive science","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Spirtes et al., 2000] Peter Spirtes, Clark N Glymour, and Richard Scheines. Causation, prediction, and search, vol- ume 81. MIT press, 2000.","title":"Causation, prediction, and search","authors":"Peter Spirtes, Clark N Glymour, Richard Scheines","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Žliobaitė et al., 2011] Indre Žliobaitė, Faisal Kamiran, and Toon Calders. Handling conditional discrimination. In ICDM, pages 992-1001. IEEE, 2011.","title":"Handling conditional discrimination","authors":"Žliobaitė","year":"2011","venue":"ICDM","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Zhang et al., 2016a] Lu Zhang, Yongkai Wu, and Xintao Wu. A causal framework for discovering and remov- ing direct and indirect discrimination. arXiv preprint arXiv:1611.07509, 2016.","title":"A causal framework for discovering and removing direct and indirect discrimination","authors":"Zhang","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Zhang et al., 2016b] Lu Zhang, Yongkai Wu, and Xintao Wu. On discrimination discovery using causal networks. In Proceedings of SBP-BRiMS 2016, 2016.","title":"On discrimination discovery using causal networks","authors":"Zhang","year":"2016","venue":"Proceedings of SBP-BRiMS 2016","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Zhang et al., 2016c] Lu Zhang, Yongkai Wu, and Xintao Wu. Situation testing-based discrimination discovery: a causal inference approach. In Proceedings of IJCAI, 2016.","title":"Situation testing-based discrimination discovery: a causal inference approach","authors":"Zhang","year":"2016","venue":"Proceedings of IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2016/386"}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}