{"type":"ScholarlyArticle","title":"Metamorphic Testing and Certified Mitigation of Fairness Violations in NLP Models","authors":[{"name":"Pingchuan Ma","slug":"pingchuan-ma","given":"Pingchuan","family":"Ma","affiliation":"The Hong Kong University of Science and Technology; Beijing Electronic Science and Technology Institute"},{"name":"Shuai Wang","slug":"shuai-wang","given":"Shuai","family":"Wang","affiliation":"The Hong Kong University of Science and Technology"},{"name":"Jin Liu","slug":"jin-liu","given":"Jin","family":"Liu","affiliation":"Beijing Electronic Science and Technology Institute"}],"abstract":"Natural language processing (NLP) models have been increasingly used in sensitive application domains including credit scoring, insurance, and loan assessment. Hence, it is critical to know that the decisions made by NLP models are free of unfair bias toward certain subpopulation groups. In this paper, we propose a novel framework employing metamorphic testing, a well-established software testing scheme, to test NLP models and find discriminatory inputs that provoke fairness violations. Furthermore, inspired by recent breakthroughs in the certified robustness of machine learning, we formulate NLP model fairness in a practical setting as (ε, k)-fairness and accordingly smooth the model predictions to mitigate fairness violations. We demonstrate our technique using popular (commercial) NLP models, and successfully flag thousands of discriminatory inputs that can cause fairness violations. We further enhance the evaluated models by adding certified fairness guarantee at a modest cost.","doi":"10.24963/ijcai.2020/64","url":"https://ijcai.org/proceedings/2020/64","pdf":"https://ijcai.org/proceedings/2020/0064.pdf","markdown":"https://ijcai.org/proceedings/2020/64.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/64","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"AI Ethics","track":"Main track","number":64,"pages":{"first":458,"last":465},"keywords":["AI Ethics: Fairness","Natural Language Processing: NLP Applications and Tools"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/24792/file_24792.m3u8","length_sec":253,"provider":"hls"}],"references":[{"position":1,"raw":"[Aggarwal et al., 2019] Aniya Aggarwal, Pranay Lohia, Seema Nagar, Kuntal Dey, and Diptikalyan Saha. Black box fairness testing of machine learning models. In ACM ESEC/FSE, pages 625-635. ACM, 2019.","title":"Black box fairness testing of machine learning models","authors":"Aniya Aggarwal, Pranay Lohia, Seema Nagar, Kuntal Dey, Diptikalyan Saha","year":"2019","venue":"ACM ESEC/FSE","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Barocas et al., 2019] Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and Machine Learning. fairmlbook.org, 2019. http://www.fairmlbook.org.","title":"Fairness and Machine Learning. fairmlbook","authors":"Solon Barocas, Moritz Hardt, Arvind Narayanan","year":"2019","venue":null,"doi":null,"url":"http://www.fairmlbook.org","internal_url":null},{"position":3,"raw":"[Bolukbasi et al., 2016] Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman is to home- maker? debiasing word embeddings. In NIPS, pages 4349-4357, 2016.","title":"Man is to computer programmer as woman is to homemaker? debiasing word embeddings","authors":"Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, Adam T Kalai","year":"2016","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bozic and Wotawa, 2019] Josip Bozic and Franz Wotawa. Testing chatbots using metamorphic relations. IFIP- ICTSS, pages 41-55, 2019.","title":"Testing chatbots using metamorphic relations","authors":"Wotawa Bozic, Josip Bozic, Franz Wotawa","year":"2019","venue":"IFIP-ICTSS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chen et al., 1998] Tsong Y Chen, Shing C Cheung, and Shiu Ming Yiu. Metamorphic testing: a new approach for generating next test cases. Technical report, Technical Report HKUST-CS98-01, 1998.","title":"Metamorphic testing: a new approach for generating next test cases","authors":"Shing C Tsong Y Chen, Cheung, Ming Shiu, Yiu","year":"1998","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cohen et al., 2019] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via random- ized smoothing. In ICML, pages 1310-1320, 2019.","title":"Certified adversarial robustness via randomized smoothing","authors":"Jeremy Cohen, Elan Rosenfeld, Zico Kolter","year":"2019","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Dixon et al., 2018] Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Measuring and miti- gating unintended bias in text classification. 2018.","title":"Measuring and mitigating unintended bias in text classification","authors":"Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, Lucy Vasserman","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dwarakanath et al., 2018] Anurag Dwarakanath, Manish Ahuja, Samarth Sikand, Raghotham M. Rao, R. P. Ja- gadeesh Chandra Bose, Neville Dubash, and Sanjay Pod- der. Identifying implementation bugs in machine learning based image classifiers using metamorphic testing. ISSTA 2018, pages 118-128, 2018.","title":"Identifying implementation bugs in machine learning based image classifiers using metamorphic testing","authors":"Anurag Dwarakanath, Manish Ahuja, Samarth Sikand, M Raghotham, R P Rao, Chandra Jagadeesh, Neville Bose, Sanjay Dubash, Podder","year":"2018","venue":"ISSTA 2018","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Ethayarajh et al., 2019] Kawin Ethayarajh, David Duve- naud, and Graeme Hirst. Towards understanding linear word analogies. ACL, 2019.","title":"Towards understanding linear word analogies","authors":"Kawin Ethayarajh, David Duvenaud, Graeme Hirst","year":"2019","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Galhotra et al., 2017] Sainyam Galhotra, Yuriy Brun, and Alexandra Meliou. Fairness testing: testing software for discrimination. In ACM ESEC/FSE, pages 498-510. ACM, 2017.","title":"Fairness testing: testing software for discrimination","authors":"Galhotra","year":"2017","venue":"ACM ESEC/FSE","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Garg et al., 2019] Sahaj Garg, Vincent Perot, Nicole Limti- aco, Ankur Taly, Ed H Chi, and Alex Beutel. Counterfac- tual fairness in text classification through robustness. In AIES, pages 219-226. ACM, 2019.","title":"Counterfactual fairness in text classification through robustness","authors":"Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Ed H Chi, Alex Beutel","year":"2019","venue":"AIES","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Ge et al., 2018] Tao Ge, Furu Wei, and Ming Zhou. Fluency boost learning and inference for neural grammatical error correction. In ACL, pages 1055-1065, 2018.","title":"Fluency boost learning and inference for neural grammatical error correction","authors":"Tao Ge, Furu Wei, Ming Zhou","year":"2018","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2020] Pinjia He, Clara Meister, and Zhendong Su. Structure-invariant testing for machine translation. ICSE '20, 2020.","title":"Structure-invariant testing for machine translation","authors":"He","year":"2020","venue":"ICSE '20","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Huang et al., 2019] Po-Sen Huang, Robert Stanforth, Jo- hannes Welbl, Chris Dyer, Dani Yogatama, Sven Gowal, Krishnamurthy Dvijotham, and Pushmeet Kohli. Achiev- ing verified robustness to symbol substitutions via inter- val bound propagation. arXiv preprint arXiv:1909.01492, 2019.","title":"Achieving verified robustness to symbol substitutions via interval bound propagation","authors":"Huang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Jia et al., 2019] Robin Jia, Aditi Raghunathan, Kerem Göksel, and Percy Liang. Certified robustness to adversar- ial word substitutions. arXiv preprint arXiv:1909.00986, 2019.","title":"Certified robustness to adversarial word substitutions","authors":"Robin Jia, Aditi Raghunathan, Kerem Göksel, Percy Liang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kairouz et al., 2016] Peter Kairouz, Sewoong Oh, and Pramod Viswanath. Extremal mechanisms for local dif- ferential privacy. JMLR, 17(1):492-542, 2016.","title":"Extremal mechanisms for local differential privacy","authors":"Peter Kairouz, Sewoong Oh, Pramod Viswanath","year":"2016","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kiritchenko and Mohammad, 2018] Svetlana Kiritchenko and Saif M. Mohammad. Examining gender and race bias in two hundred sentiment analysis systems. CoRR, abs/1805.04508, 2018.","title":"Examining gender and race bias in two hundred sentiment analysis systems","authors":"Svetlana Kiritchenko, Saif M Mohammad","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Kusner et al., 2017] Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fer- gus, S. Vishwanathan, and R. Garnett, editors, NIPS, pages 4066-4076.","title":"Counterfactual fairness","authors":"Matt J Kusner, Joshua Loftus, Chris Russell, Ricardo Silva","year":null,"venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"Curran Associates, Inc., 2017.","title":null,"authors":"Curran Associates, Inc","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lecuyer et al., 2019] Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, and Suman Jana. Cer- tified robustness to adversarial examples with differential privacy. In IEEE S&P, pages 656-672. IEEE, 2019.","title":"Certified robustness to adversarial examples with differential privacy","authors":"Mathias Lecuyer, Vaggelis Atlidakis, Roxana Geambasu, Daniel Hsu, Suman Jana","year":"2019","venue":"IEEE S&P","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Maas et al., 2011] Andrew L Maas, Raymond E Daly, Pe- ter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In ACL, pages 142-150. Association for Computational Lin- guistics, 2011.","title":"Learning word vectors for sentiment analysis","authors":"Raymond E Andrew L Maas, Peter T Daly, Dan Pham, Andrew Y Huang, Christopher Ng, Potts","year":"2011","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Manning et al., 2014] Christopher D. Manning, Mihai Sur- deanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. The Stanford CoreNLP natural language processing toolkit. In ACL System Demonstrations, pages 55-60, 2014.","title":"The Stanford CoreNLP natural language processing toolkit","authors":"D Christopher, Mihai Manning, John Surdeanu, Jenny Bauer, Steven J Finkel, David Bethard, Mcclosky","year":"2014","venue":"ACL System Demonstrations","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Martinez, 2018] Janio Martinez. Kaggle Bank Marketing Dataset, 2018.","title":"Kaggle Bank Marketing Dataset","authors":"Janio Martinez","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Mu et al., 2019] J. Mu, P. Liang, and N. Goodman. Shaping visual representations with language for few-shot classifi- cation. arXiv preprint arXiv:1911.02683, 2019.","title":"Shaping visual representations with language for few-shot classification","authors":"J Mu, P Liang, N Goodman","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Park et al., 2018] Ji Ho Park, Jamin Shin, and Pascale Fung. Reducing gender bias in abusive language detection. arXiv preprint arXiv:1808.07231, 2018.","title":"Reducing gender bias in abusive language detection","authors":"Ji Ho, Park, Jamin Shin, Pascale Fung","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Pei et al., 2017] Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. Deepxplore: Automated whitebox testing of deep learning systems. In SOSP, pages 1-18. ACM, 2017.","title":"Deepxplore: Automated whitebox testing of deep learning systems","authors":"Kexin Pei, Yinzhi Cao, Junfeng Yang, Suman Jana","year":"2017","venue":"SOSP","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Rhue, 2018] Lauren Rhue. Racial influence on automated perceptions of emotions. Available at SSRN 3281765, 2018.","title":"Racial influence on automated perceptions of emotions","authors":"Lauren Rhue","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Sheng et al., 2019] Emily Sheng, Kai-Wei Chang, Premku- mar Natarajan, and Nanyun Peng. The woman worked as a babysitter: On biases in language generation. arXiv:1909.01326, 2019.","title":"The woman worked as a babysitter: On biases in language generation","authors":"Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, Nanyun Peng","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Speer et al., 2017] Robert Speer, Joshua Chin, and Cather- ine Havasi. Conceptnet 5.5: An open multilingual graph of general knowledge. In AAAI, 2017.","title":"Conceptnet 5.5: An open multilingual graph of general knowledge","authors":"Robert Speer, Joshua Chin, Catherine Havasi","year":"2017","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Sun and Zhou, 2018] Liqun Sun and Zhi Quan Zhou. Meta- morphic testing for machine translations: MT4MT. ASWEC' 18, pages 96-100, 2018.","title":"Metamorphic testing for machine translations: MT4MT","authors":"Zhou ; ] Liqun Sun, Zhi Quan Sun, Zhou","year":"2018","venue":"ASWEC","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Sun et al., 2019] Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. Mitigating gender bias in natural language process- ing: Literature review. arXiv:1906.08976, 2019.","title":"Mitigating gender bias in natural language processing: Literature review","authors":"Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai Elsherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, William Yang, Wang","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Tian et al., 2018] Yuchi Tian, Kexin Pei, Suman Jana, and Baishakhi Ray. DeepTest: Automated testing of deep- neural-network-driven autonomous cars. ICSE '18, 2018.","title":"DeepTest: Automated testing of deepneural-network-driven autonomous cars","authors":"Yuchi Tian, Kexin Pei, Suman Jana, Baishakhi Ray","year":"2018","venue":"ICSE '18","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Udeshi et al., 2018] Sakshi Udeshi, Pryanshu Arora, and Sudipta Chattopadhyay. Automated directed fairness test- ing. In ACM/IEEE ASE, pages 98-108. ACM, 2018.","title":"Automated directed fairness testing","authors":"Sakshi Udeshi, Pryanshu Arora, Sudipta Chattopadhyay","year":"2018","venue":"ACM/IEEE ASE","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wang et al., 2019] Jingyi Wang, Guoliang Dong, Jun Sun, Xinyu Wang, and Peixin Zhang. Adversarial sample de- tection for deep neural network through model mutation testing. ICSE '19, pages 1245-1256, 2019.","title":"Adversarial sample detection for deep neural network through model mutation testing","authors":"Jingyi Wang, Guoliang Dong, Jun Sun, Xinyu Wang, Peixin Zhang","year":"2019","venue":"ICSE '19","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Warner, 1965] Stanley L Warner. Randomized response: A survey technique for eliminating evasive answer bias. Journal of the American Statistical Association, 60(309):63-69, 1965.","title":"Randomized response: A survey technique for eliminating evasive answer bias","authors":"L Stanley, Warner","year":"1965","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Xie et al., 2018] Xiaofei Xie, Lei Ma, Felix Juefei-Xu, Hongxu Chen, Minhui Xue, Bo Li, Yang Liu, Jian- jun Zhao, Jianxiong Yin, and Simon See. Coverage- guided fuzzing for deep neural networks. arXiv preprint arXiv:1809.01266, 2018.","title":"Coverageguided fuzzing for deep neural networks","authors":"Xiaofei Xie, Lei Ma, Felix Juefei-Xu, Hongxu Chen, Minhui Xue, Bo Li, Yang Liu, Jianjun Zhao, Jianxiong Yin, Simon See","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Yan et al., 2019] Boyang Yan, Brian Yecies, and Zhi Quan Zhou. Metamorphic relations for data validation: a case study of translated text messages. In 2019 IEEE/ACM 4th International Workshop on Metamorphic Testing (MET), pages 70-75. IEEE, 2019.","title":"Metamorphic relations for data validation: a case study of translated text messages","authors":"Boyang Yan, Brian Yecies, Zhi Quan Zhou","year":"2019","venue":"IEEE/ACM 4th International Workshop on Metamorphic Testing (MET)","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhang et al., 2018] Mengshi Zhang, Yuqun Zhang, Ling- ming Zhang, Cong Liu, and Sarfraz Khurshid. Deep- Road: GAN-based Metamorphic Testing and Input Valida- tion Framework for Autonomous Driving Systems. ASE, 2018.","title":"Deep-Road: GAN-based Metamorphic Testing and Input Validation Framework for Autonomous Driving Systems","authors":"Zhang","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zhao et al., 2018] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Gender bias in coreference resolution: Evaluation and debiasing methods. arXiv:1804.06876, 2018.","title":"Gender bias in coreference resolution: Evaluation and debiasing methods","authors":"Zhao","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}