{"type":"ScholarlyArticle","title":"Evaluating and Aggregating Feature-based Model Explanations","authors":[{"name":"Umang Bhatt","slug":"umang-bhatt","given":"Umang","family":"Bhatt","affiliation":"University of Cambridge; Carnegie Mellon University"},{"name":"Adrian Weller","slug":"adrian-weller","given":"Adrian","family":"Weller","affiliation":"University of Cambridge; The Alan Turing Institute"},{"name":"José M. F. Moura","slug":"jose-m-f-moura","given":"José M. F.","family":"Moura","affiliation":"Carnegie Mellon University"}],"abstract":"A feature-based model explanation denotes how much each input feature contributes to a model's output for a given data point. As the number of proposed explanation functions grows, we lack quantitative evaluation criteria to help practitioners know when to use which explanation function. This paper proposes quantitative evaluation criteria for feature-based explanations: low sensitivity, high faithfulness, and low complexity. We devise a framework for aggregating explanation functions. We develop a procedure for learning an aggregate explanation function with lower complexity and then derive a new aggregate Shapley value explanation function that minimizes sensitivity.","doi":"10.24963/ijcai.2020/417","url":"https://ijcai.org/proceedings/2020/417","pdf":"https://ijcai.org/proceedings/2020/0417.pdf","markdown":"https://ijcai.org/proceedings/2020/417.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/417","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":417,"pages":{"first":3016,"last":3022},"keywords":["Machine Learning: Interpretability","Machine Learning: Explainable Machine Learning","AI Ethics: Explainability"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/25206/file_25206.m3u8","length_sec":302,"provider":"hls"}],"references":[{"position":1,"raw":"[Baehrens et al., 2010] David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus-Robert Muller. How to explain individual classifi- cation decisions. JMLR, 11(Jun):1803-1831, 2010.","title":"How to explain individual classification decisions","authors":"David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, Klaus-Robert Muller","year":"2010","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Batterman and Rice, 2014] Robert W Batterman and Collin C Rice. Minimal model explanations. Philosophy of Science, 81(3):349-376, 2014.","title":"Minimal model explanations","authors":"W Robert, Collin C Batterman, Rice","year":"2014","venue":"Philosophy of Science","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Berger, 2013] James O Berger. Statistical decision theory and Bayesian analysis. Springer Science & Business Me- dia, 2013.","title":"Statistical decision theory and Bayesian analysis","authors":"O James, Berger","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bhatt et al., 2019a] Umang Bhatt, Pradeep Ravikumar, et al. Building human-machine trust via interpretability. In Proceedings of the AAAI Conference on Artificial Intel- ligence, volume 33, pages 9919-9920, 2019.","title":"Building human-machine trust via interpretability","authors":"Bhatt","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bhatt et al., 2019b] Umang Bhatt, Pradeep Ravikumar, and José M. F. Moura. Towards aggregating weighted feature attributions. arXiv:1901.10040, 2019.","title":"Towards aggregating weighted feature attributions","authors":"Bhatt","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bhatt et al., 2020] Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, José M. F. Moura, and Peter Ecker- sley. Explainable machine learning in deployment. ACM Conference on Fairness, Accountability, and Transparency (FAT*), 2020.","title":"Explainable machine learning in deployment","authors":"Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, M F José, Peter Moura, Eckersley","year":"2020","venue":"ACM Conference on Fairness, Accountability, and Transparency (FAT*)","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Camburu et al., 2019] Oana-Maria Camburu, Eleonora Giunchiglia, Jakob Foerster, Thomas Lukasiewicz, and Phil Blunsom. Can I trust the explainer? Verifying post-hoc explanatory methods. arXiv:1910.02065, 2019.","title":"Can I trust the explainer? Verifying post-hoc explanatory methods","authors":"Oana-Maria Camburu, Eleonora Giunchiglia, Jakob Foerster, Thomas Lukasiewicz, Phil Blunsom","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chang et al., 2019] Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud. Explaining image clas- sifiers by counterfactual generation. In International Con- ference on Learning Representations, 2019.","title":"Explaining image classifiers by counterfactual generation","authors":"Chun-Hao Chang, Elliot Creager, Anna Goldenberg, David Duvenaud","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Chen et al., 2019] Jianbo Chen, Le Song, Martin J Wain- wright, and Michael I Jordan. L-shapley and C-shapley: Efficient model interpretation for structured data. Interna- tional Conference on Learning Representations, 2019.","title":"L-shapley and C-shapley: Efficient model interpretation for structured data","authors":"Jianbo Chen, Le Song, Martin J Wainwright, Michael I Jordan","year":"2019","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dua and Graff, 2017] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.","title":"Dheeru Dua and Casey Graff. UCI machine learning repository","authors":"Graff Dua","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Ghorbani et al., 2019] Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is frag- ile. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3681-3688, 2019.","title":"Interpretation of neural networks is fragile","authors":"Ghorbani","year":"2019","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Goold and Lipkin Jr, 1999] Susan Dorr Goold and Mack Lipkin Jr. The doctor-patient relationship: challenges, opportunities, and strategies. Journal of general internal medicine, 14(Suppl 1):S26, 1999.","title":"The doctor-patient relationship: challenges, opportunities, and strategies","authors":"Lipkin Goold, ; Jr, Susan Dorr Goold, Mack Lipkin","year":"1999","venue":"Journal of general internal medicine","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Hooker et al., 2019] Sara Hooker, Dumitru Erhan, Pieter- Jan Kindermans, and Been Kim. A benchmark for inter- pretability methods in deep neural networks. In Advances in Neural Information Processing Systems, pages 9734- 9745, 2019.","title":"A benchmark for interpretability methods in deep neural networks","authors":"Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, Been Kim","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Johnson et al., 2016] Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo An- thony Celi, and Roger G Mark. Mimic-iii, a freely ac- cessible critical care database. Scientific Data, 2016.","title":"Mimic-iii, a freely accessible critical care database","authors":"E W Alistair, Tom J Johnson, Lu Pollard, Li-Wei H Shen, Mengling Lehman, Mohammad Feng, Benjamin Ghassemi, Peter Moody, Leo Szolovits, Roger G Anthony Celi, Mark","year":"2016","venue":"Scientific Data","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lage et al., 2019] Isaac Lage, Emily Chen, Jeffrey He, Menaka Narayanan, Been Kim, Samuel J Gershman, and Finale Doshi-Velez. Human evaluation of models built for interpretability. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, volume 7, pages 59-67, 2019.","title":"Human evaluation of models built for interpretability","authors":"Isaac Lage, Emily Chen, Jeffrey He, Menaka Narayanan, Been Kim, Finale Samuel J Gershman, Doshi-Velez","year":"2019","venue":"Proceedings of the AAAI Conference on Human Computation and Crowdsourcing","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lipton, 2003] Peter Lipton. Inference to the best explana- tion. Routledge, 2003.","title":"Inference to the best explanation","authors":"Peter Lipton","year":"2003","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lundberg and Lee, 2017] Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 30 (NeurIPS 2017), pages 4765-4774, 2017.","title":"A unified approach to interpreting model predictions","authors":"M Scott, Su-In Lundberg, Lee","year":"2017","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Melis and Jaakkola, 2018] David Alvarez Melis and Tommi Jaakkola. Towards robust interpretability with self- explaining neural networks. In Advances in Neural Infor- mation Processing Systems (NeurIPS 2018), 2018.","title":"Towards robust interpretability with selfexplaining neural networks","authors":"Jaakkola ; ] Melis, Alvarez David, Tommi Melis, Jaakkola","year":"2018","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Purushotham et al., 2018] Sanjay Purushotham, Chuizheng Meng, Zhengping Che, and Yan Liu. Benchmarking deep learning models on large healthcare datasets. Journal of Biomedical Informatics, 83:112-134, 2018.","title":"Benchmarking deep learning models on large healthcare datasets","authors":"Sanjay Purushotham, Chuizheng Meng, Zhengping Che, Yan Liu","year":"2018","venue":"Journal of Biomedical Informatics","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ruben, 2015] David-Hillel Ruben. Explaining explanation. Routledge, 2015.","title":"Explaining explanation","authors":"David-Hillel Ruben","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Shapley, 1953] Lloyd S Shapley. A value for n-person games. In Contributions to the Theory of Games II, pages 307-317, 1953.","title":"A value for n-person games","authors":"S Lloyd, Shapley","year":"1953","venue":"Contributions to the Theory of Games II","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Shrikumar et al., 2017] Avanti Shrikumar, Peyton Green- side, and Anshul Kundaje. Learning important features through propagating activation differences. In Proceed- ings of the 34th International Conference on Machine Learning-Volume 70 (ICML 2017), pages 3145-3153. Journal of Machine Learning Research, 2017.","title":"Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences","authors":"Shrikumar","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Štrumbelj and Kononenko, 2014] Erik Štrumbelj and Igor Kononenko. Explaining prediction models and individ- ual predictions with feature contributions. Knowledge and Information Systems, 41(3):647-665, 2014.","title":"Erik Štrumbelj and Igor Kononenko. Explaining prediction models and individual predictions with feature contributions","authors":"Kononenko Štrumbelj","year":"2014","venue":"Knowledge and Information Systems","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Sundararajan et al., 2017] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep net- works. In Proceedings of the 34th International Confer- ence on Machine Learning-Volume 70 (ICML 2017), pages 3319-3328. Journal of Machine Learning Research, 2017.","title":"Axiomatic attribution for deep networks","authors":"Mukund Sundararajan, Ankur Taly, Qiqi Yan","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Yang and Kim, 2019] Mengjiao Yang and Been Kim. BIM: Towards quantitative evaluation of interpretability meth- ods with ground truth. arXiv:1907.09701, 2019.","title":"BIM: Towards quantitative evaluation of interpretability methods with ground truth","authors":"Yang, Kim ;, Mengjiao Yang, Been Kim","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Yeh et al., 2019] Chih-Kuan Yeh, Cheng-Yu Hsieh, Arun Suggala, David I Inouye, and Pradeep K Ravikumar. On the (in) fidelity and sensitivity of explanations. In Ad- vances in Neural Information Processing Systems, pages 10965-10976, 2019.","title":"On the (in) fidelity and sensitivity of explanations","authors":"Chih-Kuan Yeh, Cheng-Yu Hsieh, Arun Suggala, David I Inouye, Pradeep K Ravikumar","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}