{"type":"ScholarlyArticle","title":"Statistically Significant Concept-based Explanation of Image Classifiers via Model Knockoffs","authors":[{"name":"Kaiwen Xu","slug":"kaiwen-xu","given":"Kaiwen","family":"Xu","affiliation":"University of Tsukuba; RIKEN AIP"},{"name":"Kazuto Fukuchi","slug":"kazuto-fukuchi","given":"Kazuto","family":"Fukuchi","affiliation":"University of Tsukuba; RIKEN AIP"},{"name":"Youhei Akimoto","slug":"youhei-akimoto","given":"Youhei","family":"Akimoto","affiliation":"University of Tsukuba; RIKEN AIP"},{"name":"Jun Sakuma","slug":"jun-sakuma","given":"Jun","family":"Sakuma","affiliation":"Tokyo Institute of Technology; RIKEN AIP"}],"abstract":"A concept-based classifier can explain the decision process of a deep learning model by human understandable concepts in image classification problems. However, sometimes concept-based explanations may cause false positives, which misregards unrelated concepts as important for the prediction task. Our goal is to find the statistically significant concept for classification to prevent misinterpretation. In this study, we propose a method using a deep learning model to learn the image concept and then using the knockoff sample to select the important concepts for prediction by controlling the False Discovery Rate (FDR) under a certain value. We evaluate the proposed method in our experiments on both synthetic and real data. Also, it shows that our method can control the FDR properly while selecting highly interpretable concepts to improve the trustworthiness of the model.","doi":"10.24963/ijcai.2023/58","url":"https://ijcai.org/proceedings/2023/58","pdf":"https://ijcai.org/proceedings/2023/0058.pdf","markdown":"https://ijcai.org/proceedings/2023/58.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/58","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI Ethics, Trust, Fairness","track":"Main Track","number":58,"pages":{"first":519,"last":526},"keywords":["Machine Learning: ML: Explainable/Interpretable machine learning","AI Ethics, Trust, Fairness: ETF: Explainability and interpretability","AI Ethics, Trust, Fairness: ETF: Trustworthy AI"],"videos":[],"references":[{"position":1,"raw":"[Bahadori and Heckerman, 2021] Mohammad Taha Ba- hadori and David Heckerman. Debiasing concept-based explanations with causal analysis. In ICLR, 2021.","title":"Debiasing concept-based explanations with causal analysis","authors":"Heckerman ; Bahadori, Taha Mohammad, David Bahadori, Heckerman","year":"2021","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Candes et al., 2018] Emmanuel Candes, Yingying Fan, Lu- cas Janson, and Jinchi Lv. Panning for gold:'model- x'knockoffs for high dimensional controlled variable se- lection. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2018.","title":"Panning for gold:'modelx'knockoffs for high dimensional controlled variable selection","authors":"Emmanuel Candes, Yingying Fan, Lucas Janson, Jinchi Lv","year":"2018","venue":"Journal of the Royal Statistical Society: Series B (Statistical Methodology","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2018] Tian Qi Chen, Xuechen Li, Roger B. Grosse, and David Duvenaud. Isolating sources of disen- tanglement in variational autoencoders. In NeurIPS, pages 2615-2625, 2018.","title":"Isolating sources of disentanglement in variational autoencoders","authors":"Tian Qi, Chen, Xuechen Li, Roger B Grosse, David Duvenaud","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chen et al., 2020] Zhi Chen, Yijie Bei, and Cynthia Rudin. Concept whitening for interpretable image recognition. Nat. Mach. Intell., 2020.","title":"Concept whitening for interpretable image recognition","authors":"Zhi Chen, Yijie Bei, Cynthia Rudin","year":"2020","venue":"Nat. Mach. Intell","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Ding et al., 2020] Zheng Ding, Yifan Xu, Weijian Xu, Gau- rav Parmar, Yang Yang, Max Welling, and Zhuowen Tu. Guided variational autoencoder for disentanglement learn- ing. In CVPR, 2020.","title":"Guided variational autoencoder for disentanglement learning","authors":"Zheng Ding, Yifan Xu, Weijian Xu, Gaurav Parmar, Yang Yang, Max Welling, Zhuowen Tu","year":"2020","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Goyal et al., 2019] Yash Goyal, Uri Shalit, and Been Kim. Explaining classifiers with causal concept effect (cace). CoRR, 2019.","title":"Explaining classifiers with causal concept effect (cace)","authors":"Yash Goyal, Uri Shalit, Been Kim","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Higgins et al., 2017] Irina Higgins, Loïc Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a con- strained variational framework. In ICLR, 2017.","title":"beta-vae: Learning basic visual concepts with a constrained variational framework","authors":"Irina Higgins, Loïc Matthey, Arka Pal, Christopher P Burgess, Xavier Glorot, Matthew M Botvinick, Shakir Mohamed, Alexander Lerchner","year":"2017","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Kazhdan et al., 2020] Dmitry Kazhdan, Botty Dimanov, Mateja Jamnik, Pietro Liò, and Adrian Weller. Now you see me (CME): concept-based model extraction. In CIKM (Workshops), 2020.","title":"Now you see me (CME): concept-based model extraction","authors":"Dmitry Kazhdan, Botty Dimanov, Mateja Jamnik, Pietro Liò, Adrian Weller","year":"2020","venue":"CIKM (Workshops)","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kim and Mnih, 2018] Hyunjik Kim and Andriy Mnih. Dis- entangling by factorising. In ICML, Proceedings of Ma- chine Learning Research, 2018.","title":"Hyunjik Kim and Andriy Mnih. Disentangling by factorising","authors":"Mnih Kim","year":"2018","venue":"ICML, Proceedings of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kim et al., 2018] Been Kim, Martin Wattenberg, Justin Gilmer, Carrie J. Cai, James Wexler, Fernanda B. Viégas, and Rory Sayres. Interpretability beyond feature attribu- tion: Quantitative testing with concept activation vectors (TCAV). In ICML, 2018.","title":"Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV)","authors":"Kim","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kingma and Welling, 2014] Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In ICLR, 2014.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2014","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Koh et al., 2020] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In ICML, 2020.","title":"Concept bottleneck models","authors":"Pang Wei Koh, Thao Nguyen, Siang Yew, Stephen Tang, Emma Mussmann, Been Pierson, Percy Kim, Liang","year":"2020","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[LeCun et al., 1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning ap- plied to document recognition. Proceedings of the IEEE, 1998.","title":"Gradient-based learning applied to document recognition","authors":"Yann Lecun, Léon Bottou, Yoshua Bengio, Patrick Haffner","year":"1998","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Liu et al., 2015] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV, 2015.","title":"Deep learning face attributes in the wild","authors":"Ziwei Liu, Ping Luo, Xiaogang Wang, Xiaoou Tang","year":"2015","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Locatello et al., 2020] Francesco Locatello, Michael Tschannen, Stefan Bauer, Gunnar Rätsch, Bernhard Schölkopf, and Olivier Bachem. Disentangling factors of variations using few labels. In ICLR, 2020.","title":"Disentangling factors of variations using few labels","authors":"Francesco Locatello, Michael Tschannen, Stefan Bauer, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem","year":"2020","venue":"ICLR","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[O'Shaughnessy et al., 2020] Matthew R. O'Shaughnessy, Gregory Canal, Marissa Connor, Christopher Rozell, and Mark A. Davenport. Generative causal explanations of black-box classifiers. In NeurIPS, 2020.","title":"Generative causal explanations of black-box classifiers","authors":"R Matthew, Gregory O'shaughnessy, Marissa Canal, Christopher Connor, Mark A Rozell, Davenport","year":"2020","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Romano et al., 2020] Yaniv Romano, Matteo Sesia, and Emmanuel Candès. Deep knockoffs. Journal of the Amer- ican Statistical Association, 2020.","title":"Deep knockoffs","authors":"Yaniv Romano, Matteo Sesia, Emmanuel Candès","year":"2020","venue":"Journal of the American Statistical Association","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Sarkar et al., 2022] Anirban Sarkar, Deepak Vijaykeerthy, Anindya Sarkar, and Vineeth N. Balasubramanian. A framework for learning ante-hoc explainable models via concepts. In CVPR, 2022.","title":"A framework for learning ante-hoc explainable models via concepts","authors":"Anirban Sarkar, Deepak Vijaykeerthy, Anindya Sarkar, Vineeth N Balasubramanian","year":"2022","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Selvaraju et al., 2017] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In ICCV, 2017.","title":"Grad-cam: Visual explanations from deep networks via gradient-based localization","authors":"R Ramprasaath, Michael Selvaraju, Abhishek Cogswell, Ramakrishna Das, Devi Vedantam, Dhruv Parikh, Batra","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Stammer et al., 2021] Wolfgang Stammer, Patrick Schramowski, and Kristian Kersting. Right for the right concept: Revising neuro-symbolic concepts by interacting with their explanations. In CVPR, 2021.","title":"Right for the right concept: Revising neuro-symbolic concepts by interacting with their explanations","authors":"Stammer","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Tran et al., 2022] Thien Q. Tran, Kazuto Fukuchi, Youhei Akimoto, and Jun Sakuma. Unsupervised causal binary concepts discovery with VAE for black-box model expla- nation. In AAAI, 2022.","title":"Unsupervised causal binary concepts discovery with VAE for black-box model explanation","authors":"Q Thien, Kazuto Tran, Youhei Fukuchi, Jun Akimoto, Sakuma","year":"2022","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Yang et al., 2021] Mengyue Yang, Furui Liu, Zhitang Chen, Xinwei Shen, Jianye Hao, and Jun Wang. Causalvae: Disentangled representation learning via neural structural causal models. In CVPR, 2021.","title":"Causalvae: Disentangled representation learning via neural structural causal models","authors":"Yang","year":"2021","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Zhang et al., 2022] Wencan Zhang, Mariella Dimiccoli, and Brian Y. Lim. Debiased-cam to mitigate image perturba- tions with faithful visual explanations of machine learning. In CHI, 2022.","title":"Debiased-cam to mitigate image perturbations with faithful visual explanations of machine learning","authors":"Zhang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhou et al., 2016] Bolei Zhou, Aditya Khosla, Àgata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In CVPR, 2016.","title":"Learning deep features for discriminative localization","authors":"Zhou","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}