{"type":"ScholarlyArticle","title":"Attribution Quality Metrics with Magnitude Alignment","authors":[{"name":"Chase Walker","slug":"chase-walker","given":"Chase","family":"Walker","affiliation":"Department of Electrical and Computer Engineering , University of Central Florida , Orlando , FL , USA"},{"name":"Dominic Simon","slug":"dominic-simon","given":"Dominic","family":"Simon","affiliation":"Department of Electrical and Computer Engineering , University of Central Florida , Orlando , FL , USA"},{"name":"Kenny Chen","slug":"kenny-chen","given":"Kenny","family":"Chen","affiliation":"Lockheed Martin , Orlando , FL , USA"},{"name":"Rickard Ewetz","slug":"rickard-ewetz","given":"Rickard","family":"Ewetz","affiliation":"Department of Electrical and Computer Engineering , University of Central Florida , Orlando , FL , USA"}],"abstract":"Attribution algorithms play an instrumental role in human interpretation of AI models. The methods measure the importance of the input features to the model output decision, which can be displayed as an attribution map for image classifiers. Perturbation tests are the state-of-the-art approach to evaluate the quality of an attribution map. Unfortunately, we observe that perturbation tests fail to consider attribution magnitude, which translates into inconsistent quality scores. In this paper, we propose Magnitude Aligned Scoring (MAS), a new attribution quality metric that measures the alignment between the magnitude of the attributions and the model response. In particular, the metric accounts for both the relative ordering and the magnitude of the pixels within an attribution. In the experimental evaluation, we compare the MAS metric with existing metrics across a wide range of models, datasets, attributions, and evaluations. The results demonstrate that the MAS metric is 4x more sensitive to attribution changes, 2x more consistent, and 1.6x more invariant to baseline modifications. Our code and the referenced appendix are publicly available via https://github.com/chasewalker26/Magnitude-Aligned-Scoring.","doi":"10.24963/ijcai.2024/59","url":"https://ijcai.org/proceedings/2024/59","pdf":"https://ijcai.org/proceedings/2024/0059.pdf","markdown":"https://ijcai.org/proceedings/2024/59.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/59","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"AI Ethics, Trust, Fairness","track":"Main Track","number":59,"pages":{"first":530,"last":538},"keywords":["Machine Learning: ML: Explainable/Interpretable machine learning","AI Ethics, Trust, Fairness: ETF: Explainability and interpretability","Computer Vision: CV: Interpretability and transparency","AI Ethics, Trust, Fairness: ETF: Trustworthy AI"],"videos":[],"references":[{"position":1,"raw":"[Adebayo et al., 2018] Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. NeurIPS, 31, 2018.","title":"Sanity checks for saliency maps","authors":"Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, Been Kim","year":"2018","venue":"NeurIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ancona et al., 2018] Marco Ancona, Enea Ceolini, Cengiz Öztireli, and Markus Gross. Towards better understand- ing of gradient-based attribution methods for deep neural networks. In International Conference on Learning Rep- resentations, 2018.","title":"Towards better understanding of gradient-based attribution methods for deep neural networks","authors":"Marco Ancona, Enea Ceolini, Cengiz Öztireli, Markus Gross","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Borji et al., 2013] Ali Borji, Dicky N. Sihite, and Laurent Itti. Quantitative analysis of human-model agreement in visual saliency modeling: A comparative study. IEEE Transactions on Image Processing, 22(1):55-69, 2013.","title":"Quantitative analysis of human-model agreement in visual saliency modeling: A comparative study","authors":"Ali Borji, N Dicky, Laurent Sihite, Itti","year":"2013","venue":"IEEE Transactions on Image Processing","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Bylinskii et al., 2019] Zoya Bylinskii, Tilke Judd, Aude Oliva, Antonio Torralba, and Frédo Durand. What do dif- ferent evaluation metrics tell us about saliency models? IEEE Transactions on Pattern Analysis and Machine In- telligence, 41(3):740-757, 2019.","title":"What do different evaluation metrics tell us about saliency models?","authors":"Zoya Bylinskii, Tilke Judd, Aude Oliva, Antonio Torralba, Frédo Durand","year":"2019","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chefer et al., 2021] Hila Chefer, Shir Gur, and Lior Wolf. Transformer interpretability beyond attention visualiza- tion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 782-791, 2021.","title":"Transformer interpretability beyond attention visualization","authors":"Hila Chefer, Shir Gur, Lior Wolf","year":"2021","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cheng et al., 2017] Gong Cheng, Junwei Han, and Xiao- qiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10):1865-1883, 2017.","title":"Remote sensing image scene classification: Benchmark and state of the art","authors":"Cheng","year":"2017","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Das and Rad, 2020] Arun Das and Paul Rad. Opportunities and challenges in explainable artificial intelligence (xai): A survey, 2020.","title":"Opportunities and challenges in explainable artificial intelligence (xai): A survey","authors":"Arun Das, Paul Rad","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hao et al., 2021] Yaru Hao, Li Dong, Furu Wei, and Ke Xu. Self-attention attribution: Interpreting information inter- actions inside transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12963-12971, 2021.","title":"Self-attention attribution: Interpreting information interactions inside transformer","authors":"Yaru Hao, Li Dong, Furu Wei, Ke Xu","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hooker et al., 2019a] Sara Hooker, Dumitru Erhan, Pieter- Jan Kindermans, and Been Kim. A benchmark for inter- pretability methods in deep neural networks. Advances in neural information processing systems, 32, 2019.","title":"A benchmark for interpretability methods in deep neural networks","authors":"Hooker","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hooker et al., 2019b] Sara Hooker, Dumitru Erhan, Pieter- Jan Kindermans, and Been Kim. A benchmark for inter- pretability methods in deep neural networks. Advances in neural information processing systems, 32, 2019.","title":"A benchmark for interpretability methods in deep neural networks","authors":"Hooker","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kapishnikov et al., 2019] Andrei Kapishnikov, Tolga Bolukbasi, Fernanda Viegas, and Michael Terry. Xrai: Better attributions through regions. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4947-4956, Los Alamitos, CA, USA, nov 2019. IEEE Computer Society.","title":"Xrai: Better attributions through regions","authors":"Andrei Kapishnikov, Tolga Bolukbasi, Fernanda Viegas, Michael Terry","year":"2019","venue":"IEEE/CVF International Conference on Computer Vision (ICCV)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kapishnikov et al., 2021a] Andrei Kapishnikov, Tolga Bolukbasi, Fernanda Viegas, and Michael Terry. Xrai code repository, 2021. Accessed: 2022-10-15.","title":"Xrai code repository","authors":"Kapishnikov","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Kapishnikov et al., 2021b] Andrei Kapishnikov, Subhashini Venugopalan, Besim Avci, Ben Wedin, Michael Terry, and Tolga Bolukbasi. Guided integrated gradients: an adap- tive path method for removing noise. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5048-5056, Los Alamitos, CA, USA, jun 2021. IEEE Computer Society.","title":"Guided integrated gradients: an adaptive path method for removing noise","authors":"Kapishnikov","year":"2021","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Krippendorff, 2004] Klaus Krippendorff. Content Analysis: An Introduction to Its Methodology (second edition). Sage Publications, 2004.","title":"Content Analysis: An Introduction to Its Methodology","authors":"Krippendorff","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Kümmerer et al., 2014] Matthias Kümmerer, Thomas Wal- lis, and Matthias Bethge. How close are we to un- derstanding image-based saliency? arXiv preprint arXiv:1409.7686, 2014.","title":"How close are we to understanding image-based saliency","authors":"Matthias Kümmerer, Thomas Wallis, Matthias Bethge","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Miglani et al., 2020] Vivek Miglani, Narine Kokhlikyan, Bilal Alsallakh, Miguel Martin, and Orion Reblitz- Richardson. Investigating saturation effects in integrated gradients, 2020.","title":"Investigating saturation effects in integrated gradients","authors":"Vivek Miglani, Narine Kokhlikyan, Bilal Alsallakh, Miguel Martin, Orion Reblitz-Richardson","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Pan et al., 2021] Deng Pan, Xin Li, and Dongxiao Zhu. Ex- plaining deep neural network models with adversarial gra- dient integration. In Zhi-Hua Zhou, editor, Proceedings of the Thirtieth International Joint Conference on Arti- ficial Intelligence, IJCAI-21, pages 2876-2883. Interna- tional Joint Conferences on Artificial Intelligence Orga- nization, 8 2021. Main Track.","title":"Explaining deep neural network models with adversarial gradient integration","authors":"Xin Deng Pan, Dongxiao Li, Zhu","year":"2021","venue":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21","doi":null,"url":null,"internal_url":"/proceedings/2021/396"},{"position":20,"raw":"[Paszke et al., 2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In Proceedings of the 33rd International Conference on Neural Information Pro- cessing Systems.","title":"Pytorch: An imperative style, highperformance deep learning library","authors":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach Devito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, Soumith Chintala","year":null,"venue":"Proceedings of the 33rd International Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Petsiuk et al., 2018a] Vitali Petsiuk, Abir Das, and Kate Saenko. Rise code repository, 2018. Accessed: 2022-11- 01.","title":"Rise code repository","authors":"Petsiuk","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Petsiuk et al., 2018b] Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Randomized input sampling for explana- tion of black-box models. In Proceedings of the British Machine Vision Conference (BMVC), 2018.","title":"Rise: Randomized input sampling for explanation of black-box models","authors":"Petsiuk","year":"2018","venue":"Proceedings of the British Machine Vision Conference (BMVC)","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Qiang et al., 2022] Yao Qiang, Deng Pan, Chengyin Li, Xin Li, Rhongho Jang, and Dongxiao Zhu. Attcat: Explain- ing transformers via attentive class activation tokens. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Pro- cessing Systems, volume 35, pages 5052-5064.","title":"Attcat: Explaining transformers via attentive class activation tokens","authors":"Yao Qiang, Deng Pan, Chengyin Li, Xin Li, Rhongho Jang, Dongxiao Zhu","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Ribeiro et al., 2016] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. \"why should i trust you?\" explain- ing the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowl- edge discovery and data mining, pages 1135-1144, 2016.","title":"why should i trust you?\" explaining the predictions of any classifier","authors":"Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Russakovsky et al., 2015] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. Ima- geNet Large Scale Visual Recognition Challenge. Inter- national Journal of Computer Vision (IJCV), 115(3):211- 252, 2015.","title":"Ima-geNet Large Scale Visual Recognition Challenge","authors":"Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C Berg, Li Fei-Fei","year":"2015","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Samek et al., 2016] Wojciech Samek, Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, and Klaus- Robert Müller. Evaluating the visualization of what a deep neural network has learned. IEEE transactions on neural networks and learning systems, 28(11):2660-2673, 2016.","title":"Evaluating the visualization of what a deep neural network has learned","authors":"Wojciech Samek, Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, Klaus-Robert Müller","year":"2016","venue":"IEEE transactions on neural networks and learning systems","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Selvaraju et al., 2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on com- puter vision, pages 618-626, 2017.","title":"Grad-cam: Visual explanations from deep networks via gradient-based localization","authors":"Michael Ramprasaath R Selvaraju, Abhishek Cogswell, Ramakrishna Das, Devi Vedantam, Dhruv Parikh, Batra","year":"2017","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Shrikumar et al., 2016] Avanti Shrikumar, Peyton Green- side, Anna Shcherbina, and Anshul Kundaje. Not just a black box: Learning important features through propagating activation differences. arXiv preprint arXiv:1605.01713, 2016.","title":"Avanti Shrikumar, Peyton Greenside","authors":"Shrikumar","year":"2016","venue":"Learning important features through propagating activation differences","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Simonyan et al., 2014] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps, 2014.","title":"Deep inside convolutional networks: Visualising image classification models and saliency maps","authors":"Karen Simonyan, Andrea Vedaldi, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Springenberg et al., 2015] Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Ried- miller. Striving for simplicity: The all convolutional net, 2015.","title":"Striving for simplicity: The all convolutional net","authors":"Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, Martin Riedmiller","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Sundararajan et al., 2017] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep net- works. In Proceedings of the 34th International Confer- ence on Machine Learning -Volume 70, ICML'17, page 3319-3328. JMLR.org, 2017.","title":"Axiomatic attribution for deep networks","authors":"Mukund Sundararajan, Ankur Taly, Qiqi Yan","year":"2017","venue":"Proceedings of the 34th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Tomsett et al., 2020] Richard Tomsett, Dan Harborne, Supriyo Chakraborty, Prudhvi Gurram, and Alun Preece. Sanity checks for saliency metrics. In AAAI, volume 34, pages 6021-6029, 2020.","title":"Sanity checks for saliency metrics","authors":"Richard Tomsett, Dan Harborne, Supriyo Chakraborty, Prudhvi Gurram, Alun Preece","year":"2020","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Yuan et al., 2021] Tingyi Yuan, Xuhong Li, Haoyi Xiong, Hui Cao, and Dejing Dou. Explaining information flow inside vision transformers using markov chain. In eXplain- able AI approaches for debugging and diagnosis., 2021.","title":"Explaining information flow inside vision transformers using markov chain","authors":"Tingyi Yuan, Xuhong Li, Haoyi Xiong, Hui Cao, Dejing Dou","year":"2021","venue":"eXplainable AI approaches for debugging and diagnosis","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Zeiler and Fergus, 2014] Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In European conference on computer vision, pages 818-833. Springer, 2014.","title":"Visualizing and understanding convolutional networks","authors":"D Matthew, Rob Zeiler, Fergus","year":"2014","venue":"European conference on computer vision","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}