{"type":"ScholarlyArticle","title":"Negative Flux Aggregation to Estimate Feature Attributions","authors":[{"name":"Xin Li","slug":"xin-li","given":"Xin","family":"Li","affiliation":"Department of Computer Science , Wayne State University , USA"},{"name":"Deng Pan","slug":"deng-pan","given":"Deng","family":"Pan","affiliation":"Department of Computer Science , Wayne State University , USA"},{"name":"Chengyin Li","slug":"chengyin-li","given":"Chengyin","family":"Li","affiliation":"Department of Computer Science , Wayne State University , USA"},{"name":"Yao Qiang","slug":"yao-qiang","given":"Yao","family":"Qiang","affiliation":"Department of Computer Science , Wayne State University , USA"},{"name":"Dongxiao Zhu","slug":"dongxiao-zhu","given":"Dongxiao","family":"Zhu","affiliation":"Department of Computer Science , Wayne State University , USA"}],"abstract":"There are increasing demands for understanding deep neural networks' (DNNs) behavior spurred by growing security and/or transparency concerns. Due to multi-layer nonlinearity of the deep neural network architectures, explaining DNN predictions still remains as an open problem, preventing us from gaining a deeper understanding of the mechanisms. To enhance the explainability of DNNs, we estimate the input feature's attributions to the prediction task using divergence and flux. Inspired by the divergence theorem in vector analysis, we develop a novel Negative Flux Aggregation (NeFLAG) formulation and an efficient approximation algorithm to estimate attribution map. Unlike the previous techniques, ours doesn't rely on fitting a surrogate model nor need any path integration of gradients. Both qualitative and quantitative experiments demonstrate a superior performance of NeFLAG in generating more faithful attribution maps than the competing methods.  Our code is available at https://github.com/xinli0928/NeFLAG.","doi":"10.24963/ijcai.2023/50","url":"https://ijcai.org/proceedings/2023/50","pdf":"https://ijcai.org/proceedings/2023/0050.pdf","markdown":"https://ijcai.org/proceedings/2023/50.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/50","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI Ethics, Trust, Fairness","track":"Main Track","number":50,"pages":{"first":446,"last":454},"keywords":["AI Ethics, Trust, Fairness: ETF: Trustworthy AI","AI Ethics, Trust, Fairness: ETF: Explainability and interpretability"],"videos":[],"references":[{"position":1,"raw":"[Bach et al., 2015] Sebastian Bach, Alexander Binder, Gré- goire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7), 2015.","title":"On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation","authors":"Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, Wojciech Samek","year":"2015","venue":"PloS one","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Chefer et al., 2021] Hila Chefer, Shir Gur, and Lior Wolf. Transformer interpretability beyond attention visualization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 782-791, 2021.","title":"Transformer interpretability beyond attention visualization","authors":"Hila Chefer, Shir Gur, Lior Wolf","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2021] Hugh Chen, Scott M Lundberg, and Su-In Lee. Explaining a series of models by propagating shapley values. arXiv preprint arXiv:2105.00108, 2021.","title":"Explaining a series of models by propagating shapley values","authors":"Hugh Chen, Scott M Lundberg, Su-In Lee","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Deng et al., 2009] Jia Deng, Wei Dong, Richard Socher, Li- Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255.","title":"Imagenet: A large-scale hierarchical image database","authors":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, Li Fei-Fei","year":"2009","venue":"IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Erion et al., 2021] Gabriel Erion, Joseph D Janizek, Pascal Sturmfels, Scott M Lundberg, and Su-In Lee. Improving performance of deep learning models with axiomatic at- tribution priors and expected gradients. Nature Machine Intelligence, pages 1-12, 2021.","title":"Improving performance of deep learning models with axiomatic attribution priors and expected gradients","authors":"Joseph D Gabriel Erion, Pascal Janizek, Scott M Sturmfels, Su-In Lundberg, Lee","year":"2021","venue":"Nature Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[He et al., 2015] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. corr abs/1512.03385 (2015), 2015.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Hesse et al., 2021] Robin Hesse, Simone Schaub-Meyer, and Stefan Roth. Fast axiomatic attribution for neural networks. Advances in Neural Information Processing Systems, 34, 2021.","title":"Fast axiomatic attribution for neural networks","authors":"Hesse","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hooker et al., 2019] Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. A benchmark for interpretabil- ity methods in deep neural networks. Advances in neural information processing systems, 32, 2019.","title":"A benchmark for interpretability methods in deep neural networks","authors":"Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, Been Kim","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Kapishnikov et al., 2019] Andrei Kapishnikov, Tolga Boluk- basi, Fernanda Viégas, and Michael Terry. Xrai: Better attri- butions through regions. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4948- 4957, 2019.","title":"Xrai: Better attributions through regions","authors":"Andrei Kapishnikov, Tolga Bolukbasi, Fernanda Viégas, Michael Terry","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kapishnikov et al., 2021] Andrei Kapishnikov, Subhashini Venugopalan, Besim Avci, Ben Wedin, Michael Terry, and Tolga Bolukbasi. Guided integrated gradients: An adap- tive path method for removing noise. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5050-5058, 2021.","title":"Guided integrated gradients: An adaptive path method for removing noise","authors":"Andrei Kapishnikov, Subhashini Venugopalan, Besim Avci, Ben Wedin, Michael Terry, Tolga Bolukbasi","year":"2021","venue":"Proceedings of the IEEE/CVF conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Li et al., 2021] Xin Li, Xiangrui Li, Deng Pan, and Dongx- iao Zhu. Improving adversarial robustness via probabilisti- cally compact loss with logit constraints. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 8482-8490, 2021.","title":"Improving adversarial robustness via probabilistically compact loss with logit constraints","authors":"Xin Li, Xiangrui Li, Deng Pan, Dongxiao Zhu","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lundberg and Lee, 2017] Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Advances in neural information processing systems, pages 4765-4774, 2017.","title":"A unified approach to interpreting model predictions","authors":"M Scott, Su-In Lundberg, Lee","year":"2017","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Miglani et al., 2020] Vivek Miglani, Narine Kokhlikyan, Bilal Alsallakh, Miguel Martin, and Orion Reblitz- Richardson. Investigating saturation effects in integrated gradients. arXiv preprint arXiv:2010.12697, 2020.","title":"Investigating saturation effects in integrated gradients","authors":"Vivek Miglani, Narine Kokhlikyan, Bilal Alsallakh, Miguel Martin, Orion Reblitz-Richardson","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Montavon et al., 2017] Grégoire Montavon, Sebastian La- puschkin, Alexander Binder, Wojciech Samek, and Klaus- Robert Müller. Explaining nonlinear classification deci- sions with deep taylor decomposition. Pattern Recognition, 65:211-222, 2017.","title":"Explaining nonlinear classification decisions with deep taylor decomposition","authors":"Grégoire Montavon, Sebastian Lapuschkin, Alexander Binder, Wojciech Samek, Klaus-Robert Müller","year":"2017","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Pan et al., 2020] Deng Pan, Xiangrui Li, Xin Li, and Dongx- iao Zhu. Explainable recommendation via interpretable feature mapping and evaluation of explainability. In Chris- tian Bessiere, editor, Proceedings of the Twenty-Ninth Inter- national Joint Conference on Artificial Intelligence, IJCAI 2020, pages 2690-2696. ijcai.org, 2020.","title":"Explainable recommendation via interpretable feature mapping and evaluation of explainability","authors":"Xiangrui Deng Pan, Xin Li, Dongxiao Li, Zhu","year":"2020","venue":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020","doi":null,"url":null,"internal_url":"/proceedings/2020/373"},{"position":16,"raw":"[Pan et al., 2021] Deng Pan, Xin Li, and Dongxiao Zhu. Ex- plaining deep neural network models with adversarial gradi- ent integration. In Thirtieth International Joint Conference on Artificial Intelligence (IJCAI), 2021.","title":"Explaining deep neural network models with adversarial gradient integration","authors":"Xin Deng Pan, Dongxiao Li, Zhu","year":"2021","venue":"Thirtieth International Joint Conference on Artificial Intelligence (IJCAI)","doi":null,"url":null,"internal_url":"/proceedings/2021/396"},{"position":17,"raw":"[Petsiuk et al., 2018] Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421, 2018.","title":"Rise: Randomized input sampling for explanation of black-box models","authors":"Abir Vitali Petsiuk, Kate Das, Saenko","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Qiang et al., 2022] Yao Qiang, Deng Pan, Chengyin Li, Xin Li, Rhongho Jang, and Dongxiao Zhu. Attcat: Explain- ing transformers via attentive class activation tokens. In Advances in Neural Information Processing Systems, 2022.","title":"Attcat: Explaining transformers via attentive class activation tokens","authors":"Yao Qiang, Deng Pan, Chengyin Li, Xin Li, Rhongho Jang, Dongxiao Zhu","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Ribeiro et al., 2016] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. \" why should i trust you?\" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge dis- covery and data mining, pages 1135-1144, 2016.","title":"why should i trust you?\" explaining the predictions of any classifier","authors":"Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Shah et al., 2021] Harshay Shah, Prateek Jain, and Praneeth Netrapalli. Do input gradients highlight discriminative features? Advances in Neural Information Processing Systems, 34:2046-2059, 2021.","title":"Do input gradients highlight discriminative features?","authors":"Shah","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Shrikumar et al., 2017] Avanti Shrikumar, Peyton Green- side, and Anshul Kundaje. Learning important features through propagating activation differences. In Interna- tional Conference on Machine Learning, pages 3145-3153. PMLR, 2017.","title":"Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences","authors":"Shrikumar","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Simonyan et al., 2013] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional net- works: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013.","title":"Deep inside convolutional networks: Visualising image classification models and saliency maps","authors":"Karen Simonyan, Andrea Vedaldi, Andrew Zisserman","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Smilkov et al., 2017] Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smooth- grad: removing noise by adding noise. arXiv preprint arXiv:1706.03825, 2017.","title":"Smoothgrad: removing noise by adding noise","authors":"Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, Martin Wattenberg","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Srinivas and Fleuret, 2019] Suraj Srinivas and François Fleuret. Full-gradient representation for neural network visualization. Advances in neural information processing systems, 32, 2019.","title":"Full-gradient representation for neural network visualization","authors":"Suraj Srinivas, François Fleuret","year":"2019","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sturmfels et al., 2020] Pascal Sturmfels, Scott Lundberg, and Su-In Lee. Visualizing the impact of feature attribution baselines. Distill, 2020. https://distill.pub/2020/attribution- baselines.","title":"Visualizing the impact of feature attribution baselines","authors":"Pascal Sturmfels, Scott Lundberg, Su-In Lee","year":"2020","venue":"Distill","doi":null,"url":"https://distill.pub/2020/attribution-baselines","internal_url":null},{"position":27,"raw":"[Sundararajan et al., 2017] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep net- works. In International Conference on Machine Learning, pages 3319-3328. PMLR, 2017.","title":"Axiomatic attribution for deep networks","authors":"Mukund Sundararajan, Ankur Taly, Qiqi Yan","year":"2017","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Szegedy et al., 2015] Christian Szegedy, Vincent Van- houcke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wo- jna. Rethinking the inception architecture for computer vision. corr abs/1512.00567 (2015), 2015.","title":"Rethinking the inception architecture for computer vision","authors":"Szegedy","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}