---
title: "Concept-Level Causal Explanation Method for Brain Function Network Classification"
authors:
  - "Jinduo Liu"
  - "Feipeng Wang"
  - "Junzhong Ji"
doi: "10.24963/ijcai.2024/342"
url: "https://ijcai.org/proceedings/2024/342"
pdf: "https://ijcai.org/proceedings/2024/0342.pdf"
venue: "Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence"
year: 2024
pages: "3087-3096"
track: "Main Track"
keywords:
  - "Humans and AI: HAI: Brain sciences"
  - "Data Mining: DM: Networks"
  - "AI Ethics, Trust, Fairness: ETF: Explainability and interpretability"
  - "Machine Learning: ML: Causality"
---

# Concept-Level Causal Explanation Method for Brain Function Network Classification

Jinduo Liu, Feipeng Wang, Junzhong Ji

*Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence*, pages 3087-3096. DOI: [10.24963/ijcai.2024/342](https://doi.org/10.24963/ijcai.2024/342)

## Abstract

Using deep models to classify brain functional networks (BFNs) for the auxiliary diagnosis and treatment of brain diseases has become increasingly popular. However, the unexplainability of deep models has seriously hindered their applications in computer-aided diagnosis. In addition, current explanation methods mostly focus on natural images, which cannot be directly used to explain the deep model for BFN classification. In this paper, we propose a concept-level causal explanation method for BFN classification called CLCEM. First, CLCEM employs the causal learning method to extract concepts that are meaningful to humans from BFNs. Second, it aggregates the same concepts to obtain the contribution of each concept to the model output. Finally, CLCEM adds the contribution of each concept to make a diagnosis. The experimental results show that our CLCEM can not only accurately identify brain regions related to specific brain diseases but also make decisions based on the concepts of these brain regions, which enables humans to understand the decision-making process without performance degradation.

## Full text

## 1 Introduction

The brain functional network (BFN) is usually constructed from functional magnetic resonance imaging (fMRI) data, which can reveal the patterns of brain functional activity. Current studies have shown that functional connectivity (FC) is an effective biomarker for the diagnosis of mental disorders, so it is of great significance and value to diagnose mental disorders based on the BFN [Seguin et al.2023, Steward et al.2023,Liu et al.2022]. In recent years, using deep learning methods to classify BFN has realized the auxiliary diagnosis and treatment of brain diseases, which has become a research hotspot in the medical imaging area. However, the black-box nature (uninterpretability) of deep learning methods has led to a fatal flaw in real applications [Gu et al.2020]. Therefore, it is necessary to explore a novel explanation method for deep learning methods in BFN classification [Hu et al.2021].

∗Corresponding Author

Current explanation methods can be roughly divided into two categories: post-hoc methods, and built-in methods. The post-hoc methods aim to explain the decision-making process of a well-trained model by some means, including saliency map methods [Selvaraju et al.2019], example-based methods [Kim et al.2016], and perturbation-based methods [Wagner et al.2019]. Among them, the saliency map methods reversepropagate the model to obtain the derivatives of input features and measure the contributions of input features to model outputs. The example-based methods pick a specific example from the dataset to explain the predictions and data distribution of models. The perturbation-based methods observe changes in the output of the model by perturbing some input data, determine the weight of the perturbation data based on the degree of changes, and explain by the perturbation data and its weight. Overall, these methods have difficulty in automatically explaining how a deep model makes decisions.

By comparison, built-in methods are apt to reveal the reasoning process of models through some customized structures or restrictions in training, which can automatically produce an explanation that humans can understand. The existing builtin methods can be roughly divided into three types: activation map-based methods [Woo et al.2018], protype-based methods [Li et al.2018] and concept-based methods [Ghorbani et al.2019]. Activation map-based methods use feature upsampling to obtain a weighted combination of activation maps to indicate the important pixels. Essentially, activation mapbased methods belong to the pixel-level explanation, which is difficult for humans to understand. The prototype-based methods use special structures to extract prototypes in the training process, and make a decision by comparing the input samples with each prototype. However, their prototype extraction abilities may poor. Thus, they always deal with relatively simple gray image datasets, e.g., MNIST. In contrast, concept-based methods can explain the reasoning process and result in a way that humans easily understand, and the explanation effects for these methods have nothing to do with the complexity of the data. Therefore, from the perspective of convenience, the concept-based methods are better than other built-in methods in real applications [Yao et al.2023]. This provides a clue for explaining BFN classification. Even so, current concept-based methods usually bind extracted features with concepts by visual means, which are suitable for natural pictures. In fact, BFN data have no visual meaning, so there is an urgent need for a new method, which can bind the extracted features with concepts that are meaningful to clinicians without visual means and provide a reasonable explanation for the BFN classification process.

To accommodate this need, we propose a novel conceptlevel causal explanation method for BFN classification called CLCEM in this paper. The proposed method can bind the extracted features from BFN with the concepts of BFN by using the strength of causal relationships between them, and give a reasonable explanation from the view of concepts. In detail, CLCEM includes three phases: concept extraction process, concept aggregation process and decision process. CLCEM first extracts human-understandable concepts (regions of interest) from the BFN by means of causal concept loss in the concept extraction process. After that, it utilizes the location connection to aggregate the same concepts to obtain the contribution of every concept to the output of the model in the concept aggregation process. Finally, CLCEM computes the contribution of each concept for outputs to obtain a probability of whether the subject is a patient or not in the decision process. The main contributions of our paper can be summarized as follows:

- To the best of our knowledge, the proposed CLCEM is the first work that develops a concept-level causal explanation method to explain the inference process of deep learning models in the brain network classification task.
- The proposed method employs the causal concept loss, which can automatically extract human-understandable concepts (regions of interest) from BFN without the need for manual intervention.
- Systematic experiments have been conducted to verify that CLCEM can effectively explain the decision process of the BFN classification model without losing the classification performance of original deep learning models.

## 2 RelatedWorks

## 2.1 BFNClassificationMethods

Current studies have shown that the connection between brain regions is an effective biomarker for the diagnosis of brain disorders [Liu et al.2024, Zhang et al.2024]. Therefore, an increasing number of BFN classification methods have been proposed for the computer-aided diagnosis (CAD) of mental illness in the past several years. These methods can be roughly divided into two categories: traditional machine learning methods and deep learning methods. The traditional machine learning methods including linear regression (LR), support vector machine (SVM), random forest (RF), often achieve low performance without manually extracting features [de Vos et al.2018, Sen and Parhi2021]. Different from traditional machine learning methods, deep learning methods can automatically extract high-level features to obtain good classification performance. Therefore, using the deep learning method to classify BFNs has received increasing popularity from related researchers. Henaff [Henaff et al.2015] pointed out that a convolution neural network (CNN) can effectively extract the topology information of BFN with fewer parameters. Kawahara [Kawahara et al.2017] proposed a new CNN called BrainNetCNN that is more suitable for the special topology of BFNs by using three new convolution layers: e2e, e2n, and n2g. Ji [Ji et al.2021] proposed a new convolution kernel CNNEW with an elementwise weighting mechanism to extract hierarchical topological features of brain networks. Each weight is assigned to an element with a unique neuroscience significance. Although deep learning methods tend to obtain better performance more easily than traditional machine learning methods, there is a fatal defect in them called uninterpretability, which makes people unable to understand the decision process of deep models and hinders their widespread adoption in CAD [Kwon et al.2019].

## 2.2 ExplanationMethods

At present, the explanation methods can be divided into two types: post-hoc, and built-in. post-hoc methods treat the entire deep model as a black box and try to explain its decisions by analyzing the external behavior of the model. However, the explanations given by the post-hoc methods illustrate particular areas valued by deep models [Selvaraju et al.2019, Kim et al.2016, Wagner et al.2019, Byrne2023]. These post-hoc methods regard the deep model as a black box and cannot explain the decision-making process of the models. By comparison, built-in methods [Woo et al.2018,Li et al.2018, Ghorbani et al.2019], which are integrated into deep models, can explain their decision-making processes. In recent years, a multitude of concept-based built-in methods have been introduced [Goyal et al.2019,O'Shaughnessy et al.2020,Heskes et al.2020], owing to their advanced functionality in comparison to other methods. However, current concept-based methods focus on tasks related to natural pictures and are not applicable to BFN classification [Huai et al.2022]. Therefore, there is an urgent need to develop novel concept-based explaination methods for BFN classification.

## 3 Preliminaries

## 3.1 ProblemDescription

BFN can be described as a two-dimensional matrix, where each element (feature) represents the functional connectivity strength between two brain regions. Since most existing human-brain medical studies focus on brain regions, which are also known as regions of interest (ROIs), we define the ROI as the concept to facilitate understanding the decisionmaking process of deep models for BFN classification. The corresponding elements of each ROI in the BFN are shown in Figure. 1, where ROI i corresponds to the elements in the i-th row and i-th column of the BFN.

Figure 1: The corresponding elements of each ROI in BFN.

![Image](https://ijcai.org/proceedings/2024/fulltext/342/image_000000_af2ff5bfe45438f39b95b99cd33d1228e9835e864c8a5b1bd5ddb11a2dee3576.png)

Figure 2: The schematic diagram of CLCEM. The CLCEM first utilizes the causal concept loss to extract m concepts from the BFN in the concept extraction process, subsequently aggregates the same concepts to obtain the contribution of each concept to the output of the model in the concept aggregation process by the location connection, and finally makes decisions by computing these contributions of concepts to the output in the decision process. Notably, grey balls represent common features (The features may be extracted from any deep learning model that needs to be explained. In the experiment section, we employed the most commonly used convolutional neural network (CNN) model), each color ball represents one of n concepts, and bicolor balls represent the contributions of concepts to the output, in which the left color represents the concept and the right color represents the category.

![Image](https://ijcai.org/proceedings/2024/fulltext/342/image_000001_20079d62c34a280192160511142f43766741194c0a3cb730be9477ab5faf83dc.png)

Current explanation works mostly focus on traditional natural image classification tasks, and the pixel set is easy for humans to understand, such as with a bird's head and feathers. However, for BFN classification, each element has a unique neurological significance. Limited by current human brain medical studies, the element set may be difficult for people to understand. This has also caused great obstacles to the explanation works. Therefore, it is necessary to define the concept to provide an understandable explanation in BFN classification tasks.

## 3.2 Notations

Let X ∈ Rn×n be the matrix of a BFN, where n is the number of ROIs. The label of the BFN is given by Y. The concept set of a BFN can be denoted as {C0, C1, ..., Cn-1}, where n is the total number of ROIs. The feature set of the concept extraction process can be represented as {v0, v1, ..., vm-1}, where m is the number of concepts which are extracted by the method. According to the definition, n and m may be different. Note that we treat ROIs (brain regions) as humanunderstandable concepts, so numerically they are considered equal in this paper (m= n).

## 4 Methodology

## 4.1 Main Idea

In this section, we propose a concept-level explanation method based on causality called CLCEM to provide an explanation of deep models for BFN classification. The proposed method can automatically extract humanunderstandable concepts from BFN through depth structure and use these concepts to make decisions to explain deep models. Figure. 2 shows the schematic diagram of the CLCEM. The feature extractor only plays the role of a deep feature extractor without any special requirement, which can be artificial neural network (ANN), CNN and so on. Therefore, in this paper, we focus on the concept extraction process, concept aggregation and decision process. Next, we will introduce them in detail.

## 4.2 Concept Extraction

From Figure. 2, we can see that the concept extraction process takes the deep features extracted from BFN as inputs and outputs human-understandable concepts. In detail, the concept extraction process first uses the causal estimator to evaluate the causal relationship strength between the deep features extracted from BFN and the FC strength of n concepts in BFN. Then, it utilizes the causal relationship strength to calculate the causal concept loss. By optimizing it, the concept extraction process can automatically extract the concepts that are meaningful to humans. Next, we introduce the causal estimator and causal concept loss, respectively.

## Causal Estimator

The causal estimator is consists of n multilayer perceptrons (MLPs), which can be regarded as nonlinear structural equation modeling (SEM). They can evaluate the causal strength between n concepts of BFN and each of m features in the concept extraction process. Next, we explain how to evaluate causal strength by causal estimators.

The SEM used by the causal estimator is a special form of a Bayesian network model, and is skilled in verifying the causal relationship between continuous variables. Generally, SEM can be roughly divided into two categories: linear and nonlinear. Considering the complex nonlinear relationship between the input features and the outputs of hidden layer neurons, we choose nonlinear SEM to evaluate causal strength in the causal estimator.

For concept Ci and feature vj, we can use nonlinear SEM

to evaluate the causal strength between them as

<!-- formula-not-decoded -->

where fCi,vj (·) is a nonlinear fitting function for vj by Ci, and ϵi,j is the fitting error. By Eq. (1), the loss for the causal estimator can be written as

<!-- formula-not-decoded -->

where m is the quantity of features in the concept extraction process and n is the number of ROIs or concepts in the BFN.

After fitting well by minimizing CEL, ϵi,j is inversely proportional to the causal strength fromCi to vj. By Eq. (1), we can obtain the causal strength fromCi to vj as

<!-- formula-not-decoded -->

And the si,j can be used to calculate the causal concept loss for CLCEM.

## Causal Concept Loss

The causal concept loss can be calculated by the causal relationship strength evaluated by the causal estimator. By minimizing it, the concept extraction process can automatically extract m concepts from BFN. Next, we will introduce how to calculate the causal concept loss.

Through the causal estimator, we have obtained the causal relationship strength from the ith concept of BFN to the jth feature in the concept extraction process as si,j. We can obtain the causal concept loss as

<!-- formula-not-decoded -->

where ` sort(.) is the descending sort function. According to Eqs.(3) and (4), we can finally obtain the causal concept loss as

<!-- formula-not-decoded -->

where ´ sort(.) is the ascending sort function. By minimizing CCL, each feature of the concept extraction process is forced to maintain a strong causal relationship with one of the n concepts in BFN and weaker causal relationships with the other concepts. Therefore, we can think that there are m concepts automatically extracted from BFN by minimizing the causal concept loss in the concept extraction process.

## 4.3 Concept Aggregation and Decision

Through the concept extraction process, we obtained m concepts extracted from the deep features of BFN. The causal concept loss CCL cannot guarantee that there are no duplicates in m concepts. This will lead to a misunderstanding if the model uses these concepts to make decisions directly. We cannot explain the difference among the duplicate concepts. So we need the concept aggregation process to aggregate the same concepts to obtain the contribution of every concept to outputs. Then the deep model can make decisions based on these contributions of concepts in the decision process.

In the concept aggregation process, CLCEM uses location connections to aggregate the same concepts. There are 2n aggregated concepts in it as {αi,0, αi,1|i ∈ [0, n)}, and each kind of concept corresponds to two aggregated concepts, which represent its contribution to the subject as a healthy person or as a patient. When aggregating, CLCEM connects only the neurons representing the ith concept in the concept extraction process to αi,0 and αi,1.

During the decision process, CLCEM can calculate the probability that the subject is a patient or healthy person through addition as follows:

<!-- formula-not-decoded -->

where y = 0 or 1 represents that the subject is a healthy person or patient, respectively.

In other words, through the concept extraction process, concept aggregation process and decision process, we can say that the CLCEM has a transparent decision-making process, reflects the contribution of various concepts to decisionmaking and is highly explanatory.

## 4.4 Model Training

In training, the causal estimator attempts to use n concepts to fit the deep features extracted from BFN by the feature extractor. This increases the accuracy of the estimation of causal relationship strength. The causal concept loss for the feature extractor tries to extract m concepts by making each of the m features in the concept extraction process fit well by only one of the n concepts using the causal estimator. Therefore, we can also regard this training process as adversarial training, and the loss for the causal estimator and feature extractor can be separately written as Loss1 and Loss2

<!-- formula-not-decoded -->

where λ (set to 0.1) is the weight of causal concept loss for concept extraction, CE(., .) is the cross entropy loss for classification, and Y ′ or Y represents the output label or real label vector, respectively.

The whole process of CLCEM's training is shown in Algorithm 1, where nbepochs is the number of training epochs, lr is the learning rate and wd represents the L2 regularity coefficient.

In detail, the algorithm takes the training set (Xt, Yt) and verification set (Xv, Yv) as inputs and the parameters of the total network Θ as the output. Xt and Xv are the BFNs in the training and verification sets, respectively. Yt or Yv is the corresponding label set in the training or verification set, respectively. {v0, v1, ..., vm-1} is the output of the feature extractor, which can be bound to concepts by the concept extraction process. fCi,vj (Ci) is the output of the causal estimator in the concept extraction process, which tries to use concept i of BFN to fit vj .

Algorithm 1 can be roughly divided into three stages in each epoch: first, train the causal estimator in the concept

## Algorithm 1 Model Training of CLCEM

Input: Xt, Yt, Xv, Yv

Parameter: nbepochs, lr, wd, λ

Output: Θ

- 1: Initialize Adam optimizer.
- 2: Let minloss = 1e5 .
- 3: for in range(nepochs) do
- 4: for in range(2) do
- 5: Input Xt into the feature extractor to get the output as {v0, v1, ..., vm-1}.
- 6: Input Xt into the causal estimator to get the output as {fCi,vj (Ci), i ∈ [0, n), j ∈ [0, m)}.
- 7: Calculate Loss1 by Eqs.(1), (2) and (7).
- 8: Minimize Loss1 by optimizer1.
- 9: end for
- 10: Input Xt into the causal estimator to get the output as {fCi,vj (Ci), i ∈ [0, n), j ∈ [0, m)}.
- 11: Input Xt into the feature extractor to get the output as {v0, v1, ..., vm-1}.
- 12: Calculate Loss2 by Eqs.(1), (5) and (7).
- 13: Minimize Loss2 by optimizer2.
- 14: Input Xv into the causal estimator and feature extractor separately.
- 15: Calculate Loss2 by Eqs.(1), (5) and (7).
- 16: if minloss &gt; Loss2 then
- 17: minloss = Loss2.
- 18: Save the parameters of the total network as Θ.
- 19: end if
- 20: end for
- 21: return Θ.

extraction process on the training set by minimizing Loss1 (5-10). Then, calculate Loss2 by the output of the causal estimator and feature extractor on the training set and minimize it to train the feature extractor (11-14). Finally, keep the best parameters of the total network with minimal Loss2 on the verification set (15-20). To clarify the training process of CLCEM, we have drawn it as Figure. 3. In practical usage, CLCEM as an explanation method, is trained end-to-end together with a BFN classification model (e.g., CNN).

Figure 3: The training process of CLCEM.

![Image](https://ijcai.org/proceedings/2024/fulltext/342/image_000002_a34c249c618b60d87cc86788c83a201990126e5352e2c08828bc37adb7c808d3.png)

## 5 Results and Discussion

In the experiments, our initial evaluation focuses on assessing the strength of the causal relationship between BFN concepts and the features of the concept extraction process. This evaluation aims to verify whether CLCEM can automatically extract human-understandable concepts based on the causal concept loss. Subsequently, the explanation's accuracy is confirmed through an assessment of the contribution of ROIs to classification. This involves a comparison of evaluation results with existing medical studies, followed by the identification of key ROIs to guide the classification process. The code is available at https://github.com/bjutAILab/CLCEM.

## 5.1 Configuration of the Device Operating Environment

This section outlines the computational framework used for the experiments, which includes both hardware and software components. The hardware setup features a highperformance NVIDIA GeForce RTX 3090 and RTX 3080 Ti GPUs, paired with an AMD Ryzen 9 5950X 16-Core Processor. This combination forms a powerful computing environment. The system operates on Ubuntu 20.04 LTS, providing a stable foundation for our projects. It is equipped with 64GB of high-speed DDR4 RAM, ensuring exceptional responsiveness and efficiency. For deep learning operations, we use the PyTorch framework, version 1.12.1, which includes GPU support via CUDA version 11.4.

## 5.2 Dataset Description

The dataset used in this paper is obtained from the ABIDE I database. It has both functional and structural brain imaging data of 1112 individuals including 539 autism spectrum disorder (ASD) patients and 573 typically developing (TD) controls, which were collected from 16 different sites around the world. Since the data processing of fMRI is very flexible, the preprocessed connectomes project provides the preprocessed fMRI data of ABIDE I, and is available at http://preprocessed-connectomes-project.org/abide/.

In this study, we chose 505 ASD patients and 530 TD controls from the dataset after removing subjects who did not have complete phenotypic information. The data employs automated anatomical labeling (AAL, 90 ROIs except for cerebellum) template and regard the mean value of all voxels in an ROI as its signal.

## 5.3 Metric Description

## Concept Extraction Metric

In this paper, the metrics we use can be divided into two categories: causal metrics and performance metrics. For the causal metric, we mainly use it to evaluate the causal relationship strength between the concepts of BFN and the features of the concept extraction process. By this, we can verify whether the causal concept loss in CLCEM works. For a given causal structure G, the causal metric Mcl can be calculated as

<!-- formula-not-decoded -->

Figure 4: The Mcl between the randomly selected 50 concepts of BFN and the features of concept extraction process. Note that the 50 concepts represent the causal relationships between features in the concept extraction process, not the final selection of 90 concepts (ROIs).

![Image](https://ijcai.org/proceedings/2024/fulltext/342/image_000003_385fc3494f64c26d256d2e779693d9c730a76651c6b445ccda613ab325e125d4.png)

where X ∈ Rτ×dθ represents the given data, ˆ θ is the maximum likelihood estimation of G's parameter θ, dθ is the dimensionality of θ or the amount of variables in G and τ denotes the amount of samples in X.

Then, we choose regression models to describe every causal relationship and assign the ˆxki as the corresponding estimate for xki , which is the ith variable in the kth observed sample. Thus we can obtain Mcl as

<!-- formula-not-decoded -->

where RSSi = ∑τ-1 k=0 (ˆxki -xki )2 represents the fitting error of the regression models. Moreover, the Eq. (9) is equivalent to the log-likelihood objective used by GraN-DAG, and we further assume that the variances of noise are equal. Thus, the Mcl can finally be written as

<!-- formula-not-decoded -->

where dθ is the dimensionality of θ or the amount of variables in G and τ denotes the amount of samples in X. RSSi = ∑τ-1 k=0 (ˆxki -xki )2 represents the fitting error of the regression models. The smaller theMcl, the stronger the causal relationship.

## Classification Performance Metric

We use six performance metrics primarily to verify the correctness of the explanation and evaluate the influence of CLCEM on classification performance. Four common performance metrics include accuracy (Acc), recall (Rec), precision (Pre) and F1-score (F1). In addition, we also utilize balanced accuracy (b-Acc) and the area under the curve (AUC) to further evaluate the expected performance.

b-Acc is a metric used to evaluate the performance of classification models, particularly when dealing with imbalanced datasets where the number of instances in each class is not evenly distributed. It can be calculated by:

<!-- formula-not-decoded -->

where TP, FP, TN and FN are the number of true positive subjects, false positive subjects, true negative subjects and false negative subjects, respectively.

AUC refers specifically to the area under ROC curve in this paper, which takes false positive rate (FPR) as abscissa and true positive rate (TPR) as ordinate. They can be calculated by:

<!-- formula-not-decoded -->

<!-- formula-not-decoded -->

## 5.4 Effectiveness of Causal Concept Loss in CLCEM for Concept Extraction

To verify that CLCEM can automatically extract humanunderstandable concepts by the causal concept loss to explain the decision process of deep models, we evaluate the causal relationship strength between the concepts of BFN and features of the concept extraction process. In this paper, we use Mcl to accomplish this goal.

The Mcl between the concepts of the input BFN and the features of the concept extraction process are shown in Figure. 4, where the abscissa represents the serial number of features, and the ordinate represents the value of Mcl. In detail, the Mcl is calculated by Eq. (10), and then we standardized them by

<!-- formula-not-decoded -->

For the demonstration purposes, we randomly selected only 50 features from all features. Because too many features will make the graph too dense and difficult to understand. There are two different types of graphs in Figure. 4. The histogram represents theMcl of the strongest causal relationship between the corresponding feature and n BFN concepts, and the box diagram represents the distribution of Mcl between the corresponding feature and the other n - 1 BFN concepts. We find that all bar charts are much lower than the lower edge of the corresponding box chart. BecauseMcl is inversely proportional to the strength of causal relationship, we can assert that every feature maintains a strong causal relationship with one of n BFN concepts and weaker ones with the others. It is worth noting that there are thousands of features in brain network classification involving 90 brain regions. We randomly displayed 50 to convey that the relationships extracted by our method are effective.

Therefore, we can conclude that by causal concept loss, CLCEM can automatically extract concept-level causal explanations for deep models' decision processes.

## 5.5 Explanation Analysis of CLCEM in ASD Classification

To verify the correctness of the explanation generated by CLCEM, we used the explanation to locate the ROI highly related to ASD. Then, we checked these ROIs based on existing brain science research results. Finally, we relied only on these ROI for classification to more intuitively show the correctness of the explanation.

Next, we introduce how to use the explanation generated by CLCEM to locate the ROI highly related to ASD or contributing most to the CAD of ASD. The concept aggregation process uses location connections to aggregate the same concepts to obtain the contribution of each concept to the output. Let us abstract the concept extraction process and concept aggregation process of CLCEM into a nonlinear function Γ. Then, the output α ∈ R2n of the concept aggregation process for an input can be written as

<!-- formula-not-decoded -->

where α = {αi,j}(i ∈ [0, n), j ∈ [0, 2), and x represents an input subject. αi,j is the probability of the input subject's label equaling j according to the concept i.

From the perspective of classification, we think that the concept whose probability differs greatly for different classes is important. Therefore, we can use α to locate the ROI that contributes greatly to classification by

<!-- formula-not-decoded -->

where si is the score of ROI i and is positively correlated with the contribution to classification, αk i,j is the probability of the kth input subject's label equaling to j according to the concept i and q is the quantity of input subjects.

Using Eq. (16), we can obtain all ROI contributions to classification. For ease of presentation, we standardized them according to Eq. (14). For a comprehensive and detailed presentation, we use Table 1 and the Figure. 5 to display the location of the top 10 important ROIs in the human brain, which can show the important ROI more figuratively. In Table 1, "No." is the default order of ROI in the AAL template, and "score" indicates scientific counting.

To more clearly illustrate the important ROIs, Figure 5 is presented to display the locations of these ROIs. From Figure. 5 we can find that most of the functions of the ROI are closely related to some typical manifestations of ASD patients, such as lack of social contact, difficulty in communicating, stereotyped or repeated behavior and interest, lack of sensitivity to facial features and slow development of perception. Correspondingly, the LING.R is related to visual signal processing, logic analysis and visual memory. The STG.R is the language center and plays an important role in social cognition. The ANG.L is mainly involved in the language and digit processing of human brain and in processes related to attention. The CAL.L is related to visual function. The ORBsupmed.R is believed to contribute to cognitive function. The MTG.R is associated with different processes that recognize known faces, semantic memory processing, integrate multimodal sensory, etc. Finally, the FFG.R is important in object and face recognition.

Table 1: The score of the top 10 ROIs evaluated by CLCEM.

|   No. | ROI                                      | abbreviation   |   score |
|-------|------------------------------------------|----------------|---------|
|    48 | Lingual gyrus                            | LING.R         |  1.00e0 |
|    82 | Superior temporal gyrus                  | STG.R          | 9.99e-1 |
|    21 | Olfactory cortex                         | OLF.L          | 4.37e-1 |
|    65 | Angular gyrus                            | ANG.L          | 1.47e-1 |
|    43 | Calcarine fissure and surrounding cortex | CAL.L          | 1.46e-1 |
|    26 | Superior frontal gyrus (medial orbital)  | ORBsupmed.R    | 1.14e-1 |
|    86 | Middle temporal gyrus                    | MTG.R          | 7.80e-2 |
|    56 | Fusiform gyrus                           | FFG.R          | 7.24e-2 |
|    44 | Calcarine fissure and surrounding cortex | CAL.R          | 6.83e-2 |
|    88 | Temporal pole: middle temporal gyrus     | TPOmid.R       | 6.36e-2 |

Figure 5: Visualization of top 10 important ROIs selected by CLCEM in human brain.

![Image](https://ijcai.org/proceedings/2024/fulltext/342/image_000004_b23e468d361d084ac2c76673922fa4fcf2a2fa83394512c0c93004745efd84f3.png)

Several existing medical studies showed that changes in these ROIs are directly and significantly associated with the onset of ASD. Keehn [Jao Keehn et al.2017] found that compared with TD, the ASD group showed a bilateral cluster of activation with peak intensity in the right lingual gyrus, which has a more significant effect on the activity of the left lingual gyrus. Kana [Kana et al.2016] studied the FC difference in age- and IQ-matched adults with and without ASD and found that the brain activation level of the superior temporal gyrus in ASD patients decreased significantly during implicit emotion processing. David [Menassa et al.2017] believes that ASD is characterized by sensory abnormalities, including impaired olfactory recognition, and changes in the cellular structure of the olfactory cortex may be the basis for the olfactory differences in ASD. Li [Li et al.2014] thinks that the local connectivity of the angular gyrus in ASD patients is stronger to support their cognitive function. Libero [Libero et al.2014] collected fMRI data from ASD patients and TD individuals when judging the model's actions and intentions, and found that patients with ASD had significantly reduced activation in the calcarine sulcus compared with TD individuals when attending to the intentions of actions. Yerys [Yerys et al.2015] found that children with ASD demonstrated increased activation in the superior frontal gyrus during the switch-vs-stay contrast compared to controls. Cauda [Cauda et al.2011] analyzed the voxel-based morphometry findings and found that gray matter in the middle temporal gyrus was significantly increased in ASD patients. Van [van Kooten et al.2008] found that the activity of the fusiform gyrus and other cortical regions supporting face processing in ASD patients decreased.

Table 2: The BFN classification performance of CNN using various ROIs selected by different explanation methods. The AAL atlas comprises all 90 ROIs, while other explanation methods select the top 30 most important ROIs from 90 ROIs.

| ROI Selection Methods   | Acc(%)↑    | Rec(%)↑     | Pre(%)↑    | F1(%)↑     | b-Acc(%)↑   | AUC(%)↑    |
|-------------------------|------------|-------------|------------|------------|-------------|------------|
| AAL (All regions)       | 65.02±4.47 | 64.05±8.84  | 64.62±6.44 | 63.82±5.43 | 65.25±4.45  | 69.45±4.58 |
| LIME (2016)             | 66.09±3.87 | 69.80±11.26 | 64.92±3.31 | 66.84±5.86 | 65.95±4.11  | 70.23±4.78 |
| Grad-CAM (2019)         | 65.80±3.81 | 66.52±12.88 | 64.98±5.46 | 64.85±6.35 | 66.09±3.98  | 69.63±4.62 |
| FGVis (2019)            | 65.26±2.92 | 65.11±8.98  | 66.87±4.87 | 65.39±3.35 | 65.33±3.11  | 69.78±4.16 |
| AutoRMI (2022)          | 65.96±2.43 | 64.08±8.02  | 66.15±4.05 | 64.71±4.14 | 66.04±2.45  | 69.98±2.88 |
| CMIC (2023)             | 66.18±3.78 | 65.92±9.97  | 66.06±5.06 | 65.09±6.82 | 66.31±3.81  | 69.94±5.15 |
| CLCEM (Ours)            | 68.37±2.37 | 71.27±6.07  | 67.62±3.73 | 69.20±3.03 | 68.16±2.57  | 71.75±2.67 |

## 5.6 Comparative Analysis of ExplanationMethods

Through the above demonstration, we have proven the correctness of the explanation given by CLCEM from the perspective of the brain mechanism. However, it is still uncertain whether CLCEM is more effective in aligning with deep classification models of BFN when elucidating significant brain regions, compared to other explanation methods. Indeed, CLCEM is a general explanation method for BFN classification (suitable for most deep models), rendering direct comparisons with established methods challenging. To validate its advantages in explaining BFN classification tasks, we indirectly assess whether the features explained by CLCEM are the most significant for model decisions, in comparison with other explanation methods.

In our experiments, we utilized extensively adopted and high-performing deep learning models CNN [Henaff et al.2015] for brain network classification. The explanation methods encompass: LIME [Ribeiro et al.2016], Grad-CAM [Selvaraju et al.2019], FGVis [Wagner et al.2019], AutoRMI [Huai et al.2022], and CMIC [Yao et al.2023]. In detail, we use the explanations provided by CLCEM and other state-ofthe-art methods to identify the top 30 most important ROIs, and then employ CNN to classify BFNs using only these ROIs. We choose the top 30 ROIs, because most explanation methods could yield approximately optimal results. The number of brain regions selected can be 10, 30, 70, or any number less than the total of 90 brain regions. We chose 30 for the presentation of results because preliminary testing with different quantities showed that selecting 30 regions provides a balance where the number is not too high, yet the performance is significantly improved. In particular, the dataset was randomly divided into 10 equal parts, and we conducted a 10-fold cross validation. For fair comparison, different numbers of ROIs have the same network structure except that the input layer was finely tuned to accommodate different scales of BFN. The results are shown in Table 2 by mean±stddev.

In Table 2, we can see that the performance of models using the top 30 most important ROIs is better than the performance of models using all ROIs for most of the methods. And CLCEM demonstrates the most outstanding performance, achieving optimal results across all metrics. Therefore, we can conclude that the explanation (important ROIs) obtained from CLCEM is consistent with the characteristics that the deep classification model actually values.

In summary, through a series of experiments and analysis, we have established the correctness and advantages of CLCEM's exposition across three dimensions: brain neural mechanism, existing medical research, and classification model performance. Considering these factors, we posit that CLCEM's explanation is comprehensible to humans and aligns with the decision logic of the deep model.

## 6 Conclusion and FutureWork

To explain deep models for BFN classification, we proposed a novel concept-level causal explanation method called CLCEM. The proposed method can automatically extract human-understandable concepts from BFN under the constraints of causal concept loss and use these concepts to explain the decision logic of deep models. CLCEM provides a better explanation than other kinds of explanation methods and does not require the input data to be humanunderstandable natural pictures.

In the future, we will study the theoretical aspects and extensibility of CLCEM in BFN classification, and explore the use of CLCEM in multi-class brain disease diagnosis tasks. Additionally, we will also attempt to incorporate other easily comprehensible brain information as concepts.

## Acknowledgements

Thanks for the professional and constructive suggestions from the reviewers. This work was partly supported by National Natural Science Foundation of China Research Program (62106009, 62276010), in part by R&amp;D Program of Beijing Municipal Education Commission (KM202210005030, KZ202210005009).

## References

1. [Byrne, 2023] Ruth MJ Byrne. Good explanations in ex- plainable artificial intelligence (xai): Evidence from hu- man explanatory reasoning. In Proceedings of the Thirty- Second International Joint Conference on Artificial Intel- ligence, pages 6536-6544, 2023.
2. [Cauda et al., 2011] Franco Cauda, Elisabetta Geda, Katius- cia Sacco, Federico D'Agata, Sergio Duca, Giuliano Gem- iniani, and Roberto Keller. Grey matter abnormality in autism spectrum disorder: an activation likelihood estima- tion meta-analysis study. Journal of Neurology, Neuro- surgery & Psychiatry, 82(12):1304-1313, 2011.
3. [de Vos et al., 2018] Frank de Vos, Marisa Koini, Tijn M Schouten, Stephan Seiler, Jeroen van der Grond, Anita Lechner, Reinhold Schmidt, Mark de Rooij, and Serge ARB Rombouts. A comprehensive analysis of rest- ing state fmri measures to classify individual patients with alzheimer's disease. Neuroimage, 167:62-72, 2018.
4. [Ghorbani et al., 2019] Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim. Towards automatic concept- based explanations. Advances in Neural Information Pro- cessing Systems, 32:9277-9286, 2019.
5. [Goyal et al., 2019] Yash Goyal, Amir Feder, Uri Shalit, and Been Kim. Explaining classifiers with causal concept ef- fect (cace). arXiv preprint arXiv:1907.07165, 2019.
6. [Gu et al., 2020] Donghao Gu, Yaowei Li, Feng Jiang, Zhao- jing Wen, Shaohui Liu, Wuzhen Shi, Guangming Lu, and Changsheng Zhou. Vinet: A visually interpretable im- age diagnosis network. IEEE Transactions on Multimedia, 22(7):1720-1729, 2020.
7. [Henaff et al., 2015] Mikael Henaff, Joan Bruna, and Yann LeCun. Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163, 2015.
8. [Heskes et al., 2020] Tom Heskes, Evi Sijben, Ioan Gabriel Bucur, and Tom Claassen. Causal shapley values: Exploit- ing causal knowledge to explain individual predictions of complex models. Advances in neural information process- ing systems, 33:4778-4789, 2020.
9. [Hu et al., 2021] Wenxing Hu, Xianghe Meng, Yuntong Bai, Aiying Zhang, Gang Qu, Biao Cai, Gemeng Zhang, Tony W Wilson, Julia M Stephen, Vince D Calhoun, et al. Interpretable multimodal fusion networks reveal mecha- nisms of brain cognition. IEEE transactions on medical imaging, 40(5):1474-1483, 2021.
10. [Huai et al., 2022] Mengdi Huai, Jinduo Liu, Chenglin Miao, Liuyi Yao, and Aidong Zhang. Towards automat- ing model explanations with certified robustness guaran- tees. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 6935-6943, 2022.
11. [Jao Keehn et al., 2017] R Joanne Jao Keehn, Sandra S Sanchez, Claire R Stewart, Weiqi Zhao, Emily L Grenesko-Stevens, Brandon Keehn, and Ralph-Axel Müller. Impaired downregulation of visual cortex during auditory processing is associated with autism symptoma- tology in children and adolescents with autism spectrum disorder. Autism Research, 10(1):130-143, 2017.
12. [Ji et al., 2021] Junzhong Ji, Xinying Xing, Yao Yao, Jun- wei Li, and Xiaodan Zhang. Convolutional kernels with an element-wise weighting mechanism for identifying ab- normal brain connectivity patterns. Pattern Recognition, 109:107570, 2021.
13. [Kana et al., 2016] Rajesh K Kana, Michelle A Patriquin, Briley S Black, Marie M Channell, and Bruno Wicker. Al- tered medial frontal and superior temporal response to im- plicit processing of emotions in autism. Autism Research, 9(1):55-66, 2016.
14. [Kawahara et al., 2017] Jeremy Kawahara, Colin J Brown, Steven P Miller, Brian G Booth, Vann Chau, Ruth E Grunau, Jill G Zwicker, and Ghassan Hamarneh. Brain- netcnn: Convolutional neural networks for brain net- works; towards predicting neurodevelopment. NeuroIm- age, 146:1038-1049, 2017.
15. [Kim et al., 2016] Been Kim, Rajiv Khanna, and Oluwasanmi O Koyejo. Examples are not enough, learn to criticize! criticism for interpretability. Advances in neural information processing systems, 29:2288-2296, 2016.
16. [Kwon et al., 2019] Bum Chul Kwon, Min-Je Choi, Joanne Taery Kim, Edward Choi, Young Bin Kim, Soon- wook Kwon, Jimeng Sun, and Jaegul Choo. Retainvis: Visual analytics with interpretable and interactive recur- rent neural networks on electronic medical records. IEEE transactions on visualization and computer graphics, 25(1):299-309, 2019.
17. [Li et al., 2014] Hai Li, Zhong Xue, Timothy M Ellmore, Richard E Frye, and Stephen TC Wong. Network-based analysis reveals stronger local diffusion-based connectiv- ity and different correlations with oral language skills in brains of children with high functioning autism spectrum disorders. Human brain mapping, 35(2):396-413, 2014.
18. [Li et al., 2018] Oscar Li, Hao Liu, Chaofan Chen, and Cynthia Rudin. Deep learning for case-based reasoning through prototypes: A neural network that explains its pre- dictions. In Proceedings of the AAAI Conference on Arti- ficial Intelligence, volume 32, 2018.
19. [Libero et al., 2014] Lauren E Libero, Jose O Maximo, Hrishikesh D Deshpande, Laura G Klinger, Mark R Klinger, and Rajesh K Kana. The role of mirroring and mentalizing networks in mediating action intentions in autism. Molecular autism, 5(1):1-13, 2014.
20. [Liu et al., 2022] Jinduo Liu, Junzhong Ji, Guangxu Xun, and Aidong Zhang. Inferring effective connectivity net- works from fmri time series with a temporal entropy-score. IEEE transactions on neural networks and learning sys- tems, 33(10):5993-6006, 2022.
21. [Liu et al., 2024] Jinduo Liu, Lu Han, and Junzhong Ji. Mcan: Multimodal causal adversarial networks for dy- namic effective connectivity learning from fmri and eeg data. IEEE Transactions on Medical Imaging, 2024.
22. [Menassa et al., 2017] David A Menassa, Carolyn Sloan, and Steven A Chance. Primary olfactory cortex in autism and epilepsy: increased glial cells in autism. Brain Pathol- ogy, 27(4):437-448, 2017.
23. [O'Shaughnessy et al., 2020] Matthew O'Shaughnessy, Gregory Canal, Marissa Connor, Christopher Rozell, and Mark Davenport. Generative causal explanations of black-box classifiers. Advances in Neural Information Processing Systems, 33:5453-5467, 2020.
24. [Ribeiro et al., 2016] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. " why should i trust you?" explain- ing the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowl- edge discovery and data mining, pages 1135-1144, 2016.
25. [Seguin et al., 2023] Caio Seguin, Olaf Sporns, and Andrew Zalesky. Brain network communication: concepts, models and applications. Nature reviews neuroscience, 24(9):557- 574, 2023.
26. [Selvaraju et al., 2019] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In- ternational Journal of Computer Vision, 128(2):336-359, 2019.
27. [Sen and Parhi, 2021] Bhaskar Sen and Keshab K Parhi. Pre- dicting biological gender and intelligence from fmri via dynamic functional connectivity. IEEE Transactions on Biomedical Engineering, 68(3):815-825, 2021.
28. [Steward et al., 2023] Anna Steward, Davina Biel, Matthias Brendel, Anna Dewenter, Sebastian Roemer, Anna Rubin- ski, Ying Luan, Martin Dichgans, Michael Ewers, Nicolai Franzmeier, et al. Functional network segregation is as- sociated with attenuated tau spreading in alzheimer's dis- ease. Alzheimer's & Dementia, 19(5):2034-2046, 2023.
29. [van Kooten et al., 2008] Imke AJ van Kooten, Saskia JMC Palmen, Patricia von Cappeln, Harry WM Steinbusch, Hu- bert Korr, Helmut Heinsen, Patrick R Hof, Herman van Engeland, and Christoph Schmitz. Neurons in the fusiform gyrus are fewer and smaller in autism. Brain, 131(4):987- 999, 2008.
30. [Wagner et al., 2019] Jorg Wagner, Jan Mathias Kohler, To- bias Gindele, Leon Hetzel, Jakob Thaddaus Wiedemer, and Sven Behnke. Interpretable and fine-grained visual ex- planations for convolutional neural networks. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9097-9107, 2019.
31. [Woo et al., 2018] Sanghyun Woo, Jongchan Park, Joon- Young Lee, and In So Kweon. Cbam: Convolutional block attention module. In Proceedings of the European confer- ence on computer vision (ECCV), pages 3-19, 2018.
32. [Yao et al., 2023] Liuyi Yao, Yaliang Li, Sheng Li, Jin- duo Liu, Mengdi Huai, Aidong Zhang, and Jing Gao. Concept-level model interpretation from the causal aspect. IEEE Transactions on Knowledge and Data Engineering, 35(9):8799-8810, 2023.
33. [Yerys et al., 2015] Benjamin E Yerys, Ligia Antezana, Rachel Weinblatt, Kathryn F Jankowski, John Strang, Chandan J Vaidya, Robert T Schultz, William D Gaillard, and Lauren Kenworthy. Neural correlates of set-shifting in children with autism. Autism Research, 8(4):386-397, 2015.
34. [Zhang et al., 2024] Zuozhen Zhang, Junzhong Ji, and Jin- duo Liu. Metarlec: Meta-reinforcement learning for dis- covery of brain effective connectivity. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10261-10269, 2024.