--- title: "Learning Causal Effects on Hypergraphs (Extended Abstract)" authors: - "Jing Ma" - "Mengting Wan" - "Longqi Yang" - "Jundong Li" - "Brent Hecht" - "Jaime Teevan" doi: "10.24963/ijcai.2023/721" url: "https://ijcai.org/proceedings/2023/721" pdf: "https://ijcai.org/proceedings/2023/0721.pdf" venue: "Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence" year: 2023 pages: "6463-6467" track: "Sister Conferences Best Papers" keywords: - "Sister Conferences Best Papers: Machine Learning" - "Sister Conferences Best Papers: Data Mining" --- # Learning Causal Effects on Hypergraphs (Extended Abstract) Jing Ma, Mengting Wan, Longqi Yang, Jundong Li, Brent Hecht, Jaime Teevan *Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence*, pages 6463-6467. DOI: [10.24963/ijcai.2023/721](https://doi.org/10.24963/ijcai.2023/721) ## Abstract Hypergraphs provide an effective abstraction for modeling multi-way group interactions among nodes, where each hyperedge can connect any number of nodes. Different from most existing studies which leverage statistical dependencies, we study hypergraphs from the perspective of causality. Specifically, we focus on the problem of individual treatment effect (ITE) estimation on hypergraphs, aiming to estimate how much an intervention (e.g., wearing face covering) would causally affect an outcome (e.g., COVID-19 infection) of each individual node. Existing works on ITE estimation either assume that the outcome of one individual should not be influenced by the treatment of other individuals (i.e., no interference), or assume the interference only exists between connected individuals in an ordinary graph. We argue that these assumptions can be unrealistic on real-world hypergraphs, where higher-order interference can affect the ITE estimations due to group interactions. We investigate high-order interference modeling, and propose a new causality learning framework powered by hypergraph neural networks. Extensive experiments on real-world hypergraphs verify the superiority of our framework over existing baselines. ## Full text ## 1 Introduction Group interactions among individuals exist in many scenarios, e.g., massive gathering events. Although the conventional pairwise graph definition covers various applications (e.g., physical contact networks or social networks [Mitchell, 1974]), it fails to capture the complete information of group interactions (where each interaction may involve more than two individuals) [Bai et al., 2021; Feng et al., 2019]. Hypergraphs can be introduced to address this limitation. Consider a hypergraph example that individuals are connected via in-person social events, each event can be represented as a hyperedge (Fig. 1a). Each hyperedge can connect any number of individuals, unlike an ordinary edge connecting exactly two nodes (Fig. 1b). ∗This is an extended abstract of a paper [Ma et al., 2022] that won the Best Paper Award at KDD 2022. 𝑢ହ ![Image](https://ijcai.org/proceedings/2023/fulltext/721/image_000000_5d526a41d75ccb87f44664718da83f1e711942a5c8384d97cc514a59726ce02e.png) - (c) First, second, and third-order interferences with u1. Figure 1: (a) An illustrative example of group interactions on a hypergraph, where each circle represents a hyperedge (group); (b) An ordinary graph projected from this hypergraph; (c) Illustration of interferences with u1 from its neighbors on the hypergraph. Note interference on (b) is pairwise (first-order only) while higher-order interference exists on the original hypergraph (a). While many studies have been devoted to utilizing such a generalized hypergraph structure to facilitate machine learning tasks [Bai et al., 2021; Feng et al., 2019], the majority were still executed at the statistical correlation level. A critical limitation here is the lack of causality, which is particularly important for understanding the impact of a policy intervention (e.g., wearing face covering) on an outcome of interest (e.g., COVID-19 infection). For individuals connected as in Fig. 1a, one may ask "how would each individual's face covering practice (treatment) causally influence their infection risk (outcome)?" This causal problem is particularly hard on hypergraphs, since the outcome of each individual is not only affected by their own treatment, but also influenced by the treatment of other individuals on the hypergraph (e.g., face covering practice of other individuals who may physically contact the target individual through a gathering event). We focus on learning causal effects on hypergraphs. Specifically, we aim to estimate the individual treatment effect (ITE) under hypergraph interference from observational data. Classic ITE estimation relies on the Stable Unit Treatment Value (SUTVA) assumption [Fisher, 1936; SplawaNeyman et al., 1990] that there is no interference [Tchetgen and VanderWeele, 2012; Hudgens and Halloran, 2008] (i.e., spillover effect) among individuals/instances (also referred to as units in causal inference literature). That means the outcomes for any instance are not affected by the treatment of other instances. This assumption is often impractical in the real world, especially on graphs where the interference is ubiquitous [Ahluwalia et al., 2001; Yilmaz et al., 2002]. Most existing studies of interference [Aronow and Samii, 2017; Basse and Feller, 2018; Imai et al., 2020; Kohavi et al., 2013; Tchetgen and VanderWeele, 2012; Ugander et al., 2013; Yuan et al., 2021; Ma and Tresp, 2021] assume the interference only exists in a pairwise way on ordinary graphs. This pairwise interference notion is insufficient to characterize the high-order interference on hypergraphs. As shown in Fig. 1c, within a gathering event (hyperedge) between u1, u2 and u3, an individual's (u1) infection outcome can be affected by the first-order interference from other individuals (u2 → u1 and u3 → u1) as well as the high-order interference from the interactions among other individuals (the interaction between u2 and u3 may also act on influencing the exposure of the virus to u1; consequently, u1's infection risk can be affected by this second-order interaction effect, i.e., u2 × u3 → u1). This demands techniques capable of modeling high-order interference, but to the best of our knowledge, very little work has been done in this area. We propose a novel framework- Causal Inference under Spillover Effects in Hypergraphs (HyperSCI)-to model high-order interference and promote ITE estimation performance. At a high level, this framework controls for the confounders and models high-order interference based on representation learning, then estimates the outcomes based on the learned representations. We evaluate HyperSCI through extensive experiments under different scenarios of high-order interference, and provide in-depth analysis of how our framework acts on different nodes and hyperedges. ## 2 ProblemDefinition Definition 1. (Hypergraph) A hypergraph H = {V, E} consists of a set of n nodes V = {vi}ni=1 and a set of m hyperedges E = {ek}mk=1 . Each hyperedge can connect any number of nodes. In the studied problem, the given observational data is denoted by {X,H,T,Y}. Here, X = {xi}ni=1 , T = {ti}ni=1 and Y = {yi}ni=1 represent node features, treatment assignments, and observed outcomes, respectively. H = {hi,e} ∈ Rn×m is an incidence matrix for hypergraph H. Here, hi,e = 1 if node i is in hyperedge e, otherwise hi,e = 0. The treatment assignment for each node is binary (i.e., ti ∈ {0, 1}). We use H,X, T to denote the random variables for the hypergraph structure, features, and treatment for any node. Definition 2. (Potential outcome) The potential outcome [Rubin, 1980] of the unit i (denoted by y1 i or y0 i ) is defined as the outcome which would be realized for unit i under treatment ti = 1 or ti = 0. These potential outcomes can be obtained via a transformation function Y Ti i = ΦY (Ti,Xi, T-i,X-i,H). Here, ΦY is a (non-deterministic) function, i.e., yti i = ΦY (ti, xi,T-i,X-i,H), where (·)-i denotes all other nodes on H except i. Our aim is to estimate ITE in a hypergraph. The ITE in the studied problem is defined as follows: Definition 3. For each node i on the hypergraph H, the individual treatment effect (ITE) τ(xi,T-i,X-i,H) is defined by the difference between potential outcomes corresponding to ti = 1 and ti = 0: Here, we give our main assumptions and theoretical analysis for this study: Assumption 1. (Expressiveness of summary function) There is an expressive summary function oi = SMR(H,T-i,X-i) to summarize the environmental information for each node. For any node i, any values of H,X-i, and T-i, if the output oi is determined, then the value of the potential outcomes y1 i and y0 i with feature xi are also determined. Assumption 2. (Unconfoundedness) For any node i, given the node features, the potential outcomes are independent with the treatment assignment and summary of neighbors, i.e., Y 1 i , Y 0 i ⊥⊥ Ti,Oi|Xi. Theorem 1. (Identifiability) The defined ITE can be identifiable from observational data under the above assumptions. We refer readers to [Ma et al., 2022] for a full derivation. ## 3 ProposedMethod Fig. 2 shows an overview of our framework HyperSCI, which contains three main components: confounder representation learning, interference modeling, and outcome prediction. ## 3.1 Confounder Representation Learning HyperSCI learns representations of confounders by mapping the node features xi into a latent space with a multilayer perceptron (MLP) module, i.e., zi = MLP(xi). The confounder representations for all the nodes are denoted by Z = {zi}ni=1 . Similar as [Shalit et al., 2017], a Wasserstein-1 distancebased representation balancing method is used to minimize the distance between the representation distributions of the treatment group and control group, aiming to improve the treatment effect estimation performance. ## 3.2 Interference Modeling An interference modeling module is developed to model the high-order interference among nodes in the hypergraph. More specifically, a function Ψ(·) is learned via a hypergraph neural network module to obtain the interference representations (pi) for each node i, i.e., pi = Ψ(Z,H,T-i, ti). The illustration of this module is shown in Fig. 3. This module is implemented based on a hypergraph convolutional network [Bai et al., 2021; Yadati et al., 2019] as well as a hypergraph attention mechanism [Bai et al., 2021; Zhang et al., 2019; Ding et al., 2020]. ![Image](https://ijcai.org/proceedings/2023/fulltext/721/image_000001_04e8c607daffb5e59861db667e1a6572cfa9da8f0beb26fda1b65cac58845c91.png) Hypergraph module ![Image](https://ijcai.org/proceedings/2023/fulltext/721/image_000002_c7a9ff14aae940703780b46d2cb021a883859075b760a8ddeb7bc42d21dc041f.png) $# Interference representation Figure 3: An illustration of the hypergraph module in HyperSCI. Here node v1 (highlighted in yellow) is taken as an example. To learn the interference representations for each node, the treatment and confounder representations are propagated through the hypergraph structure. A vanilla Laplacian matrix for the given hypergraph H can be calculated as: where D ∈ Rn×n is a diagonal matrix in which each element stands for the node degree (i.e., ∑me=1 hi,e). B ∈ Rm×m is a diagonal matrix in which each element corresponds to the size of each hyperedge (∑ni=1 hi,e). The hypergraph convolution operation is defined as: where P(l) denotes the representations in the l-th layer of the hypergraph module. The input of the first layer is the confounder representation masked by the treatment assignment, i.e., p(0) i = ti ∗ zi. Here, ∗ is element-wise multiplication. W(l+1) ∈ Rd(l)×d(l+1) represents the parameter matrix in the (l+1)-th layer of the hypergraph module, where d(l) and d(l+1) are the dimensionality of the l-th layer and (l+1)-th layer, respectively. While the hypergraph convolution layer allows for interference modeling through hyperedges, it lacks flexibility to consider the varying significance of interference on different nodes via different hyperedges. To address this, a hypergraph attention mechanism [Bai et al., 2021; Ding et al., 2020; Zhang et al., 2019] is utilized to capture the intrinsic relationship between nodes and hyperedges. Specifically, the attention weights are learned for each node and its corresponding hyperedges, which allows for a better understanding of how certain individuals, such as those participating in group events, may have a greater influence on or be influenced by others in these groups within the context of a hypergraph, as where σ(·) is an activation function, Ei is the set of hyperedges which contain the node i. ze is the representation for each hyperedge e, obtained by aggregating across the representations of its associated nodes. Wa denotes a parameter matrix to compute the node-hyperedge attention. sim(·) denotes a similarity function, which can be implemented as: ![Image](https://ijcai.org/proceedings/2023/fulltext/721/image_000003_f3622bb388a6e89598a819ceb1cb87d630da2ce3abd76dcc949bbc49c345eb9c.png) Here, a is a weight vector, [·, ·] is a concatenation operation. The attention scores are used to model the different significance of interference. More specifically, the original incidence matrix H of the hypergraph in Eq. 2 is replaced with an attention-involved matrix ˜H = {˜hi,e}, where ˜hi,e = αi,ehi,e. ## 3.3 Outcome Prediction Based on the confounder representations and the interference representations, the potential outcomes are predicted by: where f1(·) and f0(·) are learnable functions which are trained to predict potential outcomes for treatment assignment 1 and 0, respectively. The ITE for each node i is then estimated by: ˆ τi = ˆ y1 i - ˆ y0 i . The prediction for the observed outcome is obtained by ˆ yi = ˆ yti i . The final loss function is: where the first term is the outcome prediction loss, which can be implemented by standard mean squared error. Lb is the representation balancing loss, as introduced in Section 3.1. Θ denotes all the model parameters. α and λ are hyperparameters that control the weights for representation balancing and model regularization, respectively. ## 4 Experimental Evaluation It is often very hard to obtain the ground-truth counterfactuals as only one of the two potential outcomes can be obtained in the observational data. Hence, we follow a standard practice to evaluate our framework and the alternative approaches on three semi-synthetic datasets Contact, GoodReads, and Microsoft Teams. We leverage as much real-world information as possible in the simulated environment. Our datasets are all based on real-world hypergraphs, and we retain the treatment allocations as well as node features (covariates) if they are available. We simulate the outcome generation process to assess the true ITEs, which allow us to evaluate the performance of ITE estimation. Full details of the datasets, baselines, and experimental settings can be found in [Ma et al., 2022]. Table 1: ITE estimation performance. "CT", "GR" and "MS" refer to Contact, GoodReads, and Microsoft Teams datasets, respectively. | Data | Method | Linear - √ϵPEHE | Linear - ϵATE | Quadratic - √ϵPEHE | Quadratic - ϵATE | |--------|-----------|-------------------|-----------------|----------------------|--------------------| | CT | LR | 25.41 | 9.11 | 38.22 | 20.28 | | | CEVAE | 22.88 | 8.29 | 35.28 | 18.22 | | | CFR | 24.04 | 7.17 | 32.24 | 17.28 | | | Netdeconf | 10.22 | 4.29 | 21.23 | 11.39 | | | GNN-HSIC | 7.42 | 2.06 | 16.28 | 7.28 | | | GCN-HSIC | 7.28 | 2.08 | 14.23 | 6.27 | | | HyperSCI | 3.45 | 1.39 | 9.20 | 2.24 | | GR | LR | 23.01 | 13.42 | 48.56 | 31.19 | | | CEVAE | 22.69 | 12.49 | 45.21 | 29.22 | | | CFR | 20.30 | 13.21 | 41.72 | 26.28 | | | Netdeconf | 18.39 | 12.20 | 35.18 | 21.20 | | | GNN-HSIC | 17.20 | 12.18 | 27.22 | 16.87 | | | GCN-HSIC | 16.01 | 12.06 | 25.42 | 16.28 | | | HyperSCI | 15.68 | 11.81 | 19.23 | 13.33 | | MS | LR | 22.80 | 21.41 | 414.17 | 192.80 | | | CEVAE | 19.36 | 8.63 | 315.01 | 188.47 | | | CFR | 25.23 | 18.28 | 392.56 | 189.75 | | | Netdeconf | 11.11 | 9.22 | 241.02 | 147.29 | | | GNN-HSIC | 9.38 | 6.91 | 114.28 | 81.21 | | | GCN-HSIC | 8.27 | 6.60 | 109.57 | 77.75 | | | HyperSCI | 5.13 | 4.46 | 81.08 | 74.41 | ## 4.1 ITE Estimation Performance The performance of ITE estimation in hypergraph is shown in Table 1. HyperSCI outperforms all the baselines under different (linear and quadratic) simulation settings of the outcome generation function. As for the reasons, HyperSCI can leverage the hypergraph structure to model the high-order interference. In this way, it mitigates the influence of the interference on ITE estimation performance. Among baselines, some of them consider the pairwise network interference (GCN-HSIC and GNN-HSIC [Ma and Tresp, 2021]), or use the graph structure to infer the hidden confounders (Netdeconf [Guo et al., 2020]). These methods perform better than those baselines (LR, CEVAE [Louizos et al., 2017], CFR [Shalit et al., 2017]) which cannot handle graph information. Furthermore, in the simulation, the hyperparameter β controls the level of hypergraph spillover effect in the outcome simulation. The ITE estimation results under different values of β are shown in Fig. 4. When β increases, the outcome is more strongly affected by interference, and larger performance gains can be observed from HyperSCI compared with the baselines. ## 4.2 ACloser Look at High-Order Interference We further take a closer look at high-order interference. We investigate how HyperSCI responds to hyperedges with different sizes. More specifically, we remove the hyperedges with size larger than k, denote the modified hypergraph as H(k) , and vary the value of k. In Fig 5, we compare the ITE estimation performance of HyperSCI with its variant on the projected ordinary graph HyperSCI-G. We observe that: 1) When k = 2 (hyperedge size ≤ 2), the performance of HyperSCI-G is close to HyperSCI. Because when k = 2, graph convolution can be regarded as a special case of hypergraph convolution with small differences in the graph Laplacian matrix (as illustrated in [Bai et al., 2021]). Empirically this leads to a minor performance difference between HyperSCI-G and HyperSCI; 2) When k increases, the performance of ITE estimation from both methods are gradually improved, but such an improvement becomes less significant when k is larger. Besides, we notice HyperSCI consistently outperforms HyperSCI-G and such a difference becomes larger as k increases, indicating its efficacy on modeling high-order interference especially on large hyperedges. Figure 4: ITE estimation performance under different values of β in linear setting on GoodReads. ![Image](https://ijcai.org/proceedings/2023/fulltext/721/image_000004_230a9a8d4352a7884eb0403c4740996e0855a884435a94f9ed624d84a7e792fc.png) Figure 5: ITE estimation performance of HyperSCI / HyperSCI-G on GoodReads with hyperedge size no more than k. ![Image](https://ijcai.org/proceedings/2023/fulltext/721/image_000005_92b30214f1e44aeb69091118588325c85426d15ab57e8d826660ba4cfedb2981.png) ## 5 Conclusion We study an important research problem of ITE estimation with the existence of high-order interference on hypergraphs. We identify and analyze the influence of high-order interference in causal effect estimation. To address this problem, we propose a novel framework HyperSCI, which estimates the ITEs based on representation learning. More specifically, HyperSCI learns the representation of confounders, models the high-order interference with a hypergraph neural network module, then predicts the potential outcomes for each instance with the learned representations. Extensive experiments conducted on semi-synthetic data based on real-world hypergraphs consistently validate the effectiveness of HyperSCI in ITE estimation under different interference scenarios. ## References 1. [Ahluwalia et al., 2001] Rohini Ahluwalia, H Rao Unnava, and Robert E Burnkrant. The moderating role of commit- ment on the spillover effect of marketing communications. Journal of Marketing research, 38(4):458-470, 2001. 2. [Aronow and Samii, 2017] Peter M Aronow and Cyrus Samii. Estimating average causal effects under general interference, with application to a social network experi- ment. The Annals of Applied Statistics, 11(4):1912-1947, 2017. 3. [Bai et al., 2021] Song Bai, Feihu Zhang, and Philip HS Torr. Hypergraph convolution and hypergraph attention. Pattern Recognition, 110:107637, 2021. 4. [Basse and Feller, 2018] Guillaume Basse and Avi Feller. Analyzing two-stage experiments in the presence of inter- ference. Journal of the American Statistical Association, 113(521):41-55, 2018. 5. [Ding et al., 2020] Kaize Ding, Jianling Wang, Jundong Li, Dingcheng Li, and Huan Liu. Be more with less: Hyper- graph attention networks for inductive text classification. arXiv preprint arXiv:2011.00387, 2020. 6. [Feng et al., 2019] Yifan Feng, Haoxuan You, Zizhao Zhang, Rongrong Ji, and Yue Gao. Hypergraph neural networks. In Proceedings of the AAAI Conference on Ar- tificial Intelligence, volume 33, pages 3558-3565, 2019. 7. [Fisher, 1936] Ronald Aylmer Fisher. Design of experi- ments. Br Med J, 1(3923):554-554, 1936. 8. [Guo et al., 2020] Ruocheng Guo, Jundong Li, and Huan Liu. Learning individual causal effects from networked observational data. In Proceedings of the 13th Interna- tional Conference on Web Search and Data Mining, pages 232-240, 2020. 9. [Hudgens and Halloran, 2008] Michael G Hudgens and M Elizabeth Halloran. Toward causal inference with inter- ference. Journal of the American Statistical Association, 103(482):832-842, 2008. 10. [Imai et al., 2020] Kosuke Imai, Zhichao Jiang, and Anup Malani. Causal inference with interference and noncom- pliance in two-stage randomized experiments. Journal of the American Statistical Association, pages 1-13, 2020. 11. [Kohavi et al., 2013] Ron Kohavi, Alex Deng, Brian Frasca, Toby Walker, Ya Xu, and Nils Pohlmann. Online con- trolled experiments at large scale. In Proceedings of the 19th ACM SIGKDD international conference on Knowl- edge discovery and data mining, pages 1168-1176, 2013. 12. [Louizos et al., 2017] Christos Louizos, Uri Shalit, Joris M Mooij, David Sontag, Richard Zemel, and Max Welling. Causal effect inference with deep latent-variable models. In Advances in Neural Information Processing Systems, 2017. 13. [Ma and Tresp, 2021] Yunpu Ma and Volker Tresp. Causal inference under networked interference and intervention policy enhancement. In International Conference on Arti- ficial Intelligence and Statistics, pages 3700-3708. PMLR, 2021. 14. [Ma et al., 2022] Jing Ma, Mengting Wan, Longqi Yang, Jundong Li, Brent Hecht, and Jaime Teevan. Learning causal effects on hypergraphs. In ACM SIGKDD Inter- national Conference on Knowledge Discovery and Data Mining, 2022. 15. [Mitchell, 1974] J Clyde Mitchell. Social networks. Annual review of anthropology, 3(1):279-299, 1974. 16. [Rubin, 1980] Donald B Rubin. Randomization analysis of experimental data: The fisher randomization test com- ment. Journal of the American Statistical Association, 75(371):591-593, 1980. 17. [Shalit et al., 2017] Uri Shalit, Fredrik D Johansson, and David Sontag. Estimating individual treatment effect: gen- eralization bounds and algorithms. In International Con- ference on Machine Learning, 2017. 18. [Splawa-Neyman et al., 1990] Jerzy Splawa-Neyman, Dorota M Dabrowska, and TP Speed. On the application of probability theory to agricultural experiments. essay on principles. section 9. Statistical Science, pages 465-472, 1990. 19. [Tchetgen and VanderWeele, 2012] Eric J Tchetgen Tchet- gen and Tyler J VanderWeele. On causal inference in the presence of interference. Statistical methods in medical research, 21(1):55-75, 2012. 20. [Ugander et al., 2013] Johan Ugander, Brian Karrer, Lars Backstrom, and Jon Kleinberg. Graph cluster randomiza- tion: Network exposure to multiple universes. In Proceed- ings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 329-337, 2013. 21. [Yadati et al., 2019] Naganand Yadati, Madhav Nimishakavi, Prateek Yadav, Vikram Nitin, Anand Louis, and Partha Talukdar. Hypergcn: A new method for training graph convolutional networks on hypergraphs. Advances in neural information processing systems, 32, 2019. 22. [Yilmaz et al., 2002] Serdar Yilmaz, Kingley E Haynes, and Mustafa Dinc. Geographic and network neighbors: Spillover effects of telecommunications infrastructure. Journal of Regional Science, 42(2):339-360, 2002. 23. [Yuan et al., 2021] Yuan Yuan, Kristen Altenburger, and Far- shad Kooti. Causal network motifs: Identifying heteroge- neous spillover effects in a/b tests. In Proceedings of the Web Conference 2021, pages 3359-3370, 2021. 24. [Zhang et al., 2019] Ruochi Zhang, Yuesong Zou, and Jian Ma. Hyper-sagnn: a self-attention based graph neural net- work for hypergraphs. arXiv preprint arXiv:1911.02613, 2019.