{"type":"ScholarlyArticle","title":"Learning Cluster Causal Diagrams: An Information-Theoretic Approach","authors":[{"name":"Xueyan Niu","slug":"xueyan-niu","given":"Xueyan","family":"Niu","affiliation":"Cognitive Computing Lab Baidu Research 10900 NE 8th St. Bellevue , WA 98004 , USA"},{"name":"Xiaoyun Li","slug":"xiaoyun-li","given":"Xiaoyun","family":"Li","affiliation":"Cognitive Computing Lab Baidu Research 10900 NE 8th St. Bellevue , WA 98004 , USA"},{"name":"Ping Li","slug":"ping-li","given":"Ping","family":"Li","affiliation":"Cognitive Computing Lab Baidu Research 10900 NE 8th St. Bellevue , WA 98004 , USA"}],"abstract":"Many real-world phenomena arise from causal relationships among a set of variables. As a powerful tool, Bayesian Network (BN) has been successful in describing high-dimensional distributions. However, the faithfulness condition, enforced in most BN learning algorithms, is violated in the settings where multiple variables synergistically affect the outcome (i.e., with polyadic dependencies). Building upon recent development in cluster causal diagrams (C-DAGs), we initiate the formal study of learning C-DAGs from observational data to relax the faithfulness condition. We propose a new scoring function, the Clustering Information Criterion (CIC), based on information-theoretic measures that represent various complex interactions among variables. The CIC score also contains a penalization of the model complexity under the minimum description length principle. We further provide a searching strategy to learn structures of high scores. Experiments on both synthetic and real data support the effectiveness of the proposed method.","doi":"10.24963/ijcai.2022/675","url":"https://ijcai.org/proceedings/2022/675","pdf":"https://ijcai.org/proceedings/2022/0675.pdf","markdown":"https://ijcai.org/proceedings/2022/675.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/675","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Uncertainty in AI","track":"Main Track","number":675,"pages":{"first":4871,"last":4877},"keywords":["Uncertainty in AI: Bayesian Networks","Uncertainty in AI: Graphical Models","Uncertainty in AI: Uncertainty Representations","Uncertainty in AI: Causality, Structural Causal Models and Causal Inference"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986696","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984839","length_sec":363,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Anand et al., 2022] Tara V Anand, Adele H Ribeiro, Jin Tian, and Elias Bareinboim. Effect identification in cluster causal diagrams. arXiv preprint arXiv:2202.12263, 2022.","title":"Effect identification in cluster causal diagrams","authors":"Adele H Tara V Anand, Jin Ribeiro, Elias Tian, Bareinboim","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Battiston et al., 2020] Federico Battiston, Giulia Cencetti, Iacopo Iacopini, Vito Latora, Maxime Lucas, Alice Pata- nia, Jean-Gabriel Young, and Giovanni Petri. Networks beyond pairwise interactions: structure and dynamics. Physics Reports, 874:1-92, 2020.","title":"Networks beyond pairwise interactions: structure and dynamics","authors":"Federico Battiston, Giulia Cencetti, Iacopo Iacopini, Vito Latora, Maxime Lucas, Alice Patania, Jean-Gabriel Young, Giovanni Petri","year":"2020","venue":"Physics Reports","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Chen et al., 2021] Xinshi Chen, Haoran Sun, Caleb Elling- ton, Eric Xing, and Le Song. Multi-task learning of order- consistent causal graphs. Advances in Neural Information Processing Systems, 34, 2021.","title":"Multi-task learning of orderconsistent causal graphs","authors":"Xinshi Chen, Haoran Sun, Caleb Ellington, Eric Xing, Le Song","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chow and Liu, 1968] CK Chow and CN Liu. Approx- imating discrete probability distributions with depen- dence trees. IEEE Transactions on Information Theory, 14(3):462-467, 1968.","title":"Approximating discrete probability distributions with dependence trees","authors":"Liu ; ] Ck Chow, C N Chow, Liu","year":"1968","venue":"IEEE Transactions on Information Theory","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Correa and Bareinboim, 2020] Juan Correa and Elias Bareinboim. A calculus for stochastic interventions: Causal effect identification and surrogate experiments. In Proceedings of AAAI, pages 10093-10100, 2020.","title":"Juan Correa and Elias Bareinboim. A calculus for stochastic interventions: Causal effect identification and surrogate experiments","authors":"Bareinboim Correa","year":"2020","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[de Campos and Cozman, 2013] Cassio de Campos and Fabio Cozman. Complexity of inferences in polytree- shaped semi-qualitative probabilistic networks. In Pro- ceedings of AAAI, pages 217-223, 2013.","title":"Cassio de Campos and Fabio Cozman. Complexity of inferences in polytreeshaped semi-qualitative probabilistic networks","authors":"Cozman De Campos","year":"2013","venue":"Proceedings of AAAI","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[de Campos, 2006] Luis M de Campos. A scoring function for learning Bayesian networks based on mutual informa- tion and conditional independence tests. Journal of Ma- chine Learning Research, 7(10), 2006.","title":"A scoring function for learning Bayesian networks based on mutual information and conditional independence tests","authors":"Luis M De Campos","year":"2006","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Heckerman et al., 1995] David Heckerman, Dan Geiger, and David M Chickering. Learning Bayesian networks: The combination of knowledge and statistical data. Ma- chine Learning, 20(3):197-243, 1995.","title":"Learning Bayesian networks: The combination of knowledge and statistical data","authors":"David Heckerman, Dan Geiger, David M Chickering","year":"1995","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[James et al., 2016] Ryan G James, Nix Barnett, and James P Crutchfield. Information flows? a critique of transfer en- tropies. Physical Review Letters, 116(23):238701, 2016.","title":"Information flows? a critique of transfer entropies","authors":"Ryan G James, Nix Barnett, James P Crutchfield","year":"2016","venue":"Physical Review Letters","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Kalisch and Bühlman, 2007] Markus Kalisch and Peter Bühlman. Estimating high-dimensional directed acyclic graphs with the PC-algorithm. Journal of Machine Learn- ing Research, 8(3), 2007.","title":"Estimating high-dimensional directed acyclic graphs with the PC-algorithm","authors":"Markus Kalisch, Peter Bühlman","year":"2007","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Kraskov et al., 2004] Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Physical Review E, 69(6), 2004.","title":"Estimating mutual information","authors":"Alexander Kraskov, Harald Stögbauer, Peter Grassberger","year":"2004","venue":"Physical Review E","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Liu et al., 2017] Ziwei Liu, Xiaoxiao Li, Ping Luo, Chen Change Loy, and Xiaoou Tang. Deep learning markov random field for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 40(8):1814-1828, 2017.","title":"Deep learning markov random field for semantic segmentation","authors":"Ziwei Liu, Xiaoxiao Li, Ping Luo, Chen Change Loy, Xiaoou Tang","year":"2017","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Lizier et al., 2018] Joseph T Lizier, Nils Bertschinger, Jürgen Jost, and Michael Wibral. Information decompo- sition of target effects from multi-source interactions: Per- spectives on previous, current and future work, 2018.","title":"Information decomposition of target effects from multi-source interactions: Perspectives on previous, current and future work","authors":"Nils Joseph T Lizier, Jürgen Bertschinger, Michael Jost, Wibral","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[McGill, 1954] William McGill. Multivariate information transmission. Transactions of the IRE Professional Group on Information Theory, 4(4):93-111, 1954.","title":"Multivariate information transmission","authors":"William Mcgill","year":"1954","venue":"Transactions of the IRE Professional Group on Information Theory","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mokhtarian et al., 2021] Ehsan Mokhtarian, Sina Akbari, Fateme Jamshidi, Jalal Etesami, and Negar Kiyavash. Learning Bayesian networks in the presence of structural side information. arXiv preprint arXiv:2112.10884, 2021.","title":"Learning Bayesian networks in the presence of structural side information","authors":"Ehsan Mokhtarian, Sina Akbari, Fateme Jamshidi, Jalal Etesami, Negar Kiyavash","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Morningstar and Melko, 2018] Alan Morningstar and Roger G Melko. Deep learning the ising model near criticality. Journal of Machine Learning Research, 18:1-17, 2018.","title":"Deep learning the ising model near criticality","authors":"Alan Morningstar, Roger G Melko","year":"2018","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nabi et al., 2020] Razieh Nabi, Joel Pfeiffer, Murat Ali Bayir, Denis Charles, and Emre Kıcıman. Causal infer- ence in the presence of interference in sponsored search advertising. arXiv preprint arXiv:2010.07458, 2020.","title":"Causal inference in the presence of interference in sponsored search advertising","authors":"Razieh Nabi, Joel Pfeiffer, Ali Murat, Denis Bayir, Emre Charles, Kıcıman","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Niu and Quinn, 2019] Xueyan Niu and Christopher J Quinn. A measure of synergy, redundancy, and unique information using information geometry. In Proceedings of ISIT, pages 3127-3131, 2019.","title":"A measure of synergy, redundancy, and unique information using information geometry","authors":"Xueyan Niu, Christopher J Quinn","year":"2019","venue":"Proceedings of ISIT","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Parviainen and Kaski, 2017] Pekka Parviainen and Samuel Kaski. Learning structures of bayesian networks for vari- able groups. International Journal of Approximate Rea- soning, 88:110-127, 2017.","title":"Learning structures of bayesian networks for variable groups","authors":"Kaski Parviainen, Pekka Parviainen, Samuel Kaski","year":"2017","venue":"International Journal of Approximate Reasoning","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Pearl and Bareinboim, 2014] Judea Pearl and Elias Barein- boim. External validity: From do-calculus to transporta- bility across populations. Statistical Science, 29(4), 2014.","title":"Judea Pearl and Elias Bareinboim. External validity: From do-calculus to transportability across populations","authors":"Bareinboim Pearl","year":"2014","venue":"Statistical Science","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Pearl, 2000] Judea Pearl. Causality: Models, Reasoning and Inference. Cambridge University Press, 2000.","title":"Judea Pearl. Causality: Models, Reasoning and Inference","authors":"Pearl","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Reing et al., 2021] Kyle Reing, Greg Ver Steeg, and Aram Galstyan. Influence decompositions for neural network at- tribution. In Proceedings of AISTATS, pages 2710-2718, 2021.","title":"Influence decompositions for neural network attribution","authors":"Kyle Reing, Greg Ver Steeg, Aram Galstyan","year":"2021","venue":"Proceedings of AISTATS","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Robins et al., 2003] James M Robins, Richard Scheines, Peter Spirtes, and Larry Wasserman. Uniform consistency in causal inference. Biometrika, 90(3):491-515, 2003.","title":"Uniform consistency in causal inference","authors":"Richard James M Robins, Peter Scheines, Larry Spirtes, Wasserman","year":"2003","venue":"Biometrika","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Sachs et al., 2005] Karen Sachs, Omar Perez, Dana Pe'er, Douglas A Lauffenburger, and Garry P Nolan. Causal protein-signaling networks derived from multiparameter single-cell data. Science, 308(5721):523-529, 2005.","title":"Causal protein-signaling networks derived from multiparameter single-cell data","authors":"Karen Sachs, Omar Perez, Dana Pe'er, Douglas A Lauffenburger, Garry P Nolan","year":"2005","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Saeed et al., 2020] Basil Saeed, Snigdha Panigrahi, and Caroline Uhler. Causal structure discovery from distribu- tions arising from mixtures of DAGs. In Proceedings of ICML, pages 8336-8345, 2020.","title":"Causal structure discovery from distributions arising from mixtures of DAGs","authors":"Saeed","year":"2020","venue":"Proceedings of ICML","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Schwarz, 1978] Gideon Schwarz. Estimating the dimension of a model. The Annals of Statistics, pages 461-464, 1978.","title":"Estimating the dimension of a model","authors":"Gideon Schwarz, Schwarz","year":"1978","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Spirtes et al., 2000] Peter Spirtes, Clark N Glymour, Richard Scheines, and David Heckerman. Causation, Prediction, and Search. MIT Press, 2000.","title":"Causation, Prediction, and Search","authors":"Peter Spirtes, Richard Clark N Glymour, David Scheines, Heckerman","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Tikka et al., 2021] Santtu Tikka, Jouni Helske, and Juha Karvanen. Clustering and structural robustness in causal diagrams. arXiv preprint arXiv:2111.04513, 2021.","title":"Clustering and structural robustness in causal diagrams","authors":"Tikka","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Tsamardinos et al., 2006] Ioannis Tsamardinos, Laura E Brown, and Constantin F Aliferis. The max-min hill- climbing Bayesian network structure learning algorithm. Machine Learning, 65(1):31-78, 2006.","title":"The max-min hillclimbing Bayesian network structure learning algorithm","authors":"Ioannis Tsamardinos, Laura E Brown, Constantin F Aliferis","year":"2006","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Uhler et al., 2013] Caroline Uhler, Garvesh Raskutti, Peter Bühlmann, and Bin Yu. Geometry of the faithfulness as- sumption in causal inference. The Annals of Statistics, pages 436-463, 2013.","title":"Geometry of the faithfulness assumption in causal inference","authors":"Caroline Uhler, Garvesh Raskutti, Peter Bühlmann, Bin Yu","year":"2013","venue":"The Annals of Statistics","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Wellman, 1990] Michael P Wellman. Fundamental con- cepts of qualitative probabilistic networks. Artificial In- telligence, 44(3):257-303, 1990.","title":"Fundamental concepts of qualitative probabilistic networks","authors":"Michael P Wellman","year":"1990","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}