{"type":"ScholarlyArticle","title":"Learning Causally Disentangled Representations via the Principle of Independent Causal Mechanisms","authors":[{"name":"Aneesh Komanduri","slug":"aneesh-komanduri","given":"Aneesh","family":"Komanduri","affiliation":"University of Arkansas"},{"name":"Yongkai Wu","slug":"yongkai-wu","given":"Yongkai","family":"Wu","affiliation":"Clemson University"},{"name":"Feng Chen","slug":"feng-chen","given":"Feng","family":"Chen","affiliation":"University of Texas at Dallas"},{"name":"Xintao Wu","slug":"xintao-wu","given":"Xintao","family":"Wu","affiliation":"University of Arkansas"}],"abstract":"Learning disentangled causal representations is a challenging problem that has gained significant attention recently due to its implications for extracting meaningful information for downstream tasks. In this work, we define a new notion of causal disentanglement from the perspective of independent causal mechanisms. We propose ICM-VAE, a framework for learning causally disentangled representations supervised by causally related observed labels. We model causal mechanisms using nonlinear learnable flow-based diffeomorphic functions to map noise variables to latent causal variables. Further, to promote the disentanglement of causal factors, we propose a causal disentanglement prior learned from auxiliary labels and the latent causal structure. We theoretically show the identifiability of causal factors and mechanisms up to permutation and elementwise reparameterization. We empirically demonstrate that our framework induces highly disentangled causal factors, improves interventional robustness, and is compatible with counterfactual generation.","doi":"10.24963/ijcai.2024/476","url":"https://ijcai.org/proceedings/2024/476","pdf":"https://ijcai.org/proceedings/2024/0476.pdf","markdown":"https://ijcai.org/proceedings/2024/476.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/476","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Machine Learning","track":"Main Track","number":476,"pages":{"first":4308,"last":4316},"keywords":["Machine Learning: ML: Causality","Machine Learning: ML: Generative models","Machine Learning: ML: Representation learning"],"videos":[],"references":[{"position":1,"raw":"[Ahuja et al., 2023] Kartik Ahuja, Divyat Mahajan, Yixin Wang, and Yoshua Bengio. Interventional causal represen- tation learning. In Proceedings of the 40th International Conference on Machine Learning, 2023.","title":"Interventional causal representation learning","authors":"Kartik Ahuja, Divyat Mahajan, Yixin Wang, Yoshua Bengio","year":"2023","venue":"Proceedings of the 40th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bengio et al., 2013] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1798-1828, 2013.","title":"Representation learning: A review and new perspectives","authors":"Yoshua Bengio, Aaron Courville, Pascal Vincent","year":"2013","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Brehmer et al., 2022] Johann Brehmer, Pim De Haan, Phillip Lippe, and Taco Cohen. Weakly supervised causal representation learning. In Advances in Neural Informa- tion Processing Systems, 2022.","title":"Weakly supervised causal representation learning","authors":"Brehmer","year":"2022","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Comon, 1994] Pierre Comon. Independent component anal- ysis, a new concept? Signal Processing, 36(3):287-314, 1994. Higher Order Statistics.","title":"Pierre Comon. Independent component analysis, a new concept?","authors":"Comon","year":"1994","venue":"Signal Processing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Eastwood and Williams, 2018] Cian Eastwood and Christo- pher K. I. Williams. A framework for the quantitative eval- uation of disentangled representations. In International Conference on Learning Representations, 2018.","title":"A framework for the quantitative evaluation of disentangled representations","authors":"Cian Eastwood, K I Christopher, Williams","year":"2018","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Eastwood et al., 2023] Cian Eastwood, Andrei Liviu Nico- licioiu, Julius Von Kügelgen, Armin Kekić, Frederik Träuble, Andrea Dittadi, and Bernhard Schölkopf. DCI- ES: An extended disentanglement framework with con- nections to identifiability. In The Eleventh International Conference on Learning Representations, 2023.","title":"DCI-ES: An extended disentanglement framework with connections to identifiability","authors":"Cian Eastwood, Andrei Liviu Nicolicioiu, Julius Von Kügelgen, Armin Kekić, Frederik Träuble, Andrea Dittadi, Bernhard Schölkopf","year":"2023","venue":"The Eleventh International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Higgins et al., 2017] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-VAE: Learning basic visual concepts with a constrained varia- tional framework. In International Conference on Learn- ing Representations, 2017.","title":"beta-VAE: Learning basic visual concepts with a constrained variational framework","authors":"Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, Alexander Lerchner","year":"2017","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Hyvarinen et al., 2018] Aapo Hyvarinen, Hiroaki Sasaki, and Richard E. Turner. Nonlinear ica using auxiliary vari- ables and generalized contrastive learning. arXiv preprint arXiv:1805.08651, 2018.","title":"Nonlinear ica using auxiliary variables and generalized contrastive learning","authors":"Aapo Hyvarinen, Hiroaki Sasaki, Richard E Turner","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Hyvärinen and Pajunen, 1999] Aapo Hyvärinen and Petteri Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural Networks, 12(3):429-439, 1999.","title":"Nonlinear independent component analysis: Existence and uniqueness results","authors":"Pajunen Hyvärinen, Aapo Hyvärinen, Petteri Pajunen","year":"1999","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ilse et al., 2020] Maximilian Ilse, Jakub M. Tomczak, Christos Louizos, and Max Welling. Diva: Domain in- variant variational autoencoders. In Medical Imaging with Deep Learning, 2020.","title":"Diva: Domain invariant variational autoencoders","authors":"Maximilian Ilse, M Jakub, Christos Tomczak, Max Louizos, Welling","year":"2020","venue":"Medical Imaging with Deep Learning","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Joy et al., 2021] Tom Joy, Sebastian Schmon, Philip Torr, Siddharth N, and Tom Rainforth. Capturing label charac- teristics in vaes. In International Conference on Learning Representations, 2021.","title":"Capturing label charac-teristics in vaes","authors":"Tom Joy, Sebastian Schmon, Philip Torr, N Siddharth, Tom Rainforth","year":"2021","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Khemakhem et al., 2020] Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoencoders and nonlinear ica: A unifying framework. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, 2020.","title":"Variational autoencoders and nonlinear ica: A unifying framework","authors":"Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, Aapo Hyvarinen","year":"2020","venue":"Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Khemakhem et al., 2021] Ilyes Khemakhem, Ricardo Monti, Robert Leech, and Aapo Hyvarinen. Causal autoregressive flows. In Proceedings of The 24th Interna- tional Conference on Artificial Intelligence and Statistics, 2021.","title":"Causal autoregressive flows","authors":"Ilyes Khemakhem, Ricardo Monti, Robert Leech, Aapo Hyvarinen","year":"2021","venue":"Proceedings of The 24th International Conference on Artificial Intelligence and Statistics","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kingma and Welling, 2013] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.","title":"Auto-encoding variational bayes","authors":"P Diederik, Max Kingma, Welling","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Komanduri et al., 2022] Aneesh Komanduri, Yongkai Wu, Wen Huang, Feng Chen, and Xintao Wu. Scm-vae: Learning identifiable causal representations via structural knowledge. In IEEE International Conference on Big Data (Big Data), 2022.","title":"Scm-vae: Learning identifiable causal representations via structural knowledge","authors":"Aneesh Komanduri, Yongkai Wu, Wen Huang, Feng Chen, Xintao Wu","year":"2022","venue":"IEEE International Conference on Big Data (Big Data)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Komanduri et al., 2024] Aneesh Komanduri, Yongkai Wu, Feng Chen, and Xintao Wu. Learning causally disen- tangled representations via the principle of independent causal mechanisms. arXiv preprint arXiv:2306.01213, 2024.","title":"Learning causally disentangled representations via the principle of independent causal mechanisms","authors":"Aneesh Komanduri, Yongkai Wu, Feng Chen, Xintao Wu","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lachapelle et al., 2020] Sébastien Lachapelle, Philippe Brouillard, Tristan Deleu, and Simon Lacoste-Julien. Gradient-based neural dag learning. In International Conference on Learning Representations, 2020.","title":"Gradient-based neural dag learning","authors":"Sébastien Lachapelle, Philippe Brouillard, Tristan Deleu, Simon Lacoste-Julien","year":"2020","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lachapelle et al., 2022] Sebastien Lachapelle, Pau Ro- driguez, Yash Sharma, Katie E Everett, Rémi LE PRIOL, Alexandre Lacoste, and Simon Lacoste-Julien. Disentan- glement via mechanism sparsity regularization: A new principle for nonlinear ICA. In First Conference on Causal Learning and Reasoning, 2022.","title":"Disentanglement via mechanism sparsity regularization: A new principle for nonlinear ICA","authors":"Sebastien Lachapelle, Pau Rodriguez, Yash Sharma, Katie E Everett, L E Rémi, Alexandre Priol, Simon Lacoste, Lacoste-Julien","year":"2022","venue":"First Conference on Causal Learning and Reasoning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liang et al., 2023] Wendong Liang, Armin Kekić, Julius von Kügelgen, Simon Buchholz, Michel Besserve, Luigi Gresele, and Bernhard Schölkopf. Causal component anal- ysis. In Advances in Neural Information Processing Sys- tems, 2023.","title":"Causal component analysis","authors":"Wendong Liang, Armin Kekić, Simon Julius Von Kügelgen, Michel Buchholz, Luigi Besserve, Bernhard Gresele, Schölkopf","year":"2023","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Lippe et al., 2023] Phillip Lippe, Sara Magliacane, Sindy Löwe, Yuki M Asano, Taco Cohen, and Efstratios Gavves. Causal representation learning for instantaneous and tem- poral effects in interactive systems. In The Eleventh Inter- national Conference on Learning Representations, 2023.","title":"Causal representation learning for instantaneous and temporal effects in interactive systems","authors":"Phillip Lippe, Sara Magliacane, Sindy Löwe, Yuki M Asano, Taco Cohen, Efstratios Gavves","year":"2023","venue":"The Eleventh International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Locatello et al., 2019] Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. In Proceedings of the 36th International Conference on Machine Learning, 2019.","title":"Challenging common assumptions in the unsupervised learning of disentangled representations","authors":"Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, Olivier Bachem","year":"2019","venue":"Proceedings of the 36th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Locatello et al., 2020] Francesco Locatello, Ben Poole, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem, and Michael Tschannen. Weakly-supervised disentanglement without compromises. In Proceedings of the 37th Interna- tional Conference on Machine Learning, 2020.","title":"Weakly-supervised disentanglement without compromises","authors":"Francesco Locatello, Ben Poole, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem, Michael Tschannen","year":"2020","venue":"Proceedings of the 37th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Nasr-Esfahany et al., 2023] Arash Nasr-Esfahany, Moham- mad Alizadeh, and Devavrat Shah. Counterfactual iden- tifiability of bijective causal models. In Proceedings of the 40th International Conference on Machine Learning, 2023.","title":"Counterfactual identifiability of bijective causal models","authors":"Arash Nasr-Esfahany, Mohammad Alizadeh, Devavrat Shah","year":"2023","venue":"Proceedings of the 40th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Papamakarios et al., 2021] George Papamakarios, Eric Nal- isnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing flows for prob- abilistic modeling and inference. Journal of Machine Learning Research, 22(1), 2021.","title":"Normalizing flows for probabilistic modeling and inference","authors":"George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, Balaji Lakshminarayanan","year":"2021","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Parascandolo et al., 2018] Giambattista Parascandolo, Niki Kilbertus, Mateo Rojas-Carulla, and Bernhard Schölkopf. Learning independent causal mechanisms. In Proceedings of the 35th International Conference on Machine Learn- ing, 2018.","title":"Learning independent causal mechanisms","authors":"Parascandolo","year":"2018","venue":"Proceedings of the 35th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Pearl, 2009] Judea Pearl. Causality. Cambridge University Press, Cambridge, UK, 2 edition, 2009.","title":"Judea Pearl. Causality","authors":"Pearl","year":"2009","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Peters et al., 2017] Jonas Peters, Dominik Janzing, and Bernhard Schlkopf. Elements of Causal Inference: Foun- dations and Learning Algorithms. The MIT Press, 2017.","title":"Elements of Causal Inference: Foundations and Learning Algorithms","authors":"Peters","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Scholkopf et al., 2021] Bernhard Scholkopf, Francesco Lo- catello, Stefan Bauer, Nan Rosemary Ke, Nal Kalch- brenner, Anirudh Goyal, and Yoshua Bengio. Toward Causal Representation Learning. Proceedings of the IEEE, 109(5):612-634, May 2021.","title":"Toward Causal Representation Learning","authors":"Bernhard Scholkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, Yoshua Bengio","year":"2021","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Schölkopf and von Kügelgen, 2022] Bernhard Schölkopf and Julius von Kügelgen. From statistical to causal learning. arXiv preprint arXiv:2204.00607, 2022.","title":"From statistical to causal learning","authors":"Von Schölkopf, Bernhard Kügelgen, Julius Schölkopf, Von Kügelgen","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Shen et al., 2022] Xinwei Shen, Furui Liu, Hanze Dong, Qing Lian, Zhitang Chen, and Tong Zhang. Weakly super- vised disentangled generative causal representation learn- ing. Journal of Machine Learning Research, 23(241):1- 55, 2022.","title":"Weakly supervised disentangled generative causal representation learning","authors":"Xinwei Shen, Furui Liu, Hanze Dong, Qing Lian, Zhitang Chen, Tong Zhang","year":"2022","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Suter et al., 2019] Raphael Suter, Djordje Miladinovic, Bernhard Schölkopf, and Stefan Bauer. Robustly disentan- gled causal mechanisms: Validating deep representations for interventional robustness. In Proceedings of the 36th International Conference on Machine Learning, 2019.","title":"Robustly disentangled causal mechanisms: Validating deep representations for interventional robustness","authors":"Raphael Suter, Djordje Miladinovic, Bernhard Schölkopf, Stefan Bauer","year":"2019","venue":"Proceedings of the 36th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Tran et al., 2019] Dustin Tran, Keyon Vafa, Kumar Agrawal, Laurent Dinh, and Ben Poole. Discrete flows: Invertible generative models of discrete data. In Advances in Neural Information Processing Systems, 2019.","title":"Discrete flows: Invertible generative models of discrete data","authors":"Dustin Tran, Keyon Vafa, Kumar Agrawal, Laurent Dinh, Ben Poole","year":"2019","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Träuble et al., 2021] Frederik Träuble, Elliot Creager, Niki Kilbertus, Francesco Locatello, Andrea Dittadi, Anirudh Goyal, Bernhard Schölkopf, and Stefan Bauer. On disen- tangled representations learned from correlated data. In Proceedings of the 38th International Conference on Ma- chine Learning, 2021.","title":"On disentangled representations learned from correlated data","authors":"Frederik Träuble, Elliot Creager, Niki Kilbertus, Francesco Locatello, Andrea Dittadi, Anirudh Goyal, Bernhard Schölkopf, Stefan Bauer","year":"2021","venue":"Proceedings of the 38th International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[von Kügelgen et al., 2021] Julius von Kügelgen, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Schölkopf, Michel Besserve, and Francesco Locatello. Self-supervised learning with data augmentations prov- ably isolates content from style. In Advances in Neural Information Processing Systems, 2021.","title":"Self-supervised learning with data augmentations provably isolates content from style","authors":"Von Kügelgen","year":"2021","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Yang et al., 2021] Mengyue Yang, Furui Liu, Zhitang Chen, Xinwei Shen, Jianye Hao, and Jun Wang. Causalvae: Disentangled representation learning via neural structural causal models. In IEEE Conference on Computer Vision and Pattern Recognition, 2021.","title":"Causalvae: Disentangled representation learning via neural structural causal models","authors":"Yang","year":"2021","venue":"IEEE Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}