{"type":"ScholarlyArticle","title":"PHSIC against Random Consistency and Its Application in Causal Inference","authors":[{"name":"Jue Li","slug":"jue-li","given":"Jue","family":"Li","affiliation":"Institute of Big Data Science and Industry , Shanxi University , Taiyuan 030006 , China"},{"name":"Yuhua Qian","slug":"yuhua-qian","given":"Yuhua","family":"Qian","affiliation":"Institute of Big Data Science and Industry , Shanxi University , Taiyuan 030006 , China"},{"name":"Jieting Wang","slug":"jieting-wang","given":"Jieting","family":"Wang","affiliation":"Institute of Big Data Science and Industry , Shanxi University , Taiyuan 030006 , China"},{"name":"Saixiong Liu","slug":"saixiong-liu","given":"Saixiong","family":"Liu","affiliation":"Institute of Big Data Science and Industry , Shanxi University , Taiyuan 030006 , China"}],"abstract":"The Hilbert-Schmidt Independence Criterion (HSIC) based on kernel functions is capable of detecting nonlinear dependencies between variables, making it a common method for association relationship mining. However, in situations with small samples, high dimensions, or noisy data, it may generate spurious associations, causing two unrelated variables to have certain scores. To address this issue, we propose a novel criterion, named as Pure Hilbert-Schmidt Independence Criterion (PHSIC). PHSIC is achieved by subtracting the mean HSIC obtained under random conditions from the original HSIC value. We demonstrate three significant advantages of PHSIC through theoretical and simulation experiments: (1) PHSIC has a baseline of zero, enhancing the interpretability of HSIC. (2) Compared to HSIC, PHSIC exhibits lower bias. (3) PHSIC enables a fairer comparison across different samples and dimensions. To validate the effectiveness of PHSIC, we apply it to multiple causal inference tasks to measure the independence between cause and residual. Experimental results demonstrate that the causal model based on PHSIC performs well compared to other methods in scenarios involving small sample sizes and noisy data, both in real and simulated datasets.","doi":"10.24963/ijcai.2024/233","url":"https://ijcai.org/proceedings/2024/233","pdf":"https://ijcai.org/proceedings/2024/0233.pdf","markdown":"https://ijcai.org/proceedings/2024/233.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/233","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Data Mining","track":"Main Track","number":233,"pages":{"first":2108,"last":2116},"keywords":["Data Mining: DM: Exploratory data mining","Machine Learning: ML: Causality","AI Ethics, Trust, Fairness: ETF: Trustworthy AI"],"videos":[],"references":[{"position":1,"raw":"[Assaad et al., 2019] Charles Karim Assaad, Emilie Devi- jver, Eric Gaussier, and Ali Ait-Bachir. Scaling causal inference in additive noise models. In The 2019 ACM SIGKDD Workshop on Causal Discovery, pages 22-33. PMLR, 2019.","title":"Scaling causal inference in additive noise models","authors":"Charles Karim Assaad, Emilie Devijver, Eric Gaussier, Ali Ait-Bachir","year":"2019","venue":"The 2019 ACM SIGKDD Workshop on Causal Discovery","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Gretton et al., 2005] Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Schölkopf. Measuring statis- tical dependence with hilbert-schmidt norms. In Interna- tional conference on algorithmic learning theory, pages 63-77. Springer, 2005.","title":"Measuring statistical dependence with hilbert-schmidt norms","authors":"Arthur Gretton, Olivier Bousquet, Alex Smola, Bernhard Schölkopf","year":"2005","venue":"International conference on algorithmic learning theory","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Gretton et al., 2007] Arthur Gretton, Kenji Fukumizu, Choon Teo, Le Song, Bernhard Schölkopf, and Alex Smola. A kernel statistical test of independence. Ad- vances in neural information processing systems, 20, 2007.","title":"A kernel statistical test of independence","authors":"Arthur Gretton, Kenji Fukumizu, Choon Teo, Le Song, Bernhard Schölkopf, Alex Smola","year":"2007","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Hotelling, 1992] Harold Hotelling. Relations between two sets of variates. In Breakthroughs in statistics: methodol- ogy and distribution, pages 162-190. Springer, 1992.","title":"Harold Hotelling. Relations between two sets of variates","authors":"Hotelling","year":"1992","venue":"Breakthroughs in statistics: methodology and distribution","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hoyer et al., 2008] Patrik Hoyer, Dominik Janzing, Joris M Mooij, Jonas Peters, and Bernhard Schölkopf. Nonlinear causal discovery with additive noise models. Advances in neural information processing systems, 21, 2008.","title":"Nonlinear causal discovery with additive noise models","authors":"Patrik Hoyer, Dominik Janzing, M Joris, Jonas Mooij, Bernhard Peters, Schölkopf","year":"2008","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Hu et al., 2018] Shoubo Hu, Zhitang Chen, Vahid Par- tovi Nia, Laiwan Chan, and Yanhui Geng. Causal infer- ence and mechanism clustering of a mixture of additive noise models. Advances in neural information processing systems, 31, 2018.","title":"Causal inference and mechanism clustering of a mixture of additive noise models","authors":"Hu","year":"2018","venue":"Advances in neural information processing systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Jaworski et al., 2010] Piotr Jaworski, Fabrizio Durante, Wolfgang Karl Hardle, and Tomasz Rychlik. Copula the- ory and its applications, volume 198. Springer, 2010.","title":"Copula theory and its applications","authors":"Piotr Jaworski, Fabrizio Durante, Wolfgang Karl Hardle, Tomasz Rychlik","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Li et al., 2021] Guohe Li, Yong Li, Yifeng Zheng, Ying Li, Yunfeng Hong, and Xiaoming Zhou. A novel feature selection approach with pareto optimality for multi-label data. Applied Intelligence, pages 1-18, 2021.","title":"A novel feature selection approach with pareto optimality for multi-label data","authors":"Guohe Li, Yong Li, Yifeng Zheng, Ying Li, Yunfeng Hong, Xiaoming Zhou","year":"2021","venue":"Applied Intelligence","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Liaghat and Mansoori, 2019] Samaneh Liaghat and Egh- bal G Mansoori. Filter-based unsupervised feature se- lection using hilbert-schmidt independence criterion. In- ternational journal of machine learning and cybernetics, 10:2313-2328, 2019.","title":"Samaneh Liaghat and Eghbal G Mansoori. Filter-based unsupervised feature selection using hilbert-schmidt independence criterion","authors":"Mansoori Liaghat","year":"2019","venue":"International journal of machine learning and cybernetics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ma et al., 2020] Wan-Duo Kurt Ma, JP Lewis, and W Bas- tiaan Kleijn. The hsic bottleneck: Deep learning without back-propagation. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 5085-5092, 2020.","title":"The hsic bottleneck: Deep learning without back-propagation","authors":"Wan-Duo Kurt Ma, J P Lewis, W Bastiaan Kleijn","year":"2020","venue":"Proceedings of the AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Mooij et al., 2016] Joris M Mooij, Jonas Peters, Dominik Janzing, Jakob Zscheischler, and Bernhard Schölkopf. Distinguishing cause from effect using observational data: methods and benchmarks. The Journal of Machine Learn- ing Research, 17(1):1103-1204, 2016.","title":"Distinguishing cause from effect using observational data: methods and benchmarks","authors":"Mooij","year":"2016","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Niu et al., 2010] Donglin Niu, Jennifer G Dy, and Michael I Jordan. Multiple non-redundant spectral clustering views. In Proceedings of the 27th international conference on ma- chine learning (ICML-10), pages 831-838, 2010.","title":"Multiple non-redundant spectral clustering views","authors":"Donglin Niu, Jennifer G Dy, Michael I Jordan","year":"2010","venue":"Proceedings of the 27th international conference on machine learning (ICML-10)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ogarrio et al., 2016] Juan Miguel Ogarrio, Peter Spirtes, and Joe Ramsey. A hybrid causal search algorithm for latent variable models. In Conference on probabilistic graphical models, pages 368-379. PMLR, 2016.","title":"Juan Miguel Ogarrio, Peter Spirtes, and Joe Ramsey. A hybrid causal search algorithm for latent variable models","authors":"Ogarrio","year":"2016","venue":"Conference on probabilistic graphical models","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Puth et al., 2015] Marie-Therese Puth, Markus Neuhäuser, and Graeme D Ruxton. Effective use of spearman's and kendall's correlation coefficients for association between two measured traits. Animal Behaviour, 102:77-84, 2015.","title":"Effective use of spearman's and kendall's correlation coefficients for association between two measured traits","authors":"Marie-Therese Puth, Markus Neuhäuser, Graeme D Ruxton","year":"2015","venue":"Animal Behaviour","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Romano et al., 2016] Simone Romano, Nguyen Xuan Vinh, James Bailey, and Karin Verspoor. A framework to adjust dependency measure estimates for chance. In Proceedings of the 2016 SIAM international conference on data mining, pages 423-431. SIAM, 2016.","title":"A framework to adjust dependency measure estimates for chance","authors":"Simone Romano, Xuan Nguyen, James Vinh, Karin Bailey, Verspoor","year":"2016","venue":"Proceedings of the 2016 SIAM international conference on data mining","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Schweizer and Wolff, 1981] Berthold Schweizer and Ed- ward F Wolff. On nonparametric measures of dependence for random variables. The annals of statistics, 9(4):879- 885, 1981.","title":"On nonparametric measures of dependence for random variables","authors":"Wolff Schweizer, Berthold Schweizer, Edward F Wolff","year":"1981","venue":"The annals of statistics","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Shimizu et al., 2006] Shohei Shimizu, Patrik O Hoyer, Aapo Hyvärinen, Antti Kerminen, and Michael Jordan. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.","title":"A linear non-gaussian acyclic model for causal discovery","authors":"Shohei Shimizu, Patrik O Hoyer, Aapo Hyvärinen, Antti Kerminen, Michael Jordan","year":"2006","venue":"Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Song et al., 2007a] Le Song, Alex Smola, Arthur Gretton, and Karsten M Borgwardt. A dependence maximization view of clustering. In Proceedings of the 24th interna- tional conference on Machine learning, pages 815-822, 2007.","title":"A dependence maximization view of clustering","authors":"Song","year":"2007","venue":"Proceedings of the 24th international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Song et al., 2007b] Le Song, Alex Smola, Arthur Gretton, Karsten M Borgwardt, and Justin Bedo. Supervised fea- ture selection via dependence estimation. In Proceedings of the 24th international conference on Machine learning, pages 823-830, 2007.","title":"Supervised feature selection via dependence estimation","authors":"Song","year":"2007","venue":"Proceedings of the 24th international conference on Machine learning","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Spirtes et al., 2000] Peter Spirtes, Clark N Glymour, and Richard Scheines. Causation, prediction, and search. MIT press, 2000.","title":"Causation, prediction, and search","authors":"Peter Spirtes, Clark N Glymour, Richard Scheines","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Vinh et al., 2009] Nguyen Xuan Vinh, Julien Epps, and James Bailey. Information theoretic measures for cluster- ings comparison: is a correction for chance necessary? In Proceedings of the 26th annual international conference on machine learning, pages 1073-1080, 2009.","title":"Information theoretic measures for clusterings comparison: is a correction for chance necessary?","authors":"Xuan Nguyen, Julien Vinh, James Epps, Bailey","year":"2009","venue":"Proceedings of the 26th annual international conference on machine learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wang et al., 2022] Jieting Wang, Yuhua Qian, Feijiang Li, Jiye Liang, and Qingfu Zhang. Generalization perfor- mance of pure accuracy and its application in selective en- semble learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2):1798-1816, 2022.","title":"Generalization performance of pure accuracy and its application in selective ensemble learning","authors":"Wang","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Yamanishi et al., 2004] Yoshihiro Yamanishi, Jean-Philippe Vert, and Minoru Kanehisa. Heterogeneous data compar- ison and gene selection with kernel canonical correlation analysis. Kernel methods in computational biology, pages 209-229, 2004.","title":"Heterogeneous data comparison and gene selection with kernel canonical correlation analysis","authors":"Yoshihiro Yamanishi, Jean-Philippe Vert, Minoru Kanehisa","year":"2004","venue":"Kernel methods in computational biology","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhang and Hyvarinen, 2012] Kun Zhang and Aapo Hyvari- nen. On the identifiability of the post-nonlinear causal model. arXiv preprint arXiv:1205.2599, 2012.","title":"On the identifiability of the post-nonlinear causal model","authors":"Hyvarinen Zhang, Kun Zhang, Aapo Hyvarinen","year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhao et al., 2023] Boxiang Zhao, Shuliang Wang, Lianhua Chi, Chuanfeng Zhao, Hanning Yuan, Qi Li, Xiaojia Liu, Jing Geng, and Ye Yuan. Hanm: Hierarchical additive noise model for many-to-one causality discovery. IEEE Transactions on Knowledge and Data Engineering, 2023.","title":"Hanm: Hierarchical additive noise model for many-to-one causality discovery","authors":"Zhao","year":"2023","venue":"IEEE Transactions on Knowledge and Data Engineering","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}