Large Language Models for Causal Discovery: Current Landscape and Future Directions

1 Introduction

Uncovering causal relationships-understanding why things happen-is fundamental to scientific discovery and informed decision-making across diverse domains. From discovering the causes of diseases and developing effective treatments to optimizing complex systems like city traffic flow or global supply chains, knowing why something occurs is crucial for effective intervention [Kuang et al., 2020]. Causal Discovery (CD) has long relied on two pillars: statistical methods for data analysis and domain experts for knowledge integration [Pearl, 2009]. While domain experts provide invaluable insights drawn from years of experience and deep understanding, their involvement often creates bottlenecks in the discovery process. Consulting experts is time-consuming, expensive, and inherently limited by human availability and potential biases. Meanwhile, Statistical CD methods (SCD)

[Shimizu et al., 2011; Scutari and Denis, 2014; Zheng et al., 2018], while mathematically rigorous, often fall short in real-world scenarios. They typically demand vast amounts of high-quality data, which is often unavailable or expensive to acquire. Furthermore, they struggle to disentangle complex temporal dynamics inherent in many real-world systems, where causes and effects unfold over time and influence each other in intricate ways [Ban et al., 2023a]. Specifically, these methods frequently produce multiple, equally plausible causal explanations, traditionally requiring expert intervention to resolve these ambiguities.

Large Language Models (LLMs) offer a transformative approach to the challenges of causal discovery, potentially serving as valuable tools to complement and augment human expertise. Their ability to process and synthesize massive amounts of text-effectively distilling knowledge from countless documents, research papers, and expert opinions-makes them powerful aids for enhancing expert-level reasoning [Petroni et al., 2019]. Unlike traditional domain experts with specialized knowledge in specific areas, LLMs can provide broad perspective across multiple domains concurrently, working alongside human experts rather than replacing them. They can integrate information from diverse sources to help identify potential causal relationships that might benefit from further expert validation. For example, when analyzing urban traffic, an LLM could rapidly synthesize insights from traffic engineering papers, weather studies, and city planning documents to suggest potential causal factors for human experts to accelerate what could otherwise take weeks of literature review.

The integration of LLMs into CD represents a paradigm shift from both purely statistical approaches and traditional expert-dependent methods, manifesting in three primary ways: (1) LLMs can directly infer causal graphs or subgraph structures from natural language descriptions and domain knowledge [Jin et al., 2023b], effectively automating the initial expert hypothesis generation phase. (2) LLMs can function as posterior correction mechanisms, validating and refining causal relationships identified by SCD methods against their extensive knowledge base [Long et al., 2024], similar to how experts would review and adjust statistical findings. (3) They can serve as comprehensive prior information sources for traditional SCD algorithms, providing domain knowledge and contextual constraints before statistical analysis [Takayama et al., 2024]. This systematic enhancement of human expert involvement with LLM-based assistance not only accelerates the discovery process but also makes sophisticated causal analysis accessible to a broader range of researchers and without domain expertise.

Several surveys have examined how large language models (LLMs) relate to causality. However, there is still a lack of detailed survey specifically on causal discovery in the era of LLMs. Traditional causal discovery surveys like [Glymour et al., 2019] extensively explore connections with machine learning and deep learning approaches but lacks discussion with the emergence of LLMs in this domain. [Zhao et al., 2023] pioneered the discussion of LLMs in causal reasoning but primarily focuses on broader tasks such as counterfactual reasoning, cause attribution, and causal effect estimation. More recent work by [Zhang et al., 2024] examines causality in LLMs but provides a limited analysis of how these models fundamentally transform causal discovery methods. Similarly, [Yu et al., 2024] offers a comprehensive review of improving LLMs' causal reasoning capabilities but lacks specific focus on causal discovery tasks and their integration with traditional causal theory. To address this gap in literature, our survey examines the emerging intersection of LLMs and CD, providing a systematic framework for understanding their integration. We begin in Section 2 with background on both LLMs and CD, bridging knowledge gaps for researchers from either field. Section 3 analyzes how LLMs can enhance CD through direct inference, prior knowledge integration, and structural refinement, while acknowledging the methodological challenges each approach faces. Section 4 evaluates benchmark datasets and showcases applications across diverse domains, from healthcare to social sciences. Section 5 examines current limitations, explores open questions about LLMs' causal reasoning capabilities, and identifies promising research directions that could advance this rapidly evolving field.

2 Background

This section lays the groundwork for understanding fundamental concepts and methodologies of causal inference and discovery, including essential notations and definitions in causal discovery to prepare readers better understand and subsequent technical sections.

2.1 Graphical Models and Structure Learning

Directed Acyclic Graphs (DAGs) provide the foundational framework for causal modeling, defined as ordered pairs G = (V, E), where V represents variables and E ⊆ V × V contains directed edges without cycles. Structure learning aims to estimate these graphs by capturing essential dependence relationships in data [Drton and Maathuis, 2017]. While any probability measure P(X) can be factored as P(X) = P(X1)P(X2|X1) · · · P(Xn|X1, . . . ,Xn-1), this

Figure 2: Two Markov equivalent DAGs

complete representation introduces unnecessary dependencies. The goal is identifying minimal structures where edges Xi → Xj represent only necessary conditional dependencies [Pearl, 2009]. A fundamental limitation is that different DAGs can encode identical conditional independence relations, forming Markov equivalence classes. For example, given variables with relations X1 ̸⊥ X2,X2 ̸⊥ X3,X1 ̸⊥ X3,X1 ⊥ X3 | X2, multiple network structures (Figure 2) can represent these relationships, showing that causal direction cannot be uniquely determined from data alone.

2.2 Causal Discovery: Methods and Evaluation

A causal graph is a DAG where edges represent direct causal relationships-an edge Xi → Xj indicates that Xi is a direct cause of Xj. Causal Discovery (CD) is the systematic process of uncovering these causal relationships from observational data [Glymour et al., 2019], which typically involves learning Bayesian Network structures with causal interpretations when all common causes are observed. These discovered structures serve as foundations for downstream applications including effect inference [Kuang et al., 2020] and prediction [Chu et al., 2023]. Formally, a Structural Causal Model (SCM) M = (V,U, F, P) provides the framework for representing these systems [Pearl, 2009], where V represents observable variables, U unobservable variables, F causal mechanisms, and P the probability distribution. Each variable Xi ∈ V is determined by a function fi ∈ F of its direct causes Pa(Xi) and an independent noise term Ui ∈ U: Xi = fi(Pa(Xi),Ui). The goal of causal discovery is to recover these structural components, particularly the parent sets defining the underlying graph structure.

Statistical approaches to causal discovery have evolved into two methodologies: Constraint-based methods like the PC Algorithm [Spirtes et al., 2001] that test conditional independence relationships (X ⊥ Y |Z) to remove edges and orient directions, and Score-based methods that optimize scoring functions balancing model fit against complexity, exemplified by NOTEARS [Zheng et al., 2018]. Causal discovery with LLMs addresses two distinct tasks with different evaluation requirements: Causal Order Predictions (pairwise discovery) evaluates direct relationships between variable pairs using standard classification metrics like accuracy and F1-score, while Full Graph Discovery constructs complete causal networks using Structural Hamming Distance (SHD) and Normalized Hamming Distance (NHD) [Tsamardinos et al., 2006], which measure edge operations needed to transform a learned graph into the true causal structure.

3 LLMs for Causal Discovery

Large Language Models (LLMs) have revolutionized natural language processing through their advanced transformer architecture, demonstrating remarkable capabilities in reasoning, knowledge acquisition, and cross-domain generalization [Zhao et al., 2023], while serving as reliable knowledge bases

[Petroni et al., 2019] suitable for causal discovery tasks, enabling them to assume the role of human domain experts as illustrated in Figure 3. We discuss three primary approaches to incorporating LLMs in causal discovery: (a) direct inference, (b) posterior refinement on derived causal structures, and (c) knowledge integration as prior for generating causal structures, with Table 1 providing practical examples of prompts used in these approaches.

3.1 LLMs as Direct Inference

Direct causal discovery leverages LLMs' extensive knowledge acquired during pre-training to serve as automated domain experts capable of reasoning about causal relationships. Unlike traditional methods that rely on statistical patterns or require extensive human expert consultation, LLMs can utilize their broad understanding of domain concepts, scientific principles, and real-world relationships to infer causality at scale. The fundamental setting involves providing LLMs with meta-data such as the descriptive texts T = {t1, t2, . . . , tn} for variables X = {x1, x2, . . . , xn}. By comprehending these descriptions and applying learned knowledge, LLMs identify causal statements denoted as S = {(xi, xj)}, where (xi, xj) indicates that xi causes xj . This approach effectively transforms LLMs into scalable meta-data experts who can reason about causality with both breadth and precision.

Two primary approaches have emerged in this direction. First, causal order prediction, pioneered by [Willig et al., 2022] and advanced by [Kıcıman et al., 2023], focuses on determining pairwise causal relationships through direct LLM queries, primaliry using chain-of-thought style prompting [Wei et al., 2022]. Second, complete and partial causal graph discovery methods aim to identify broader causal structures through iterative pairwise discovery [Long et al., 2024; Kıcıman et al., 2023], though this naturally introduces computational challenges scaling quadratically with the number of variables. To address these efficiency barriers, [Jiralerspong et al., 2024] reduced computational complexity from O(n2 ) to O(n) through a structured three-phase process: root cause identification, relationship expansion, and logical consistency verification. For high-dimensional settings, [Sokolov et al., 2024] developed a scalable solution using hierarchical clustering based on semantic similarity, efficiently discovering causal relationships by first analyzing within-cluster connections before determining inter-cluster causality. To enhance the reliability of these approaches, researchers have pursued several complementary directions: systematic verification to mitigate hallucination [Ji et al., 2023], specialized fine-tuning for causal reasoning [Le et al., 2024], and structured prompting frameworks for consistent causal extraction [Zhang et al., 2024; Vashishtha et al., 2023].

3.2 LLMs as Posterior Correction

LLMs can serve as a expert judge by correcting and refining the learned causal structures from the traditional statistical causal discovery (SCD) methods based on contextual reasoning or additional data. Recall that most of constraint-based and score-based methods can only identify a BN up to its Markov equivalence class. Given the set of all conditional independence relations, [Long et al., 2023] introduces a method that uses LLM as an imperfect expert to progressively reduce the number of possible causal structures within the Markov equivalence class, while controlling the risk of misorienting edges. However, its application to larger datasets remains unproven and is likely hindered by the approach's complexity and computational demands. To refine causal structures beyond the Markov equivalence class using LLMs, [Long et al., 2023] formulated a constrained discrete optimization problem: minS |ME,S | subject to P(G⋆ ∈ ME,S ) ≥ 1 - η, where ME,S denotes the refined equivalence class after incorporating expert-guided orientations, and η controls the risk of excluding the true graph G⋆ . Define U to be all undirected edges whose orientations vary across DAGs in the MEC, two greedy strategies are proposed: one prioritizes reducing the size of the equivalence class, and the other minimizes the risk of eliminating, both following the iterative structure 1:

Instead of deriving the true causal graph beyond its Markov equivalence class, the Iterative LLM Supervised CSL Framework (ILS-CSL) by [Ban et al., 2023a] refines a partially learned DAG by leveraging LLM feedback iteratively to correct edge orientations. This approach efficiently integrates expert knowledge while avoiding exhaustive pairwise queries, ensuring a more accurate and robust causal structure without requiring full causal discovery from scratch.

[Takayama et al., 2024] introduced a framework in which large language models (LLMs) function both as post hoc refiners and prior knowledge generators, described in Algorithm 2: Initially, the raw adjacency matrix ˆG0, derived from certain SCD methods without prior knowledge, is input into the LLMs. For each potential edge, the LLM is queried multiple times using pairwise prompts, with each response being a binary decision (i.e., "yes" or "no"). The probability matrix P is then constructed by aggregating these responses, and is subsequently converted into a deterministic prior knowledge matrix ˆG based on predefined thresholds. Finally, ˆG is incorporated into the original SCD method to infer the final causal graph. While this framework is primarily designed for LiNGAM, it is applicable to any causal discovery algorithm that relies on a matrix of pairwise edge scores.

[Abdulaal et al., 2024] introduced the Causal Modeling Agent (CMA), a novel framework for causal discovery that combines the metadata-based reasoning of LLMs with the data-driven power of Deep Structural Causal Models (DSCMs). CMA employs an LLM to propose an initial causal graph, which then informs the fitting of a DSCM to the data. The framework iteratively refines this graph in global and local phases, again using the LLM for both prior knowledge and a critic of the model's output, enabling the discovery of causal relationships in complex, multi-modal data. Unlike purely constraint-based or scoring-based methods, CMA integrates aspects of both while leveraging LLMs. Furthermore, it can generate chain graphs to account for unmeasured confounding and has demonstrated state-of-the-art performance on datasets like Arctic Sea [Huang et al., 2021] and on synthetic data designed to prevent data leakage.

3.3 LLMs as Prior Knowledge

Similarly, LLMs can also be used in conjunction with traditional methods to provide source of prior knowledge by leveraging meta-data extracted from textual descriptions and domain-specific information. In [Ban et al., 2023b], a set of variables X along with their descriptive texts T are provided as input to an LLM, which performs causal discovery by identifying direct relationships after comprehending the semantic meaning of each variable. Traditionally, the incorporation of prior knowledge into CD procedures follows two primary approaches: the hard constraint method and the soft constraint method. The hard constraint approach strictly enforces prior knowledge by eliminating edges in BNs that conflict with constraints derived from traditional causal discovery algorithms. However, this method lacks flexibility, as any spurious prior assumptions cannot be corrected during the learning process, potentially leading to erroneous causal structures. [Chen et al., 2023] provides a systematic approach for detecting and correcting potentially erroneous prior knowledge derived from LLMs, thereby enhancing the reliability of utilizing LLMs for hard prior constraints. To address this, [Chen et al., 2023] focus on a particularly impactful class of prior errors, termed order-reversed priors, where a prior (Xj → Xi) contradicts the true edge (Xi → Xj) in the underlying DAG G0. The authors show that incorporating such priors under hard constraints induces a unique acyclic structure in the learned graph G1, called a quasi-circle: a pair of distinct directed paths ℓ1 and ℓ2 from Xs to Xt sharing only their endpoints, with an example shown in Figure 5. Building on this insight, the authors propose a post-hoc correction strategy that iteratively identifies priors involved in quasi-circles, and replaces or removes them.

In contrast, the soft constraint approach, though implemented through varying methodologies, aims to integrate prior knowledge in a fault-tolerant manner. [Ban et al., 2023b] integrates LLM-derived prior into scoring functions, such as BDeu or BIC, as a regularization term, allowing flexibility in cases where LLM priors may be inconsistent with observed data. Alternatively, [Darvariu et al., 2024] introduces a probabilistic prior framework described in 2, where the LLMderived priors consist of probabilities of the existence and direction of each edge in the causal graph, which are then incorporated into some traditional CD algorithms. In particular, the proposed approach is compatible with any causal discovery algorithm that relies on a pairwise edge score matrix, including LiNGAM and NOTEARS. Given the demonstrated effectiveness of integrating LLMs into differentiable causal discovery algorithms within this probabilistic framework, recent work by [Kampani et al., 2024] further extends this paradigm by using LLMs to initialize the continuous optimization process. Unlike many previous methods that assume a parametric model (e.g., linear-Gaussian), focus primarily on continuous data, and compute likelihoods accordingly, the LLM-DCD framework targets discrete data using a non-parametric loss called MLE-INTERP. Notably, LLMs provide an initial estimate of the adjacency matrix A, thereby guiding the continuous optimization from an informed starting point. Although their results appears to be promising on discrete data, it's unclear whether a similar framework could be generalized to continuous data.

A novel application of LLMs as providers of prior information lies in conditional independence testing, a cornerstone of constraint-based causal discovery. Rather than using traditional statistical tests, LLMs can be queried with natural language prompts representing conditional independence relationships, effectively serving as an oracle. As shown in Figure 4, this enables constainted based algorithms like PC algorithm [Spirtes et al., 2001] to leverage LLMs for guidance in causal graph construction. The chatPC method [Cohrs et al., 2023] exemplifies this approach, integrating LLMs with the PC algorithm by transforming conditional independence tests (e.g., "Is X independent of Y given Z?") into natural language prompts. The LLM's responses then guide the PC algorithm's edge removal process. This work evaluates LLM performance on such queries, proposes a statistical aggregation method to combine multiple LLM responses for increased robustness, and analyzes the resulting causal graphs. Research indicates that LLMs tend to be more conservative in their independence judgments than human experts, yet still demonstrate evidence of causal reasoning.

Remark. Probabilistic prior knowledge has never been a strong focus in the causal discovery literature. Before the era of LLMs, expert priors were often assumed to be deterministically correct. Although recent efforts have introduced imperfect expert frameworks, they often rely on simplified assumptions. For instance, [Long et al., 2023] models expert fallibility using a fixed noise level ϵ ∈ {0.1, 0.3}, assuming a uniform error rate across all edge orientations. However, in practice, LLM accuracy is highly sensitive to prompt design, semantic ambiguity, and domain-specific difficulty-none of which are captured by a static ϵ. Meanwhile, [Chen et al., 2023] focuses on quasi-circle detection to identify erroneous priors but limits the scope to cycles of length three, due to the exponential complexity O(nL-2 ) for general detection. As a result, when faced with novel tasks or unseen data, the reliability of LLM-derived causal knowledge remains uncertain and lacks any principled quantification.

Algorithm 2: Probabilistic LLM-driven Priors

Input: Data D, metadata {µi}di=1 , LLM expert E, score function f, budget B, prior strength τ Output: Estimated DAG G ⋆ Initialize G (0) ← (V, ∅), P ∈ Rd×d ; foreach unordered pair (i, j), i ̸= j do Query E on (µi, µj) to get Pi→j ; Set P[i, j] ←Pi→j ; G ⋆ ← G (0) , s⋆ ←∞ ; for b = 1 to B do G ←G (0) ; while termination not met do Let A ← valid acyclic edges not in G ; Sample ei→j ∼ softmax(i,j)∈A(P[i, j]/τ) ; G ← G ∪ {ei→j} ; Compute s ← f(G) ; if s < s⋆ then G ⋆ ← G, s⋆ ← s ; return G ⋆

4 Evaluations and Applications

4.1 Benchmarks and Datasets

Table 2 provides a comprehensive overview of benchmark datasets used to evaluate LLMs' causal reasoning capabilities. For each dataset, the table indicates (1) if for determining causal relationships between variable pairs, (2) full graph reconstruction, and (3) novel reasoning scenarios (testing on previously unseen causal patterns). The table also details key characteristics including the average number of nodes and edges per graph, along with the total number of graphs in the collection. Multi-graph benchmark datasets, such as CausalBench [Zhou et al., 2024], are particularly noteworthy as they incorporate established causal networks dataset commonly tested in the literature, such as Asia [Pearl, 1988] and Insurance [Binder et al., 1997], offering evaluation across diverse graph sizes and domains. Among these benchmarks, CORR2CAUSE [Jin et al., 2023a] addresses a crucial aspect of causal reasoning: the ability to differentiate causation from correlation and can be further used for fine-tuning LLMs to enhance their causal inference capabilities from identifying purely correlational statements.

4.2 Applications

Causal discovery has been used as a crucial tool across numerous real-world domains, with LLM-based methods significantly expanding its capabilities and applications. For instance, [Gkountouras et al., 2024] introduced a "causal world model" framework, connecting causal variables to natural language to improve reasoning in complex environments. To address the challenge of ill-defined high-level variables often found in real-world observational data, [Liu et al., 2024] enables LLMs to propose such variables, effectively extending

Note: Pair = Pairwise Discovery; Full = Full Graph Discovery; Novel = Involves a stimulator to regenerate data to avoid data leakage

Table 2: Summary of Benchmark Datasets for Evaluating Causal Discovery Tasks

causal discovery to unstructured data. These and other advances have facilitated LLM-guided causal discovery in fields like medicine [Tu et al., 2023; Cohrs et al., 2025], finance [Sokolov et al., 2024], genetics [Afonja et al., 2024], and health informatics [Patel et al., 2024]. Further research explores LLM-driven causal discovery in multi-agent systems [Abdulaal et al., 2024; Jiang et al., 2024] and multi-modal data integration [Shen et al., 2024], leveraging the richness of multi-modal data to provide additional information to better capture the complexity of real-world systems.

5 Challenges and Visions

Unified Evaluation and Domain-Specific Applications.

A significant challenge in LLM-enhanced causal discovery is the lack of standardized evaluation protocols, making it difficult to establish true state-of-the-art performance. Researchers should utilize both synthetic datasets (avoiding data leakage) and established benchmarks while developing frameworks incorporating multiple performance metrics beyond accuracy. Simultaneously, we envision significant potential in domain-specialized models that better capture fieldspecific causal relationships in critical domains like healthcare and economics. These systems could be enhanced through integration with field-specific knowledge bases, specialized reasoning modules, and domain-specific RAG incorporating scientific literature-for instance, accessing pathway databases in biology or physics-based models in climate science. Continuous learning mechanisms could ensure these systems remain current with emerging research findings.

LLM for SCM Diagnosis. Future research should expand LLMs' role beyond identifying causal relationships to verifying underlying properties of Structural Causal Models. While current approaches primarily detect causal links [Jin et al., 2023b], LLMs could verify crucial aspects like the nature of relationships (linear vs. nonlinear), functional forms, and noise distributions. This verification is particularly important since traditional methods depend on specific assumptions-DirectLiNGAM requires linear relationships [Shimizu et al., 2011], and BIC scoring becomes less reliable with nonlinear effects [Peters et al., 2017]. LLMs could leverage their language understanding to interpret domain knowledge about expected relationship characteristics, helping select appropriate algorithms and validate assumptions, thus improving reliability across diverse scenarios.

Explanability and Interpretbility. The capability of LLMs to perform genuine causal reasoning remains an open question, and studies [Zeˇcevi´c et al., 2023; Feng et al., 2024] have suggested LLMs may function more as patternmatching systems reciting embedded knowledge rather than understanding true causality, functioning as 'causal parrots' without deeper understanding [Jin et al., 2023a]. Empirical studies show LLMs often rely on correlational heuristics when faced with questions requiring understanding of confounding variables-indicating a gap between statistical association and causal understanding [Kıcıman et al., 2023; Long et al., 2024]. Future research should focus on: (1) developing interpretability methods to analyze how LLMs process causal relationships; (2) investigating relationships between pre-training data and causal capabilities; and (3) creating frameworks distinguishing between genuine reasoning and memorized patterns.

6 Conclusion

The integration of LLMs with causal discovery represents a promising yet challenging advancement in artificial intelligence. This survey has explored how LLMs can enhance traditional causal discovery through direct inference, knowledge integration, and structural refinement. However, as systems trained primarily on correlational data, LLMs face inherent limitations in genuine causal reasoning that requires interventions and counterfactuals. They may reproduce existing biases and generate plausible but incorrect causal relationships, highlighting critical areas for future research in understanding and improving LLMs' causal reasoning capabilities. Moving forward, we envision LLMs as complementary tools that assist human experts rather than replace them, ultimately accelerating scientific discovery while maintaining human expertise at the center of the causal discovery process.

Acknowledgments

The work is supported in part by the U.S. Office of Naval Research Award under Grant Number N00014-24-1-2668, and the National Science Foundation under Grants IIS-2316306 and CNS-2330215.