LLMs Uncertainty Quantification via Adaptive Conformal Semantic Entropy

1 Introduction

Despite the wide range application of Large Language Models (LLMs), they remain uncertain in their predictions and often generate incorrect or misleading outputs with high confidence [Shanahan, 2024]. This overconfidence limits their safe deployment in high-stakes domains such as healthcare [Thirunavukarasu et al., 2023], law [Teubner et al., 2023], and scientific research [Liu et al., 2023]. Reliable uncertainty quantification (UQ), defined as the process of estimating the model's confidence in its own predictions, is therefore essential for improving model reliability, safety, and trustworthiness [Chang et al., 2024].

Related Work. Most existing methods for uncertainty estimation in LLMs rely on token-level signals, such as entropy over next-token distributions [Duan et al., 2023; Fadeeva et al., 2024; Yarie et al., 2024] or average sequence log-likelihood [Yao et al., 2019; Huang et al., 2023]. While computationally efficient, these approaches capture only surface-level lexical variability rather than uncertainty over meaning [Shorinwa et al., 2025]. In autoregressive generation, LLMs often assign high probability to multiple nearsynonymous tokens, producing outputs that are probabilistically coherent yet semantically ambiguous or incorrect. As a result, token-level entropy cannot distinguish superficial phrasing variation from genuine semantic ambiguity. In particular, entropy may remain low even when probability mass is spread across responses with mutually inconsistent meanings, creating a misleading impression of confidence [Brown et al., 2020; Holtzman et al., 2020].

More recent work has shifted toward semantic-level uncertainty estimation, notably Semantic Entropy (SE) [Kuhn et al., 2023], which aggregates uncertainty by clustering semantically equivalent responses using Natural Language Inference. However, SE relies on hard cluster assignments, forcing each response into a single cluster and ignoring overlapping semantic regions, which can yield unstable uncertainty estimates. To improve reliability, conformalized approaches [Kaur et al., 2024] employ dynamic clustering with finite-sample guarantees, while the Conformalized Abstention Policy [Tayebati et al., 2025] integrates reinforcement learning to learn instancespecific risk thresholds. Despite their guarantees, these methods treat uncertainty as a scalar signal and remain insensitive to cluster brittleness. As a result, they are vulnerable to the wrong-consensus trap, in which lexically consistent but factually incorrect response clusters satisfy conformal thresholds, masking underlying semantic error.

Proposal. To address these limitations, we propose Adaptive Conformal Semantic Entropy (ACSE), a model-independent method that quantifies uncertainty in pretrained LLMs based on dispersion in meaning. Rather than relying on token-level signals, ACSE estimates prompt-level uncertainty in a continuous semantic vector space, capturing dispersion in response meaning. For a finite set of representative prompts (e.g.,

1,000 calibration prompts), we sample multiple responses per prompt, embed each response using a sentence encoder whose cosine similarity reflects semantic proximity [Reimers and Gurevych, 2019], and cluster the embeddings into coherent semantic groups. We then estimate probability mass for each group via soft assignments and derive a base uncertainty score from the normalized entropy of the resulting distribution. This base semantic uncertainty reflects the degree of meaning dispersion: higher dispersion-corresponding to multiple semantically distinct responses-indicates greater uncertainty, whereas semantically consistent responses imply lower uncertainty [Yadkori et al., 2024]. We subsequently adjust the uncertainty score when cluster structure indicates brittleness, such as weak support for the dominant cluster, high internal diversity, or other risk patterns that empirically correlate with errors under distribution shift. Finally, we calibrate the adjusted uncertainty score using conformal prediction on heldout calibration prompts [Vovk et al., 2005] to learn a cutoff that serves two purposes: (1) making prompt-level answer/abstain decisions and (2) generating response-level prediction sets to support user decision-making. Figure 1 provides an overview of the approach, which is illustrated by a toy example.

Toy Example. Consider a user querying a pretrained geographical LLM with the prompt What is the capital of Australia?. The model returns Sydney with probability 70% (high confidence) despite its semantic incorrectness, and Canberra with probability 30% (low confidence). Selecting the most likely sequence yields a wrong consensus, a common failure mode in open-ended generation. Assume the user has calibrated the model on a held-out set of geographical prompts and identified 0.58 as a conformity cutoff corresponding to 90% confidence under the assumption that new prompts are drawn from the same distribution [Vovk et al., 2005]. For a new prompt, ACSE samples 10 responses using fixed decoding settings: 9 contain Sydney and 1 contains Canberra. These responses are embedded, clustered by meaning, and assigned soft semantic memberships via normalized cosine similarity to cluster centroids. Since assignments are probabilistic, clusters are non-disjoint. Averaging memberships across responses yields mean cluster weights of 0.87 for Sydney and 0.13 for Canberra. The resulting base semantic uncertainty score, given by the normalized entropy of these weights, is

0.55. Because this score falls below the abstention threshold (0.55 < 0.58), the model would return the dominant Sydney cluster. However, semantic entropy alone can be overly optimistic when clusters are internally brittle, e.g., when cluster membership is unstable or weakly representative. Our approach accounts for such brittleness by adjusting the uncertainty score. In this example, incorporating cluster quality factors (Section 3.1) increases the score to 0.62. Since 0.62 > 0.58, the model abstains from returning the incorrect answer. While sampling and post-processing incur additional cost, this overhead is justified in safety-critical applications such as using LLMs for mental health support.

Contributions. We make the following contributions. First, we propose ACSE, a method for estimating entropy-based uncertainty at the semantic level for a given prompt. Second, we introduce an adjusted uncertainty score that leverages cluster robustness features to adaptively inflate semantic uncertainty, explicitly penalizing ambiguous response semantics and mitigating hallucinations. Third, we incorporate a post-hoc conformal calibration phase on held-out prompts to learn a cutoff over adjusted uncertainty estimates aligned with response correctness and with formal guarantee, yielding decision rules in both prompt-level and response level. Finally, we demonstrate through extensive experiments on open-domain question answering benchmarks that our adaptive semantic-level uncertainty modeling achieves superior discrimination and calibration compared to state-of-the-art baselines. The extended version of this paper can be found at https://arxiv.org/abs/2605.04295, which includes additional experimental evaluations, the computation overhead analysis, and proofs of the theorems.

2 Prompt-Level Uncertainty Estimation

ACSE calibrates a pretrained LLM to a new domain using a finite set of labeled calibration prompts Dcal of size M. Calibration begins by sampling multiple responses per prompt to assess the consistency of the model's generated meanings.

2.1 Response Sample Generation

Let V denote the token vocabulary. A generated response is a finite token sequence yi = ⟨yi,1, . . . , yi,Ti ⟩ terminated by an EOS token in Ti decoding steps. For a fixed input prompt x ∈ X, we draw a set of n independent responses Y(x) = {y1, . . . , yn} from a language model with parameters θ. At each decoding step t, the model defines a next-token probability distribution over tokens v ∈ V as,

where ∑v∈V πi,t(v) = 1 and yi,t denotes the t-th token of response yi. To balance diversity and semantic coherence, we adopt nucleus (top-η) sampling [Holtzman et al., 2020], which dynamically truncates the distribution at each step. Specifically, the nucleus Γη(x, i, t) is the smallest subset of tokens whose cumulative probability exceeds a threshold η ∈ R(0,1] ,

The truncated distribution is obtained by renormalizing over Γη ,

Tokens are sampled as yi,t ∼ π(η) i,t (·) and appended until EOS or a length limit is reached. Repeating this procedure yields the response set Y(x). The variability of Y(x) provides an empirical proxy for model confidence. When the model is confident, stochastic decoding produces lexically diverse but semantically equivalent responses; when uncertain, samples diverge into multiple, often conflicting, meanings. The sample size n controls a trade-off between computational cost and resolution: larger n captures finer-grained semantic variation, while smaller n suffices when responses concentrate on a single meaning. In the next step, we quantify this dispersion by embedding Y(x) into a semantic space where cosine similarity reflects semantic proximity.

2.2 Semantic Embedding

We represent each sampled response by its underlying meaning rather than its surface form, so that paraphrases are embedded nearby while semantically distinct answers are well separated. This representation enables reliable grouping of responses into semantic clusters for downstream uncertainty estimation. Each generated response yi ∈ Y(x) is mapped into a continuous semantic space using a pre-trained sentence encoder f : Y → Rd , producing embeddings E = {vi = f(yi) ∈ Rd ∣∣ yi ∈ Y(x)}. We ℓ2-normalize these embeddings such that ∥vi∥2 = 1 and measure similarity using cosine geometry [Cer et al., 2018; Reimers and Gurevych, 2019; Gao et al., 2021]. This choice reflects semantic relatedness across lexical and syntactic variation, provides bounded and scale-invariant similarity scores, and yields more stable neighborhoods in high-dimensional spaces than Euclidean distance. The resulting embeddings capture the latent semantic structure of model outputs and form the basis for measuring dispersion in meaning. The effectiveness of this representation depends on selecting a sentence encoder that reliably separates semantic similarity from surface variation.

Using the embeddings E, we define a binary error function e : X × Y → {0, 1} that assigns error labels to prompt-response pairs where e(x, yi) = 1{yi is incorrect}. Labels are determined automatically via semantic matching [Cer et al., 2018; Reimers and Gurevych, 2019]. Given a reference response ˆ y for prompt x and a cosine similarity threshold τcos ∈ (0, 1), a response yi is labeled correct if cos(vyi , vˆ y) ≥ τcos. In the following, we leverage semantic clusters of E to derive soft assignments that preserve the semantic ranking induced by the encoder.

2.3 Soft Clustering of Semantic Embedding

To transform the unstructured distribution of response embeddings into a meaningful uncertainty signal, we aggregate semantically proximal embeddings into clusters that represent distinct meanings. This serves two purposes: (1) collapsing lexically diverse but semantically equivalent responses into a single equivalence class, and (2) exposing higher-level semantic structure whose relative prevalence reflects model uncertainty. We apply Hierarchical Agglomerative Clustering (HAC) [Murtagh and Contreras, 2017] to the embedding set E under cosine geometry. Using unit-normalized embeddings, cosine dissimilarity reduces to

HAC constructs a dendrogram by iteratively merging clusters with minimum average pairwise dissimilarity (average linkage), balancing sensitivity to local neighborhoods and global coherence. Cutting the dendrogram at a threshold ϵ ∈ R(0,1) yields a set of K semantically coherent clusters

Lower ϵ enforces strict semantic agreement and higher ϵ allows broader conceptual grouping. This adaptive thresholding aligns cluster granularity with the intrinsic density of the embedding space.

To avoid overstating certainty near cluster boundaries, we adopt soft cluster assignments. Each cluster Ck is represented by a centroid ck = ∑v∈Ck v/|Ck|. For a response embedding vi, we compute its similarity to each centroid as,

which is a parameter-free, monotonic transformation that preserves angular geometry without temperature tuning. Then, normalizing across clusters yields soft assignments

where sik quantifies the degree to which response yi supports semantic cluster Ck. Assignments concentrate when a response aligns strongly with one cluster, and disperse when it lies between competing meanings, preserving signals of semantic ambiguity. Finally, we aggregate soft assignments across the n sampled responses for prompt x to obtain a prompt-level distribution over meanings P(Ck) = n-1 ∑ni=1 sik where ∑Kk=1 P(Ck) = 1. We define the Semantic Entropy (SE) as,

and normalize it to ensure comparability across prompts with different K,

The normalized score u(x) ∈ R[0,1] quantifies semantic dispersion: low values indicate strong agreement on a single meaning, while high values reflect uncertainty arising from competing semantic interpretations.

3 Conformalized Adaptive Semantic Entropy

Conformal Prediction. Conformal Prediction (CP) is a distribution-free framework for calibrating set-valued predictions with finite-sample guarantees. Given a calibration dataset Dcal = {(xi, yi)}Mi=1 and a test point (xM+1, yM+1) drawn from the same distribution, CP ensures

where α ∈ R(0,1) is the miscoverage level and C is the prediction set [Vovk et al., 2005]. CP relies on a non-conformity score S : X ×Y → R+ that measures how poorly a candidate output conforms to the calibration data. For confidence level 1-α, the prediction set is

where τα is the empirical (1-α)-quantile of calibration scores. This guarantee is model- and distribution-agnostic with exchangeability assumption, i.e., the joint distribution is invariant under permutations of the calibration samples. We refer to [Angelopoulos and Bates, 2023] for a comprehensive treatment. We adapt CP calibration to LLMs for two complementary purposes: (1) to derive a prompt-level decision rule that determines whether to answer or abstain, and (2) to construct a response-level prediction set over sampled outputs that supports user decision-making by identifying responses satisfying conformal coverage. Achieving these goals hinges on a critical component of CP: the fidelity of the non-conformity score. Since the quality of conformal prediction sets depends directly on this score, we introduce an adjusted uncertainty score tailored to LLMs (Section 3.1).

3.1 Adjusted Semantic Uncertainty

Although the base semantic uncertainty u(x) in (9) captures dispersion across meanings, it does not account for structural properties of semantic clusters that correlate with prediction difficulty. In particular, u(x) can underestimate uncertainty in low-sample regimes, semantically heterogeneous clusters with weak separation, and sparsely supported clusters that are vulnerable to distribution shift. To address this, we amplify u(x)

using the odds of uncertainty, O(u, x) = u(x)/(1 - u(x)), which represents the uncertainty-confidence ratio. To capture cluster brittleness, we define a prompt-level inflation factor λ : X → R[1,λmax] to scale these odds such that O(ˆu, x) = λ(x) · O(u, x), leading to the adjusted uncertainty

This bounded, monotone, and order-preserving transform ensures ˆu(x) ≥ u(x) with equality iff λ(x) = 1 or u(x) ∈ {0, 1}. Inflation increases with both u and λ, peaking at intermediate u(x) values to prioritize sensitive borderline decisions. Unlike response-level error labels e(x, y), the inflation factor λ(x) operates strictly at the prompt level by aggregating properties of the full response set Y(x).

To make λ(x) interpretable and data-driven, we synthesize five normalized, prompt-level robustness features from the embedding geometry using ℓ2-normalized response embeddings {vi}ni=1 , soft assignments sik of response yi to cluster Ck, and the agglomerative cluster set J with unit-norm centroids {ck}Kk=1 , where k⋆ = arg maxk P(Ck) denotes the dominant cluster and i⋆ = argmaxi sik⋆ denotes the response most representative of the dominant cluster Ck∗ .

1. Semantic Entropy. The normalized semantic entropy u(x) measures multimodality across clusters and serves as the primary ambiguity signal. Larger u(x) directly increases λ(x), inflating uncertainty when probability mass is spread across competing meanings.
2. Centroid Distance. We assess if the dominant response is geometrically supported by its cluster using the centroid similarity ai⋆ k⋆ in (6). The distance ˜a(x) = 1-ai⋆ k⋆ is small when the representative response lies near the cluster center and large when it is atypical, indicating structural instability. We set ˜a(x) ∝ λ(x) so that uncertainty increases whenever the dominant prediction lacks sufficient geometric support.
3. Dominant Cluster Dispersion. To quantify internal coherence, we define the intra-cluster dispersion

Low values indicate tight semantic agreement, while high values reflect weak internal consistency; dk⋆(x) increases λ(x) proportionally; thus, uncertainty is inflated whenever the dominant cluster lacks internal coherence.

1. Dominant Cluster Size. To penalize fragile consensus supported by few samples, we define the sparsity feature

where κ = medianx∈Dcal ( maxk |Ck|) is a dataset-specific scaling constant computed once on Dcal. Smaller dominant clusters yield larger gk⋆(x) directly proportional to λ(x) to inflate uncertainty under sparse semantic support.

(5) Margin to Threshold (Overconfidence Suppression).

To suppress unwarranted confidence in the low-uncertainty regime, we introduce the margin-to-threshold feature m(x). Using the calibration set Dcal, we define a label-free reference threshold τref = Quantileγ({u(x) : x ∈ Dcal}) where γ ≥ 0.5 is user specified. This threshold is fixed after calibration and reused during deployment to compute λ(x), thereby preserving the validity of subsequent conformal calibration. We define the normalized margin

which is large when u(x) ≪ τref , indicating extreme, and potentially miscalibrated confidence. We incorporate m(x) into λ(x) so that overconfident prompts receive additional inflation, nudging borderline cases toward abstention, while the max operator ensures the penalty vanishes whenever u(x) ≥ τref .

All features are computed identically during calibration and inference to ensure representational consistency. The constants κ and τref are derived once from the unlabeled calibration set Dcal and frozen thereafter, preserving conformal validity. To design our monotone inflation function λ, we introduce a nonnegative weight vector w = ⟨wu, w˜a, wd, wg, wm⟩ ∈ [0, 1]5 that encodes the relative contribution of each structural semantic feature of the feature set F = {u, ˜a, dk⋆, gk⋆,m} where ∑l∈F wl = 1. These assigned weights are a design choice and not learned parameters, accounting for prioritization over cluster brittleness features. We aggregate the feature set F by weighted average, into the normalized composite brittleness metric B(x) = ∑l∈F wl l(x) ∈ R[0,1] where larger B(x) indicates a less reliable prompt due to cluster brittleness. We then define the inflation mapping λ(x) = 2/(2 - B(x)) ∈ R[1,2] , which satisfies λmin(x) = 1 for benign prompts (B(x) = 0) and approaches the bounded maximum λmax(x) = 2 under maximal structural brittleness, while remaining strictly increasing and convex in B(x). This yields a smooth convex map from [0, 1] to [1, 2] ensuring boundedness, monotonicity, and order preservation while amplifying high-brittleness cases. Since ∂λ/∂l > 0 for all l ∈ F, any increase in cluster brittleness inflates ˆu(x) via (12). As the entire inflation procedure is label-free, conformal calibration and inference remain valid in the transformed ˆu-space, preserving finite-sample guarantees.

3.2 Prompt Acceptance and Response Set

Prompt-Level Decision Rule. For each prompt x, let define the returned response ˜ y = yi∗ as the most representative member of dominant cluster k⋆ , i.e., cluster with largest total assignment mass. We use the inflated uncertainty ˆu(x) ∈ R[0,1] as a prompt-level score so that small ˆu(x) indicates robust semantic agreement after inflation, while large ˆu(x) indicates cluster brittleness. Since CP requires one label per prompt, we define a prompt-level error label by evaluating the returned response E(x) = e(x, ˜ y) ∈ {0, 1} where E(x) = 0 indicates that the returned response is correct. Using I0 = {ˆu(x) : x ∈ Dcal ∧ E(x) = 0} as calibration prompts whose returned response is correct, we calibrate a prompt acceptance threshold ˆ τ = Quantile1-α(I0 ) which is (1 - α)-quantile of scores ˆu(.) in the multiset I0 (if I0 = ∅, we abstain on all prompts). At inference, to answer a prompt x, we

This calibration is appropriate since it aligns the prompt-level score ˆu(x) with the prompt-level LLM outcome, i.e., correctness of the actually returned response ˜ y.

Theorem 1 (Conformal coverage for prompt acceptance). Assume ˜ y and ˆu(·) are computed by the same procedure for calibration and test prompts under exchangeability of Dcal and test prompt xnew. With the (1 - α)-quantile ˆ τ defined on ˆu-space of prompts with correct returned responses, we have

Response-Level Prediction Sets. To construct CPconsistent prediction sets over sampled responses, we define a response-level non-conformity score that combines prompt uncertainty with response atypicality. Using the clustering outputs, we first define a response conformity measure

which is large when response yi is both strongly assigned to its best-matching semantic cluster and that cluster is well supported across samples. We then define the response-level non-conformity score (smaller is better) as,

which increases with prompt-level uncertainty ˆu(x) and with response atypicality. Thus, a response attains a low score only when it arises from a low-uncertainty prompt and is semantically representative within a well-supported cluster. For calibration prompt-response pairs, we compute S(xj , yji) in (19) and form the multiset S0 = {S(xj, yji) : eji = 0} of scores corresponding to correct responses. The (1 - α)- quantile threshold is then defined as ˆ q = Quantile1-α(S0). For a test prompt x with sampled responses Y(x) = {yi}ni=1 , the response-level prediction set is

where the size of Cα(x) provides a direct measure of responselevel uncertainty. If a single output is required, we return the most representative response ˜ y ∈ Cα(x).

Theorem 2 (Conformal coverage for prediction response sets). Assume S(x, y) is computed identically on calibration and test triples (x, y, e) under exchangeability with the same response sampling procedure. Considering Cα defined in (20), for a randomly drawn test triple (xnew, ynew, enew),

The proofs of the theorems and computational overhead analysis of our approach are reported in Appendices A and B of the extended version, respectively.

4 Experimental Evaluations

We evaluate the performance of ACSE using discriminative and conformal analyses, benchmarking its ability to identify hallucinations (incorrect responses) reliably through adaptive uncertainty inflation against both uncalibrated and calibrated baselines. Appendix C of the extended version describes the experimental setup and evaluation metrics.

Implementation Details. For each prompt, we generate n = 10 responses via nucleus sampling (η = 0.9) with temperature 0.3 to balance diversity and coherence and embed them using the sentence encoder all-MiniLM-L6. We perform HAC clustering with average linkage and cosine distance threshold ϵ = 0.35. We set λ to equally weigh all brittleness features as w = ⟨ 1 5 , 15 , 15 , 15 , 15 ⟩ to avoid any prioritization, and set the overconfidence penalty threshold τref to the 75th percentile of the calibration base-uncertainty distribution. Each dataset uses 2000 samples split into disjoint calibration set (60%) and test set (40%). For Size-Stratified Coverage Violation (SSCV) evaluation, we define prediction set size strata as [1-2, 3-5, 6-7, 8-10]. The source codes are available at https:// github.com/ tailabTMU/ ACSE.

Datasets. We evaluate on five benchmarks spanning opendomain retrieval, conversational QA, hallucination detection, and multi-domain reasoning: TriviaQA [Joshi et al., 2017], CoQA [Reddy et al., 2019], Natural Questions (NQ) [Kwiatkowski et al., 2019], TruthfulQA [Lin et al., 2022], and a stratified subset of MMLU [Wang et al., 2024].

LLM Models. We use Mistral-7B-Instruct [Jiang et al., 2024] which combines Grouped-Query Attention (GQA) and Sliding Window Attention (SWA) for strong performance with efficient inference. To assess model-agnosticism, we additionally evaluate Llama-2-7B-Chat [Touvron et al., 2023], Falcon7B-Instruct (Multi-Query Attention; MQA), and Qwen-7BChat [Bai et al., 2023].

Baselines. We compare ACSE against six uncertainty methods spanning lexical, semantic, geometric, and calibrated approaches: Token Entropy (TE) [Vaswani et al., 2017], P(True) [Kadavath et al., 2022], Semantic Uncertainty (SU) [Kuhn et al., 2023], and EigV [Lin et al., 2024]. We also include two CP-based calibrated frameworks: DDCRPCP [Kaur et al., 2024] and Conformal Abstention Policy (CAP) [Tayebati et al., 2025]; we emphasize comparisons among DDCRP-CP, CAP, and ACSE since they share a CP calibration protocol, enabling standardized evaluation.

4.1 Results and Interpretations

We conduct an extensive empirical evaluation of ACSE to assess whether it (1) improves hallucination detection relative to state-of-the-art baselines, (2) strictly enforces user-specified coverage via conformal mechanisms along with the selective risk while maintaining high acceptance rates, and (3) improves uncertainty quantification on hallucinations against baselines.

Hallucination Detection Performance. This evaluation assesses the ACSE's capability to distinguish between factually correct response generations and hallucinations across various datasets and LLMs. The results in Tables 1 and 2 demonstrate that ACSE achieves better discriminative performance across all benchmarks, e.g., on TriviaQA, ACSE increases AUROC to 0.88 from CAP's 0.80. This improvement is particularly notable on high-efficiency models such as Falcon-7B where ACSE reduces 0.48 FPR@95 observed by CAP to 0.31, a 35.4% relative decrease in accepted hallucinations. Low standard deviations (0.01 to 0.02) across five independent runs confirm the statistical significance of the results. We report ablation studies and additional results on a text summarization dataset, recent 8B LLMs and baselines, together with base SE comparisons, probabilistic calibration results, and statistical significance tests in Appendix D of the extended version.

Conformal Guarantees and Selective Risk. We assess ACSE reliability by varying α using both response-level and prompt-level metrics reported in Table 3, to enforce userspecified guarantees through conformal decision rules against baselines. Statistical reliability is reported through Responselevel Coverage (R-Cov) which measures the inclusion rate of correct generated responses in the prediction sets, and Promptlevel Coverage (P-Cov) which evaluates the fraction of accepted prompts with correct returned responses. The analyses in Figure 2a show that ACSE guarantees superior R-Cov across all α levels. Similarly, Figure 2b demonstrates that ACSE consistently satisfies the theoretical prompt coverage guarantee, whereas baselines frequently exhibit undercoverage violations. ACSE also maintains low Average Prediction Set Size (APS) of 1.07 compared to 1.32 APS of CAP at α = 0.10. Additionally, ACSE achieves the lowest SSCV scores across all settings, e.g., 0.030 vs. 0.042 for CAP at α = 0.10, indicating superior calibration stability across varying prediction set sizes. ACSE also consistently maximizes the Acceptance Rate (Acc.) across benchmarks, reaching 75.8% at α = 0.10, a substantial improvement over the runner-up CAP at 65.4%. Furthermore, ACSE strictly guarantees the selective risk as the conditional error among accepted prompts that hallucinate, which is restricted by α as shown in Figure 2c. While the SU baseline suffers from overconfidence and risk violations, ACSE maintains a selective risk of 0.178 at low APS. ACSE also avoids the conservativeness of DDCRP-CP, which yields marginally lower risk but produces significantly larger prediction sets (1.45 vs. 1.07), trading precision for safety.

Uncertainty Quantification Analysis. This experiment evaluates the discriminative ability of our adjusted uncertainty score (ˆu) to separate correct responses from hallucinations. Figure 3 illustrates this separation across four quadrants against each baseline. The top-left quadrant denotes the ideal abstention region for hallucinations where they exhibit both low baseline confidence and high ACSE uncertainty; conversely, the bottom-right quadrant denotes the ideal acceptance region for the correct responses. ACSE consistently assigns high uncertainty to hallucinations and low uncertainty to correct responses whereas the baselines exhibit varying ranges of confidence while hallucinating. The top-right quadrant highlights baseline failures compared to ACSE, in which hallucinations are incorrectly assigned high confidence but are correctly flagged by ACSE with high uncertainty.

5 Conclusion

We introduced ACSE to mitigate LLM overconfidence by quantifying uncertainty through semantic dispersion. Rather than treating uncertainty as a one-dimensional signal, ACSE employs an adaptive inflation mechanism to penalize semantic brittleness, ensuring reliable estimates. Evaluations across diverse benchmarks demonstrate that ACSE significantly improves hallucination detection, discriminative performance, and coverage guarantees. ACSE offers a robust solution for the reliable deployment of LLMs in safety-critical domains. Future work will explore adaptive sampling strategies to reduce computational overhead and extend the framework to conformal risk control for fine-grained quality calibration.

Acknowledgments

This research was undertaken thanks in part to funding from the Canada First Research Excellence Fund at Toronto Metropolitan University and Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery grants (#348100).