2503.15850

# Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey **Authors**: Xiaoou Liu, Tiejin Chen, Longchao Da, Chacha Chen, Zhen Lin, Hua Wei > xiaoouli@asu.eduArizona State UniversityTempeArizonaUSA > tchen169@asu.eduArizona State UniversityTempeArizonaUSA > longchao@asu.eduArizona State UniversityTempeArizonaUSA > chacha@uchicago.eduUniversity of ChicagoChicagoILUSA > zhenlin4@illinois.eduUniversity of Illinois Urbana-ChampaignChampaignILUSA > hua.wei@asu.edu0000-0002-3735-1635Arizona State UniversityTempeArizonaUSA (2025; 11 March 2025; 6 May 2025) Abstract. Uncertainty quantification (UQ) enhances the reliability of Large Language Models (LLMs) by estimating confidence in outputs, enabling risk mitigation and selective prediction. However, traditional UQ methods struggle with LLMs due to computational constraints and decoding inconsistencies. Moreover, LLMs introduce unique uncertainty sources, such as input ambiguity, reasoning path divergence, and decoding stochasticity, that extend beyond classical aleatoric and epistemic uncertainty. To address this, we introduce a new taxonomy that categorizes UQ methods based on computational efficiency and uncertainty dimensions, including input, reasoning, parameter, and prediction uncertainty. We evaluate existing techniques, summarize existing benchmarks and metrics for UQ, assess their real-world applicability, and identify open challenges, emphasizing the need for scalable, interpretable, and robust UQ approaches to enhance LLM reliability. Uncertainty Quantification; Large Language Models copyright: acmlicensed journalyear: 2025 copyright: acmlicensed conference: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2; August 3–7, 2025; Toronto, ON, Canada booktitle: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (KDD ’25), August 3–7, 2025, Toronto, ON, Canada doi: 10.1145/3711896.3736569 isbn: 979-8-4007-1454-2/2025/08 ccs: Computing methodologies Machine learning ccs: Computing methodologies Natural language processing 1. Introduction Large Language Models (LLMs) like GPT-4 (Achiam et al., 2023) have achieved remarkable capabilities in text generation, reasoning, and decision-making, driving their adoption in high-stakes domains such as healthcare diagnostics (Qiu et al., 2024; Da et al., 2024d), legal analysis (Cheong et al., 2024; Li et al., 2024b), and transportation systems (Da et al., 2024b; Lai et al., 2025; Xing et al., 2025). However, their reliability remains a critical concern: LLMs often produce plausible but incorrect or inconsistent outputs, with studies showing that over 30% of answers in medical QA tasks contain factual errors (Jin et al., 2021). In sensitive applications, these limitations pose risks ranging from misinformation to life-threatening misdiagnoses, underscoring the urgent need for robust reliability frameworks. Uncertainty quantification (UQ) emerges as an important mechanism to enhance LLM reliability by explicitly modeling confidence in model outputs. By estimating uncertainty, users can identify low-confidence predictions for human verification, prioritize high-certainty responses, and mitigate risks like overconfidence in hallucinations (Liu et al., 2024a). For instance, in clinical settings, uncertainty-aware LLMs could flag uncertain diagnoses for specialist review, reducing diagnostic errors by up to 41% (Sen et al., 2024). This capability is particularly critical as LLMs’ transition from experimental tools to production systems requiring accountability. Traditional UQ methods face significant hurdles when applied to LLMs. Bayesian approaches like Monte Carlo dropout (Gal and Ghahramani, 2016) are computationally prohibitive for trillion-parameter models and natural language generation (NLG) tasks, while ensemble methods struggle with consistency across diverse decoding strategies (Lu et al., 2024). Furthermore, LLMs introduce unique uncertainty sources, such as input ambiguity (Chen et al., 2025a; Guo et al., 2021), reasoning path divergence, and decoding stochasticity that transcend classical aleatoric and epistemic categorizations (Hüllermeier and Waegeman, 2021). The complexity of LLMs, characterized by sequence generation over vast parameter spaces and reliance on massive datasets, exacerbates uncertainty challenges. This complexity, coupled with the critical need for reliable outputs in high-stakes applications, positions UQ for LLMs as a compelling yet underexplored research frontier. Targeting the unique challenges of UQ in LLMs, this survey firstly introduces a novel taxonomy for LLM UQ, categorizing methods along two axes: (1) computational efficiency (e.g., single-pass vs. sampling-based techniques) and (2) uncertainty dimensions (input, reasoning, parametric, predictive). This framework addresses three gaps in prior works: First, it decouples uncertainty sources unique to LLMs from traditional ML contexts. Second, it evaluates methods through the lens of different dimensions of the responses from LLM: input uncertainty, reasoning uncertainty, parameter uncertainty, and prediction uncertainty. Each of these dimensions may involve aleatoric uncertainty, epistemic uncertainty, or a mixture of both. Third, it identifies understudied areas like reasoning uncertainty, challenges, and possible future directions. Connection to Existing Surveys: Prior surveys (Shorinwa et al., 2024; Huang et al., 2024b, 2025a) focus on hallucination detection or retrofitting classical UQ taxonomies, neglecting LLM-specific challenges like prompt-driven input uncertainty. Our work uniquely addresses the interplay between model scale, open-ended generation, and uncertainty dynamics, which are critical for modern LLMs but overlooked in earlier frameworks. The remainder of this survey is structured as follows: Section 2 characterizes LLM uncertainty dimensions and differentiates confidence from uncertainty. Section 3 evaluates UQ methods using our taxonomy. Section 4 introduces the evaluation of UQ methods for LLM, including benchmarks and metrics. Sections 5 and 6 introduce the applications of UQ in different domains with LLMs and identify open challenges and future directions. 2. Perliminaries 2.1. Sources of Uncertainty in LLMs 2.1.1. Aleatoric vs. Epistemic Uncertainty For UQ on traditional machine learning tasks such as classification or regression (Young et al., 2025), there are mainly two types of uncertainty (Der Kiureghian and Ditlevsen, 2009; Ye et al., 2024): aleatoric uncertainty, which models the uncertainty from noise in the dataset, and epistemic uncertainty, which arises from the model’s lack of knowledge about the underlying data distribution. Aleatoric uncertainty in LLMs primarily stems from data sources used to train LLMs, which contain inconsistencies, biases, and contradicting information. Additionally, ambiguity in natural language also contributes to aleatoric uncertainty, as different interpretations of the same prompt can lead to multiple plausible responses. On the other hand, when encountering unfamiliar topics, LLMs may exhibit high epistemic uncertainty, often manifesting as hallucinations or overconfident yet incorrect statements. Epistemic uncertainty can be reduced through domain-specific fine-tuning or retrieval-augmented generation techniques that allow the model to access external knowledge sources. 2.1.2. Uncertainty with Different Dimensions While the uncertainty for LLMs can also be classified through aleatoric and epistemic uncertainty, these two categories alone are insufficient to fully capture the complexities of uncertainty in LLMs. In particular, LLMs exhibit uncertainty not only due to training data limitations but also due to input variability and decoding mechanisms. Therefore, in the following, we formulate four dimensions of uncertainty, each of which may involve aleatoric uncertainty, epistemic uncertainty, or a combination of both. The technical methods on how to quantify these uncertainties will be discussed later in Section 3. $\bullet$ Input Uncertainty (Aleatoric Uncertainty): Input uncertainty arises when a prompt is ambiguous or underspecified, making it impossible for an LLM to generate a single definitive response. This is inherently aleatoric, as even a “perfect model” cannot resolve the ambiguity. For instance, “What is the capital of this country?” lacks sufficient context, leading to unpredictable outputs. Similarly, “Summarize this document” may yield different responses depending on different expected details. $\bullet$ Reasoning Uncertainty (Mixed Uncertainty): Reasoning uncertainty occurs when an LLM derives answers through multi-step reasoning (Mondorf and Plank, 2024) or retrieval (Li et al., 2024c), where the uncertainty of each step can lead to ambiguous or incorrect results. This uncertainty is aleatoric when the problem itself is ambiguous and epistemic when the model cannot offer robust reasoning. $\bullet$ Parameter Uncertainty (Epistemic Uncertainty): Parameter uncertainty stems from training data gaps, where the model has either never seen relevant information or has learned an incorrect representation. Unlike aleatoric uncertainty, epistemic uncertainty can be reduced by improving the model’s knowledge base. Bayesian methods (Gal and Ghahramani, 2016), deep ensembles (Lakshminarayanan et al., 2017), and uncertainty-aware training (Mukherjee and Awadallah, 2020) can help quantify and mitigate this type of uncertainty. $\bullet$ Prediction Uncertainty (Mixed Uncertainty): Prediction uncertainty refers to variability in generated outputs across different sampling runs, influenced by both aleatoric and epistemic sources. For example, when asked “What are the side effects of a new experimental drug?”, the model’s responses might vary significantly across different sampling runs, especially if no reliable data is available in its training set. A high-variance output distribution in such scenarios suggests that the model is both aware of multiple possible answers, reflecting aleatoric uncertainty, and uncertain due to incomplete knowledge, highlighting epistemic uncertainty. 2.2. Uncertainty and Confidence in LLMs 2.2.1. Classical Confidence Estimation UQ and confidence estimation are closely related yet distinct concepts. In traditional machine learning, uncertainty is a property of the model’s predictive distribution, capturing the degree of variability or unpredictability given a particular input. In contrast, confidence reflects the model’s belief in the correctness of a particular prediction. If we follow the definition in classification tasks, the confidence measure would be the predicted probability $\hat{p}(Y=y|x)$ given input $x$ (an uncertainty measure which does not depend on the particular prediction $y$ could be entropy, taking the form of $\sum_{y}-\hat{p}(Y=y|x)\log{\hat{p}(Y=y|x)}$ ). Table 1 shows a similar notation in a question-answering (QA) task in Natural Language Generation (NLG). The corresponding confidence score in NLG tasks for an auto-regressive language model would be the joint probability for the generated sequence: $$ \displaystyle C(\mathbf{x},\mathbf{s})=\hat{p}(\mathbf{s}|\mathbf{x}) \displaystyle=\prod_{i}\hat{p}(s_{i}|\mathbf{s}_{<i},\mathbf{x}). \tag{1} $$ The log of Eq. 1 is sometimes referred to as sequence likelihood (Zhao et al., 2023a). In general, an uncertainty estimate in existing literature usually takes the form of $U(\mathbf{x})$ , while confidence estimates are usually expressed as $C(\mathbf{x},\mathbf{s})$ . Note that, unlike classification tasks, not all NLG applications have the notion of a “correct” answer (e.g., summarization). Thus, while for the ease of writing we use the term correctness throughout this section, it should really be interpreted as the gold-label for the particular application. Note also that in most cases, the correct answer is not unique, and thus such gold-label typically takes the form of a “correctness function” that decides whether a particular generation $\mathbf{s}$ is good or not. We will denote such a function as $f(\mathbf{s}|\mathbf{x})$ . | $\mathbf{x}$ $\mathbf{s}$ $w_{i}$ | The question that LLMs answer Generation from LLMs i-th token in the generation $\mathbf{s}$ | | --- | --- | | $\mathcal{D}$ | Corpus of LLMs | | $U(\mathbf{x})$ | Uncertainty of question $\mathbf{x}$ | | $C(\mathbf{x},\mathbf{s})$ | Confidence of generation $\mathbf{s}$ given $\mathbf{x}$ | | $H(\mathbf{s})$ | Entropy of generation $\mathbf{s}$ | Table 1. Notations used in this paper for an exemplary QA task. 2.2.2. Confidence Improvement There are usually two dimensions along which researchers improve confidence estimates in NLG, which is unsurprisingly largely influenced by confidence scoring literature from classification (Jiang et al., 2018), especially binary classification. We refer to them as ranking performance and calibration: $\bullet$ Ranking performance refers to the discriminative power of the confidence measure on the correctness. Like in classification, LLM confidence is often evaluated by its ability to separate correct and incorrect answers, thus typically measured by evaluation metrics like AUROC (Kadavath et al., 2022) or AUARC (Lin et al., 2024) as detailed in Section 4. $\bullet$ Calibration refers to closing the gap between the confidence score and the expected correctness conditioned on confidence score. It has a long history preceding even modern machine learning (Murphy and Winkler, 1977), but bears slightly different meanings in NLP. In general, we could define a perfectly calibrated confidence measure to achieve: $∀ c,\mathbb{E}[f(\mathbf{s}|\mathbf{x})|C(\mathbf{x},\mathbf{s})=c]=c,$ where the expectation is taken over the joint distribution of $\mathbf{x}$ and generation $\mathbf{s}$ . A lot of papers focus on evaluating the calibration quality of specific language models (LMs) and tasks (Wang et al., 2020; Kumar and Sarawagi, 2019). Evaluation typically relies on variants of Expected Calibration Error (ECE) (Tian et al., 2023; Kumar and Sarawagi, 2019). Oftentimes confidence scores from classification could be directly applied (Stengel-Eskin and Van Durme, 2023) in order to evaluate whether an LM is over- or under-confident, especially for de facto classification tasks like sentiment analysis or multiple-choice QA. 2.2.3. Confidence Estimation for LLMs Confidence estimation in large language models (LLMs) refers to the task of quantifying how certain a model is about a specific generated output. In this subsection, we review three major families of approaches to confidence estimation in LLMs: $\bullet$ UQ methods with Confidence Estimation. As uncertainty and confidence are often intertwined, many approaches used in UQ have their counterpart in confidence estimation. For example, for black-box settings where the parameters of LLMs are unavailable, (Lin et al., 2023; Zhang et al., 2024b) computes a similarity matrix of sampled responses and derives confidence estimates for each generation via its degree or distance derived from the graph Laplacian, before using these scores to compute uncertainty. For white-box settings where model parameters are available, researchers mostly compute the confidence from the output logits, either through normalizing Eq. 1 with the length of $\mathbf{s}$ (Malinin and Gales, 2021), replacing the logit-sum or mean with weighted sum by attention values (Lin et al., 2024) or by importance inferred from natural language inference (NLI) models (Duan et al., 2024). Such variants of sequence likelihood could then be fed for (entropy-style) uncertainty computation (Kuhn et al., 2023; Lin et al., 2024). $\bullet$ LLM-as-a-judge. Another popular approach is asking the LM itself whether a particular free-form generation is correct (Kadavath et al., 2022). However, this formulation also poses a restriction on the confidence estimation method, as it is essentially a scalar logit. Thus, many extensions focus on applying calibration methods from classification to calibrate such self-evaluation. The few exceptions include (Ren et al., 2023b; Kadavath et al., 2022), which converts samples from free-form generation into a multiple-choice question (with generations being the options) and adds a ”None of the above” option to elicit the confidence. $\bullet$ Trainable Confidence Estimators. Since we typically care about the LM’s confidence in the “semantic space” due to semantic invariance, instead of manipulating logits, a popular approach is to perform additional training for confidence estimation. This could be done on the base LM (either fully (Zhao et al., 2023a; Jiang et al., 2021; Kapoor et al., 2024) or partially (Liu et al., 2024a)) with a different loss, or using a separate model on the internal or external representations from the base LM (Jagannatha and Yu, 2020; Ulmer et al., 2024). On the other end of the spectrum, without any training, prompting could be used to elicit verbalized confidence values (Tian et al., 2023). Finally, one could combine multiple confidence estimation methods and enjoy the benefit of ensembling (Gao et al., 2024). As with UQ evaluation (more in Section 4), the choice of correctness function has a profound impact on the conclusion of the experiments, especially for free-form generation tasks. Popular choices include using (potentially larger) LLM as judges (Liu et al., 2024a; Lin et al., 2023; Tian et al., 2023), human annotations (Ren et al., 2023b), or lexical similarities such as ROUGE (Kuhn et al., 2023; Zhao et al., 2023a). Recently, Liu et al. (2025) proposes to evaluate free-form generation confidence measures with selected multiple-choice datasets as an efficient complement. For longer generations, Huang et al. (2024a) proposes to use ordinal (not binary) correctness values to capture the ambiguity in the quality of a generation. In a similar flavor, (Baan et al., 2022) studies the issues in the evaluation of calibration when there is intrinsic human disagreement on the label. | Input clarification ensembles (Hou et al., 2024) ICL-Sample (Ling et al., 2024) SPUQ (Gao et al., 2024) | Input Uncertainty Input Uncertainty Input Uncertainty | Multi Rounds Generations Multi Rounds Generations Multi Rounds Generations + Additional Model | Black-box Black-box Black-box | No No No | | --- | --- | --- | --- | --- | | UAG (Yin et al., 2024) | Reasoning Uncertainty | Single Round Generation | White-box | No | | CoT-UQ (Zhang and Zhang, 2025) | Reasoning Uncertainty | Single Round Generation | White-box | Yes | | TouT (Mo and Xin, 2024) | Reasoning Uncertainty | Multi Rounds Generations | Black-box | No | | TopologyUQ (Da et al., 2025) | Reasoning Uncertainty | Multi Rounds Generations | Black-box | No | | Stable Explanations Confidence (Becker and Soatto, 2024) | Reasoning Uncertainty | Multi Rounds Generation | Black-box | Yes | | SAPLMA (Azaria and Mitchell, 2023) | Parameter + Prediction Uncertainty | Fine-tuning | White-box | Yes | | Supervised estimation (Liu et al., 2024c) | Parameter + Prediction Uncertainty | Fine-tuning | White-box | Yes | | UaIT (Liu et al., 2024b) | Parameter + Prediction Uncertainty | Fine-tuning | White-box | Yes | | LoRA ensembles (Balabanov and Linander, 2024) | Parameter Uncertainty | Fine-tuning | White-box | Yes | | BloB (Wang et al., 2025) | Parameter Uncertainty | Fine-tuning | White-box | Yes | | BLoRA (Yang et al., [n. d.]) | Parameter Uncertainty | Fine-tuning | White-box | Yes | | Perplexity (Mora-Cross and Calderon-Ramirez, 2024; Margatina et al., 2023) | Prediction Uncertainty | Single Round Generation | White-box | Yes | | SAR (Duan et al., 2024) | Prediction Uncertainty | Single Round Generation | White-box | Yes | | P(True) (Kadavath et al., 2022) | Prediction Uncertainty | Single Round Generation | White-box | Yes | | Response improbability (Fadeeva et al., 2024) | Prediction Uncertainty | Single Round Generation | White-box | Yes | | Average log probability (Manakul et al., [n. d.]) | Prediction Uncertainty | Single Round Generation | White-box | Yes | | Predictive Entropy (Kadavath et al., 2022) | Prediction Uncertainty | Multi Rounds Generations | White-box | Yes | | Relative Mahalanobis distance (Ren et al., 2023a) | Prediction Uncertainty | Multi Rounds Generations | White-box | Yes | | HUQ (Vazhentsev et al., 2023) | Prediction Uncertainty | Multi Rounds Generations | White-box | Yes | | Conformal Prediction (CP) (Quach et al., [n. d.]; Kumar et al., 2023) | Prediction Uncertainty | Multi Rounds Generations | White-box | No | | ConU (Wang et al., 2024a) | Prediction Uncertainty | Multi Rounds Generations | White-box | No | | Level-adaptive CP (Cherian et al., 2025) | Prediction Uncertainty | Multi Rounds Generations | White-box | No | | LoFreeCP (Su et al., 2024) | Prediction Uncertainty | Multi Rounds Generations | Black-box | No | | Ecc(J),Deg(J) (Lin et al., 2023) | Prediction Uncertainty | Multi Rounds Generations | Black-box | Yes | | Eig(J) (Lin et al., 2023) | Prediction Uncertainty | Multi Rounds Generations | Black-box | No | | Normal length predictive entropy (Malinin and Gales, 2021) | Prediction Uncertainty | Multi Rounds Generations +Additional Model | White-box | Yes | | Semantic Entropy (Kuhn et al., 2023) | Prediction Uncertainty | Multi Rounds Generations + Additional Model | White-box | Yes | | Kernel Language Entropy (Nikitin et al., 2024) | Prediction Uncertainty | Multi Rounds Generations + Additional Model | White-box | Yes | | Ecc(C),Ecc(E),Deg(C),Deg(E) (Lin et al., 2023) | Prediction Uncertainty | Multi Rounds Generations + Additional Model | Black-box | Yes | | Eig(C),Eig(E) (Lin et al., 2023) | Prediction Uncertainty | Multi Rounds Generations + Additional Model | Black-box | No | | MD-UQ (Chen et al., 2025b) | Prediction Uncertainty | Multi Rounds Generations + Additional Model | Black-box | No | | D-UE (Da et al., 2024a) | Prediction Uncertainty | Multi Rounds Generations + Additional Model | Black-box | Yes | Table 2. An overview of UQ methods discussed in this paper for different dimensions, efficiency, and model settings. Remarks. Existing literature sometimes uses the terms uncertainty and confidence interchangeably. They do often seemingly coincide: When a model’s prediction has low confidence, we naturally consider this as a high uncertainty case. This, however, is treating $U(\mathbf{x})=-\max_{\mathbf{s}}C(\mathbf{x},\mathbf{s})$ as an uncertainty estimate. In general, a model may exhibit high uncertainty over its output space but still express high confidence in a specific output. Conversely, a model could have low overall uncertainty but low confidence in a particular prediction. While the “low uncertainty low confidence case” is relatively less interesting in classification or regression tasks due to MLE point prediction, this scenario is notably more common in NLG, as the output is typically randomly sampled In fact, even if the output is greedily generated, it might not have the highest confidence as measured by Eq. 1. from the predictive distribution. There are also applications that require one but not the other (e.g. conformal language modeling (Quach et al., [n. d.]) or seletive generation (Cole et al., 2023)). In the rest of this paper, we sometimes follow the language of the original papers and treat confidence estimates as uncertainty, but will clearly mark the methods that provide confidence estimates. 3. UQ Methods for Different Dimensions 3.1. Input Uncertainty As mentioned in Section 2.1.2, input uncertainty arises from the ambiguous or incomplete input to the LLMs. While there are works in the LLMs domain that try to benchmark or deal with ambiguity (Guo et al., 2021; Zamani et al., 2020; Deng et al., 2023; Chen et al., 2025a), they did not model the uncertainties induced by ambiguity. Existing UQ methods that specifically consider input uncertainty focus on perturbing the input prompts of LLMs. For instance, (Hou et al., 2024) proposes an approach that generates multiple clarifications for a given prompt and ensembles the resulting generations by using mutual information to capture the disagreement among the predictions arising from different clarifications. Similarly, (Ling et al., 2024) proposed ICL-Sample, which quantified the input uncertainty in the setting of in-context learning using different in-context samples. (Gao et al., 2024) proposes SPUQ, which perturbs the input by techniques such as paraphrasing and dummy tokens to expose the model’s sensitivity and capture uncertainty. Specifically, SPUQ quantified the input uncertainty by using a similarity metric such as BERTScore (Zhang et al., [n. d.]) to measure how consistent the responses are across different perturbations. In general, there are only a few papers that consider input uncertainty. Since ambiguity is common and important in natural language, more effort is needed into input uncertainty and its application. 3.2. Reasoning Uncertainty Reasoning is the process of drawing conclusions based on available information. As the LLMs have demonstrated remarkable performance on tasks involving reasoning, recent research has focused on using UQ in LLM reasoning and analyzing the internal reasoning process. For example, TopologyUQ (Da et al., 2025) introduces a formal method to extract and structure LLM explanations into graph representations, quantifying reasoning uncertainty by employing graph-edit distances and revealing redundancy through stable topology measures. Stable-Explanation Confidence (Becker and Soatto, 2024) treats each possible model and its explanation pair as a test-time classifier to construct a posterior answer distribution that reflects overall reasoning confidence. CoT-UQ (Zhang and Zhang, 2025) integrates chain-of-thought reasoning into a response-level UQ framework, thereby leveraging the inherent multi-step reasoning capability of LLMs to further improve uncertainty assessment. Collectively, these approaches provide a robust and interpretable framework for enhancing LLM reasoning by quantifying uncertainty at local or global levels. The quantified uncertainty could be used to guide the exploration of reasoning steps and improving the final performance in completing the tasks. In (Mo and Xin, 2024), they propose Tree of Uncertain Thoughts (TouT), which extend the Tree of Thoughts (ToT) (Yao et al., 2023) framework by quantifying the uncertainties in intermediate reasoning steps with Monte Carlo Dropout and assigning uncertainty scores to important decision points. Similarly, (Yin et al., 2024) reduces the error accumulation in multi-step reasoning by monitoring the predicted probability of the next token at each generation step, dynamically retracting to more reliable states and incorporating certified reasoning clues when high uncertainty is detected. Their experimental results shows that integrating uncertainty enhances the precision of generated responses by integrating these local measures with global search techniques. 3.3. Parameter Uncertainty Parameter uncertainty arises when an LLM lacks sufficient knowledge due to limitations in its training data or model parameters. It reflects the model’s uncertainty about its own predictions, which can be reduced with additional training or adaptation techniques. Traditional UQ methods like Monte Carlo Dropout and Deep Ensembles have been widely used but are computationally infeasible for large-scale LLMs due to the need for multiple forward passes or model replicas. To address this, Bayesian Low-Rank Adaptation by Backpropagation (BLoB) (Wang et al., 2025) and Bayesian Low-Rank Adaptation (BLoRA) (Yang et al., [n. d.]) incorporate Bayesian modeling into LoRA adapters, allowing uncertainty estimation through parameter distributions without a full-model ensemble. However, these methods still incur significant computational costs. Finetuning-based approaches offer a more practical alternative. Techniques such as Supervised Uncertainty Estimation (Liu et al., 2024c) train auxiliary models to predict the confidence of LLM outputs based on activation patterns and logit distributions. Similarly, Uncertainty-aware Instruction Tuning (UaIT) (Liu et al., 2024b) modifies the fine-tuning process to explicitly train models to express uncertainty in their outputs. SAPLMA (Azaria and Mitchell, 2023) refines probabilistic alignment techniques to dynamically adjust model uncertainty estimates, ensuring adaptability to different downstream tasks. Additionally, LoRA ensembles (Balabanov and Linander, 2024) provide an alternative to full-model ensembles by training multiple lightweight LoRA-adapted variants of an LLM instead of retraining the entire network. 3.4. Prediction Uncertainty Most off-the-shelf UQ methods focus on prediction uncertainty since it is the most straightforward way to estimate the uncertainty. Considering the number of generations and models when estimating uncertainties, existing methods for predicting uncertainty can be categorized into the following three categories. 3.4.1. Single Round Generation Most single-round generation methods utilize the logit or hidden states during generation. With only one round of generation, these methods usually methods are usually efficient in estimating uncertainties. $\bullet$ Perplexity is a measure of how well a probabilistic language model predicts a sequence of text (Vaswani et al., 2017) while Mora-Cross and Calderon-Ramirez (2024), Margatina et al. (2023) and Manakul et al. ([n. d.]) utilize the perplexity as the uncertainty. In detail, using $w_{i}$ as the i-th token in the generation, perplexity is given by $\text{Perplexity}=\exp\left(-\frac{1}{N}\sum_{i=1}^{N}\ln p(w_{i})\right)$ . A higher perplexity means the model spreads its probability more broadly over possible words, indicating that it has a higher uncertainty. $\bullet$ Maximum Token Log-Probability. Apart from the perplexity, Maximum token log-probability (Manakul et al., [n. d.]) measures the sentence’s likelihood by assessing the least likely token in the sentence. A higher $\text{Maximum}(p)$ indicates higher uncertainty of the whole generation. It is calculated by $Max(p)=\max\limits_{i}(-\ln p(w_{i}))$ . $\bullet$ Entropy reflects how widely distributed a model’s predictions are for a given input, indicating the level of uncertainty in its outputs (Kuhn et al., 2023; Kadavath et al., 2022). Entropy for the i-th token is provided by $\mathcal{H}_{i}=-\sum_{\tilde{w}∈\mathcal{D}}p_{i}(\tilde{w})\log p_{i}(% \tilde{w})$ . Then it is possible to use the mean or maximum value of entropy as the final uncertainty (Manakul et al., [n. d.]): $Avg(\mathcal{H})=\frac{1}{N}\sum_{i=1}^{N}\mathcal{H}_{i};Max(\mathcal{H})=% \max\limits_{i}(\mathcal{H}_{i})$ . Furthermore, Shifting Attention to Relevance (SAR) (Duan et al., 2024), enhanced the performance of entropy by adjusting attention to more relevant tokens inside the sentence. In detail, SAR assigned weight for $\mathcal{H}_{i}$ and the weight $R(w_{i},s,x)$ can be obtained by: $R(w_{i},s,x)=1-\left|g(x\cup s,x\cup s\setminus\{w_{i}\})\right|,$ where $g$ is a function that measures the semantic similarity between two sentences, which can be estimated with NLI models (Duan et al., 2024). $\bullet$ Response Improbability (Fadeeva et al., 2024) uses response improbability, which computes the probability of a given sentence and subtracts the resulting value from one. In detail, response improbability is provided by $MP(s)=1-\prod_{i=1}p_{i}(w_{i}).$ If the sentence is certain (i.e., the product of token probabilities is high), $MP(s)$ will be low. $\bullet$ P(True) (Kadavath et al., 2022) measures the uncertainty of the claim by asking the LLM itself whether the generation is true or not. Specifically, P(True) is calculated The original name is P(IK), which stands for “I Know”.: $\text{P(True)}=1-p(y_{1}=\text{``True”}).$ Note that here we are using $y_{1}$ as the first token instead of $w_{1}$ because $w_{1}$ represents the first token in the generation $s$ while $y_{1}$ represents the first token when asking LLM whether the generation $s$ is correct or not. P(True) requires running the LLM twice. However, it does not require multiple generations $s$ . Therefore, we still classify this method as a single-round generation This could be considered an uncertainty estimate as the sequence to be evaluated is the prediction given the input.. 3.4.2. Multiple rounds generation Multiple rounds generation methods estimate uncertainty by generating multiple predictions from the LLMs and analyzing their consistency, similarity, or variability. These approaches assume that if a model is confident, its outputs should be stable across different sampling conditions. $\bullet$ Token-Level Entropy. Token-level entropy quantifies uncertainty in LLMs by analyzing the probability distribution of generated tokens across multiple samples. A confident model assigns high probability to a specific token, resulting in low entropy, while uncertain predictions distribute probability across multiple tokens, leading to higher entropy. Multiple responses are generated for the same input to estimate token-level entropy, and the entropy of the token probability distribution is computed. For example, predictive entropy (Kadavath et al., 2022) can also be applied to multiple response settings and shows a better uncertainty quality based on the variability of multiple outputs. Similarly, SAR (Duan et al., 2024) could also be applied to multiple responses. (Malinin and Gales, 2021) extends with Monte Carlo-based approximations and focuses on how probability distributions evolve across tokens during autoregressive generation. There are two main approaches to get the final uncertainty: one averages entropy across multiple sampled outputs, and the other decomposes sequence-level uncertainty into token-level contributions using entropy approximation. $\bullet$ Conformal Prediction. Conformal Prediction (CP) (Shafer and Vovk, 2008) is a statistical framework that provides formal coverage guarantees for uncertainty estimation in LLMs. Its distribution-free properties make it suitable for both black-box and white-box models. In the black-box setting, where model internals are inaccessible, CP estimates uncertainty using response frequency, semantic similarity, or self-consistency. One study proposes a method tailored for API-only LLMs (Su et al., 2024), using frequency-based sampling combined with normalized entropy and semantic similarity to define nonconformity scores. Another black-box CP method introduces a self-consistency-based uncertainty measure (Wang et al., 2024a), which clusters sampled generations and selects a representative response to construct prediction sets with correctness guarantees, making it particularly effective for open-ended NLG tasks. On the other hand, white-box CP methods use logits, internal activations, and calibration techniques for more refined uncertainty estimation. One study proposes Conformal Language Modeling (Quach et al., [n. d.]), which integrates CP with autoregressive text generation by dynamically calibrating a stopping rule to ensure at least one response in the generated set is statistically valid. Another work adapts CP for multiple-choice QA (Kumar et al., 2023), using model confidence scores to calibrate prediction sets, ensuring coverage with minimal set size. A more advanced technique, conditional CP (Cherian et al., 2025), dynamically adjusts coverage guarantees based on the difficulty of the input, optimizing prediction set size while maintaining reliability. $\bullet$ Consistency-Based Methods. Consistency-based uncertainty estimation methods analyze the agreement between multiple generated responses from an LLM to determine uncertainty. The underlying assumption is that if the model is confident, its responses should be consistent, while high variability among responses suggests uncertainty. (Lin et al., 2023) measures the overlap between words through Jaccard similarity in different generations. This method evaluates the deviation from self-consistency, where a high Jaccard similarity across generations implies low uncertainty. However, word-level similarity alone is insufficient, as different responses can convey the same meaning using different phrasing. Moreover, the generated response might include long reasoning steps that require detailed analysis (Golovneva et al., [n. d.]). To address this problem, some methods incorporate external models to assess semantic similarity rather than relying solely on lexical overlap. 3.4.3. Multiple Rounds Generation with External Models Semantic-based uncertainty estimation methods expand multiple generation approaches by incorporating external models, such as Natural Language Inference (NLI) or pretrained language models, to evaluate the semantic relationships among generated responses beyond surface-level similarity. $\bullet$ Distribution-based entropy methods quantify uncertainty by modeling the distribution of generated responses in a semantic space. Semantic Entropy (SE) (Kuhn et al., 2023) refines uncertainty estimation by clustering generated responses based on semantic equivalence. This approach uses an NLI model to determine entailment relationships among responses, grouping them into meaning-preserving clusters. Instead of calculating entropy over individual responses, SE computes entropy over these clusters. Kernel Language Entropy (KLE) (Nikitin et al., 2024) takes a different approach by avoiding explicit clustering. Instead, it embeds the responses in a semantic space using a positive semidefinite kernel function. By computing von Neumann entropy over these response distributions, KLE provides an even more fine-grained measure of uncertainty that considers nuanced semantic variations. $\bullet$ Pairwise similarity methods construct a pairwise semantic similarity matrix between responses and analyze its structural properties to estimate uncertainty. Methods like (Lin et al., 2023) use NLI models to score entailment and contradiction between every pair of generated outputs, forming a weighted similarity graph. A confident model yields semantically coherent responses with strong mutual agreement (high similarity), while inconsistent or ambiguous outputs lead to greater dispersion in the matrix. To quantify this dispersion, spectral graph metrics are applied: Eccentricity (Ecc) captures variability spread, Eigenvalue-based (Eig) measures assess global structure, and Degree (Deg) evaluates local consistency. Recent works further extend this by modeling the response similarity graph as directed (Da et al., 2024a) or multi-dimensional (Chen et al., 2025b), allowing for richer representation of semantic asymmetry or latent factors in uncertainty. 4. Evaluation of Uncertainty in LLMs 4.1. Benchmark Datasets Datasets used in previous studies can be organized into several categories based on their focus. An overall summary of the categorization of datasets and benchmarks for UQ is shown in Table 3. $\bullet$ Reading comprehension benchmarks include CoQA (Reddy et al., 2019) for conversational question answering tasks, RACE (Lai et al., 2017) for general reading comprehension, TriviaQA (Joshi et al., 2017) for fact-based questions, CosmosQA (Huang et al., 2019) for contextual understanding, SQuAD (Rajpurkar et al., 2016) for question answering on passages, and HotpotQA (Yang et al., 2018) for multi-hop reasoning. $\bullet$ Reasoning and math benchmarks include HotpotQA (Yang et al., 2018) and StrategyQA (Geva et al., 2021), which test multi-hop reasoning, GSM8K (Cobbe et al., 2021) for solving math problems, and CalibratedMath (Lin et al., 2022a), designed to evaluate confidence expression in arithmetic. These benchmarks are helpful to evaluate the reasoning uncertainty. $\bullet$ Factuality evaluation draws on datasets such as TruthfulQA (Lin et al., 2022b) for addressing common misconceptions, FEVER (Thorne et al., 2018) for claim verification, HaluEval (Li et al., 2023a) for detecting hallucinations, and an annotated FActScore (Min et al., 2023) dataset for evaluating the factuality of long-form text generated by LLMs. $\bullet$ General knowledge benchmarks can be adapted for UQ to test the models’ general knowledge, such as MMLU (Hendrycks et al., 2020) for a wide range of subjects, GPQA (Rein et al., 2024) for multiple-choice questions in physical sciences, and HellaSwag (Zellers et al., 2019) for common-sense reasoning through sentence completion. These benchmarks can be adapted for UQ because the tasks can be reduced to a classification problem, determining whether the model is confident or uncertain. The structured nature of these benchmarks allows for clear evaluation of the model’s confidence in its predictions. $\bullet$ Consistency and ambiguity are two additional kind of benchmarks for UQ. Consistency benchmarks such as ParaRel (Elazar et al., 2021) tests semantic consistency across 328 paraphrases for 38 relations, and datasets like AmbigQA and AmbigInst, which feature inherent ambiguities (Min et al., 2020; Hou et al., 2024; Chen et al., 2025a). Ambiguity datasets are useful in UQ evaluation because they introduce aleatoric uncertainty by highlighting cases where multiple plausible interpretations exist, helping to assess how well models distinguish between data-driven randomness and model-based uncertainty. These datasets enable a more precise decomposition of uncertainty into aleatoric and epistemic components, improving model reliability and interpretability. | Reading Comprehension | TriviaQA (Joshi et al., 2017), CoQA (Reddy et al., 2019), RACE (Lai et al., 2017), CosmosQA (Huang et al., 2019), SQuAD (Rajpurkar et al., 2016), HotpotQA (Yang et al., 2018) | | --- | --- | | Reasoning & Math | StrategyQA (Geva et al., 2021), HotpotQA (Yang et al., 2018), GSM8K (Cobbe et al., 2021), CalibratedMath (Lin et al., 2022a) | | Factuality | TruthfulQA (Lin et al., 2022b), FEVER (Thorne et al., 2018), HaluEval (Li et al., 2023a), FActScore (Min et al., 2023) | | General Knowledge | MMLU (Hendrycks et al., 2020), GPQA (Rein et al., 2024), HellaSwag (Zellers et al., 2019) | | Consistency & Ambiguity | ParaRel (Elazar et al., 2021), AmbigQA (Min et al., 2020), AmbigInst (Hou et al., 2024), Abg-SciQA (Chen et al., 2025a) | Table 3. Categorization of benchmarking datasets for UQ. Recently, there have been efforts to develop UQ benchmarks for dedicated sources of uncertainty or specific methods. For example, MAQA (Yang et al., 2025) is a dataset specifically designed to evaluate epistemic uncertainty in language models; LM-Polygraph (Fadeeva et al., 2023) was later adopted as a comprehensive uncertainty benchmark (Vashurin et al., 2025). (Ye et al., 2025) developed a benchmark for conformal prediction methods. These contributions represent specialized datasets explicitly designed to assess UQ capabilities in LLMs, rather than adapting existing general-purpose benchmarks. 4.2. Evaluation Metrics UQ is often evaluated from binary classification tasks, with the rationale being that high uncertainty should correspond to low expected accuracy. This is typically modeled by assigning a binary label to each response with a correctness function and using the uncertainty estimates to predict the label. AUROC (Area Under the Receiver Operating Characteristic curve), which measures how effectively the uncertainty score separates correct from incorrect responses, is often used. With values ranging from 0 to 1, higher AUROC s indicate better performance. Responses with confidence above the threshold are classified as predicted positives, while those below are treated as predicted negatives. Many prior studies use AUROC to evaluate how well the uncertainty score discriminates correct from incorrect predictions (Chen and Mueller, 2024; Kuhn et al., 2023; Xiong et al., 2024a; Liu et al., 2024c, 2025). Similarly, AUPRC (Area Under the Precision-Recall Curve) and AUARC (Area Under the Accuracy-Rejection Curve) (Nadeem et al., 2009) also offer further insights into UQ. AUPRC measures how well the uncertainty score separates correct from incorrect responses (Ling et al., 2024), while AUARC assesses how effectively the uncertainty measure aids in selecting accurate responses by determining which uncertain questions to reject (Lin et al., 2024). In the context of NLG where the correctness label is hard to obtain, researchers also compute heuristic-based fuzzy matching metrics such as BLEU (Papineni et al., 2002) and ROUGE (Kuhn et al., 2023) between the generated text and the reference output(s) to gauge the quality. However, these metrics often fail to capture semantic fidelity or factual correctness. Consequently, many researchers are increasingly turning to LLM-as-a-judge evaluations, wherein a large language model (e.g., GPT-4) is prompted to assess text quality or correctness. This approach can capture nuanced aspects like coherence, style, and factuality, but also introduces risks of bias and inconsistency. Human annotation, however, is expensive and is often limited to a small scale (Zhang et al., 2024b; Kuhn et al., 2023). Apart from the binary classification framework, there are also multiple evaluation methods designed for the specific treatment of uncertainty, sometimes qualitative. For example, focusing on decomposing aleatoric and epistemic uncertainty, (Hou et al., 2024) evaluates only the aleatoric part by using AmbigQA (Min et al., 2020), as high ambiguity questions should incur higher aleatoric uncertainty (whereas math questions, for examples, might have lower). The evaluation in (Giulianelli et al., 2023), on the other hand, is a comparison between the variability of human production (generation) with that of the LM. With an emphasis on UQ for longer generations, (Zhang et al., 2024b) compares the uncertainty estimate against FActScore (Min et al., 2023), as the “correctness” of a long paragraph could be ill-defined or ambiguous. 5. UQ Applications in LLMs LLMs are increasingly applied in diverse domains, offering flexibility and reasoning capabilities. However, UQ is crucial for ensuring their reliability, particularly in high-stakes applications. This section will introduce the applications that integrate the UQ of LLMs from some example domains. Many other fields like energy management, operations research, etc., employ LLMs and would require such discussions on the need for UQ as well. $\bullet$ Robotics. LLM-based robotic planning suffers from ambiguity and hallucinations, motivating the need for UQ in the planning loop. For example, closed-loop planners (Zheng et al., 2024) employ an uncertainty-based failure detector to continuously assess and adjust plans in real-time, while non-parametric UQ methods (Tsai et al., 2024) use an efficient querying strategy to improve reliability. (Mullen Jr and Manocha, 2024) integrates action feasibility checks to align the LLM’s confidence with real-world constraints, improving success rates from approximately 70% to 80%. Similarly, (Ong et al., 2024) dynamically adjusts thresholds for alternative paths in adaptive skill selection, achieving higher success rates. (Liang et al., 2024b) develops an introspective planning framework with LLMs self-assess their uncertainty to enhance safety and human-robot collaboration. $\bullet$ Transportation. Preliminary research explores how LLMs can enhance transportation systems (Da et al., 2024c; Yao et al., 2025; Da et al., 2023). For example, LLM inference has been used to bridge the sim-to-real gap in traffic signal control (Da et al., 2023, 2024b) and smooth mixed-autonomy traffic (Yao et al., 2025). However, both cases reveal the potential risk posed by hallucination. A few works have investigated the uncertainty measure while using the LLMs (de Zarzà et al., 2023), which tries to link the use of VLMs with deep probabilistic programming for UQ while conducting multimodal traffic accident forecasting tasks. $\bullet$ Healthcare. In healthcare, LLMs and VLMs can be good references for diagnosis, but uncertainty is a critical dimension that should be considered together with the generation of more reliable treatment plans (Savage et al., 2024). In (Chen et al., 2024), it quantifies uncertainty in a white-box setting, and reveals that an effective reduction of model uncertainty can be achieved by using the proposed multi-tasking and ensemble methods in EHRs. However, as (Wu et al., 2024) benchmarks popular uncertain quantification methods with different model sizes on medical question-answering datasets, the challenge of UQ for medical applications is still severe. 6. Challenges and Future Directions While significant strides have been made in integrating uncertainty quantification into LLMs, several unaddressed challenges persist. This section will explore these unresolved issues, ranging from efficiency-performance trade-offs to cross-modal uncertainty, and outline promising avenues for future research, aiming to advance the reliability of LLMs in high-stakes applications. $\bullet$ Efficiency-Performance Trade-offs. Multi-sample uncertainty methods incur prohibitive costs for trillion-parameter LLMs ($12k per million queries (Li et al., 2024a)), yet yield marginal reliability gains ( $≤ 0.02$ AUROC improvement (Xiong et al., 2024b)). Hybrid approaches combining low-cost proxies (attention variance (Heo et al., 2018), hidden state clustering (Nikitin et al., 2024)) could resolve this by achieving 90% of maximal performance at 10% computational cost. For example, precomputing uncertainty “hotspots” during inference could trigger targeted multi-sampling only for high-risk outputs like medical diagnoses. $\bullet$ Interpretability Deficits. Users struggle to distinguish whether uncertainty stems from ambiguous inputs, knowledge gaps, or decoding stochasticity. Modular architectures that decouple uncertainty estimation layers (Huang et al., 2025b; Sensoy et al., 2018) or employ causal tracing of transformer attention pathways (Wang et al., 2024b) could clarify uncertainty origins. For instance, perturbing model weights (Gal and Ghahramani, 2016) might reveal parametric uncertainty in low-resource languages, while input modules flag underspecified terms for clarification. $\bullet$ Cross-Modality Uncertainty. Integrating vision, text, and sensor data introduces misaligned confidence estimates between modalities: LVLMs exhibit $2.4×$ higher uncertainty in visual vs. textual components (Zhang et al., 2024a), causing 63% of errors in multi-modal QA (Zhang et al., 2024d). Dynamic contrastive decoding and uncertainty-aware fusion protocols show promise (Huo et al., 2024; Suo et al., 2025), but require domain-specific adaptations (e.g., aligning radiology images with reports (Li et al., 2023b; Zhao et al., 2023b)). Future work must develop unified uncertainty embeddings to harmonize modality confidence scales and adversarial training against cross-modal backdoor attacks (Zhang et al., 2024c; Liang et al., 2024a). $\bullet$ System-level Uncertainty in Agents and Reasoning. As LLMs are increasingly deployed as autonomous agents or reasoning engines, the propagation and accumulation of uncertainty across steps becomes critical. Errors in early steps can lead to cascading failures, especially when the model expresses misplaced confidence. However, most existing UQ methods operate at one round of outputs from LLM, lacking mechanisms to capture uncertainty over multi-step reasoning chains or multi-action plans. As studies suggest that LLMs often fail to revise earlier decisions when presented with contradictory information (Creswell et al., [n. d.]), there is a need for temporally-aware uncertainty tracking. Enhancing LLMs with structured memory or model-based planning, or leveraging graph-based representations to trace and revise uncertain steps (Madaan et al., 2023; Yao et al., 2023), could possibly provide more reliable behavior. $\bullet$ UQ Evaluation. Evaluating the quality of UQ remains a fundamental challenge. While the binary classification metrics introduced in Section 4.2 are widely used, they are not always suitable: many tasks, especially in NLG, cannot be easily reduced to binary correctness. Even for structured tasks like question answering, determining whether a free-form generation is correct can be nontrivial due to semantic variability and ambiguity. This issue becomes even more pronounced in open-ended tasks. Moreover, LLM-as-a-judge evaluation approaches are themselves subject to systematic biases (Panickssery et al., 2024; Lin et al., 2022b; Zheng et al., 2023). In addition, common evaluation metrics such as AUROC and AUARC often fail to capture what might be considered “meaningful” uncertainty. These metrics typically assess a model’s ability to distinguish between correct and incorrect outputs, but do not differentiate between confidently wrong responses and those accompanied by an appropriate level of uncertainty. 7. Conclusion In this survey, we offer a comprehensive overview of uncertainty quantification (UQ) in Large Language Models (LLMs). We first introduce the fundamental concepts relevant to both UQ and LLMs, highlighting the importance of reliability in high-stakes applications. Following this, we propose a detailed taxonomy for characterizing uncertainty dimensions in LLMs, including input, reasoning, parameter, and prediction uncertainty. We systematically introduce existing UQ methods using our novel taxonomy, reviewing their effectiveness across different uncertainty types. Ultimately, we identify and discuss some of the persistent challenges in UQ for LLMs, providing insightful directions for future research. The primary goal of this survey is to promote the integration of UQ techniques into LLM development, motivating both machine learning researchers and practitioners to participate in this rapidly advancing area. ACKNOWLEDGMENTS The work was partially supported by NSF awards #2421839. The views and conclusions contained in this paper are those of the authors and should not be interpreted as representing any funding agencies. We thank Amazon Research Awards and OpenAI for providing us with API credits under the Researcher Access program. References - (1) - Achiam et al. (2023) Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023). - Azaria and Mitchell (2023) Amos Azaria and Tom Mitchell. 2023. The Internal State of an LLM Knows When It‘s Lying. (2023), 967–976. - Baan et al. (2022) Joris Baan, Wilker Aziz, Barbara Plank, and Raquel Fernández. 2022. Stop Measuring Calibration When Humans Disagree. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 1892–1915. - Balabanov and Linander (2024) Oleksandr Balabanov and Hampus Linander. 2024. Uncertainty quantification in fine-tuned LLMs using LoRA ensembles. arXiv preprint arXiv:2402.12264 (2024). - Becker and Soatto (2024) Evan Becker and Stefano Soatto. 2024. Cycles of thought: Measuring llm confidence through stable explanations. arXiv preprint arXiv:2406.03441 (2024). - Chen and Mueller (2024) Jiuhai Chen and Jonas Mueller. 2024. Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthiness. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. 5186–5200. - Chen et al. (2025a) Tiejin Chen, Kuan-Ru Liou, Mithun Shivakoti, Aaryan Gaur, Pragya Kumari, Meiqi Guo, and Hua Wei. 2025a. Abg-SciQA: A dataset for Understanding and Resolving Ambiguity in Scientific Questions. In ICLR 2025 Workshop on Navigating and Addressing Data Problems for Foundation Models. - Chen et al. (2025b) Tiejin Chen, Xiaoou Liu, Longchao Da, Jia Chen, Vagelis Papalexakis, and Hua Wei. 2025b. Uncertainty Quantification of Large Language Models through Multi-Dimensional Responses. arXiv preprint arXiv:2502.16820 (2025). - Chen et al. (2024) Zizhang Chen, Peizhao Li, Xiaomeng Dong, and Pengyu Hong. 2024. Uncertainty Quantification for Clinical Outcome Predictions with (Large) Language Models. arXiv preprint arXiv:2411.03497 (2024). - Cheong et al. (2024) Inyoung Cheong, King Xia, KJ Kevin Feng, Quan Ze Chen, and Amy X Zhang. 2024. (A) I am not A lawyer, but…: engaging legal experts towards responsible LLM policies for legal advice. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency. 2454–2469. - Cherian et al. (2025) John Cherian, Isaac Gibbs, and Emmanuel Candes. 2025. Large language model validity via enhanced conformal prediction methods. Advances in Neural Information Processing Systems 37 (2025), 114812–114842. - Cobbe et al. (2021) Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training Verifiers to Solve Math Word Problems. arXiv preprint arXiv:2110.14168 (2021). - Cole et al. (2023) Jeremy Cole, Michael Zhang, Dan Gillick, Julian Eisenschlos, Bhuwan Dhingra, and Jacob Eisenstein. 2023. Selectively Answering Ambiguous Questions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 530–543. - Creswell et al. ([n. d.]) Antonia Creswell, Murray Shanahan, and Irina Higgins. [n. d.]. Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning. In The Eleventh International Conference on Learning Representations. - Da et al. (2024a) Longchao Da, Tiejin Chen, Lu Cheng, and Hua Wei. 2024a. Llm uncertainty quantification through directional entailment graph and claim level response augmentation. arXiv preprint arXiv:2407.00994 (2024). - Da et al. (2024b) Longchao Da, Minquan Gao, Hao Mei, and Hua Wei. 2024b. Prompt to transfer: Sim-to-real transfer for traffic signal control with prompt learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 82–90. - Da et al. (2024c) Longchao Da, Kuanru Liou, Tiejin Chen, Xuesong Zhou, Xiangyong Luo, Yezhou Yang, and Hua Wei. 2024c. Open-ti: Open traffic intelligence with augmented language model. International Journal of Machine Learning and Cybernetics 15, 10 (2024), 4761–4786. - Da et al. (2025) Longchao Da, Xiaoou Liu, Jiaxin Dai, Lu Cheng, Yaqing Wang, and Hua Wei. 2025. Understanding the Uncertainty of LLM Explanations: A Perspective Based on Reasoning Topology. arXiv preprint arXiv:2502.17026 (2025). - Da et al. (2023) Longchao Da, Hao Mei, Romir Sharma, and Hua Wei. 2023. Uncertainty-aware grounded action transformation towards sim-to-real transfer for traffic signal control. In 2023 62nd IEEE Conference on Decision and Control (CDC). 1124–1129. - Da et al. (2024d) Longchao Da, Rui Wang, Xiaojian Xu, Parminder Bhatia, Taha Kass-Hout, Hua Wei, and Cao Xiao. 2024d. Segment as You Wish–Free-Form Language-Based Segmentation for Medical Images. arXiv preprint arXiv:2410.12831 (2024). - de Zarzà et al. (2023) Irene de Zarzà, Joachim de Curtò, Gemma Roig, and Carlos T Calafate. 2023. LLM multimodal traffic accident forecasting. Sensors 23, 22 (2023), 9225. - Deng et al. (2023) Yang Deng, Shuaiyi Li, and Wai Lam. 2023. Learning to ask clarification questions with spatial reasoning. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2113–2117. - Der Kiureghian and Ditlevsen (2009) Armen Der Kiureghian and Ove Ditlevsen. 2009. Aleatory or epistemic? Does it matter? Structural safety 31, 2 (2009), 105–112. - Duan et al. (2024) Jinhao Duan, Hao Cheng, Shiqi Wang, Alex Zavalny, Chenan Wang, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu. 2024. Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 5050–5063. - Elazar et al. (2021) Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics 9 (2021), 1012–1031. - Fadeeva et al. (2024) Ekaterina Fadeeva, Aleksandr Rubashevskii, Artem Shelmanov, Sergey Petrakov, Haonan Li, Hamdy Mubarak, Evgenii Tsymbalov, Gleb Kuzmin, Alexander Panchenko, Timothy Baldwin, et al. 2024. Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification. In Findings of the Association for Computational Linguistics ACL 2024. 9367–9385. - Fadeeva et al. (2023) Ekaterina Fadeeva, Roman Vashurin, Akim Tsvigun, Artem Vazhentsev, Sergey Petrakov, Kirill Fedyanin, Daniil Vasilev, Elizaveta Goncharova, Alexander Panchenko, Maxim Panov, et al. 2023. LM-Polygraph: Uncertainty Estimation for Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 446–461. - Gal and Ghahramani (2016) Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning. PMLR, 1050–1059. - Gao et al. (2024) Xiang Gao, Jiaxin Zhang, Lalla Mouatadid, and Kamalika Das. 2024. SPUQ: Perturbation-Based Uncertainty Quantification for Large Language Models. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2336–2346. - Geva et al. (2021) Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies. Transactions of the Association for Computational Linguistics (2021), 346–361. - Giulianelli et al. (2023) Mario Giulianelli, Joris Baan, Wilker Aziz, Raquel Fernández, and Barbara Plank. 2023. What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variability. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 14349–14371. - Golovneva et al. ([n. d.]) Olga Golovneva, Moya Peng Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Fazel-Zarandi, and Asli Celikyilmaz. [n. d.]. ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning. In The Eleventh International Conference on Learning Representations. - Guo et al. (2021) Meiqi Guo, Mingda Zhang, Siva Reddy, and Malihe Alikhani. 2021. Abg-coqa: Clarifying ambiguity in conversational question answering. 3rd Conference on Automated Knowledge Base Construction (2021). - Hendrycks et al. (2020) Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 (2020). - Heo et al. (2018) Jay Heo, Hae Beom Lee, Saehoon Kim, Juho Lee, Kwang Joon Kim, Eunho Yang, and Sung Ju Hwang. 2018. Uncertainty-aware attention for reliable interpretation and prediction. Advances in neural information processing systems 31 (2018). - Hou et al. (2024) Bairu Hou, Yujian Liu, Kaizhi Qian, Jacob Andreas, Shiyu Chang, and Yang Zhang. 2024. Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling. In Forty-first International Conference on Machine Learning. - Huang et al. (2024b) Hsiu-Yuan Huang, Yutong Yang, Zhaoxi Zhang, Sanwoo Lee, and Yunfang Wu. 2024b. A survey of uncertainty estimation in llms: Theory meets practice. arXiv preprint arXiv:2410.15326 (2024). - Huang et al. (2025b) Jingwang Huang, Jiang Zhong, Qin Lei, Jinpeng Gao, Yuming Yang, Sirui Wang, Peiguang Li, and Kaiwen Wei. 2025b. Latent Distribution Decoupling: A Probabilistic Framework for Uncertainty-Aware Multimodal Emotion Recognition. arXiv preprint arXiv:2502.13954 (2025). - Huang et al. (2019) Lifu Huang, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2019. Cosmos QA: Machine reading comprehension with contextual commonsense reasoning. arXiv preprint arXiv:1909.00277 (2019). - Huang et al. (2025a) Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2025a. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems 43, 2 (2025), 1–55. - Huang et al. (2024a) Yukun Huang, Yixin Liu, Raghuveer Thirukovalluru, Arman Cohan, and Bhuwan Dhingra. 2024a. Calibrating Long-form Generations From Large Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2024. 13441–13460. - Hüllermeier and Waegeman (2021) Eyke Hüllermeier and Willem Waegeman. 2021. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine learning 110, 3 (2021), 457–506. - Huo et al. (2024) Fushuo Huo, Wenchao Xu, Zhong Zhang, Haozhao Wang, Zhicheng Chen, and Peilin Zhao. 2024. Self-introspective decoding: Alleviating hallucinations for large vision-language models. arXiv preprint arXiv:2408.02032 (2024). - Jagannatha and Yu (2020) Abhyuday Jagannatha and Hong Yu. 2020. Calibrating Structured Output Predictors for Natural Language Processing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2078–2092. - Jiang et al. (2018) Heinrich Jiang, Been Kim, Maya Gupta, and Melody Y. Guan. 2018. To trust or not to trust a classifier. In Advances in Neural Information Processing Systems. - Jiang et al. (2021) Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. 2021. How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering. Transactions of the Association for Computational Linguistics (2021), 962–977. - Jin et al. (2021) Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2021. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences 11, 14 (2021), 6421. - Joshi et al. (2017) Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 1601–1611. - Kadavath et al. (2022) Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. 2022. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221 (2022). - Kapoor et al. (2024) Sanyam Kapoor, Nate Gruver, Manley Roberts, Arka Pal, Samuel Dooley, Micah Goldblum, and Andrew Wilson. 2024. Calibration-Tuning: Teaching Large Language Models to Know What They Don‘t Know. In Proceedings of the 1st Workshop on Uncertainty-Aware NLP (UncertaiNLP 2024). 1–14. - Kuhn et al. (2023) Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation. In The Eleventh International Conference on Learning Representations. - Kumar and Sarawagi (2019) Aviral Kumar and Sunita Sarawagi. 2019. Calibration of encoder decoder models for neural machine translation. arXiv preprint arXiv:1903.00802 (2019). - Kumar et al. (2023) Bhawesh Kumar, Charlie Lu, Gauri Gupta, Anil Palepu, David Bellamy, Ramesh Raskar, and Andrew Beam. 2023. Conformal prediction with large language models for multi-choice question answering. arXiv preprint arXiv:2305.18404 (2023). - Lai et al. (2017) Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. RACE: Large-scale ReAding Comprehension Dataset From Examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 785–794. - Lai et al. (2025) Siqi Lai, Zhao Xu, Weijia Zhang, Hao Liu, and Hui Xiong. 2025. LLMLight: Large Language Models as Traffic Signal Control Agents. 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining (2025). - Lakshminarayanan et al. (2017) Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems 30 (2017). - Li et al. (2024a) Baolin Li, Yankai Jiang, Vijay Gadepally, and Devesh Tiwari. 2024a. Llm inference serving: Survey of recent advances and opportunities. arXiv preprint arXiv:2407.12391 (2024). - Li et al. (2023a) Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023a. HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 6449–6464. - Li et al. (2024b) Lincan Li, Jiaqi Li, Catherine Chen, Fred Gui, Hongjia Yang, Chenxiao Yu, Zhengguang Wang, Jianing Cai, Junlong Aaron Zhou, Bolin Shen, et al. 2024b. Political-llm: Large language models in political science. arXiv preprint arXiv:2412.06864 (2024). - Li et al. (2023b) Yaowei Li, Bang Yang, Xuxin Cheng, Zhihong Zhu, Hongxiang Li, and Yuexian Zou. 2023b. Unify, align and refine: Multi-level semantic alignment for radiology report generation. In Proceedings of the IEEE/CVF international conference on computer vision. 2863–2874. - Li et al. (2024c) Zixuan Li, Jing Xiong, Fanghua Ye, Chuanyang Zheng, Xun Wu, Jianqiao Lu, Zhongwei Wan, Xiaodan Liang, Chengming Li, Zhenan Sun, et al. 2024c. UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generation. arXiv preprint arXiv:2410.02719 (2024). - Liang et al. (2024b) Kaiqu Liang, Zixu Zhang, and Jaime Fisac. 2024b. Introspective Planning: Aligning Robots’ Uncertainty with Inherent Task Ambiguity. Advances in Neural Information Processing Systems 37 (2024), 71998–72031. - Liang et al. (2024a) Siyuan Liang, Jiawei Liang, Tianyu Pang, Chao Du, Aishan Liu, Ee-Chien Chang, and Xiaochun Cao. 2024a. Revisiting backdoor attacks against large vision-language models. arXiv preprint arXiv:2406.18844 (2024). - Lin et al. (2022a) Stephanie Lin, Jacob Hilton, and Owain Evans. 2022a. Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205.14334 (2022). - Lin et al. (2022b) Stephanie Lin, Jacob Hilton, and Owain Evans. 2022b. TruthfulQA: Measuring How Models Mimic Human Falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 3214–3252. - Lin et al. (2023) Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. 2023. Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models. Transactions on Machine Learning Research (2023). - Lin et al. (2024) Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. 2024. Contextualized Sequence Likelihood: Enhanced Confidence Scores for Natural Language Generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 10351–10368. - Ling et al. (2024) Chen Ling, Xujiang Zhao, Xuchao Zhang, Wei Cheng, Yanchi Liu, Yiyou Sun, Mika Oishi, Takao Osaki, Katsushi Matsuda, Jie Ji, Guangji Bai, Liang Zhao, and Haifeng Chen. 2024. Uncertainty Quantification for In-Context Learning of Large Language Models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 3357–3370. - Liu et al. (2024c) Linyu Liu, Yu Pan, Xiaocheng Li, and Guanting Chen. 2024c. Uncertainty estimation and quantification for llms: A simple supervised approach. arXiv preprint arXiv:2404.15993 (2024). - Liu et al. (2024b) Shudong Liu, Zhaocong Li, Xuebo Liu, Runzhe Zhan, Derek Wong, Lidia Chao, and Min Zhang. 2024b. Can LLMs learn uncertainty on their own? expressing uncertainty effectively in a self-training manner. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 21635–21645. - Liu et al. (2024a) Xin Liu, Muhammad Khalifa, and Lu Wang. 2024a. LitCab: Lightweight Language Model Calibration over Short- and Long-form Responses. In The Twelfth International Conference on Learning Representations. - Liu et al. (2025) Xiaoou Liu, Zhen Lin, Longchao Da, Chacha Chen, Shubhendu Trivedi, and Hua Wei. 2025. MCQA-Eval: Efficient Confidence Evaluation in NLG with Gold-Standard Correctness Labels. arXiv preprint arXiv:2502.14268 (2025). - Lu et al. (2024) Jinliang Lu, Ziliang Pang, Min Xiao, Yaochen Zhu, Rui Xia, and Jiajun Zhang. 2024. Merge, ensemble, and cooperate! a survey on collaborative strategies in the era of large language models. arXiv preprint arXiv:2407.06089 (2024). - Madaan et al. (2023) Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems 36 (2023), 46534–46594. - Malinin and Gales (2021) Andrey Malinin and Mark Gales. 2021. Uncertainty Estimation in Autoregressive Structured Prediction. In International Conference on Learning Representations. - Manakul et al. ([n. d.]) Potsawee Manakul, Adian Liusie, and Mark Gales. [n. d.]. SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models. In 2023 Conference on Empirical Methods in Natural Language Processing. - Margatina et al. (2023) Katerina Margatina, Timo Schick, Nikolaos Aletras, and Jane Dwivedi-Yu. 2023. Active Learning Principles for In-Context Learning with Large Language Models. In Findings of the Association for Computational Linguistics. 5011–5034. - Min et al. (2023) Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit yyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 12076–12100. - Min et al. (2020) Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020. AmbigQA: Answering Ambiguous Open-domain Questions. In EMNLP. - Mo and Xin (2024) Shentong Mo and Miao Xin. 2024. Tree of uncertain thoughts reasoning for large language models. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 12742–12746. - Mondorf and Plank (2024) Philipp Mondorf and Barbara Plank. 2024. Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models-A Survey. In First Conference on Language Modeling. - Mora-Cross and Calderon-Ramirez (2024) Maria Mora-Cross and Saul Calderon-Ramirez. 2024. Uncertainty estimation in large language models to support biodiversity conservation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 6: Industry Track). 368–378. - Mukherjee and Awadallah (2020) Subhabrata Mukherjee and Ahmed Awadallah. 2020. Uncertainty-aware self-training for few-shot text classification. Advances in Neural Information Processing Systems 33 (2020), 21199–21212. - Mullen Jr and Manocha (2024) James F Mullen Jr and Dinesh Manocha. 2024. LAP, Using Action Feasibility for Improved Uncertainty Alignment of Large Language Model Planners. arXiv preprint arXiv:2403.13198 (2024). - Murphy and Winkler (1977) Allan H. Murphy and Robert L. Winkler. 1977. Reliability of Subjective Probability Forecasts of Precipitation and Temperature. Journal of The Royal Statistical Society Series C-applied Statistics 26 (1977), 41–47. - Nadeem et al. (2009) Malik Sajjad Ahmed Nadeem, Jean-Daniel Zucker, and Blaise Hanczar. 2009. Accuracy-rejection curves (ARCs) for comparing classification methods with a reject option. In Machine Learning in Systems Biology. 65–81. - Nikitin et al. (2024) Alexander Nikitin, Jannik Kossen, Yarin Gal, and Pekka Marttinen. 2024. Kernel language entropy: Fine-grained uncertainty quantification for LLMs from semantic similarities. Advances in Neural Information Processing Systems 37 (2024), 8901–8929. - Ong et al. (2024) Hyobin Ong, Youngwoo Yoon, Jaewoo Choi, and Minsu Jang. 2024. A Simple Baseline for Uncertainty-Aware Language-Oriented Task Planner for Embodied Agents. In 2024 21st International Conference on Ubiquitous Robots (UR). 677–682. - Panickssery et al. (2024) Arjun Panickssery, Samuel Bowman, and Shi Feng. 2024. Llm evaluators recognize and favor their own generations. Advances in Neural Information Processing Systems 37 (2024), 68772–68802. - Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics. 311–318. - Qiu et al. (2024) Jianing Qiu, Kyle Lam, Guohao Li, Amish Acharya, Tien Yin Wong, Ara Darzi, Wu Yuan, and Eric J Topol. 2024. LLM-based agentic systems in medicine and healthcare. Nature Machine Intelligence 6, 12 (2024), 1418–1420. - Quach et al. ([n. d.]) Victor Quach, Adam Fisch, Tal Schuster, Adam Yala, Jae Ho Sohn, Tommi S Jaakkola, and Regina Barzilay. [n. d.]. Conformal Language Modeling. In The Twelfth International Conference on Learning Representations. - Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016). - Reddy et al. (2019) Siva Reddy, Danqi Chen, and Christopher D Manning. 2019. Coqa: A conversational question answering challenge. Transactions of the Association for Computational Linguistics 7 (2019), 249–266. - Rein et al. (2024) David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling. - Ren et al. (2023a) Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mohammad Saleh, Balaji Lakshminarayanan, and Peter J Liu. 2023a. Out-of-Distribution Detection and Selective Generation for Conditional Language Models. In The Eleventh International Conference on Learning Representations. - Ren et al. (2023b) Jie Ren, Yao Zhao, Tu Vu, Peter J. Liu, and Balaji Lakshminarayanan. 2023b. Self-Evaluation Improves Selective Generation in Large Language Models. In Proceedings on ”I Can’t Believe It’s Not Better: Failure Modes in the Age of Foundation Models” at NeurIPS 2023 Workshops, Vol. 239. 49–64. - Savage et al. (2024) Thomas Savage, John Wang, Robert Gallo, Abdessalem Boukil, Vishwesh Patel, Seyed Amir Ahmad Safavi-Naini, Ali Soroush, and Jonathan H Chen. 2024. Large language model uncertainty measurement and calibration for medical diagnosis and treatment. medRxiv (2024), 2024–06. - Sen et al. (2024) Sagar Sen, Victor Gonzalez, Erik Johannes Husom, Simeon Tverdal, Shukun Tokas, and Svein O Tjøsvoll. 2024. ERG-AI: enhancing occupational ergonomics with uncertainty-aware ML and LLM feedback. Applied Intelligence 54, 23 (2024), 12128–12155. - Sensoy et al. (2018) Murat Sensoy, Lance Kaplan, and Melih Kandemir. 2018. Evidential deep learning to quantify classification uncertainty. Advances in neural information processing systems 31 (2018). - Shafer and Vovk (2008) Glenn Shafer and Vladimir Vovk. 2008. A tutorial on conformal prediction. Journal of Machine Learning Research 9, 3 (2008). - Shorinwa et al. (2024) Ola Shorinwa, Zhiting Mei, Justin Lidard, Allen Z Ren, and Anirudha Majumdar. 2024. A survey on uncertainty quantification of large language models: Taxonomy, open research challenges, and future directions. arXiv preprint arXiv:2412.05563 (2024). - Stengel-Eskin and Van Durme (2023) Elias Stengel-Eskin and Benjamin Van Durme. 2023. Calibrated Interpretation: Confidence Estimation in Semantic Parsing. Transactions of the Association for Computational Linguistics 11 (2023), 1213–1231. - Su et al. (2024) Jiayuan Su, Jing Luo, Hongwei Wang, and Lu Cheng. 2024. API Is Enough: Conformal Prediction for Large Language Models Without Logit-Access. In Findings of the Association for Computational Linguistics: EMNLP 2024. 979–995. - Suo et al. (2025) Wei Suo, Lijun Zhang, Mengyang Sun, Lin Yuanbo Wu, Peng Wang, and Yanning Zhang. 2025. Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding. arXiv preprint arXiv:2503.00361 (2025). - Thorne et al. (2018) James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and VERification. arXiv preprint arXiv:1803.05355 (2018). - Tian et al. (2023) Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher Manning. 2023. Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 5433–5442. - Tsai et al. (2024) Yao-Hung Hubert Tsai, Walter Talbott, and Jian Zhang. 2024. Efficient Non-Parametric Uncertainty Quantification for Black-Box Large Language Models and Decision Planning. arXiv preprint arXiv:2402.00251 (2024). - Ulmer et al. (2024) Dennis Ulmer, Martin Gubri, Hwaran Lee, Sangdoo Yun, and Seong Oh. 2024. Calibrating Large Language Models Using Their Generations Only. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 15440–15459. - Vashurin et al. (2025) Roman Vashurin, Ekaterina Fadeeva, Artem Vazhentsev, Lyudmila Rvanova, Daniil Vasilev, Akim Tsvigun, Sergey Petrakov, Rui Xing, Abdelrahman Sadallah, Kirill Grishchenkov, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, Maxim Panov, and Artem Shelmanov. 2025. Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph. Transactions of the Association for Computational Linguistics (2025), 220–248. - Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017). - Vazhentsev et al. (2023) Artem Vazhentsev, Gleb Kuzmin, Akim Tsvigun, Alexander Panchenko, Maxim Panov, Mikhail Burtsev, and Artem Shelmanov. 2023. Hybrid uncertainty quantification for selective text classification in ambiguous tasks. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 11659–11681. - Wang et al. (2024b) Boshi Wang, Xiang Yue, Yu Su, and Huan Sun. 2024b. Grokking of implicit reasoning in transformers: A mechanistic journey to the edge of generalization. Advances in Neural Information Processing Systems 37 (2024), 95238–95265. - Wang et al. (2020) Shuo Wang, Zhaopeng Tu, Shuming Shi, and Yang Liu. 2020. On the Inference Calibration of Neural Machine Translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. - Wang et al. (2025) Yibin Wang, Haizhou Shi, Ligong Han, Dimitris Metaxas, and Hao Wang. 2025. Blob: Bayesian low-rank adaptation by backpropagation for large language models. Advances in Neural Information Processing Systems 37 (2025), 67758–67794. - Wang et al. (2024a) Zhiyuan Wang, Jinhao Duan, Lu Cheng, Yue Zhang, Qingni Wang, Xiaoshuang Shi, Kaidi Xu, Heng Tao Shen, and Xiaofeng Zhu. 2024a. ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees. In Findings of the Association for Computational Linguistics. 6886–6898. - Wu et al. (2024) Jiaxin Wu, Yizhou Yu, and Hong-Yu Zhou. 2024. Uncertainty Estimation of Large Language Models in Medical Question Answering. arXiv preprint arXiv:2407.08662 (2024). - Xing et al. (2025) Shuo Xing, Yuping Wang, Peiran Li, Ruizheng Bai, Yueqi Wang, Chan-wei Hu, Chengxuan Qian, Huaxiu Yao, and Zhengzhong Tu. 2025. Re-Align: Aligning Vision Language Models via Retrieval-Augmented Direct Preference Optimization. arXiv preprint arXiv:2502.13146 (2025). - Xiong et al. (2024a) Miao Xiong, Zhiyuan Hu, Xinyang Lu, YIFEI LI, Jie Fu, Junxian He, and Bryan Hooi. 2024a. Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs. In The Twelfth International Conference on Learning Representations. - Xiong et al. (2024b) Miao Xiong, Andrea Santilli, Michael Kirchhof, Adam Golinski, and Sinead Williamson. 2024b. Efficient and effective uncertainty quantification for LLMs. In Neurips Safe Generative AI Workshop 2024. - Yang et al. ([n. d.]) Adam X Yang, Maxime Robeyns, Xi Wang, and Laurence Aitchison. [n. d.]. Bayesian Low-rank Adaptation for Large Language Models. In The Twelfth International Conference on Learning Representations. - Yang et al. (2025) Yongjin Yang, Haneul Yoo, and Hwaran Lee. 2025. MAQA: Evaluating Uncertainty Quantification in LLMs Regarding Data Uncertainty. In Findings of the Association for Computational Linguistics: NAACL 2025. 5846–5863. - Yang et al. (2018) Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600 (2018). - Yao et al. (2025) Huaiyuan Yao, Longchao Da, Vishnu Nandam, Justin Turnau, Zhiwei Liu, Linsey Pang, and Hua Wei. 2025. Comal: Collaborative multi-agent large language models for mixed-autonomy traffic. In Proceedings of the 2025 SIAM International Conference on Data Mining (SDM). 409–418. - Yao et al. (2023) Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems 36 (2023), 11809–11822. - Ye et al. (2025) Fanghua Ye, Mingming Yang, Jianhui Pang, Longyue Wang, Derek Wong, Emine Yilmaz, Shuming Shi, and Zhaopeng Tu. 2025. Benchmarking llms via uncertainty quantification. Advances in Neural Information Processing Systems (2025), 15356–15385. - Ye et al. (2024) Kai Ye, Tiejin Chen, Hua Wei, and Liang Zhan. 2024. Uncertainty regularized evidential regression. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 16460–16468. - Yin et al. (2024) Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Junqi Dai, Qinyuan Cheng, Xuan-Jing Huang, and Xipeng Qiu. 2024. Reasoning in flux: Enhancing large language models reasoning through uncertainty-aware adaptive guidance. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2401–2416. - Young et al. (2025) Spencer Young, Porter Jenkins, Lonchao Da, Jeff Dotson, and Hua Wei. 2025. Flexible heteroscedastic count regression with deep double poisson networks. International Conference on Machine Learning (2025). - Zamani et al. (2020) Hamed Zamani, Susan Dumais, Nick Craswell, Paul Bennett, and Gord Lueck. 2020. Generating clarifying questions for information retrieval. In Proceedings of the web conference 2020. 418–428. - Zellers et al. (2019) Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. HellaSwag: Can a Machine Really Finish Your Sentence?. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 4791–4800. - Zhang and Zhang (2025) Boxuan Zhang and Ruqi Zhang. 2025. CoT-UQ: Improving Response-wise Uncertainty Quantification in LLMs with Chain-of-Thought. arXiv preprint arXiv:2502.17214 (2025). - Zhang et al. (2024b) Caiqi Zhang, Fangyu Liu, Marco Basaldella, and Nigel Collier. 2024b. LUQ: Long-text Uncertainty Quantification for LLMs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 5244–5262. - Zhang et al. (2024d) Ruiyang Zhang, Hu Zhang, and Zhedong Zheng. 2024d. VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimation. arXiv preprint arXiv:2411.11919 (2024). - Zhang et al. ([n. d.]) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. [n. d.]. BERTScore: Evaluating Text Generation with BERT. In International Conference on Learning Representations. - Zhang et al. (2024a) Yuan Zhang, Tao Huang, Chun-Kai Fan, Hongyuan Dong, Jiawen Li, Jiacong Wang, Kuan Cheng, Shanghang Zhang, Haoyuan Guo, et al. 2024a. Unveiling the tapestry of consistency in large vision-language models. Advances in Neural Information Processing Systems 37 (2024), 118632–118653. - Zhang et al. (2024c) Zheng Zhang, Xu Yuan, Lei Zhu, Jingkuan Song, and Liqiang Nie. 2024c. BadCM: Invisible backdoor attack against cross-modal learning. IEEE Transactions on Image Processing (2024). - Zhao et al. (2023b) Guosheng Zhao, Zijian Zhao, Wuxian Gong, and Feng Li. 2023b. Radiology report generation with medical knowledge and multilevel image-report alignment: A new method and its verification. Artificial Intelligence in Medicine 146 (2023), 102714. - Zhao et al. (2023a) Yao Zhao, Mikhail Khalman, Rishabh Joshi, Shashi Narayan, Mohammad Saleh, and Peter J Liu. 2023a. Calibrating Sequence likelihood Improves Conditional Language Generation. In The Eleventh International Conference on Learning Representations. - Zheng et al. (2023) Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. 2023. Large language models are not robust multiple choice selectors. arXiv preprint arXiv:2309.03882 (2023). - Zheng et al. (2024) Zhi Zheng, Qian Feng, Hang Li, Alois Knoll, and Jianxiang Feng. 2024. Evaluating uncertainty-based failure detection for closed-loop llm planners. arXiv preprint arXiv:2406.00430 (2024).