Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach (2404.15993v4)

Abstract: In this paper, we study the problem of uncertainty estimation and calibration for LLMs. We begin by formulating the uncertainty estimation problem, a relevant yet underexplored area in existing literature. We then propose a supervised approach that leverages labeled datasets to estimate the uncertainty in LLMs' responses. Based on the formulation, we illustrate the difference between the uncertainty estimation for LLMs and that for standard ML models and explain why the hidden neurons of the LLMs may contain uncertainty information. Our designed approach demonstrates the benefits of utilizing hidden activations to enhance uncertainty estimation across various tasks and shows robust transferability in out-of-distribution settings. We distinguish the uncertainty estimation task from the uncertainty calibration task and show that better uncertainty estimation leads to better calibration performance. Furthermore, our method is easy to implement and adaptable to different levels of model accessibility including black box, grey box, and white box.

• The paper introduces a supervised calibration framework leveraging LLM hidden activations to quantify uncertainty in responses.
• The methodology integrates white-box and grey-box features to yield improved AUROC performance over unsupervised benchmarks.
• The approach offers practical insights for enhancing LLM trustworthiness across tasks like question answering and machine translation.

Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach

This paper, titled "Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach," addresses the critical problem of estimating the uncertainty in responses generated by LLMs. The authors propose a supervised method to quantify and calibrate uncertainty, exploiting hidden activations within LLMs. By systematically evaluating different architectures and methodologies, this research enhances understanding about LLMs' inherent uncertainties, aiming to improve trustworthiness and applicability across various tasks.

Introduction

The rapid advancements in natural language processing powered by LLMs have markedly improved capabilities in understanding and generating human-like text. However, these models often generate unreliable outputs, leading to potential misinformation. Traditional machine learning uncertainty estimation approaches, focusing on fixed-dimensional outputs, face challenges when applied to the variable outputs typical of natural language generation (NLG) tasks. This paper introduces a novel, supervised approach that leverages hidden activations from LLMs, setting it apart from traditional methods which typically don't utilize such deep insights.

Methodology

The proposed methodology involves a supervised framework to estimate uncertainty, distinguishing it fundamentally from existing black-box metrics that rely on outputs like entropy or similarity without access to internal activations. The method takes advantage of white-box features by utilizing internal activations to derive uncertainty metrics, thus offering a more nuanced understanding than entropy-based methods.

Problem Setup

For given input prompts $\bm{x}$, LLMs generate responses $\bm{y}$ through a series of probabilistic token selections, modeled here for tasks such as question answering. The task of uncertainty estimation is then defined as predicting the scoring function $s(\bm{y}, \bm{y}_{\text{true}})$ using learned functions $g(\bm{x}, \bm{y})$, thereby estimating the expected correctness of generated responses.

Supervised Calibration

The supervised calibration model draws features from both white-box (hidden activations) and grey-box (e.g., entropy-based features) sources, resulting in a structured dataset that informs the learning model for uncertainty estimation. This approach effectively extracts richer uncertainty insights compared to unsupervised methods.

Figure 1: Features from Gemma-7B.

Evaluation and Results

The authors implement their approach across several LLMs, including LLaMA2-7B and Gemma-7B, and evaluate it on tasks such as question answering (TriviaQA, CoQA) and machine translation (WMT 2014). By employing area under the receiver operating characteristic curve (AUROC) as a performance metric, the paper clearly demonstrates the superior performance of their method over existing benchmarks, which often overly rely on single-metric unsupervised features.

Robust Performance

The empirical results consistently show that the supervised approach, leveraging internal activations, outperforms existing unsupervised methods. This includes not only improved AUROC scores but also better calibration of uncertainty estimates across both in-distribution and out-of-distribution (OOD) scenarios.

Figure 2: Uncertainty scores of different methods on the MMLU dataset for answers provided by the Gemma-7B model.

Discussion

Layer and Architecture Insights

The research highlights the advantage of extracting information from middle-layer activations, which apparently encapsulate more useful uncertainty information compared to last-layer activations focused on immediate token generation tasks. The scaling effect indicates no significant performance difference between different model sizes, suggesting that larger model sizes don't necessarily correlate with improved uncertainty insights.

Practical Implications and Future Directions

By successfully demonstrating that internal activations can be effectively harnessed for uncertainty estimation, the paper paves the way for applications in improving LLM trustworthiness. Furthermore, the approach opens new avenues for improving closed-source LLMs' uncertainty predictions via publicly available models.

Conclusion

This paper systematically explores uncertainty estimation for LLMs through a novel supervised approach, demonstrating significant enhancements over existing methods. The findings emphasize the value of utilizing hidden activations, thereby improving the reliability and robustness of LLM outputs across diverse NLP tasks. Future work may explore fine-tuning models to tailor uncertainty estimations for more specific applications, extending the approach to open-domain questions beyond structured NLP datasets.