TrustLLM: Trustworthiness in Large Language Models (2401.05561v6)

Abstract: LLMs, exemplified by ChatGPT, have gained considerable attention for their excellent natural language processing capabilities. Nonetheless, these LLMs present many challenges, particularly in the realm of trustworthiness. Therefore, ensuring the trustworthiness of LLMs emerges as an important topic. This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions. Specifically, we first propose a set of principles for trustworthy LLMs that span eight different dimensions. Based on these principles, we further establish a benchmark across six dimensions including truthfulness, safety, fairness, robustness, privacy, and machine ethics. We then present a study evaluating 16 mainstream LLMs in TrustLLM, consisting of over 30 datasets. Our findings firstly show that in general trustworthiness and utility (i.e., functional effectiveness) are positively related. Secondly, our observations reveal that proprietary LLMs generally outperform most open-source counterparts in terms of trustworthiness, raising concerns about the potential risks of widely accessible open-source LLMs. However, a few open-source LLMs come very close to proprietary ones. Thirdly, it is important to note that some LLMs may be overly calibrated towards exhibiting trustworthiness, to the extent that they compromise their utility by mistakenly treating benign prompts as harmful and consequently not responding. Finally, we emphasize the importance of ensuring transparency not only in the models themselves but also in the technologies that underpin trustworthiness. Knowing the specific trustworthy technologies that have been employed is crucial for analyzing their effectiveness.

• The paper presents a rigorous multi-dimensional framework benchmarking 16 LLMs on six trustworthiness aspects using over 30 curated datasets.
• It reveals a correlation between model utility and trustworthiness, highlighting trade-offs such as over-alignment that may sacrifice benign content.
• Empirical findings expose vulnerabilities in robustness, privacy leakage, and fairness, urging the need for transparent and scalable evaluation methods.

TrustLLM: A Comprehensive Framework for Evaluating Trustworthiness in LLMs

Introduction and Motivation

Trustworthiness in LLMs is foundational for their responsible deployment in both general and high-stakes domains. Despite remarkable advances in utility, the complexity, scale, and heterogeneity of LLMs impart significant challenges with respect to safety, fairness, privacy, robustness, truthfulness, ethics, transparency, and accountability. "TrustLLM: Trustworthiness in LLMs" (2401.05561) systematically addresses these concerns by introducing a rigorous, multi-dimensional benchmarking suite, providing empirical analysis on 16 prominent LLMs across six quantifiable dimensions, and establishing a taxonomy for a holistic assessment of trustworthy behavior.

Taxonomy and Principles of Trustworthiness

With a literature-driven approach, the authors distill eight core dimensions as pillars of LLM trustworthiness: truthfulness, safety, fairness, robustness, privacy, machine ethics, transparency, and accountability. TrustLLM operationalizes and benchmarks the first six, given the difficulties of scalable quantitative evaluation for transparency and accountability. For each dimension, the paper curates specific datasets, prompt designs, and metrics, spanning over 30 datasets and 18 subcategories, reflecting tasks from misinformation detection and stereotype recognition to adversarial robustness and privacy leakage.

Benchmark Design and Evaluation Protocols

The TrustLLM benchmark is designed with methodological rigor, employing a diverse suite of proprietary and open-weight models (including GPT-4, Llama2, ERNIE, Vicuna, etc.), and spanning a comprehensive array of task archetypes: classification, open/free-form generation, knowledge-intensive QA, sycophancy, privacy probing, out-of-distribution detection, and more. Evaluation is performed with a blend of classifier-assisted automation (e.g., the Longformer classifier for refusal detection), large model evaluation (e.g., using GPT-4 as a meta-judge), and carefully calibrated prompt engineering to minimize prompt-sensitivity artifacts.

Figure 1: The TrustLLM benchmark design, illustrating the six trustworthiness aspects, task and dataset diversity, and evaluation methodology.

Key Empirical Findings

1. Positive Correlation of Trustworthiness and Utility

Experimental results reveal a pronounced, though not absolute, correlation between model utility and trustworthiness scores: state-of-the-art (SOTA) models (GPT-4, Llama2-70b, ERNIE) consistently score high across both axes. However, over-alignment in certain models leads to exaggerated safety, causing benign content to be misclassified as harmful, thus degrading utility.

Figure 2: Trustworthiness rankings of 16 LLMs, highlighting clear stratification between proprietary and open-weight models.

2. Proprietary vs. Open-Weight Gap

Proprietary LLMs dominate overall trustworthiness, but select open-weight models (notably Llama2-70b) demonstrate parity—or even superiority—in specific tasks without auxiliary moderation mechanisms, challenging the narrative that proprietary control is a prerequisite for trust.

3. Over-Alignment and Safety–Utility Trade-off

Certain models, especially Llama2-7b (57% refusal on benign prompts), display a high rate of false positives in harmful content detection. This is largely a byproduct of alignment via rejection learning, which can sacrifice useful responses for the sake of strong safety guarantees.

Figure 3: Illustration of jailbreak attack vectors (left) and the exaggerated safety (right) in leading LLMs.

4. Robustness and OOD Generalization

Robustness to perturbations displays high variance, especially in open-ended or OOD tasks. The best LLMs (GPT-4, Llama2-70b) maintain high F1 and refusal-to-answer (RtA) rates under OOD inputs (>80% and >92%, respectively), while smaller or less-aligned models show catastrophic failures.

Figure 4: Heatmap of the average performance drop under various input perturbations, exposing vulnerabilities in open-weight models.

5. Privacy and Leakage

Despite an apparent awareness of privacy policies, leakage is still observable. For instance, models like GPT-4 will, under multi-shot prompting, reproduce training dataset artifacts (Enron Emails) at non-trivial rates, indicating the persistence of memorization in large-scale pretraining.

Figure 5: Left—privacy awareness under explicit policy prompts; Right—privacy leakage rates on benchmark datasets.

6. Hallucination, Sycophancy, Machine Ethics

State-of-the-art models still hallucinate on open-generation and zero-shot commonsense tasks. Sycophancy remains pervasive and correlates with factual correction failure rates. In ethical evaluations, even the top model (GPT-4) achieves below 70% alignment with human-labeled judgments in ambiguous scenarios.

Figure 6: Types of sycophancy explored in TrustLLM and their operationalization during benchmarking.

Figure 7: Contrasts between implicit and explicit ethical evaluation protocols operationalized in TrustLLM.

7. Stereotypes, Fairness

No evaluated model achieved >65% accuracy in stereotype recognition. While blocking outputs for toxic or disparaging prompts is common, this sometimes comes at the expense of wrongly flagging culturally sensitive or neutral responses as unsafe, highlighting the challenge of balancing inclusivity and fairness with over-censorship.

Figure 8: Stereotype assessment task design, capturing three subtasks: agreement, recognition, and refusal.

8. Model and Technology Transparency

Performance disparities are correlated with transparency: open disclosure of architectures, training data, and alignment strategies is essential for systematic improvement and democratized safety benchmarking.

Open Challenges and Future Research Directions

Prompt Sensitivity and Generalization: Even minor token or format changes in prompts can cause discontinuities in output safety, factuality, and classification, undermining robust deployment.

Certification of Trustworthiness: The lack of scalable, certified guarantees for safety, robustness, and privacy in LLMs—exacerbated by high compute and model complexity—remains unsolved. Advances in cryptographic secure computation, differential privacy for pretraining, and robust optimization offer partial remedies but entail efficiency–security trade-offs.

Cross-Language and Cultural Generalization: TrustLLM evaluations are predominantly English-centric. Strong cultural and linguistic variation may further degrade trustworthiness in non-English deployments, especially for open-weight models with limited multilingual data.

Temporal and Knowledge Drift: Hallucination and truthfulness degrade under temporal shift and knowledge obsolescence. Retrieval-augmented architectures and online model editing (RA-GPT, SELF-INFO) offer potential for mitigation but require evaluation at scale.

Inter-dimensional Conflicts: Optimizing safety often degrades utility and fairness; robustly balancing these trade-offs and designing alignment techniques resistant to jailbreak or replay attacks remains an open problem.

Implications and Prospects for AI Research

The TrustLLM framework sets a practical precedent for multi-aspect, standardized evaluation of LLM trustworthiness. Its empirically motivated dimensions and task granularity have several key implications:

• Industry and Regulatory Benchmarking: TrustLLM provides a rigorous, reproducible suite and leaderboard for both proprietary and open-weight LLM developers to calibrate safety claims and compliance with impending regulatory regimes.
• Model Selection and Fine-Tuning: Insights from correlation between subdimension scores and utility inform more efficient model selection, instruction tuning, and downstream LLM alignment, especially where social and legal risk is high.
• Research on Transparent and Open Safety Techniques: The recommendation to open-source not just models but also alignment and safety methodologies is critical for fostering community-driven innovation and ensuring trustworthy LLM deployment in low-resource or high-assurance contexts.
• Generalization to Multimodal and Agentic Systems: Many principles (e.g., transparency, OOD detection, privacy) readily extend to evaluation of general-purpose multimodal foundation models and agentic architectures, which are likely to exacerbate trust challenges.

Conclusion

TrustLLM advances the field by providing a nuanced, scalable, and multidimensional assessment framework for LLM trustworthiness. Notably, strong empirical results demonstrate that while substantial progress has been made, critical vulnerabilities—including privacy leakage, lack of consistent fairness, and robustness weaknesses—persist, particularly in open-weight deployments. Effective improvement in LLM trustworthiness necessitates not only architectural and training innovation but also systematic transparency, credible benchmarking, and interdisciplinary collaboration at the intersection of AI, security, human–computer interaction, and policy.

The work thus constitutes both a reference taxonomy for evaluating LLM trustworthiness and a call to action for the transparent, systematic development of safe, robust, and socially aligned LLMs.