JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMs (2402.05668v3)

Abstract: Jailbreak attacks aim to bypass the LLMs' safeguards. While researchers have proposed different jailbreak attacks in depth, they have done so in isolation -- either with unaligned settings or comparing a limited range of methods. To fill this gap, we present a large-scale evaluation of various jailbreak attacks. We collect 17 representative jailbreak attacks, summarize their features, and establish a novel jailbreak attack taxonomy. Then we conduct comprehensive measurement and ablation studies across nine aligned LLMs on 160 forbidden questions from 16 violation categories. Also, we test jailbreak attacks under eight advanced defenses. Based on our taxonomy and experiments, we identify some important patterns, such as heuristic-based attacks could achieve high attack success rates but are easy to mitigate by defenses, causing low practicality. Our study offers valuable insights for future research on jailbreak attacks and defenses. We hope our work could help the community avoid incremental work and serve as an effective benchmark tool for practitioners.

• The paper presents the first large-scale taxonomy and evaluation of 13 jailbreak methods, revealing high attack success rates across multiple LLMs.
• It employs a rigorous framework using multi-label GPT-4 judging and a systematic forbidden prompt dataset covering 16 violation categories.
• Findings underscore the transferability of attacks and expose significant defense gaps in models with advanced alignment and safety protocols.

Comprehensive Assessment of Jailbreak Attacks Against LLMs: A Technical Evaluation

Overview and Motivation

The proliferation of LLMs in real-world systems has accelerated efforts to align output with ethical, legal, and societal standards. Despite substantial advancements in alignment, a persistent challenge remains: adversaries continue to develop "jailbreak" attack techniques that circumvent safety measures and induce LLMs to produce outputs violating content guidelines or embedded policies. The paper "JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMs" (2402.05668) presents the first large-scale, systematic evaluation and taxonomy of jailbreak attacks spanning 13 state-of-the-art methods, 16 violation categories, and six widely deployed LLMs.

Figure 1: Examples of contemporary jailbreak attack methods, including those employing jailbreak prompts and those exploiting generation settings.

Methodological Framework

Jailbreak Attack Taxonomy

The paper structures jailbreak attacks into four primary classes, reflecting both the prompt construction process and level of system access:

• Human-based: Prompts crafted by humans or discovered in the wild (e.g., "DAN" prompts, Developer Mode variants).
• Obfuscation-based: Prompts altered by encoding or translation (text in Base64, low-resource languages, synthetic combination strategies).
• Optimization-based: Prompts discovered/optimized via algorithms (genetic algorithms, coordinate descent, or multi-agent strategies).
• Parameter-based: Attacks leveraging only decoding/generation parameter manipulation without prompt alteration.

Measurement and Evaluation Pipeline

A meticulous experimental design underpins the empirical analysis. This involves:

• A dataset of 160 forbidden questions across 16 violation types, synthesized using a systematic prompt generation approach, ensuring comprehensive policy coverage.

  Figure 2: The prompt used to generate forbidden questions per violation category, enabling construction of a broad forbidden dataset.

• Attack success rates (ASR) computed for each (method, LLM, violation category) triple.
• Success adjudication using an advanced LLM judge (GPT-4), provided with nuanced labeling prompts employing multi-label criteria (rephrasing, informativeness, and factual answer provision).

Experimental Analysis

Direct and Transfer Attacks

• Direct jailbreak attacks reveal high overall ASRs, consistently exceeding baseline refusal rates. For multiple LLMs—most notably ChatGLM3, Vicuna, and GPT-3.5—optimized/jailbreak prompt sets (both human-based and optimization-based) achieve ASR > 0.80 in many categories.
• Notably, parameter-based attacks (modulating decoding/generation hyperparameters) enable illicit generations even without adversarial prompts, exposing new attack surfaces in models with strong decoding flexibility.
• Obfuscation-based jailbreaks (e.g., Zulu, Base64) are model-specific: highly effective on models with strong multilingual capability or lenient text processing (GPT-3.5, GPT-4), but poorly transferable.

  Figure 3: Heatmap of average direct attack ASR across LLMs, methods, and violation categories; highest vulnerability observed for optimization-based and parameter-based methods.

• In transfer scenarios, prompts developed on a highly vulnerable model (Vicuna) are reused on others, retaining significant attack potency. Both AutoDAN and TAP exhibit robust transferability, indicating shared alignment blind spots across architectures.

  Figure 4: Fine-grained transfer attack ASR heatmap, illustrating method-policy correlations across five LLMs.

Violation Category Analysis

Several violation categories—Political Activities, Well-being Infringement, and AI Usage Disclosure—remain highly susceptible to jailbreaks, even though providers claim explicit policy coverage. Inverse, Physical Harm, Terrorist Content, and Illegal Activities remain more resistant, but ASR is still non-trivial and increases significantly under attack.

Figure 5: Category-specific breakdown of direct attack ASR by method, highlighting that policy coverage does not equate to robust defense.

Token and Cost Efficiency

• Human-based and optimization-based attacks have high prompt token lengths (often > 600), reflecting the tendency for complex, verbose prompts to induce failure of safety filters.
• Obfuscation-based and parameter-based methods are more token efficient, with the latter requiring only the bare forbidden question and achieving high ASR via sampling.
• There is a demonstrated trade-off between attack efficacy and efficiency: optimization- and parameter-based methods can be computationally expensive (AutoDAN, GCG), whereas human/obfuscation approaches are rapid but, in some categories/LLMs, less reliable.

Figure 6: Distribution of average token counts for various jailbreak methods, showing drastic differences between approaches.

Evaluation Methodology

• The paper critiques prior ASR determination methods (string match heuristics, single-label judges) for leading to high misclassification (false positive/negative) rates. By leveraging a multi-label LLM-based judge and benchmarking it against human annotation (Krippendorff's alpha = 0.87, >94% agreement), the authors demonstrate superior evaluation fidelity.

  Figure 7: The nuanced multi-perspective prompt employed for GPT-4-based response judging.

Implications and Recommendations

Alignment and Policy Gaps

The results indicate all major LLMs—even those with documented, comprehensive alignment pipelines (e.g., RLHF, red teaming)—remain highly vulnerable to jailbreaks. Notably, the gap between stated policy coverage and empirical defense is pronounced, especially for high-risk or politically sensitive categories.

Adversarial and Transfer Robustness

• Attacks based on prompt optimization and sampling hyperparameters are consistently robust and, critically, transfer effectively across both model families and LLM service providers.
• Current defense strategies insufficiently address the diversity and adaptiveness of adversarial prompt construction. Prompt-level and decoding-level safeguard circumvention remains practical at scale.

Operational and Economic Considerations

• The token-intensive nature of many current successful attacks (except parameter-based/obfuscation) has cost implications for adversaries and defenders relying on API-call-based LLM access.
• Transferability of attacks means backdoor/jailbreak discoveries on one (open) model directly threaten others, especially in cases where architectures or pretraining data overlap.

Challenges and Open Problems

• Measuring alignment efficacy: Current static policy documentation does not map to empirical resistance to adversarial queries, and automated, dynamic red-teaming pipelines remain immature.
• Defense effectiveness: External filters (moderation LLMs), context distillation, and reward-model fine-tuning require further research to address adversarial generalization.
• Prompt dataset curation: The on-going development and public sharing of effective wild jailbreak prompts pose a moving target for defenders.

Future Directions

• The development of standardized, dynamic jailbreak evaluation benchmarks (as established by this paper) is required for ongoing model assessment post-release.
• Research into adaptive, generative defense systems (e.g., adversarial training on dynamically harvested jailbreak prompts, or meta-prompting filters) appears urgent.
• The intersection of policy, legal compliance, and LLM architecture necessitates new models of formal verification and interpretability for safety mechanisms.

Conclusion

"JailbreakRadar" demonstrates that contemporary LLM alignment is systematically insufficient against a wide spectrum of jailbreak attacks. The comprehensive taxonomy, empirical evaluation, and methodological rigor highlight major weaknesses that persist across architectures, violation types, and prompt complexities. The report calls for materially more robust, adaptive, and holistic defense strategies—spanning prompt-level, decoding-level, and alignment-level interventions—if regulatory and ethical standards are to be enforced in open-access LLM deployments.