· Trending Papers
Emergent Mind

Universal and Transferable Adversarial Attacks on Aligned Language Models

(2307.15043)
Published Jul 27, 2023 in cs.CL , cs.AI , cs.CR , and cs.LG

Abstract

Because "out-of-the-box" LLMs are capable of generating a great deal of objectionable content, recent work has focused on aligning these models in an attempt to prevent undesirable generation. While there has been some success at circumventing these measures -- so-called "jailbreaks" against LLMs -- these attacks have required significant human ingenuity and are brittle in practice. In this paper, we propose a simple and effective attack method that causes aligned language models to generate objectionable behaviors. Specifically, our approach finds a suffix that, when attached to a wide range of queries for an LLM to produce objectionable content, aims to maximize the probability that the model produces an affirmative response (rather than refusing to answer). However, instead of relying on manual engineering, our approach automatically produces these adversarial suffixes by a combination of greedy and gradient-based search techniques, and also improves over past automatic prompt generation methods. Surprisingly, we find that the adversarial prompts generated by our approach are quite transferable, including to black-box, publicly released LLMs. Specifically, we train an adversarial attack suffix on multiple prompts (i.e., queries asking for many different types of objectionable content), as well as multiple models (in our case, Vicuna-7B and 13B). When doing so, the resulting attack suffix is able to induce objectionable content in the public interfaces to ChatGPT, Bard, and Claude, as well as open source LLMs such as LLaMA-2-Chat, Pythia, Falcon, and others. In total, this work significantly advances the state-of-the-art in adversarial attacks against aligned language models, raising important questions about how such systems can be prevented from producing objectionable information. Code is available at github.com/llm-attacks/llm-attacks.

Adversarial prompt elicits harmful behaviors from commercial models like ChatGPT, Claude, Bard, and Llama-2.

Overview

  • The paper investigates automated adversarial attacks that can induce aligned language models like GPT-3 and BERT to produce harmful content.

  • A novel method combining greedy and gradient-based techniques is proposed for generating adversarial prompts that can deceive language models.

  • The adversarial prompts exhibit a high degree of transferability across different language models, including closed-source models like OpenAI's ChatGPT and Google's Bard.

  • Prior to publication, the researchers engaged with AI labs to discuss the ethical implications and shared their findings to help develop better defenses.

  • The study's revelations underscore the need for ongoing efforts to improve the security of language models to resist adversarial manipulations.

Introduction to Adversarial Attacks on Language Models

Language models, such as GPT-3 and BERT, have advanced to a stage where they're increasingly being used in various applications, providing users with information, entertainment, and interaction. At the same time, it's crucial to ensure these models do not generate harmful or objectionable content. Organizations developing these models have put in considerable effort to "align" their outputs with socially acceptable standards. Despite these efforts, certain inputs, known as adversarial attacks, can lead to model misalignment, causing the generation of undesirable content. This article explores a new method that automates the process of creating these adversarial attacks, revealing vulnerabilities in these aligned models.

Crafting Automated Adversarial Prompts

Researchers have proposed a novel adversarial method that exploits the weaknesses in language models and provokes them into generating content that is generally filtered out for being objectionable. Unlike previous techniques, which mainly depended on human creativity and were not highly adaptable, the new method uses a clever combination of greedy and gradient-based techniques to automatically produce adversarial prompts. These prompts include a suffix that, when attached to otherwise innocuous queries, substantially increases the probability that the language model wrongfully responds with harmful content. This method surpasses past automated prompt-generation methods by successfully inducing a range of language models to generate such objectionable content with high consistency.

Transferability of Adversarial Prompts

What makes these findings even more compelling is the high degree of transferability observed. The adversarial prompts designed for one model were found to be effective on others, including closed-source models available publicly, such as OpenAI's ChatGPT and Google's Bard. Specifically designed by optimizing against several smaller LLMs, these adversarial prompts maintain their efficacy when tested on larger and more sophisticated models. This surprising level of transferability highlights a broad vulnerability in LLMs, which raises important questions about the methods used to align them and their robustness against such insidious inputs.

Ethical Considerations and Potential Consequences

As one might expect, the ethical implications of this research are significant. The authors addressed this by engaging with various AI labs and sharing their findings before publication. Introducing these vulnerabilities into public discourse is critical, as understanding these potential attack vectors can lead to better defenses. Nonetheless, it is essential to note that the results also point to the need for a continued search for more secure and foolproof methods to prevent adversarial attacks on language models, which are becoming more integrated into our digital lives.

In conclusion, this study marks a significant step forward in the field of machine learning security. By automating the generation of adversarial attacks and revealing their high transferability between models, it opens new avenues towards strengthening the alignment of language models, ensuring they adhere to ethical guidelines and resist manipulation despite the increasing complexity and the evolving landscape of AI-driven communication.

Subscribe by Email

Get summaries of trending comp sci papers delivered straight to your inbox:

Unsubscribe anytime.

YouTube
References

  1. Demystifying limited adversarial transferability in automatic speech recognition systems. In International Conference on Learning Representations, 2022. https://openreview.net/forum?id=l5aSHXi8jG5.

  2. Generating Natural Language Adversarial Examples
  3. Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
  4. Constitutional AI: Harmlessness from AI Feedback
  5. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pages 2397–2430. PMLR
  6. Evasion attacks against machine learning at test time. In Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2013, Prague, Czech Republic, September 23-27, 2013, Proceedings, Part III 13, pages 387–402. Springer
  7. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM workshop on artificial intelligence and security, pages 3–14, 2017a.
  8. Towards evaluating the robustness of neural networks, 2017b
  9. Are aligned neural networks adversarially aligned?

  10. CarperAI. Stable-vicuna 13b, 2023. https://huggingface.co/CarperAI/stable-vicuna-13b-delta.

  11. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30
  12. Certified adversarial robustness via randomized smoothing. In international conference on machine learning. PMLR
  13. QLoRA: Efficient Finetuning of Quantized LLMs
  14. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 320–335
  15. HotFlip: White-Box Adversarial Examples for Text Classification
  16. Improving alignment of dialogue agents via targeted human judgements
  17. Explaining and Harnessing Adversarial Examples
  18. Gradient-based Adversarial Attacks against Text Transformers

  19. Aligning {ai} with shared human values. In International Conference on Learning Representations, 2021. https://openreview.net/forum?id=dNy_RKzJacY.

  20. Adversarial examples are not bugs, they are features. In Advances in Neural Information Processing Systems (NeurIPS)
  21. Adversarial Examples for Evaluating Reading Comprehension Systems
  22. Automatically Auditing Large Language Models via Discrete Optimization
  23. Pretraining language models with human preferences. In International Conference on Machine Learning, pages 17506–17533. PMLR
  24. Scalable agent alignment via reward modeling: a research direction
  25. Globally-robust neural networks. In International Conference on Machine Learning. PMLR
  26. The Power of Scale for Parameter-Efficient Prompt Tuning
  27. Sok: Certified robustness for deep neural networks. In 2023 IEEE Symposium on Security and Privacy (SP)
  28. Exploring Targeted Universal Adversarial Perturbations to End-to-end ASR Models

  29. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018. https://openreview.net/forum?id=rJzIBfZAb.

  30. Black Box Adversarial Prompting for Foundation Models
  31. Universal adversarial perturbations. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pages 1765–1773
  32. Universal Adversarial Perturbations for Speech Recognition Systems
  33. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744
  34. Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
  35. The limitations of deep learning in adversarial settings. In 2016 IEEE European symposium on security and privacy (EuroS&P), 2016b.
  36. The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only
  37. Cold decoding: Energy-based constrained text generation with langevin dynamics. Advances in Neural Information Processing Systems, 35:9538–9551
  38. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
  39. Intriguing properties of neural networks. In International Conference on Learning Representations
  40. MosaicML NLP Team. Introducing mpt-7b: A new standard for open-source, commercially usable llms, 2023. www.mosaicml.com/blog/mpt-7b. Accessed: 2023-05-05.
  41. Llama 2: Open Foundation and Fine-Tuned Chat Models
  42. The Space of Transferable Adversarial Examples
  43. Universal Adversarial Triggers for Attacking and Analyzing NLP
  44. Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
  45. Jailbroken: How Does LLM Safety Training Fail?
  46. Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery
  47. Fundamental Limitations of Alignment in Large Language Models
  48. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
  49. PromptBench: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts

Show All 49