Training-free Lexical Backdoor Attacks on Language Models (2302.04116v1)

Abstract: Large-scale LLMs have achieved tremendous success across various NLP applications. Nevertheless, LLMs are vulnerable to backdoor attacks, which inject stealthy triggers into models for steering them to undesirable behaviors. Most existing backdoor attacks, such as data poisoning, require further (re)training or fine-tuning LLMs to learn the intended backdoor patterns. The additional training process however diminishes the stealthiness of the attacks, as training a LLM usually requires long optimization time, a massive amount of data, and considerable modifications to the model parameters. In this work, we propose Training-Free Lexical Backdoor Attack (TFLexAttack) as the first training-free backdoor attack on LLMs. Our attack is achieved by injecting lexical triggers into the tokenizer of a LLM via manipulating its embedding dictionary using carefully designed rules. These rules are explainable to human developers which inspires attacks from a wider range of hackers. The sparse manipulation of the dictionary also habilitates the stealthiness of our attack. We conduct extensive experiments on three dominant NLP tasks based on nine LLMs to demonstrate the effectiveness and universality of our attack. The code of this work is available at https://github.com/Jinxhy/TFLexAttack.