ImgTrojan: Jailbreaking Vision-Language Models with ONE Image (2403.02910v3)
Abstract: There has been an increasing interest in the alignment of LLMs with human values. However, the safety issues of their integration with a vision module, or vision LLMs (VLMs), remain relatively underexplored. In this paper, we propose a novel jailbreaking attack against VLMs, aiming to bypass their safety barrier when a user inputs harmful instructions. A scenario where our poisoned (image, text) data pairs are included in the training data is assumed. By replacing the original textual captions with malicious jailbreak prompts, our method can perform jailbreak attacks with the poisoned images. Moreover, we analyze the effect of poison ratios and positions of trainable parameters on our attack's success rate. For evaluation, we design two metrics to quantify the success rate and the stealthiness of our attack. Together with a list of curated harmful instructions, a benchmark for measuring attack efficacy is provided. We demonstrate the efficacy of our attack by comparing it with baseline methods.
- Flamingo: a visual language model for few-shot learning. ArXiv preprint, abs/2204.14198.
- Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966.
- Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419.
- Sharegpt4v: Improving large multi-modal models with better captions. ArXiv preprint, abs/2311.12793.
- Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.
- Gradient-based adversarial attacks against text transformers. arXiv preprint arXiv:2104.13733.
- Sleeper agents: Training deceptive llms that persist through safety training. arXiv preprint arXiv:2401.05566.
- Obelics: An open web-scale filtered dataset of interleaved image-text documents.
- M33{}^{3}start_FLOATSUPERSCRIPT 3 end_FLOATSUPERSCRIPTIT: A large-scale dataset towards multi-modal multilingual instruction tuning. ArXiv preprint, abs/2306.04387.
- Red teaming visual language models. arXiv preprint arXiv:2401.12915.
- Improved baselines with visual instruction tuning.
- Improved baselines with visual instruction tuning. arXiv preprint arXiv:2310.03744.
- Visual instruction tuning. ArXiv preprint, abs/2304.08485.
- Visual instruction tuning. arXiv preprint arXiv:2304.08485.
- Autodan: Generating stealthy jailbreak prompts on aligned large language models. arXiv preprint arXiv:2310.04451.
- Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860.
- OpenAI. 2023. Gpt-4v(ision) system card.
- Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318.
- Visual adversarial examples jailbreak large language models. arXiv preprint arXiv:2306.13213.
- Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 8748–8763.
- Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. ArXiv preprint, abs/2111.02114.
- Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models.
- How many unicorns are in this image? a safety evaluation benchmark for vision llms. arXiv preprint arXiv:2311.16101.
- Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575.
- Jailbroken: How does llm safety training fail? arXiv preprint arXiv:2307.02483.
- Watch out for your agents! investigating backdoor threats to llm-based agents. arXiv preprint arXiv:2402.11208.
- Weak-to-strong jailbreaking on large language models. arXiv preprint arXiv:2401.17256.
- Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592.
- Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
