LLM Agents can Autonomously Exploit One-day Vulnerabilities (2404.08144v2)

Abstract: LLMs have becoming increasingly powerful, both in their benign and malicious uses. With the increase in capabilities, researchers have been increasingly interested in their ability to exploit cybersecurity vulnerabilities. In particular, recent work has conducted preliminary studies on the ability of LLM agents to autonomously hack websites. However, these studies are limited to simple vulnerabilities. In this work, we show that LLM agents can autonomously exploit one-day vulnerabilities in real-world systems. To show this, we collected a dataset of 15 one-day vulnerabilities that include ones categorized as critical severity in the CVE description. When given the CVE description, GPT-4 is capable of exploiting 87% of these vulnerabilities compared to 0% for every other model we test (GPT-3.5, open-source LLMs) and open-source vulnerability scanners (ZAP and Metasploit). Fortunately, our GPT-4 agent requires the CVE description for high performance: without the description, GPT-4 can exploit only 7% of the vulnerabilities. Our findings raise questions around the widespread deployment of highly capable LLM agents.

• The paper demonstrates GPT-4's ability to exploit one-day vulnerabilities with an 87% success rate when provided CVE details.
• The methodology uses an agent framework that integrates CVE descriptions to compare GPT-4's performance against GPT-3.5, open-source LLMs, and traditional vulnerability scanners.
• The findings underscore critical cybersecurity implications, calling for updated guidelines and ethical safeguards in the deployment of high-capability LLM agents.

Autonomous Exploitation of One-day Vulnerabilities by LLM Agents

Introduction

LLMs have demonstrated substantial advancements in various domains, exhibiting capabilities ranging from software engineering to scientific discovery. This paper investigates the autonomous ability of LLM agents, specifically GPT-4, to exploit one-day vulnerabilities in real-world systems. These one-day vulnerabilities refer to issues disclosed in the Common Vulnerabilities and Exposures (CVE) database that have yet to be patched.

Methodology

The study utilizes a dataset comprising 15 real-world one-day vulnerabilities, including critical severity cases as per CVE descriptions. To assess the capability of LLM agents, the researchers implemented an agent framework leveraging tools and the CVE description provided to GPT-4. The performance was compared against other models, such as GPT-3.5, open-source LLMs, and traditional vulnerability scanners like ZAP and Metasploit.

Figure 1: System diagram of our LLM agent.

Results

The experimental results indicate a significant disparity in capabilities between GPT-4 and other models. GPT-4 exploited 87% of the vulnerabilities when given a CVE description. No other models, including GPT-3.5 and 8 open-source LLM variants, achieved any successful exploitations. This discrepancy underscores the potential of GPT-4 as a highly adept tool in identifying and exploiting cybersecurity vulnerabilities autonomously. Interestingly, removal of CVE descriptions led to a dramatic drop in GPT-4's success rate from 87% to 7%, highlighting the reliance on CVE details for effective exploitation.

Implications

These findings pose critical considerations for cybersecurity practices, emphasizing the need to reassess the deployment strategies of high-capability LLM agents. Moreover, they bring to light the alarming possibility of emergent capabilities in AI models, which can be utilized for both benign and malicious purposes in cybersecurity contexts. Given the current limitations when CVE descriptions are absent, further investigation into enhancing autonomous vulnerability detection capabilities is warranted, potentially incorporating advanced planning mechanisms and subagents.

Future Directions

The study sparks a dialogue on whether LLM agents can be fortified further through enhanced planning and exploration frameworks to better detect vulnerabilities independently. Additionally, the exploration of stronger ethical guidelines and security protections around the use of such models becomes imperative, given their dual-use potential in cybersecurity.

Conclusions

In conclusion, the research successfully demonstrates that GPT-4 possesses the nascent ability to exploit one-day vulnerabilities in real-world systems—a task that has eluded other models tested. This capability, however, heavily depends on the access to detailed vulnerability information, urging a cautious approach to the integration and future deployment of LLM agents across cybersecurity landscapes. The study serves as both a profound demonstration of LLM potential and a call to action for the broader cybersecurity community to address the implications of these advanced computational tools.