Prompt Fuzzing for Fuzz Driver Generation

Writing high-quality fuzz drivers is time-consuming and requires a deep understanding of the library. However, the performance of the state-of-the-art automatic fuzz driver generation techniques leaves a lot to be desired. Fuzz drivers, which are learned from consumer code, can reach deep states but are restricted to their external inputs. On the other hand, interpretative fuzzing can explore most APIs but requires numerous attempts in a vast search space. We propose PromptFuzz, a coverage-guided fuzzer for prompt fuzzing that iteratively generates fuzz drivers to explore undiscovered library code. To explore API usage in fuzz drivers during prompt fuzzing, we proposed several key techniques: instructive program generation, erroneous program sanitization, coverage-guided prompt mutation, and constrained fuzzer fusion. We implemented PromptFuzz and evaluated its effectiveness on 14 real-world libraries, comparing it against OSS-Fuzz and the state-of-the-art fuzz driver generation solution (i.e., Hopper). The experiment results demonstrate that the fuzz drivers generated by PromptFuzz achieve higher branch coverage that is 1.61 times greater than that of OSS-Fuzz and 1.67 times greater than that of Hopper. In addition, the fuzz drivers generated by PromptFuzz successfully detect 33 true bugs out of a total of 44 crashes, which were previously unknown, and 27 of these bugs have been confirmed by the respective communities.