Hyperfuzzing: black-box security hypertesting with a grey-box fuzzer (2308.09081v1)

Abstract: Information leakage is a class of error that can lead to severe consequences. However unlike other errors, it is rarely explicitly considered during the software testing process. LeakFuzzer advances the state of the art by using a noninterference security property together with a security flow policy as an oracle. As the tool extends the state of the art fuzzer, AFL++, LeakFuzzer inherits the advantages of AFL++ such as scalability, automated input generation, high coverage and low developer intervention. The tool can detect the same set of errors that a normal fuzzer can detect, with the addition of being able to detect violations of secure information flow policies. We evaluated LeakFuzzer on a diverse set of 10 C and C++ benchmarks containing known information leaks, ranging in size from just 80 to over 900k lines of code. Seven of these are taken from real-world CVEs including Heartbleed and a recent error in PostgreSQL. Given 20 24-hour runs, LeakFuzzer can find 100% of the leaks in the SUTs whereas existing techniques using such as the CBMC model checker and AFL++ augmented with different sanitizers can only find 40% at best.