Functional testing is, for the most part, positive testing, meaning that the tests are set up to supply valid inputs, and the output is compared against a correct or expected value. Functional testing verifies that everything works correctly under normal circumstances. But what about users or systems that send inputs that make no sense or are out of order? What about attackers who are probing the application for weaknesses?
Fuzzing is negative testing and is complementary to functional testing. It delivers deliberately malformed inputs and checks to see if a failure is triggered. This makes especially good sense for internet-facing applications like BIND 9, which are likely to encounter all kinds of garbage inputs in the real world.
BIND 9 is set up to use two open source fuzzers, namely, American Fuzzy Lop (afl) and libFuzzer. Both of these are in-process, coverage-guided function-level fuzzers. This means that they create test cases (inputs), run those inputs into an entry point function in the application, and see if a process crash ensues. They use code coverage to analyze the code pathway that a given test case follows. Test cases are mutated to create new test cases, and when a new pathway is discovered, a new set of mutations is performed.
Last year, the Synopsys Cybersecurity Research Center team performed DNS protocol fuzzing with Defensics during the development effort leading up to BIND 9.17.7 and uncovered a vulnerability. The BIND 9 team has a long history of collaboration with Defensics, going back at least as far as a vulnerability located in BIND 9.10.0 in 2014.
Defensics performs fuzzing very differently from afl and libFuzzer, but the underlying principle is the same. Defensics uses an internal network protocol data model to create test cases. It knows exactly how each field of each message should look according to the protocol specification, which means Defensics can create test cases that are very realistic but still anonymized in specific ways.
Defensics delivers test cases over the network, in effect working precisely on the exposed attack surface of the application. It monitors the tested application for process crashes but can also be configured to look for memory errors, other types of resource consumption, and other indicators of application health.
As with SAST, different fuzzers find different bugs, so running multiple types of fuzzers can help flush out vulnerabilities and reduce overall risk.
One way that the BIND 9 developers can reassure users their software is clear of obvious vulnerabilities is by publishing the results of these ongoing tests publicly, via “badges” posted in the open repository that link to the latest status. Note the badge in Figure 1 indicating a “passing” status with Coverity.