Debunking the top 5 Defensics fuzz testing myths

Written in coordination with Chris Clark, Defensics product manager

Over the last year, we’ve noticed a rise in Defensics myths. Admittedly, this doesn’t surprise me. Myths abound in technology markets, where facts and figures often stand in contrast to conventional wisdom, and the fuzz testing market is a particularly challenging one to navigate. I suspect this difficulty exists for three main reasons:

• The fuzz testing market is fragmented. From one end of the market to the other, you’ll come across commercial and open source offerings that, for the most part, employ different fuzzing techniques. As a result, when comparing two fuzzing solutions, prospects may not realize they’re actually comparing apples to oranges (or peaches).
• Fuzzing isn’t as well-known as other software testing techniques. Fuzzing was first introduced in 1989, so it’s a software security testing technique that’s been around for a while. But despite its age, it’s not as well-known or mainstream as SAST, DAST, penetration testing, software composition analysis, and others.
• Fuzzing can get pretty technical. Fuzzing is often used by those with a software security background, whether they’re security researchers, white hat hackers, black hat hackers, or security engineers. Understanding the nuances of the various fuzzing offerings in the market can be tough for those who don’t have an extensive security background.

Listed below are just a handful of the myths we’ve run into, and we’re here to debunk them all. Have you run into any that aren’t mentioned here? Let us know.

Join the conversation in the Synopsys Community.

Myth 1: Defensics can’t be integrated into the product development life cycle

Most development, security, and QA teams assume that Defensics won’t fit into the product development life cycle because it’s a black box fuzzer. That’s only partially true.

Although it is a black box fuzzer, Defensics contains workflows that enable it to fit almost any environment from a technological and process standpoint. Whether you employ a traditional SDL or CI development life cycle, Defensics brings fuzz testing into development, allowing you to catch vulnerabilities early and cost-effectively. Got an unconventional development life cycle? We have an experienced professional services team that can help you identify fuzz testing checkpoints, define fuzz testing metrics, and establish a fuzz testing maturity program with Defensics. Let’s talk.

Myth 2: Defensics isn’t cost effective

Defensics aims to locate vulnerabilities, primarily unknown and zero-day, before software is released into the wild. Unlike other fuzzers in the market, Defensics has a deep understanding of the input type, meaning it can identify key weaknesses in the rules that govern communication and deliver tests that exploit those weaknesses. Because Defensics generates tests that are likely to trigger vulnerabilities, users can efficiently and effectively find and fix them before they lead to costly patches and recalls. We have a proven track record that attests not only to the quality of our test suites but also to the quality of results our customers can expect.

The Total Economic Impact of Synopsys Software Testing Tools: Coverity and Defensics, a Synopsys-commissioned report, details how an entertainment and communications technology company avoided remediation expenses of $1.8 million by fuzzing with Defensics over 3 years. For more information on Defensics’ ROI, read the full report.

Myth 3: Defensics isn’t flexible because you’re stuck with predefined test suites

It’s true that Defensics is known for its predefined test suites, and this is because predefined test suites for protocols, file formats, and interfaces is where Defensics put out its first roots. However, Defensics has grown and matured since its inception about 15 years ago.

Today, Defensics is a comprehensive fuzzing solution. Not only does it offer advanced template fuzzers for file formats (Universal Fuzzer) and protocols (Traffic Capture Fuzzer), but we also provide a Defensics fuzzing framework (Defensics SDK) so users can create their own test suites. With Universal Fuzzer, Traffic Capture Fuzzer, or Defensics SDK, users can augment our library of 250+ test suites with test cases and test suites they’ve built for file formats, protocols, and their own custom or proprietary input types. Also, don’t forget about our data sequence editor, which enables users to fine-tune our predefined test suites to capture those corner cases they might come across. This flexibility lets users rely on the same Defensics fuzzing engine and user interface for all their fuzzing needs.

Some may knock Defensics’ prebuilt test suites, but we’ve found that our customers can start fuzzing quicker with them. When provided only fuzzing frameworks, organizations have to write their own test suites, outsourced or in-house, before they can start testing. With this approach, just getting started can take a while, especially if you want quality tests. We’re not asserting that one approach is better than the other. Frankly, it comes down to what users are looking for. Our observations have shown that prebuilt test suites result in quicker time to fuzz and less test suite maintenance headaches for users. These benefits are preferred particularly by organizations and teams who aren’t fluent in fuzzing, don’t wish to specialize in fuzzing, or don’t have a dedicated fuzzing team. Still, advanced users and security teams who are knowledgeable about fuzzing may prefer having more control and flexibility over their test suites by writing their own, which is possible with Defensics SDK.

Myth 4: Defensics isn’t automated

Defensics sports a logical user interface that guides users through each step of the fuzz testing process. Here’s how Defensics works:

1. Select the appropriate software test suite. We support more than 250 protocols, file formats, and other interfaces. In addition, Synopsys employs a team of security researchers and engineers to refine existing test suites and add new ones as input types become available.
2. Point the fuzzer at the target. After selecting the appropriate test suite to run, select the corresponding software target to test. Defensics test suites contain device explorers that automatically scan for test targets. So you don’t have to fish for devices by inputting IP addresses.
3. Begin testing. Once Defensics establishes a successful connection, users can rely on automation to run the tests. Defensics determines the number of layers it needs to connect to and then runs the most effective tests based on the target’s health state. During this period, Defensics provides real-time status updates to users and logs potential vulnerabilities and other relevant data.
4. Run it again. Using the streamlined workflow, users can save their configuration to a test plan or set file. Repeat workflows or use the test plan to drive your test framework via a robust CLI or RESTful API.

And that’s how Defensics employs automation—some apparent and some subtle—throughout its testing process to make advanced fuzz testing easy for anyone.

Myth 5: Defensics lacks monitoring capabilities

When monitoring is mentioned in the context of fuzzing, it also refers to instrumentation. This is the fuzzer’s capability of picking up anomalous behavior in the test target.

Defensics offers instrumentation for the following:

• Valid case
• Syslog
• Agent
• SNMP
• Custom scripting at each step of testing execution

Defensics also offers SafeGuard checkers, the feature that found the infamous Heartbleed vulnerability, which detect the following anomalous behaviors:

• Amplification
• Authentication bypass
• Blind LDAP injection
• Blind SQL injection
• Certificate validation
• Compressed signer’s name in RRSIG record
• Cross-site request forgery
• Cross-site scripting
• Extra cookie compared to valid case
• Heartbleed
• Information leakage
• Insufficient randomness
• LDAP injection in response
• Malformed HTTP
• Remote execution
• SQL injection in response
• Unexpected data
• Unprotected credentials
• Weak cryptography

Fuzz smarter, remediate faster, and release safer with Defensics

In this article, we’ve focused heavily on Defensics myths and what makes them untrue. Now that we’ve discussed what Defensics isn’t, let’s talk about what Defensics is.

Defensics is a comprehensive, powerful, and automated black box solution that enables organizations to effectively and efficiently discover and remediate security weaknesses in software. By taking a systematic and intelligent approach to negative testing, Defensics allows organizations to ensure software security without compromising on product innovation, increasing time to market, or inflating operational costs. Fuzz smarter, remediate faster, and release safer with Defensics.

Want to learn more about Defensics?

Fuzz smarter