How effective is your vulnerability detection methodology?
When you use the right vulnerability detection methodology at the right time, you can decrease vulnerabilities and identify them earlier in the SDLC.
To develop secure software, an organization must take several steps throughout the software development life cycle (SDLC) to ensure that they build security in from the beginning. Here, we’ll explore these security touchpoints and discuss how to apply them to software under development to detect, resolve, and prevent vulnerabilities early in the SDLC.
Get to know the vulnerabilities: Bugs vs. flaws
A software vulnerability is a weakness in the design or implementation of the application that, when exploited, can result in a violation of a security policy. Not all software security defects are the result of an implementation error. In fact, roughly half of all exploitable security vulnerabilities in a system are the result of architectural flaws in the design of the software.
Vulnerabilities such as buffer overflows, race conditions, and injection vulnerabilities belong to the “implementation bug” category. By contrast, issues such as broken access control and business logic violations are architectural flaws. These different types of vulnerabilities require completely different software security testing strategies to identify. Vulnerability detection during penetration testing is great—but the ultimate goal is to put in place policies and procedures that detect these issues early and stop them from making their way into the product in the first place.
Build secure software
Everyone who has taken a software development course has seen Boehm’s curve (which shows how bug remediation costs increase further along in the SDLC). It is much more cost-effective to find a bug in code review than it is to find a bug during integration. Security vulnerabilities work the same way. The goal of effective vulnerability detection is to discover them early and keep them out of the deployed product.
The key to preventing defects and vulnerabilities is to build security into the software from the very beginning, which means detecting them throughout the SDLC. It all begins with training the people responsible for building the software and testing the application. Interestingly, many developers with computer science degrees have never taken a software security course.
If you ask a new developer to create a “Hello World” web application that asks the user to enter their first and last name and, on form submission, prints “Hello {First} {Last}” to the screen, the developer will probably implement it incorrectly—and introduce a cross-site scripting (XSS) vulnerability. Why? They didn’t learn about XSS in their degree program, and they don’t yet have much real-world experience, so they have no idea what XSS is. To prevent injection attacks, a developer must intentionally perform input sanitation and output encoding before using the user’s input. It’s best to offer software security foundations and role-based security courses to everyone involved with building the application—including QA teams.
On an organizational level, effective vulnerability detection requires integrating the software security touchpoints into every step of the SDLC. Create security requirements, and treat them as equally critical to the success of an application as the functional requirements. During architecture design reviews, include security professionals who can help perform a risk analysis of all proposed design decisions. These reviews can identify architectural flaws early in the development process, which can help prevent delayed releases, not to mention saving time and money.
Before developers check in any code, perform a manual secure code review. Code reviews are the first line of defense against implementation defects and help prevent the introduction of these vulnerabilities into the system. Once the code is buildable, use static analysis tools to detect additional implementation bugs that the manual code reviews missed.
Have you caught all those vulnerabilities yet?
Once you have a deployed, functional application, it’s time to begin dynamic analysis and manual penetration testing. Automated dynamic analysis of a web application can discover most cases of cross-site scripting. However, it will not flag the fact that customer A can see customer B’s account information, or that a nonadministrator account can forcefully browse to an interface meant for administrators only. Why? Automated tools don’t understand business logic requirements. You can usually discover these types of architectural flaws only through manual penetration testing, code reviews, and architectural analysis by trained security individuals who have a strong understanding of the business risk and application use cases.
Regular penetration testing is not enough to certify that an application is “secure.” In fact, some people refer to penetration tests as badness-ometers, which tell an organization how bad their software is, not how secure it is. According to former Synopsys VP of security technology Gary McGraw, “They provide a reading in a range from ‘deep trouble’ to ‘who knows,’ but they do not provide a reading into the security range at all.”
Measure vulnerability detection effectiveness
The next step is to document and categorize all instances of vulnerabilities found during the security activities. A successful organization will identify the top 10 categories of vulnerabilities discovered during each iteration of the SDLC. This way, they can take steps to reduce the likelihood of recurrence.
It’s also important to identify when in the SDLC the defect was discovered (architectural analysis, code review, penetration testing) and the root cause of each vulnerability (developer mistake, requirements not understood, etc.). That way, you can put in place processes and procedures to prevent the same problem from happening again in the future. Over time, these vulnerabilities will decrease, and you’ll be able to identify them earlier in the SDLC.
Note that 134 defects discovered in the last iteration versus 98 defects this iteration is not necessarily an improvement, especially if the development team created 10K LOC in the last iteration and only 5K LOC in this one. Consider defect density when judging the effectiveness of the policies and procedures you’ve put in place to prevent defects.
