Software Integrity Blog


How effective is your vulnerability detection methodology?

In order to develop secure software, an organization must take several steps throughout the software development life cycle (SDLC) to ensure that security is built-in from the beginning. Here, we’ll explore these security touchpoints and discuss how they are applied to a piece of software under development to identify, resolve, and prevent vulnerabilities early in the SDLC.

Get to know the vulnerabilities: bugs vs. flaws.

A software vulnerability is a weakness in the design or implementation of the application that, when exploited, can result in a violation of a security policy. It is important to understand that not all software security defects are a result of an implementation error. In fact, roughly only half of all exploited security vulnerabilities in a system are implementation bugs. The rest are a result of an architectural flaw in the design of the software.

For example, we can categorize vulnerabilities such as buffer overflows, race conditions, and injection vulnerabilities into the implementation bug category. Issues such as broken access control and business logic violations are considered architectural flaws. These different types of vulnerabilities require completely different software security testing strategies to identify. Identification of existing vulnerabilities during penetration testing is great—but the ultimate goal should be putting policies and procedures in place that help prevent these issues from making their way into the product in the first place.

Build secure software.

Everyone who has taken a software development course before has seen Boehm’s Curve (the diagram that outlines how bug remediation costs increase further along in the SDLC). It is much more cost effective to find a bug in code review than it is to find a bug during integration. Security vulnerabilities work the same way, and they should be discovered early and prevented from making their way into the deployed product.

The key to defect and vulnerability prevention is to build security into the software from the very beginning. It all begins with training the people who are responsible for building the software and testing the application. Interestingly, many developers straight out of college with computer science degrees have never taken a software security course.

If a new developer is asked to create a Hello World web application that simply asks the user to enter their first and last name, and upon form submission, prints “Hello {First} {Last}” to the screen, there is a very good chance that they will implement it incorrectly and introduce a cross-site scripting vulnerability. Why? It wasn’t taught during their university program, they don’t yet have any real-world experience, and they have no idea what cross-site scripting is. Preventing injection attacks requires a developer to intentionally perform input sanitation and output encoding to the user’s input prior to use. Software security foundations and role-based security courses should be given to everyone involved with building the application—including QA teams.

On an organizational level, it’s essential to integrate the software security touchpoints into every step of the SDLC. Create and treat security requirements as equally critical to the success of an application as the functional requirements. Include security professionals who can help perform a risk analysis of all proposed design decisions into the architecture design reviews. These reviews can help locate architectural flaws early in the development process. And remember, finding them early can save your firm from delayed releases, not to mention time and money.

Before any code gets checked in, perform manual secure code reviews. These code reviews are the first line of defense against implementation defects, and help prevent the introduction of these vulnerabilities into the system. Once the code is buildable, static analysis tools are used to help find additional implementation bugs that the manual code reviews missed.

Have you caught all those vulnerabilities yet?

Once there is a deployed, functional application, it’s time to begin dynamic analysis and manual penetration testing procedures. An automated dynamic analysis of a bank web application can help discover most cases of cross-site scripting. However, it will not flag the fact that customer A can see customer B’s account information, or that a non-administrator account has the ability to forcefully browse to an interface that was meant for administrators only. Why? Automated tools don’t understand the business logic requirements. These types of architectural flaws can usually only be discovered through manual penetration testing, code reviews, and architectural analysis by trained security individuals who have a strong understanding of the business risk and application use cases.

Regular penetration testing is not enough to certify an application as ‘secure.’ In fact, penetration tests are sometimes referred to as “badness-ometers” that simply provide an organization with a scale of how bad their software is. According to Synopsys VP of Security Technology, Gary McGraw, “they provide a reading in a range from ‘deep trouble’ to ‘who knows,’ but they do not provide a reading into the security range at all.

Measure vulnerability detection effectiveness.

All instances of vulnerabilities located during the security activities are then documented and categorized. A successful organization can identify the top 10 categories of vulnerabilities discovered during each iteration of the SDLC. This way, steps can be taken to reduce the likelihood of recurrence.

It’s also important to identify when in the SDLC the defect was discovered (architectural analysis, code review, penetration testing) and the root cause of each vulnerability (developer mistake, requirements not understood, etc.) so processes and procedures can be put in place to prevent the same problem from happening again in the future. The aim is that over time, these vulnerabilities will decrease and that they will be identified earlier in the SDLC.

However, 134 defects discovered last iteration versus 98 defects this iteration is not necessarily an improvement, especially if last iteration the development teams created 10KLOC and this iteration they only created 5KLOC. Consider defect density when judging the effectiveness of the policies and procedures put in place to prevent the defects.

Learn how to implement security measures earlier in your SDLC.

More by this author