Finding software security flaws at scale
So you know the difference between bugs and flaws and you know you can use techniques like threat modeling and architecture risk analysis to find those flaws. But those techniques can be difficult to scale across the enterprise as they require deep design and software security expertise. And yet, doing no type of design analysis for a large percentage of your applications because it’s difficult to scale just means you are unaware of the numerous security architecture flaws that likely exist in your software. Fortunately, there is something you can do that can not only scale across your enterprise, but can also provide valuable feedback about the flaws that exist in the software you are building.
Where do I begin?
If you are not specifically looking for flaws in your software, you probably don’t know what flaws exist in your software. And if you don’t know what flaws exist in the software you are building, how do you know what to look for? One option is to look inside your own organization for clues. Identify the common defects found by other techniques (e.g., code review or penetration testing) and see if a change in the design of the software could avoid some or all of those defects. Here’s a real-life example of what one organization did to reduce the frequency of cross-site scripting bugs.
A second option is to look outside of your organization at common flaws identified by other organizations.
Don’t try to boil the ocean.
Doing an architecture risk analysis of a piece of software requires among other things, a deep understanding of the software, its interactions with other systems, and a thorough understanding of the risks to the business. There are numerous steps involved in order to be proficient at doing architecture risk analysis. But everyone has to start somewhere.
An extremely valuable place to start looking for design flaws is how the software makes use of security controls. Examples of security controls are authentication, authorization, cryptography, auditing, and logging. For any control you are reviewing, there are a few things you want to try and find out:
- Is the control weak?
- Is the control missing?
- Is the control inappropriate?
- Can the control be bypassed?
Is there more you might want to know? Absolutely! But, remember the goal. You’re trying to scale this across the enterprise, AND you are just getting started. With that goal in mind, start your analysis with just a few security controls. You can add more controls to your analysis and more areas of interest to be analyzed over time, but start small until you create whatever artifacts or checklists you need to be efficient.
Define what “success” looks like.
You’re about to spend time and money trying to identify flaws in your software. Make sure you manage expectations of what it means to have this exercise be viewed as a success. Maybe it is just the creation of your own Top N flaws list. Maybe it is identifying one or two defects that exist across a large number of applications that were not found though a code review and/or penetration test that was performed on that application. Maybe it is being able to do this analysis in such a way that it can be injected into the design phase of all your applications.
But, remember one very important fact. This analysis, like many other types of security analysis techniques, doesn’t really tell you that your software is good or safe. Just because you did not find some defect does not mean the software is free of defects.
The best time to do a design review was at the beginning of your project. The next best time is right now.
If you had a choice, you probably would have looked for flaws in the design of your software in the early stages of the development process. For whatever reason, that doesn’t always happen. Unfortunately, any flaws that existed when the software was first built are likely still there, just waiting for some malicious entity to take advantage of that flaw (assuming it is not already being taken advantage of…).
So, there’s no better time than the present to look over your software portfolio and see if some flaw is routinely being designed into your software. If you find there is one or more, now seems like a good time to stop doing that.
