A security toolbox has a variety of static code analysis tools that may all look alike at first glance but actually serve different purposes.
Buying a house is interesting because it forces you to take a look at everything that you may have taken for granted and ignored. Recently, while I was packing my tools in preparation for a move, I realized that I have eight different hammers in my toolbox. Each hammer serves a different purpose and not all of them include driving nails. Some of these hammers were handed down from my grandpa. I bought others to complete recent projects. As I was packing them up, I had to evaluate whether I still had a use for each one. Moving is a chance to go through your belongings and decide if what you have still works for you. Just like my physical toolbox, my security toolbox has a variety of tools that may all look alike at first glance but actually serve very different purposes.
Every static application security testing (SAST) tool in my security toolbox is different. Directly comparing these tools isn’t a matter of picking out the best one. Instead, it’s a process of finding which one works best for you. Before shopping around for new tools, answer the following questions to shape your selection process.
If you are running a PHP development operation, using a tool that only scans C++ doesn’t make much sense. Some languages like Java or C# have many tools to choose from. On the other hand, languages like Perl only have support from one commercial tool.
Some enterprises have a variety of programming languages that they use in-house. As such, buying one commercial tool that can scan 35 of the most common languages makes sense. Developers within the enterprise may also specialize in a single language and may find better support from an open source offering.
Enterprise support and team tools include features that you never realize you’re missing until you don’t have them at your disposal. A stand-alone code scanner can generate results that are just as good as enterprise enabled tools. However, they may fall short when trying to track vulnerabilities across teams, providing metrics, or facilitating peer review.
Depending on how large your security operation is, paying extra for centralized scanning repositories may be worthwhile. At certain scales, lacking connectivity to a centralized database server may disqualify a tool from deployment.
Some security scanning tools work best when they are provided with the complete source code and libraries required to build an application. These tools may take hours to run and need to be run outside of the regular development process or as part of a task on an automated build server.
These periodic scans provide coverage over an entire application. They can find vulnerabilities that emerge as data is traced through an application. However, the first time a scan is run, they may generate hundreds or thousands of false positives. These need to be filtered out and fed into subsequent scans. This level of comprehensiveness does come at a cost. There is a delay between when a security vulnerability is coded and when it is discovered and resolved.
In real time as code is being written?
To close the feedback loop associated with periodic scans, some tools run on the developer’s workstation. They should either be run before the developer checks code back into the repository or as they type code into their IDE. This real time code scanning may miss some data flow-related issues. However, it provides a better learning experience aiding developers to stop writing vulnerabilities altogether. The downside is that these real time solutions may not be ideal for scanning legacy applications.
I wish I could tell you to not worry about budget and focus on security. However, finding an ideal security tool that costs more than your annual security budget can be disheartening. Then again, patching together a solution of tools to provide adequate coverage isn’t inexpensive either. Having a variety of tools means providing a variety of training and dealing with multiple reporting formats. That complexity takes time which may not be noticeable in a small operation with only a few projects, but that can change as the company grows.
When the training and time costs become noticeable, it may be worthwhile to buy a license or support contract to get those resources back. It also may not be advisable to buy the most expensive scanning software as soon as a purchase is required. Buying less expensive software that still provides value may support real gains to security-minded companies.
When making a decision about software security tools, the closed source vs. open source question is often asked right after “What types of tools should we get?” Instead of ruling out one category or another, find a tool that suits your environment, technical requirements, working style, and budget. Then you can worry about the licensing type.
If you can find open source software that supports your language, is mature enough to not hinder operations, and has acceptable manpower costs associated with self-support as time goes on, an open source solution may be best for you. Otherwise, you may choose a commercial (closed source) solution.
In the end, picking the right SAST tool for the job doesn’t mean picking a winner in the open source vs. closed source software debate. Rather, it means evaluating your organizational, stakeholder, and software requirements and then picking the best solution that will work the most efficiently and effectively toward resolving your firm’s security vulnerabilities. Our white paper Enterprise or Open Source: Which SAST Tool Is Right for You? discusses these factors and offers recommendations.