Unless you build your own AppSec tools, you need to know how to choose an application security vendor and whether to opt for individual tools or a suite.
If you want to build something properly, you need the right tools.
Not only is that true of houses, but it’s also true of things built with, and powered by, software. Computer code may be ephemeral—users don’t see it or touch it. But if it’s defective, the products that rely on it won’t work correctly. Just as faulty plumbing and wiring inside the walls of a house can endanger its occupants, code with vulnerabilities and flaws can put users’ privacy and even physical safety in danger.
Which raises the obvious questions: What are the right tools to make software safe and secure? How do you choose between application security vendors? And should you opt for individual tools or go for a whole suite?
Gartner addresses those questions in a recent report titled Hype Cycle for Application Services, 2019 (July 2019).
What belongs in a software security toolkit? There is consensus that no all-in-one, magical tool will help developers find and fix bugs and other defects throughout the SDLC (software development life cycle). But there is ongoing discussion about the strengths, weaknesses, and usefulness of individual tools.
Among them, IAST (interactive application security testing) is one of the newer kids on the block.
“Traditional SAST and DAST technologies, while well-established, are known to require significant effort to run and tune such that they find vulnerabilities without high false-positive or false-negative rates,” the Gartner report said. “IAST seeks to remedy these by using an instrumentation approach for higher combined sensitivity and speed.”
According to the research, “IAST should be considered for high-assurance testing by IT and DevOps organizations that develop their own applications.”
Another tool that should be mandatory is SCA (software composition analysis), which is devoted to identifying open source components in software. SCA helps developers find open source code with known vulnerabilities and remedy potential licensing conflicts.
Next question: Where to get the tools? There are two basic options: Build your own custom versions or buy/lease a suite of varied technologies from a single application security vendor.
There are pros and cons to both.
If an organization creates its own toolkit, it can build exactly what it needs. It doesn’t have to take tools made for a general market and try to tweak them for its own, perhaps unique, purposes.
But it makes little sense for a homeowner to create a specialized set of carpentry, plumbing, or electrical tools to build and maintain a house. Likewise, it doesn’t make sense to create the kinds of application security testing (AST) tools that multiple specialized vendors have already created.
Meera Rao, senior principal consultant at Synopsys, noted what is true of most tools and systems: Building it is only the beginning.
“Building a tool is easy. But customizing the tool, maintaining the tool, and keeping the tool up to date with the latest and greatest technology changes has been and will always be a huge challenge,” she said.
And of course, you have to make sure that your custom tools play well with others. “I have seen several organizations that built versions of static analysis tools, and when new languages and frameworks came along, they couldn’t keep up with them. Slowly and over time, they lost their core developers who had built these tools, and these tools are sitting somewhere on the shelf,” she said.
Which leads to the next decision: If an organization is going to purchase or lease tools, should it use one vendor for an entire suite of tools? Or should it pick and choose the individual tools it considers the best from multiple application security vendors?
Neither approach is perfect. “Using multiple vendors can allow organizations to pursue best-of-breed technologies in each category,” the Gartner report said, “but the drawback is they often require learning different systems, as well as using separate dashboards to manage testing and application risk across the enterprise. They also must integrate the various solutions into the SDLC.”
In most cases, an AST suite that includes static analysis, dynamic analysis, SCA, and IAST integrates them all within a single enterprise console and reporting framework.
The potential downside of that option, the report said, is that “AST suites may be particularly strong in one technology but lacking in another.”
Rao agrees, saying that after 11 years working in application security, “I have rarely seen an organization using all AST tools from the same vendor. Each one has a tool that is their flagship product. My suggestion has always been to do a tool bake-off and use the best.”
But she acknowledges that it is crucial to make sure they don’t conflict with one another.
An organization building an application security testing suite should “make sure to abstract any customizations as much as possible so when they replace a tool or integrate a new tool, the changes are minimal and have little to no impact to the development teams,” she said, adding that different types of report formatting can cause conflicts as well.
“One tool may provide results in PDF and XML formats, another tool may provide them in PDF and JSON,” she said. “Still a third tool might provide PDF and HTML reports. This poses a huge challenge when it comes to customization for common activities such as defect tracking, updating metrics dashboards, and breaking the build.
“So when piloting an AST tool, identify whether the tool supports different report formats.”
Rao provided a checklist that can help organizations select tools that will work best for their development team. Key attributes to consider include:
Accuracy and speed. Without these, the chances of a tool being adopted successfully are nil. Poor accuracy causes developers to waste time chasing down false positives. Slow assessments cause developers to wait hours or even days for results.
Automation. This is key to helping developers balance the competing pressures of speed and security without requiring deep security domain expertise.
Bug scans. Tools that identify common quality and security issues give developers a chance to remedy them before passing their code along.
Hi-fi results. When testing tools provide results with high fidelity, they reduce a mountain of potential risks to a manageable list and point the developer to fixes that can affect multiple instances of shared code at once.
Magic of multiples. Different types of security testing tools fit into the SDLC in different ways and can identify different types of potential problems.
Learning while you work. The best tools allow developers to learn and improve as they do their work. Also, developers respond to practical examples. Tools must offer tangible, actionable advice.
Simplify, simplify. Developers may be more motivated to try new things and correct mistakes if they can solve defects on their own before anyone else sees their code. The easier testing tools are for developers to use, the better the chances that they will take personal responsibility for fixes.
Stay on the inside. Developers traditionally prefer to stay within their own environment. Tools that sit outside the development toolchain yield results that get lost or ignored.
Positive reinforcement. Developers, managers, and executives want to see progress. Reporting capabilities should showcase results and allow managers to give developers an incentive to keep improving.
Gartner, Hype Cycle for Application Services, 2019, 25 July 2019, Jim Longwood
Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, express or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
Taylor Armerding is an award-winning journalist who left the declining field of mainstream newspapers in 2011 to write in the explosively expanding field of information security. He has previously written for CSO Online and the Sophos blog Naked Security. When he’s not writing he hikes, bikes, golfs, and plays bluegrass music.