Posted by Taylor Armerding on October 5, 2018
You’ve probably seen the commercials. Different situations but always the same theme. In one of them, a guy tells his neighbor, “I need to get my roof repaired. Do you know any contractors?”
“Uh, yeah, I might,” the neighbor replies.
“Great,” says the first guy. “Can you do a free background check on him for me, get me some additional quotes, research the average price for the job in this area, and book the job for two weeks from now…on a Wednesday? No, Tuesday—yeah, Tuesday’s better.”
At which point he walks off, leaving the neighbor with his mouth agape.
The pitch, of course, is that while it would be both crazy and rude to make such a demand of a neighbor, there’s an app for that—to help you navigate the uncertain, sometimes treacherous world of finding a quality building contractor.
The BSIMM—Building Security In Maturity Model—is not an app. It’s an annual report, now in its ninth iteration (BSIMM9). But when it comes to the even more treacherous world of developing an effective SSI—software security initiative—it is as good as an app—likely the best tool available to cut through at least some of the confusion and uncertainty, which is massive.
Indeed, besides constant and evolving cyber threats, the number of vendors in the software security industry is information overload all by itself. At this year’s RSA conference in April, there were more than 600 vendors in the exhibit halls, virtually all of them pitching security solutions.
As Sammy Migues, principal scientist at Synopsys, who has been one of the authors of the BSIMM report since the beginning, put it, “From a consumer perspective, the vendor marketplace is fragmented; you have to know exactly what you want so that you can get the best parts from multiple vendors and assemble them yourself.
“Tool output is often useful only to those who already know the answer,” he added, “not to the people who don’t understand security issues.”
There is no simple answer to bridge that knowledge gap, and the BSIMM doesn’t pretend that it can close it entirely. It makes it clear up front that it is not a “how to” on developing an SSI. Nor is it even a “what to do.”
But it is very much a “what’s happening now” guide—based on observations of 120 participating companies in eight verticals: financial, independent software vendors (ISVs), tech, healthcare, Internet of Things (IoT), insurance, cloud, and retail.
Those observations cover 116 SSI activities, grouped into 12 practices, which, in turn, belong to four domains: Governance, Intelligence, SSDL (Secure Software Development Lifecycle), and Deployment.
The data collected and organized are available to any organization for free—BSIMM reports are licensed under the Creative Commons Attribution-ShareAlike—and essentially report what the participating organizations are doing and what tools they are using to enable their SSIs. In other words, you can see what activities and tools are already working, or perhaps not working, for others in your industry.
Which can take you a long way toward knowing what to do. Also scattered through the practices are references to particular tools to improve security, which include those for automation, various types of software analysis, fuzz testing, and penetration testing.
Still, even though SSI activities are covered in detail, there are no specific recommendations on exactly which tools to use, or when, or which vendors might be better qualified than others.
Why not? Dr. Gary McGraw, vice president of security technology at Synopsys, who, with Migues, has been one of the authors of every BSIMM report, said that was actually one of the original goals.
“The idea in the initial days was that we could gather information and tell people what to do. As 10 years have shown, that isn’t possible,” he said.
“We’re not saying this is a bunch of disorganized data. But even among the mature firms, there is no one way they do it. They don’t even all use the same static analysis tool.”
Still, there is obvious value in such a comprehensive list of SSI activities.
“For example, lots of other people are doing SAST [static application security testing], using external pen testers, and using black box security tools in the QA [quality assurance] process,” Migues said.
“However, the BSIMM doesn’t always make them smarter about understanding which products really work and which products work in a way that’s right for them over time.
“They really need to do bake-offs,” he said. “They need to ask hard questions about what the products actually accomplish and what’s the total cost of ownership for getting that done, including the disruption to everyone’s day.”
He added that evaluating and recommending specific vendors “is Gartner’s and Forrester’s job. We wouldn’t want to attempt that.”
McGraw said there is definitely a “should do” element to the BSIMM, but it is global, and not specific to a single firm or even a single industry. That, of course, is to build more secure software.
“We’re saying, ‘Here’s what you should do for the world, all together,’” he said.
The BSIMM’s third author, Jacob West, vice president of cloud operations at Oracle, acknowledges the difficulty of creating an SSI. “Organizations still face tough questions around how and when to deploy tools as part of their software security initiatives,” he wrote in a prologue to this year’s edition. “However, armed with the BSIMM, they no longer suffer the uncertainty of answering those questions in a near vacuum.”
Get the latest Software Integrity news, thought leadership, and more.