Software Integrity


BSIMM6 brings science to software security

The sixth iteration of the Building Security In Maturity Model project is a tool you can use as a measuring stick for software security initiatives.

By now, you should have heard about the Building Security In Maturity Model (BSIMM) project, especially if you are a software security person. Maybe you’ve even downloaded a copy of your own to peruse (it’s free under the Creative Commons license).

Either way, it’s time to get a new copy, because BSIMM6 has just been released. Remember, because BSIMM is completely data driven, the BSIMM6 document is different than what you may have read in the past. That’s how science goes.

In this short piece, we’re going to focus on BSIMM6 facts and figures. The numbers are about real software security initiatives doing real work to secure the software that you use every day. This is no ephemeral top ten list from the bug parade. This is a set of facts about the real state of commercial software security on planet Earth.

Who is the BSIMM community anyway?

The BSIMM project is spearheaded by three co-authors (the same three who wrote this piece you’re reading now). We are directly involved in gathering data in person from each of the BSIMM firms. The data we gather directly through observation describes the work of 78 software security initiatives, from firms including: Adobe, Aetna, ANDA, Autodesk, Bank of America, Black Knight Financial Services, BMO Financial Group, Box, Capital One, Cisco, Citigroup, Comerica, Cryptography Research, Depository Trust and Clearing Corporation, Elavon, EMC, Epsilon, Experian, Fannie Mae, Fidelity, F-Secure, HP Fortify, HSBC, Intel Security, JPMorgan Chase & Co., Lenovo, LinkedIn, Marks & Spencer, McKesson, NetApp, NetSuite, Neustar, Nokia, NVIDIA, PayPal, Pearson Learning Technologies, Qualcomm, Rackspace, Salesforce, Siemens, Sony Mobile, Symantec, The Advisory Board, The Home Depot, TomTom, Trainline, U.S. Bank, Vanguard, Visa, VMware, Wells Fargo, and Zephyr Health.

By the way, we added a data freshness constraint to the model with BSIMM6. We now exclude measurements older than 42 months to better align with business cycles. This requirement caused 21 firms to be removed when we created BSIMM6.

What is the BSIMM?

The BSIMM is a measuring stick for software security. The best way to use the BSIMM is to compare and contrast your own initiative with the data contained in the model, which show what other organizations are doing. You can then identify goals and objectives of your own and look to the BSIMM to determine which further activities make sense for you.

The BSIMM is not a software security methodology. To make this clear, consider that the BSIMM can be used to measure Microsoft’s SDL, but it is by no means a replacement for the Microsoft SDL.

BSIMM by the numbers

At this stage of the game, the BSIMM describes the work of 1,084 full-time software security professionals who are attempting to help 287,006 developers build more secure software. They have help from the “satellite,” which is made up of developers, architects, and people in the organization directly engaged in and promoting software security, but not as full-time software security group (SSG) members.

Ever wonder how big your firm’s SSG should be? We wonder also, but we do know how big the SSGs are at 78 firms. If we average all the ratios of SSG size to Development size, we get an “SSG average of averages” of 1.51% (median 0.7%). Table 1 below contains some additional interesting data.

BSIMM6 Real-world data
Table 1

Table 2 below shows just how many firms make use of each of the 112 activities in the BSIMM. Each activity has a label (like SM1.1) and is described in detail in the BSIMM6 report. See, it turns out we do know how to do software security! We even know who is doing what. Now what we need to do is spread adoption of software security to all firms creating software. You can help.

BSIMM Scorecard
Table 2

How does your firm compare?

Here’s what happens when you measure a new firm using the BSIMM measuring stick. You can directly compare how your software security initiative stacks up against the other 78 firms in BSIMM6.

Is your firm a financial services institution? Well, we can compare you to 33 other financial services firms. Are you an ISV? We can compare you directly to 27 other ISVs. BSIMM6 also marks the introduction of the healthcare industry with the inclusion of 10 firms. Measurement is a powerful tool that drives both budgets and improvement.

Nobody wants to be the slowest zebra in the zebra pack. Is your firm the slowest zebra? You can get your own scorecard like the one in Table 3 and do some analysis to find out.

BSIMM6 Scorecard for Fake Firm
Table 3

We also create a spider diagram (Figure 1) as a way of visualizing a comparison based on 12 practices. The 112 activities in the model fit directly into the 12 practices.

BSIMM6 Fake Firm Spider Chart
Figure 1

Our spider-graph-yielding “high-water mark” approach (based on three levels per practice) is sufficient to get a low-resolution feel for maturity, especially when working with data from a particular vertical or geography.

One meaningful comparison is to chart your own firm’s maturity high-water mark against the averages we have published to see how your initiative compares.

The BSIMM community

The 78 firms participating in BSIMM6 make up the BSIMM community. A moderated private mailing list with over 250 members allows SSG leaders participating in the BSIMM to discuss solutions with others who face the same issues, discuss strategy with someone who has already addressed an issue, seek out mentors from those further along a career path, and band together to solve hard problems.

The BSIMM community also hosts annual private conferences in the United States and Europe where representatives from each firm gather together in an off-the-record forum to discuss software security initiatives.

Become part of the community today and take advantage of these unique resources.

The BSIMM website includes a credentialed BSIMM community section where information from the conferences, working groups, and mailing-list-initiated studies are posted.

Would you like your firm to be included in the BSIMM community? Give us a shout. BSIMM6 is the latest snapshot of a growing and evolving set of real data about software security. The more data we have, the better off we all are. It’s science time.

Written in coordination with Gary McGraw and Jacob West.

This was first published in October 2015.