Evaluating the progress of your software security journey is essential, but it can be a considerable challenge. Tracking operational metrics doesn’t tell you whether you are doing the right things. Analyst reports are often too general to provide tactical direction. And companies hold their security plans so close to the vest, competitive research is nearly impossible. Benchmarking can help you get a new software security initiative off the ground or navigate an existing one. It is different from other measurement techniques because it focuses on excellence, includes detailed comparisons, and pools confidential information among numerous organizations. Benchmarking your software security initiative can tell you if you are keeping pace with your peers, or if you should accelerate your efforts to rise above the competition. The results of a benchmarking assessment can help you identify new security strategies and prioritize scarce resources to be most effective.
Consider these 10 tips to get the most out of your benchmarking assessment.
1. Select the right instruments.
When you choose a methodology to assess your program, make sure you select a transparent model that is commonly used by security experts and reflects the latest practices in the industry. The terminology will be more easily understood, the assessment will be more comprehensive, and the results will garner more respect.
2. Evaluate real-world conditions.
An assessment based on current data from real-world companies will be more accurate than a checklist of theoretical issues. Look beyond the high-level findings and ask: What companies are included in the benchmarks? Are they companies whose example I want to follow?
3. Learn from experienced pilots.
If you operate in an industry that has not historically invested in security, you may have an outdated idea of what is necessary to mitigate risk. Look to industries that are considered leaders to get inspired by ideas you can adapt to your own software security initiative.
4. Verify your launch point.
Quick surveys such as online assessments are a great way to launch your benchmarking strategy. They can give you an initial read on where you stand. Unfortunately, they may also give you a false sense of security. To capture your current security posture in detail, a follow-up assessment should include interviews with multiple parties and documented activities to verify specifics. You may find that elements of your security plan are not actually being carried out in practice, or activities are different from what you expect.
5. Beware of overinflation.
Internal-only assessments can unintentionally inflate results based on assumptions and take you off course. A third party that has no stake in the outcome can evaluate your security processes with an unbiased perspective.
6. Weigh everything in your basket.
You’ll want an aggregate assessment of your security posture, but you should also look at the details. For example, an “average” result may hide the fact that a single business unit has particular strengths while another has certain weaknesses. Deconstruct your results or consider separate assessments.
7. Take a 360-degree view.
Consider your results in context. Not every element of the framework you choose may apply to your business. For example, if you don’t rely on third parties to develop software, you don’t need to develop vendor service-level agreements.
8. Reflect on your journey.
Don’t spend all your time collecting and measuring data. Your results are simply numbers on a page until you devote time to analysis. Make sure you leave some room in your budget and your timeline to apply results and prepare your maneuvers.
9. Share your experience.
Any time you invest in an external audit of your business operations, executives will want to know what the results mean. Have a plan to communicate your results with business context to increase your leadership’s understanding of software security and build support for the resources you need to evolve your program.
10. Test the wind at different altitudes.
Most companies find that it makes sense to do an in-depth assessment about every two years to track their progress. During that time, you’ll be able to see improvements in more resource-intensive, time-consuming activities.
Benchmarking your security strategies against the activities of real-world organizations provides meaningful context to help you make decisions.
Building Security in Maturity Model (BSIMM) is an assessment framework based on data gathered from 130 currently active software security initiatives. It categorizes 125 software security activities into three maturity levels based on their rate of observation and complexity.
A BSIMM assessment gives you insight into how other organizations value security activities and an unbiased perspective on the strengths and weaknesses of your own program.
The "BSIMM13 Insights and Trends" report provides an inside glimpse at this year's top software security initiatives.
Benchmark your security program against best practices.
The 10 Most Common Web Application VulnerabilitiesLearn more
5-Step Blueprint for Launching a Successful Software Security InitiativeLearn more
Secure software requires more than just tools.Watch the video
Learn where to look for ROI in an AppSec program to maximize your investment.Read the blog post