Software Integrity


Understanding software risk is as simple as changing a flat tire

There are a lot of terms and techniques for dealing with risk and we use them regularly in software security. Risk is a vector with two components: impact and likelihood. Impact is the bad stuff that is going to happen to us if the risk is realized. Likelihood is a (frequently subjective) notion of how likely that bad stuff is to happen. The two are intertwined. When we deal with a risk, we have a few choices:

  1. Avoid the risk entirely,
  2. Reduce the likelihood to an acceptable level,
  3. Reduce the impact to an acceptable level,
  4. Transfer the impact to make it someone else’s problem, or
  5. Simply “self-insure” and accept the risk.

Dealing with risk: A worked example

Car tires make a great analogy for discussing risk. The impact of getting a flat tire is fairly high. It’s at least an inconvenience and sometimes leads to a crash. Flat tires are impactful enough and likely enough that most of us do something about them.

Corrective control: Carrying a spare

The first thing you might think of when contemplating a flat tire is that most cars have a spare tire. However, a spare doesn’t have much to do with the risk of getting a flat. That is, having a spare tire doesn’t reduce the damage from a flat tire or make getting one any less likely. Spare tires reduce the after-the-fact impact of getting a flat tire because it means we can change the tire and be back on our way fairly quickly. In the language of security controls, a spare tire is a “corrective control.”

In the software world, after-the-fact fraud detection is a good example of a corrective control. If we audit transactions to detect and possibly roll back fraudulent ones, then we are doing the software equivalent of changing a flat tire.

Reducing likelihood: Driving safely

Driving safely and avoiding objects in the road while driving helps prevent flat tires. We reduce the risk of a flat tire by reducing its likelihood. This can include choosing the type of terrain we drive on, using our headlights and learning how to recognize hazardous objects. Although we reduce the likelihood of it happening, we don’t necessarily reduce the impact to us when it does. The severity of the risk is reduced, even if the impact is not.

Similarly, with software risks, we have many ways to reduce likelihood. We can use cryptographically secure randomness to make identifiers harder to guess. We can limit access to software interfaces through network access, user access or authorization models. We can reduce the number of humans who have access to something, or increase the minimum amount of effort to carry out an attack. All of these approaches reduce the likelihood of the software risk. Do we eliminate the risk? No. We make it less likely.

Reducing impact: Run-flat tires

One of the possible impacts of getting a flat tire is losing control of the vehicle and crashing. Run-flat tires are more stable and make cars easier to drive even when they are punctured. Thus, if we hit debris in the road, we still get a flat tire that we have to repair. But, we will have better control of the car, possibly with the ability to drive for some distance rather than immediately pulling the car over.

One way to reduce impact in software is by imposing rate limits. We know that bad things can happen (i.e. fraudulent transactions, password guesses and malicious file uploads), so we limit how fast or how many of these things can be done. Limits to how much money can be taken out of an ATM per day reduces the impact of a stolen ATM card. Limits to how many credit cards can be associated with a shopping account limits the impact of credit card fraud.

Avoiding risk: Work from home

We don’t always have to engage in risky behavior. Sometimes the business can choose to accomplish its mission in another way. Likewise, someone who works from home one day will avoid the risk of flat tires on that day. With software, we can avoid some risks by simply omitting support or rejecting requests that attempt to do something too risky. Making a manual process for changing a customer’s birth date is a reasonable control instead of supporting it in software, if that reduces fraud.

Transferring risk: Roadside assistance

In the London area, my car has no spare tire at all. I carry insurance that includes “breakdown coverage.” Up to four times a year, I can have a breakdown of any kind (not just a flat tire) and call for free recovery. They’ll tow me and my car to a reasonable destination of my choosing. I will spend more time in recovery than I would changing a tire at the roadside. On the other hand, I get two extra seats that can fold down into the back of my car (into the space traditionally occupied by a spare tire). I use those two seats far more often than I would use the spare tire. I have increased the impact of a flat tire a little bit, but gained some utility in the vehicle. The insurance/spare tire scheme does nothing to address the likelihood of flat tires at all; instead, it transfers the risk to someone else.

Accepting risk: No spare at all

If I decide not to pay for breakdown insurance and I drive a car that has no spare, then I am accepting the risk. I might do this if I have judged the likelihood to be exceptionally low (I place great faith in my driving skill). I might do it if I estimate the impact to be very low (maybe I only drive very short distances from home on well-maintained roads). With no spare tire and no insurance, I will find myself stopped at the roadside trying to arrange recovery if I actually get a flat. It might cost me more time and money, but it also might not happen at all.

We see risk acceptance in web sites and software quite often. In these cases, businesses have few or no technical controls around critical capabilities. They might assume certain attacks are rare or difficult when they are common and easy. Sometimes the landscape shifts dramatically and technology previously thought to be reasonable is suddenly found to be vulnerable. Likelihoods or impacts can shift suddenly.

Residual risk: Bad stuff still happens

No matter what precautions we take (i.e. driving well, good tires and clean roads), there is always the residual risk that something will happen anyway. Run-flat tires, for example, enable the driver to have more control when a puncture occurs, but a driver could still crash under some circumstances. The possibility of damage to the car (or driver) from a crash when a run-flat tire is punctured is residual risk.


Software serves a mission. Risks affect its ability to deliver on that mission. Impact largely hinges on what happens to the business. Likelihood, though, incorporates other factors like an attacker’s ability, connectivity and the number of people who could possibly carry out the attack—just to name a few. When evaluating the impact and likelihood of a software risk, evaluate the impact and likelihood much like we’ve just done with a flat tire. Consider what types of software security controls your software already has in place as well as corrective controls the business might have in place. Not every risk is handled the same way, and you might have more options than you realized.