While software developers have long used third-party web app and API pen tests to find application security defects, pen tests are also a great way to gauge the health of an SDLC. Third-party automated pen tests, which are often mandated by regulatory agencies, require very little experience with tooling or training on the part of the customer, and are fairly thorough in identifying security flaws that need to be fixed. This is why they’re popular. From both parties' points of view, it's a pretty fulfilling activity; the pen tester gets to hand over a number of alarming-sounding vulnerabilities, and the developers receive a number of defects to add to the backlog.
However, what third-party automated pen tests cannot replace is the human behind pen testing execution. While humans are slow and more expensive than automated defect discovery tooling, because they can mimic human hackers, they are better at evaluating an application's response to a pen test, and can potentially catch responses that automated software may miss.
One way to think about how you are using pen tests is to look at the cost of finding defects. For example, finding a cross-site scripting defect using a SAST tool is much more efficient than finding it using a pen test. Using pen tests for vulnerability detection means that each one costs a percentage of your limited testing time, a fraction of your checklist, and a portion of your report writeup effort. And when companies run pen tests on an annual basis, the cost per defect increases as vulnerabilities get patched, and the critical and high vulnerabilities dwindle to nothing. So if pen testing shows diminishing returns as a defect discovery tool, how can you use the expensive but more accurate human element of pen tests to take full advantage of their value?
Where pen testing really begins to pay off is when organizations leave routine defect discovery to automated tools and shift the human effort toward AppSec program assurance. Just as there are multiple points in a piece of software that could validate, detect, or encode a cross-site scripting attack, there are multiple points in an SDLC that can anticipate, prevent, detect, or mitigate various classes of vulnerabilities. By shifting pen testing efforts to finding those inflection points in your SDLC where vulnerabilities can be prevented, you’re using the human and financial costs of pen testing more efficiently than if you’re just focused on defect discovery. Using pen testing this way can help you detect the processes in your SDLC that allow vulnerabilities to creep in, so if you begin fixing those processes, you’ll also minimize future vulnerabilities.
Rather than relying on pen tests to detect security flaws that must be patched individually, pen testing should be used to perform a blameless postmortem, and analyze whether improvements are needed to ensure that potential failures are recognized at specific points in the SDLC. There are many defect identification systems that can find single types of issues, but they are usually not great at finding edge cases. For that, you need human beings, and this is where the pen test can really work for your organization.