Software Integrity Blog


What is continuous testing?

Continuous testing means testing an application continuously throughout the software life cycle. It’s a critical part of CI/CD and—like any other part—should be automated.

What is continuous testing in the CI/CD pipeline?

The increasing demands and expectations from digital-native users drive businesses to continuously enhance their software with new, better features and bug fixes. The need for more rapid, responsive development cycles has led many organizations to adopt agile development, create a DevOps culture, and practice continuous integration/continuous development. CI/CD enables daily, even hourly, software updates, so it’s become ubiquitous across organizations that need fast turnaround. But organizations practicing CI/CD can’t sacrifice security for speed. To ensure that the speed of testing aligns with the speed of CI/CD, organizations need to implement automated continuous testing.

What is continuous testing?

Continuous testing is the process of testing an application continuously throughout the development cycle. Continuous testing starts early in the pipeline, far before traditional testing. It occurs often, at every stage where code is written, built, or run. But you can’t expect your teams to track whether they’ve tested every file in every configuration of every build. They have better things to do than to click buttons all day. So continuous testing, like all other components of CI/CD, requires automation to be effective.

CI/CD enables daily, even hourly, software updates.

Why do I need continuous testing?

If you practice CI/CD, you need to perform continuous testing for two main reasons:

  1. It allows you to fix issues early in development and integration, which is faster, easier, and cheaper than fixing them after deployment.
  2. It reduces the chance that quality and security issues will make it into your production apps.

On the enterprise applications front, the software development life cycle (SDLC) has changed more in the last five years under CI/CD than it did during the previous 20 years. In the old days, under the waterfall model, a developer would meet with a business line manager or business analyst and listen to their user story about their business needs. Then the SDLC would begin. Through compartmentalized steps, development proceeded from business requirements to software architecture design, implementation, verification and testing, and finally production and maintenance.

Traditionally, neither functional nor acceptance testing would occur until the “package” had been delivered. But that’s when fixing software bugs, mitigating security vulnerabilities, and modifying code in response to change requests is most difficult and expensive—and time-consuming. And when development teams are up against a deadline, security testing is often the first thing to go. In the new world order of Agile, DevOps, and CI/CD, testing must happen throughout the SDLC—continuously and automatically.

Do all CI/CD pipelines include automated continuous testing?

You might expect that development teams would embrace any testing that happens automatically. That’s fewer activities to track, fewer buttons to click, more time to do the fun stuff, like coding. But automated continuous testing is conducted on only 24% of functional performance test cases. And within software sprints, the proportion is even a tick lower still, at 22% of test cases.

In CI/CD, testing must happen continuously and automatically.

Automated continuous testing hasn’t come as far as the software development community hoped. Some developers don’t feel comfortable without at least a few manual testing steps. But as automated continuous testing technology improves, it’ll earn the trust of more developers and become an integrated part of more CI/CD pipelines.

Why do I need automated continuous testing?

Automated continuous testing fits use cases where manual testing doesn’t offer a benefit over an automated solution, such as everything under the UI: unit tests, component tests, API tests, and so on. As OWASP and Agent Smith say, “You should never send a human to do a machine’s job.”

The range of use cases for automated continuous testing is vast:

  • Consistency. It allows you to apply quality and security requirements more consistently. If you record a manual security test and then automate it, it becomes a security requirement you can enforce on every build.
  • Speed. With automated continuous testing powered by scalable tools, developers can find and fix issues in real time throughout the SDLC. Doing so speeds up application development and reduces errors common to manual testing.
  • Scale. To scale manual testing, you need more manual testers. To scale automated testing, you only need more apps and builds to test.

Adding automated continuous testing to your CI/CD pipeline requires an application testing tool that’s easy to integrate with the build, test automation, and CI/CD tools you already use and that has extensive web API support. For enterprise organizations that are looking for an optimal tool that helps with security testing as part of the automated continuous testing effort, an interactive application security testing (IAST) tool is a powerful way to perform continuous security testing across their entire application portfolio.

In search of an IAST solution?

Seeker® is an IAST solution that can scale to thousands of apps. And unlike dynamic analysis (or DAST) solutions, Seeker complements static analysis, detecting vulnerabilities using instrumentation to analyze code, dataflow, and memory in the background while the app is under test.

Learn more about interactive application security testing


More by this author