Static analysis tools: Are they the best for finding bugs?

Before we can dig deeper into the topic of static analysis, we must first understand how it works. Once a foundation has been established, we’ll then analyze a variety of scenarios to determine when static analysis tools are the best method to find security bugs.

What is static analysis?

Static analysis refers to the examination of a piece of software without executing it. In the world of security, it refers to discovering security related bugs in software without actually running the software. Static code analysis is a white box method of testing, meaning that the tester has access to the underlying framework, design, and structure of the software. The process typically includes examining the code structure, studying the various data and control flows, and referring to the configuration settings to discover various types of security bugs.

Static analysis tools vs. manual review

Static code analysis can be automated or conducted manually. An automated review uses static analysis tools to discover bugs. It is faster than a manual review and generally provides better code coverage. Static analysis tools are effective at finding common security bugs. A manual review, on the other hand, is better at discovering complex bugs such as those related to authentication. The manual approach can also be very effective at analyzing business logic for security bugs. Manual reviews take more time but they're more thorough and the bugs discovered have a very high confidence rate.

Considerations for static analysis tools

Static analysis tools provide developers with accurate and timely code feedback and are often integrated near the end of the software development life cycle (SDLC). Tools can provide excellent insight into the quality of the developed code.

The following considerations must be made when adopting the usage of static analysis tools:

1. For code analysis, access to the underlying codebase is necessary. Analysis is not possible if the underlying static codebase is unavailable. (Although, today some tools are able to work with binaries or compiled code.)
2. Typically, proprietary applications cannot be statically analyzed due to their licensing models and unavailability of code.
3. The tools must support the underlying framework of the software. Most static analysis tools support popular languages and technologies like Java and .Net.
4. Tools will scan the application code only. Third-party libraries are not scanned, but they should be included as dependencies.
5. Since static analysis tools require the complete codebase, if the code cannot be compiled correctly, the bugs discovered are often incomplete and inaccurate. If the compile process fails before the scan, tools offer the option to halt the scan process.
6. Since static code analysis works with a static codebase, the scan is able to discover compile-time bugs only. Static analysis tools do not have any insight into the runtime environment and cannot be used to discover bugs that occur during runtime as well as any environment related issues.
7. Static code analysis tools indicate the place of the vulnerable code and can provide a code trace. Since they don’t execute the attack, there is no proof of attack. Thus, bugs discovered with tools are harder to explain.
8. There is a high false positive rate when using automated tools to detect bugs. All findings must be triaged manually. The triaging analyst must understand different coding techniques and must have expertise in the security domain.