Security should be explicitly at the requirements level. Security requirements must cover both overt functional security (say, the use of applied cryptography) and emergent characteristics. One great way to cover the emergent security space is to build abuse cases. Similar to use cases, abuse cases describe the system’s behavior under attack; building them requires explicit coverage of what should be protected, from whom, and for how long.
At the design and architecture level, a system must be coherent and present a unified security architecture that takes into account security principles (such as the principle of least privilege). Designers, architects, and analysts must clearly document assumptions and identify possible attacks. At both the specifications-based architecture stage and at the class-hierarchy design stage, risk analysis is a necessity—security analysts should uncover and rank risks so that mitigation can begin. Disregarding risk analysis at this level will lead to costly problems down the road. External review (outside the design team) is often necessary.
At the code level, we should focus on implementation flaws, especially those that static analysis tools—tools that scan source code for common vulnerabilities—can discover. Several vendors now address this space, and tools should see market-driven improvement and rapid maturity later this year. As stated earlier, code review is a necessary, but not sufficient, practice for achieving secure software. Security bugs (especially in C and C++) can be deadly, but architectural flaws are just as big a problem.
Security testing must encompass two strategies: testing security functionality with standard functional testing techniques, and risk-based security testing based on attack patterns and threat models. A good security test plan (with traceability back to requirements) uses both strategies. Security problems aren’t always apparent, even when we probe a system directly, so standard-issue quality assurance is unlikely to uncover all the pressing security issues.
Penetration testing is also useful, especially if an architectural risk analysis is specifically driving the tests. The advantage of penetration testing is that it gives a good understanding of fielded software in its real environment. However, any black box penetration testing that doesn’t take the software architecture into account probably won’t uncover anything deeply interesting about software risk. Software that falls prey to canned black box testing—which simplistic application security testing tools on the market today practice—is truly bad. This means that passing a cursory penetration test reveals very little about your real security posture, but failing an easy canned penetration test tells you that you’re in very deep trouble indeed.
Operations people should carefully monitor fielded systems during use for security breaks. Simply put, attacks will happen, regardless of the strength of design and implementation, so monitoring software behavior is an excellent defensive technique. Knowledge gained by understanding attacks and exploits should be cycled back into the development organization, and security practitioners should explicitly track both threat models and attack patterns.
Note that risks crop up during all stages of the software life cycle, so a constant risk analysis thread, with recurring risk tracking and monitoring activities, is highly recommended.