Problems are more expensive to fix at the end of the life cycle. Economics dictates finding defects as early as you possibly can. Have a flaw in your idea? Redesign it in your mind (presumably free). Have a bug in your code? Find it while it’s being typed in, not later during the build process. Want to find vulnerabilities in your software? Why wait until it’s shipped?
Penetration testing is about testing a system in its final production environment. As such, it’s best suited to probing configuration problems and other environmental factors that deeply impact software security. Driving tests that concentrate on other factors such as some knowledge of risk analysis results is the most effective approach.
One reason why so many organizations turn to penetration testing first is that it’s an attractive late-life cycle activity that can be carried out in an outside-in manner. Like the canned test, in some instances, you don’t really need to know that much about the software being tested. As a result, basic penetration testing is a common activity that can be carried out on a completed application, under time constraints specified by the operations team, to fill a security testing checkbox at the end of the SDLC. Of course, fixing things at this stage of the game is, more often than not, prohibitively expensive (and in some cases involves configuration Band-Aids rather than construction-based cures).
Bottom line: Outside-in testing is great as long as it’s not the only testing you do.