Using the cloud as a cheap DevSecOps lab environment

A DevSecOps lab gives you valuable hands-on experience with the tools and technologies you need to evaluate. Thanks to the cloud, it’s cheap to create one.

This entry in our BSIMM Monthly Insights series was contributed by guest author Brian Higdon with Freddie Mac. The views expressed are the author’s own and do not reflect the views of Freddie Mac.

In my continuing quest to “find a better way,” I’ve learned that tool and technology evaluation is relatively easy; it’s the people and processes involved that are the hard part. Setting up an on-site lab within your organization can be a challenge: Bureaucracy makes for a time-consuming exercise in frustration, and cooperation with people in other silos often seems impossible when it comes time to integrate your tools with theirs (or when you simply want to experiment with a new technique to improve your own).

Fortunately, many of the tools in the DevSecOps toolchain are FOSS (free and open source software), include a community edition version, or are available as software-as-a-service offerings, making the integration work relatively painless. In addition, eager vendors sometimes provide commercial tools on a temporary basis, if you want to play with something more familiar.

Why set up your own DevSecOps lab?

In the past, I’ve used a home lab to run my experiments because I can set up several VMs at once and install what I like on them, without worrying about impacting anything else in the organization. The only problem is sharing my results with others and giving them hands-on time with my lab setup.

To make life simpler on both fronts, I’ve largely moved my lab to the cloud. The variety of options makes it easy to spin up whatever kind of environment I need, install the software, make connections to SaaS tools, share my results with others, and give those people access to my lab to poke holes in my strawman. When I’m done, I tear it all down until the next time.

Tips for creating a cloud-based lab

My lab in the cloud has been a great experience, but I’ve learned a few things along the way that might help others:

1. You shouldn’t expect the free-tier instances that cloud vendors offer to be enough. Their servers are often underpowered and don’t have the muscle you’ll need to run your lab.
2. Creating a template for your environment is very helpful for spinning up quickly, but software tools often require significant configuration to make them represent a working lab, something that isn’t always easily done. Therefore, it’s best to create “golden images” of your servers once you’ve got them just right, both to preserve what you’ve done and to make the next lab easy to spin up. Should your experiments go horribly wrong, you’ll always have those images to roll back to and start again.
3. Given points 1 and 2, you’ll probably end up spending some cash to get your lab up and going. Hopefully, your company will support what you’re doing and reimburse you for your costs.
4. You’ll end up being an admin for your instances, especially for identity and access management, and you’ll probably end up doing a lot of sysadmin work that you might not be completely comfortable with. Google is definitely your friend here, as well as O’Reilly Safari, which is my constant companion.

How to use your DevSecOps lab

I’ve used my lab to perform bakeoffs among different vendors that my organization was evaluating. Bakeoffs are usually time-constrained and require access that I wouldn’t normally have or tools that I don’t usually administer, but my lab makes the hands-on access simple, and the cloud feature makes the results easy to share.

In my quest for DevSecOps automation, I’ve also created dozens of CLI tools in the recent past to improve my company’s processes. I find that experimenting in a lab of my own making, trying out new open source libraries without the need for approval, is a great way to get things done quickly.

Finally, and this may be the most important point, I believe that having hands-on experience with tools and still actively doing development enhances my credibility as an application security professional. I’ve come across many “pros” in the field who have done little to no development, have no practical experience in administration, or have never delivered a software project on a tight deadline. Real-world experience in these areas is necessary to better understand the teams we work with, their needs, and the pressure they feel to get things done. It’s all part of breaking down the silos among the DevSecOps groups and making us better, more knowledgeable partners.

Join the BSIMM Community for more insights.