Software Engineering Institute

At a conference in 2020, the SEI’s Tim Chick was asked, “How do you assure DevSecOps pipelines?” Based on their experience in the field, Chick and his SEI colleagues concluded that most organizations were making indefensible assertions about their pipeline’s assurance and what it would provide for product assurance. While one can assure a software product, the concept of a DevSecOps pipeline lacked enough definition and substance to be assured, or verified to behave as expected, and to have its cybersecurity risks quantified.

A few months later, bad actors exploited a supply chain flaw to deliver malware to thousands of systems running SolarWinds software. This attack validated the SEI’s conclusions: Both the product and pipeline need to be assured.

While there are many theories and tools for DevSecOps, there is no practical framework for its implementation and evaluation. “There’s no holistic view of how you bring it all together,” said Chick. Filling this gap is especially critical for major Department of Defense programs because they rely on the DevSecOps pipeline to repeatedly perform key assurance activities to address the scale and complexity of their software systems.

To bridge this gap between research and practice, the SEI developed a DevSecOps model. It includes 10 capability areas covering every stage of the DevSecOps lifecycle. Chick’s team mapped requirements to capabilities and divided them into four levels. This structure enables organizations to quantifiably evaluate their DevSecOps capabilities, from planning to quality assurance. To fully encompass the socio-technical aspects of the pipeline, the model defines goals and measurements for the roles and responsibilities within the organization.