icon-carat-right menu search cmu-wordmark

Managing Security and Resilience Risks Across the Lifecycle

Software is a growing component of today’s mission-critical systems. As organizations become more dependent on software-driven technology, security and resilience risks to their missions also increase. Managing these risks is too often deferred until after deployment due to competing priorities, such as satisfying cost and schedule objectives. However, failure to address these risks early in the systems lifecycle can not only increase operational impact and mitigation costs, but it can also severely limit management options.

For Department of Defense (DoD) weapon systems, it is especially important to manage software security and resilience risks. Proactively identifying and correcting software vulnerabilities and weaknesses minimizes the risk of cyber-attacks, weapons system failures, and other disruptions that could jeopardize DoD missions. The GAO has identified software and cybersecurity as persistent challenges across the portfolio of DoD weapon systems. To address these challenges, acquisition programs should start managing a system’s security and resilience risks early in the lifecycle and continue throughout the system’s lifespan.

This post introduces the Security Engineering Framework, a detailed schema of software-focused engineering practices that acquisition programs can use to manage security and resilience risks across the lifecycle of software-reliant systems.

Software Assurance

Software assurance is a level of confidence that, throughout its lifecycle, software functions as intended and is free of vulnerabilities, either intentionally or unintentionally designed or inserted as part of the software. Software assurance is increasingly important to organizations across all sectors because of software’s increasing influence in mission-critical systems. Managing software assurance is a challenge because of the growth in capability, complexity, and interconnection among software-reliant systems.

For example, consider how the size of flight software has increased over the years. Between 1960 and 2000, the extent of overall system functionality that software provides to military aircraft pilots increased from 8 percent to 80 percent. At the same time, the size of software in military aircraft grew from 1,000 lines of code in the F-4A to 1.7 million lines of code (MLOC) in the F-22 and 8 million lines in the F-35. This trend is expected to continue over time. As software exerts more control over complex systems (e.g., military aircraft), the potential risk posed by vulnerabilities will increase correspondingly.

Software Defects and Vulnerabilities: A Lifecyle Perspective

Figure 1 below highlights the rate of defect introduction and identification across the lifecycle. This was derived from data presented in the SEI report Reliability Validation and Improvement Framework. Studies of safety-critical systems, particularly DoD avionics software systems, show that 70 percent of all errors are introduced during requirements and architecture design activities. However, only 20 percent of the errors are found by the end of code development and unit test, while 80.5 percent of the errors are discovered at or after integration testing. The rework effort to correct requirements and design problems in later phases can be as high as 300 to 1,000 times the effort of in-phase correction. Even after the rework, undiscovered errors are likely to remain.

figure1_07232025
Figure 1: Rate of Defect Introduction and Identification across the Lifecycle

Given the complexities involved in developing large-scale, software-reliant systems, it is understandable that no software is free of risks. Defects exist even in the highest quality software. For example, best-in-class code can have up to 600 defects per MLOC, while average-quality code generally has around 6,000 defects per MLOC, and some of these defects are weaknesses that can lead to vulnerabilities. Research indicates that roughly 5 percent of software defects are security vulnerabilities. As a result, best-in-class code can have up to 30 vulnerabilities per MLOC. For average-quality code, the number of security vulnerabilities can be as high as 300 per MLOC. It is important to note that the defect rates cited here are estimates that provide general insight into the issue of code quality and number of vulnerabilities in code. Defect rates in specific projects can vary greatly. However, these estimates highlight the importance of reducing security vulnerabilities in code during software development. Secure coding practices, code reviews, and code analysis tools are important ways to identify and correct known weaknesses and vulnerabilities in code.

As illustrated in Figure 1, security and resilience must be managed across the lifecycle, starting with the development of high-level system requirements through operations and sustainment (O&S). Program and system stakeholders should apply leading practices for acquiring, engineering, and deploying secure and resilient software-reliant systems. In 2014, the SEI initiated an effort to document leading practices for managing security and resilience risks across the systems lifecycle, providing an approach for building security and resilience into a system rather than bolting them on after deployment. This effort produced several cybersecurity engineering solutions, most notably the Security Engineering Risk Analysis (SERA) method and the Acquisition Security Framework (ASF). Late last year, the SEI released the Security Engineering Framework.

Security Engineering Framework (SEF)

The SEF is a collection of software-focused engineering practices for managing security and resilience risks across the systems lifecycle, starting with requirements definition and continuing through O&S. It provides a roadmap for building security and resilience into software-reliant systems prior to deployment and maintaining these capabilities during O&S. The SEF builds on the foundational research of SERA and the ASF, providing in-depth guidance that elaborates on leading engineering practices and how to perform them.

SEF practices help ensure that engineering processes, software, and tools are secure and resilient, thereby reducing the risk that attackers will disrupt program and system information and assets. Acquisition programs can use the SEF to assess their current engineering practices and chart a course for improvement, ultimately reducing security and resilience risks in deployed software-reliant systems.

Security and Resilience

At its core, the SEF is a risk-based framework that addresses both security and resilience:

Risk management provides the foundation for managing security and resilience. In fact, risk management methods, tools, and techniques are used to manage both. However, security and resilience view risk from different perspectives: Security considers risks from a protection point of view, whereas resilience considers risk from a perspective of adapting to conditions, stresses, attacks, and compromises. As shown in Figure 2, there is some overlap between the risk perspectives of security and resilience. At the same time, security and resilience each have unique risks and mitigations.

figure2_07232025
Figure 2: Risk Perspectives: Security Versus Resilience

The SEF specifies practices for managing security and resilience risks. The perspective the organization adopts—security, resilience, or a blend of the two—influences the risks an acquisition organization considers during an assessment and the set of controls that are available for risk mitigation. Because of the related nature of security and resilience, the SEF (and the remainder of this blog post) uses the term security/resilience throughout.

SEF Structure

As illustrated in Figure 3, the SEF has a hierarchy of domains, goals, and practices:

  • Domains occupy the top level of the SEF hierarchy. A domain captures a unique management or technical perspective of managing security/resilience risks across the systems lifecycle. Each domain is supported by two or more goals, which form the next level of the SEF hierarchy.
  • Goals define the capabilities that a program leverages to build security/resilience into a software-reliant system. Related goals belong to the same SEF domain.
  • Practices inhabit the final and most detailed level in the hierarchy. Practices describe actions that support the achievement of SEF goals. The SEF phrases practices as questions. Related practices belong to the same SEF goal.
figure3_07232025
Figure 3: SEF Organization and Structure

The SEF comprises 3 domains, 13 goals, and 119 practices. The next section describes the SEF’s domains and goals.

Domain 1: Engineering Management

This domain provides a foundation for success by ensuring that security/resilience activities are planned and managed. The objective of Domain 1 is to manage security/resilience risks effectively in the system being acquired and developed.

Program and engineering managers combine their technical expertise with their business and mission knowledge to provide technical management and organizational leadership for engineering projects. Managers are tasked with planning, organizing, and directing an acquisition program’s engineering and development activities. Engineering management is a specialized type of management that is needed to lead engineering or technical personnel and projects successfully. Domain 1 comprises the following three goals:

  • Goal 1.1: Engineering Activity Management. Security/resilience engineering activities across the lifecycle are planned and managed.
  • Goal 1.2: Engineering Risk Management. Security/resilience risks that can affect the system are assessed and managed during system design and development.
  • Goal 1.3: Independent Assessment. An independent assessment of the program or system is conducted.

Domain 2: Engineering Activities

This domain addresses the day-to-day practices that are essential for building security/resilience into a software-reliant system. The objective of Domain 2 is to integrate security/resilience into the program’s existing engineering practices. All systems lifecycles address a common set of engineering activities, beginning with requirements specification and continuing through system O&S. Domain 2 expands the focus of a program’s systems lifecycle model to include security/resilience. Domain 2 comprises the following eight goals:

  • Goal 2.1: Requirements. Security/resilience requirements for the system and its software components are specified, analyzed, and managed.
  • Goal 2.2: Architecture. Security/resilience risks resulting from the system and software architectures are assessed and mitigated.
  • Goal 2.3: Third-Party Components. Security/resilience risks that can affect third-party components are identified and mitigated.
  • Goal 2.4: Implementation. Security/resilience controls are implemented, and weaknesses and vulnerabilities in software code are assessed and managed.
  • Goal 2.5: Test and Evaluation. Security/resilience risks that can affect the integrated system are identified and remediated during test and evaluation.
  • Goal 2.6: Authorization to Operate. The operation of the system is authorized, and the residual risk to operations is explicitly accepted.
  • Goal 2.7: Deployment. Security/resilience is addressed in transition and deployment activities.
  • Goal 2.8: Operations and Sustainment. Security/resilience risks and issues are identified and resolved as the system is used and supported in the operational environment.

Domain 3: Engineering Infrastructure

This domain manages security/resilience risks in the engineering, development, test, and training environments. The objectives of Domain 3 are to use software, tools, and technologies that support the program’s engineering and development activities and to manage security/resilience risks in the engineering infrastructure. Engineers and developers use a variety of software, tools, and technologies to support their design and development activities. Security/resilience engineering software, tools, and technologies need to be procured, installed, and integrated with the program’s existing engineering infrastructure.

The engineering infrastructure is the part of the IT infrastructure that supports engineering and development activities performed by personnel from the acquisition program, contractors, and suppliers. As a result, the engineering infrastructure can be an attack vector into the software-reliant system that is being acquired and developed. IT support teams need to ensure that they are applying security/resilience practices when managing the engineering infrastructure to ensure that risk is being managed appropriately. Domain 3 comprises the following two goals:

  • Goal 3.1: Engineering Software, Tools, and Technologies. Security/resilience engineering software, tools, and technologies are integrated with the engineering infrastructure.
  • Goal 3.2: Infrastructure Operations and Sustainment. Security/resilience risks in the engineering infrastructure are identified and mitigated.

SEF Practices and Guidance

SEF domains provide the organizing structure for the framework’s technical content, which is the collection of goals and practices. The SEF’s in-depth guidance for all goals and practices describes the capability represented by each goal, including its purpose, relevant context, and supporting practices. SEF guidance also defines the key concepts and background information needed to understand the intent of each practice.

Security Engineering Framework (SEF): Managing Security and Resilience Risks Across the Systems Lifecycle contains in-depth guidance for all goals and practices.

Partner with the SEI to Manage Security and Resilience Risks

The SEF documents leading engineering practices for managing security/resilience risks across the systems lifecycle. The SEI provides open access to SEF guidance, methods, and materials. Future work related to the SEF will focus primarily on transitioning SEF concepts and practices to the community. The SEI plans to work with DoD programs to pilot the SEF and incorporate lessons learned into future version of the framework.

Finally, the SEF development team continues to seek feedback on the framework, including how it is being used and applied. This information will help influence the future direction of the SEF as well as the SEI’s work on documenting leading practices for software security.

Get updates on our latest work.

Each week, our researchers write about the latest in software engineering, cybersecurity and artificial intelligence. Sign up to get the latest post sent to your inbox the day it's published.

Subscribe Get our RSS feed