Applying Causal Learning to Improve Software Cost Estimation and Project Control

Created March 2021

Software plays a major role for the Department of Defense (DoD), and its importance for DoD systems is only increasing. As the DoD looks to acquire and develop new software systems, developing plans for the future depends on having an accurate analysis of software cost. Many cost estimates, however, are often inaccurate, causing programs to break their plans as well as their budgets. To improve cost estimation, SEI researchers are exploring how to apply causal learning and discovery methods as an alternative to traditional cost analysis. Causal-learning approaches show greater success in helping software development and acquisition programs stay on track to achieve their goals.

Inaccurate Estimates Break Budgets and Derail Planning

When planning software systems development or acquisition, the DoD has run into problems like increased costs and delays. The recent DoD Cost Estimating Guide emphasizes the importance of developing “cost estimates that support the planning, programming, budgeting, acquisition, and requirements generation processes.” To make sure programs can stay on track and on budget to deliver the software capabilities the DoD needs to plan for the future, accurate cost estimates are key.

Usually, traditional cost estimation focuses only on predicting costs. These estimation methods rely on tools such as statistical correlation, regression, and even complex machine-learning applications—all of which are effective at finding associations in data to make good predictions. What these methods fail to do, however, is to understand what is driving those costs and what causes them to change. If and when costs fluctuate, these methods don’t provide a good way to manage them or understand why.

The goal to improve planning is to develop a method that can find the causes that drive costs. When a method provides such insight, organizations can take back control to salvage their planning and avoid rising costs by changing policies or taking management action. To control project outcomes, we need to know which factors truly drive them so we can manage the issues that lead to late delivery, cost overruns, and performance gaps before they ruin projects.

Getting a Handle on Software Cost

At the SEI, we have been developing modeling tools based on causal learning to identify and evaluate the issues that lead to delays, broken budgets, and other poor outcomes. Causal learning focuses on studying the relationships between a cause and its possible effects so that we can understand how we arrive at certain outcomes and how we might control them better. Research has shown that causal models for cost estimation are superior to traditional statistical models because, by identifying causal factors, proactive control of project and task outcomes becomes possible.

Over a three-year period, SEI researchers have worked to help the DoD—especially the U.S. Army and USC—to apply causal learning to control costs in software development and sustainment. We also explored how to deal with sources of uncertainty in project planning and studied traditional cost estimation models. Thanks to this work, we have been able to identify causal factors in software and systems engineering that provide ways for controlling program costs. As our research in causal learning continues to evolve, it will also help organizations better negotiate software contracts, design more effective policies and incentives, and provide more actionable information for affordability reports.

During our work with project planning, we’ve also uncovered several pitfalls that many organizations face, including poor communications, unclear planning, and ineffective hiring practices. Many organizations are often unaware of these issues, and are therefore unable to mitigate them. The SEI can help your organization identify these kinds of issues, as well as improve planning and invest your resources more effectively to chart a clear path toward achieving your program’s objectives.

Software Engineering Institute