Cost-Benefit Analysis Method

NEWS AT SEI

This library item is related to the following area(s) of work:

Software Architecture

This article was originally published in News at SEI on: March 1, 2002

A software architecture is an essential part of a complex software-intensive system. For more than five years, the SEI has been performing software architecture analyses to help software developers to examine the consequences of architectural strategies (AS) before committing resources to implementing them.

Architecture analysis, using the SEI’s Architecture Tradeoff Analysis Method  (ATAM) focuses on a system’s quality attributes, such as performance, modifiability, usability, and availability. The ATAM provides software architects, while designing or maintaining a software system, a framework to reason about the tradeoffs among these quality attributes. But the biggest tradeoffs in large, complex systems always have to do with economics: How should an organization invest its resources to maximize its gains and minimize its risks?

The Cost-Benefit Analysis Method (CBAM) picks up where the ATAM leaves off, adding costs, benefits, and schedule as attributes to be considered among the tradeoffs when a software system is being planned.

About the CBAM

The ATAM uncovers the architectural decisions that are made (or are being considered) for the system, and links these decisions to business goals and quality attributes. The CBAM builds on this foundation by enabling engineers to determine the costs and benefits associated with these decisions. Given this information, the stakeholders could then decide, for example, whether to use redundant hardware, checkpointing, or some other method to address concerns about the system’s reliability. Or the stakeholders could choose to invest their finite resources in some other quality attribute—perhaps believing that higher performance will have a better benefit/cost ratio.

A system always has a limited budget for creation or upgrade; so every architectural choice, in some sense, competes with every other one for inclusion. The CBAM does not make decisions for the stakeholders; it simply helps them elicit and document costs, benefits, and uncertainty and gives them a rational decision-making process. This process is typically performed in two stages. The first stage is for triage, and the cost and benefit judgments used here are only rough estimates. The second stage operates on a much smaller set of scenarios and architectural decisions (also called architectural strategies), which are examined in greater detail.

Using the CBAM

The CBAM consists of six steps, each of which can be executed in the first (triage) and second (detailed examination) phases:

  1. choose scenarios and architectural strategies
  2. assess QA benefits
  3. quantify the architectural benefits of the strategies
  4. quantify the architectural costs and schedule implications of the strategies
  5. calculate desirability
  6. make decisions

In the first step, scenarios of concern to the system’s stakeholders are chosen for scrutiny, and architectural strategies are designed that address these scenarios. For example, if there were a scenario that called for increased availability, then an architectural strategy might be proposed that added some redundancy and a failover capability to the system.

In the second and third steps, benefit information is elicited from the relevant stakeholders: QA benefits from managers (who, presumably, best understand the business implications of changing how the system operates and performs); and architectural strategy benefits from the architects (who, presumably, best understand the degree to which a strategy will, in fact, achieve a desired level of a quality attribute).

In the fourth step, cost and schedule information is elicited from the stakeholders. In step five these elicited values are used to calculate a desirability metric (a ratio of benefit divided by cost) for each architectural strategy. Furthermore, the inherent uncertainty in each of these values can be calculated, which aids in the final step, making decisions.

Given these six steps, the elicited values can be used as a basis for a rational decision-making process—one that includes not only the technical measures of an architectural strategy (which is what the ATAM produces) but also business measures that determine whether a particular change to the system will provide a sufficiently high return on investment.

An important feature of the next version of the CBAM will be the ability to perform multiple iterations, where each iteration adds some information and pares down the space of scenarios considered. For example, separate iterations will consider the side effects of ASs, between ASs. Jai Asundi, one of the developers of CBAM, says, “If you don’t have the resources to do multiple iterations, you can do just one or two, and you’ll still derive benefits.”

Case Study

The Earth Observing System is a constellation of NASA satellites that gathers data about the Earth for the U. S. Global Change Research Program and other scientific communities worldwide. The Earth Observing System Data Information System (EOSDIS) Core System, also called the ECS, collects data from various satellite downlink stations for further processing. The ECS consists of about 1.2 million lines of code in 12,000 modules and about 50 commercial off-the-shelf (COTS) products. The SEI and members of the ECS project performed a CBAM, demonstrating the feasibility of the method for large-scale projects. The CBAM helped structure an unstructured architecture design problem and offered the project manager a disciplined process aimed at arriving at a manageable set of architectural solutions to choose from.

Find Us Here

Find us on Youtube  Find us on LinkedIn  Find us on twitter  Find us on Facebook

Share This Page

Share on Facebook  Send to your Twitter page  Save to del.ico.us  Save to LinkedIn  Digg this  Stumble this page.  Add to Technorati favorites  Save this page on your Google Home Page 

For more information

Contact Us

info@sei.cmu.edu

412-268-5800

Help us improve

Visitor feedback helps us continually improve our site.

Please tell us what you
think with this short
(< 5 minute) survey.