An Architectural Approach to Software Cost Modeling

NEWS AT SEI

Authors

Jai Asundi

Rick Kazman

Mark H. Klein

This library item is related to the following area(s) of work:

Software Architecture

This article was originally published in News at SEI on: March 1, 2000

The aim of a successful software project is to maximize the difference between the value of the software product and its cost. The value of a complex software-intensive system results from the interaction of the functionality and quality of the software, and the marketplace. Traditional cost-estimation models [Londeix 87] have concentrated on methods to estimate the "as-built" costs of simply building the software products. Some cost-estimation techniques at the design stages [Jones 98] only consider the costs of functionality through the use of function points. However, the costs incurred for meeting the system’s quality goals are never estimated. These techniques also do not consider the costs that occur during the entire lifetime of the product, including stochastic events such as the failure or change of components, change of platforms, or even change of methods of communication among components.

A software architecture is a critical part of complex software-intensive system design. As Shaw states, "in a situation of increasing size and complexity, the design and specification of overall system structure becomes more significant" (emphasis added) [Shaw 96]. The term "overall system structure" refers to the software architecture. Software architects usually do not have well-developed structures to reason about the costs of design options. The Architecture Tradeoff Analysis Method (ATAM) [Kazman 99] provides software architect with a framework for reasoning about the technical tradeoffs. The attribute-based architectural styles (ABAS) framework [Klein 99] used within the ATAM helps an architect reason, both quantitatively and qualitatively, about the quality attributes and the stimulus/response characteristics of the system. For example, ABASs allow one to reason about important system stimuli (the failure of a component, a user request, a change to the software) and the associated system responses (mean time to failure, latency, or person-months of labor, respectively).

Functionality is key in satisfying marketplace needs, but functionality is worthless if it is not endowed with the right quality attributes; and it is these qualities that shape the architecture and, consequently, dictate cost. We propose that an architectural approach to cost estimation is necessary to enable reasoning about a system’s long-term costs as well as the coupling of costs and technical tradeoffs in the architectural design.

Problem Framework

Our understanding of the cost-estimation problem arises from the idea that any software project is the result of a set of business goals that emerge from a desire to exploit a niche in the marketplace with a new software product. Take, for example, the development of an application server that caters to on-demand software. The business goals of having a robust, high-performance, secure server lead to a set of architectural decisions whose goal is to realize specific quality-attribute requirements of the system (e.g., using tri-modular redundancy to satisfy the availability requirements, a dynamic load-balancing mechanism to meet the performance requirements, and a 256-bit encryption scheme to satisfy the security requirements). Each architecture A that results from a set {Ai} of architectural decisions has a different set of costs C{Ai}(Fig. 1). The choice of a particular set of architectural decisions maps to system qualities that can be described in terms of a particular set of stimulus/response characteristics of the system {Qi}, i.e., Ai -> Qi. (For example, the choice of using concurrent pipelines for servicing requests in this system leads to a predicted worst-case latency of 500 ms, given a specific rate of server requests.) The "value" of any particular stimulus/response characteristic chosen is the revenue that could be earned by the product in the marketplace owing to that characteristic. We believe that the software architect should attempt to maximize the difference between the value generated by the product and its cost.

Figure 1: Business goals drive the architectural decisions {Ai}, which determine the quality attributes {Qi}.Value(Va) depends on Qi and Cost(C) depends on Ai.

Figure 1: Business goals drive the architectural decisions {Ai}, which determine the quality attributes {Qi}.Value(Va) depends on Qi and Cost(C) depends on Ai.

Developing a Cost Estimate

The cost of a system can be broken up into cost of the initial design, cost of implementation, maintenance and evolution of the system, and the future costs1. Design activity could be of two kinds: routine and innovative. For routine systems, cost estimation is not a very difficult task because the architects/designers know a lot of details about previously built systems and have a good handle on costs. Traditional cost-estimation techniques are very good for these kinds of systems and can easily provide estimates of the cost of the implementation, maintenance, and evolution of the system. However, thinking about the probable future costs—such as complete systemic changes or even costs arising due to stochastic events like the loss of support for a commercial-off-the-shelf (COTS) component—is more challenging. On one hand, what time scale should one choose to ascertain probable future costs? Given that the costs should be represented by a probability distribution, how does one obtain those numbers to do quantitative analysis? We believe that the choice of time scale and appropriate elicitation techniques are critical in obtaining data for meaningful analysis. In the case of innovative designs, these problems are compounded by the fact that the experts know very little about the system that they are going to build.

The cost, C, of implementing a particular architecture is dependent on the set of architectural decisions {C(Ai)}. A project manager is expected to be able to estimate the cost of implementing various designs. By using traditional estimation techniques, an experienced project manager should not have difficulty coming up with a reasonably accurate estimate, but it may be a time-consuming and costly task—especially when several alternatives are being considered. To aid in this estimation exercise, rough implementation cost-characteristic curves that show how costs will behave with respect to each architectural decision may be of considerable help. For example, we may2 observe that the cost of the system behaves like a step function every time a new processor is added within the architecture design. On the other hand, costs due to functional or implementation complexity may be smoother.

Determining the Stimulus/Response Characteristics from Architectural Decisions

One topic of keen interest among many in the software engineering community is the stimulus/response characteristics of architectures. The attribute-based architectural styles (ABAS) framework [Klein 99] aids the software architect in reasoning about the characteristics of a system that are quality-attribute specific. Using ABASs, the architect can map the relationship between Ai, the architectural decisions, and Qi, the resulting quality attributes of the system.

Making Value Judgments Based on Stimulus/Response Characteristics

As mentioned earlier, value is derived from functionality and quality. Using Qi, the stimulus/response characteristics of the system, the marketing managers and the various stakeholders can generate value judgments to obtain Va. (A statement such as "We will have no users if serving a request takes more than 2 seconds" tells us that the value Va of the system is approximately zero if the system's latency is greater than 2 seconds.) In this manner, for a particular system, the characteristic value curves can be elicited 3. Some typical examples are shown in Figure 2. This figure gives the architect an idea of the sensitivity and criticality of the system’s quality attributes.


Figure 2: Value vs. Performance and Value vs. Availability

Figure 2: Value vs. Performance and Value vs. Availability

Importance of Choosing an Appropriate Time Scale

If we do not choose an appropriate time scale, the economics of a project cannot be properly assessed. The architects who design the architectures, the marketers who make the value judgments of functionality and stimulus/response, and the project managers who make the cost estimates for these architectures must have a common time basis for decision making. The system will end up badly designed if the time scales used by stakeholders are not the same4. Similarly, to assess its economic viability, an architecture that will form the basis for a product line will require a different time scale than would an architecture for a single product. Frequently the system’s key stakeholders do not properly understand or agree on what is expected of the product in terms of its lifetime and so cannot make these (cost-critical) decisions appropriately.

Developing Tools for Analysis

From this exercise we see that we will need analytic tools to do some meaningful analysis. First, we need a reasonable expert-elicitation technique through which we can obtain the time/cost/value structure characteristics of the system in which we are interested. The elicitation technique could be adapted from Morgan to suit cost estimation for software systems [Morgan 90]. This technique will be used in two places: for determining the value judgments of stimulus/response characteristics from stakeholders/marketers, and for determining the probable future costs of the system contingent on the architectural decisions.

Another important tool will be an appropriate method for discounting future costs. In traditional finance literature [Brearley 81], one uses the risk-free rate of return-on-investment to calculate the discount rate5. However, is this an appropriate instrument when analyzing software engineering projects? Considering the high-risk nature of most software projects, should the rate of return be higher than that used in financial markets?

Directions for Future Work

The development of a structure that incorporates the value judgment, cost-structure characteristics, and probable future costs with a consistent time scale is the essence of this work. Our areas of focus are in developing a good expert-elicitation technique as well as developing an appropriate method of discounting future costs.

Presently this work is being pursued as an extension to the ATAM. Cost will be considered as another tradeoff attribute. Additional stakeholders will be the marketing and sales managers who could give value judgments about the product and the expected lifetime of the product. We hope to validate this approach in the near future using case studies. An organization that is trying to reason about the various candidate software architectures for its software product and attempting cost-modeling at the design stage would be an ideal testbed for our approach. The benefit of our approach will be observed if it gives the software architect a business reason to make certain architectural decisions for the product—that is, the software architect could state in dollar terms6 the reason for adopting a particular architectural choice.

  1. Here we consider the entire lifetime that the product is sold/maintained or supported.
  2. Whether the increment in cost is like a step function or just a minor increase depends on the total cost of the system and will have to be treated accordingly.
  3. These curves will be approximate and elicited from experience. They will be used togive the architect a rough idea about the sensitivity of the market to certain quality attributes.
  4. The architecture for a product that is going to be used only for a 3-month period might be designed in an unmodifiable, non-scalable way, as opposed to the more modifiable approach that the architects would take if the system were to be used for 15
    years.
  5. If "r" is the rate of return, then the discount rate D=(1+r)-t or e-rt
  6. Even though this dollar value may be an expected or mean value.

References

[Brearley 81] R. A. Brearley, and S. C. Myers, Principles of Corporate Finance, McGraw Hill, 1981.

[Jones 98] T. Capers Jones, Estimating Software Costs, McGraw-Hill, 1999

[Kazman 99] R. Kazman, M. Barbacci, M. Klein, S. J. Carriere, and S. G. Woods, "Experience with Performing Architecture Tradeoff Analysis," Proceedings of ICSE 21 (Los Angeles, CA), May 1999, 54-63.

[Klein 99] M. Klein and R. Kazman, Attribute-Based Architectural Styles, CMU/SEI-99-TR-022, Software Engineering Institute, Carnegie Mellon University, 1999.

[Londeix 87] B. Londeix, Cost Estimation for Software Development, Addison-Wesley, 1987.

[Morgan 90] G. M. Morgan and M. Henrion, Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis, Cambridge University Press, New York, 1990.

[Shaw 96] M. Shaw and D. Garlan, Software Architecture: Perspectives on an Emerging Discipline, Prentice-Hall, 1996.

About the Authors

Jai Asundi is a doctoral candidate at the Department of Engineering and Public Policy at Carnegie Mellon University. His research interests lie in issues of cost modeling for software systems and issues of globalization of software development. He has a B. Tech. degree in chemical engineering from the Indian Institute of Technology, Bombay, and has worked for a software company prior to commencing his graduate studies.

Rick Kazman is a senior member of the technical staff at the SEI, where he is a technical lead in the Architecture Tradeoff Analysis Initiative. He is also an adjunct professor at the Universities of Waterloo and Toronto. His primary research interests within software engineering are software architecture, design tools, and software visualization. He is the author of more than 50 papers and co-author of several books, including a book recently published by Addison-Wesley entitled Software Architecture in Practice. Kazman received a BA and MMath from the University of Waterloo, an MA from York University, and a PhD from Carnegie Mellon University.

Mark Klein is a senior member of the technical staff at the SEI where he is a technical lead in the Architecture Tradeoff Analysis Initiative. He has more than 20 years of experience in research and technology transition on various facets of software engineering and real-time systems. Klein's most recent work focuses on the analysis of software architectures. Klein's work in real-time systems involved the development of rate monotonic analysis (RMA), the extension of the theoretical basis for RMA, and its application to realistic systems. He has co-authored many papers and is a co-author of the RMA Handbook, A Practitioner's Handbook for Real-Time Analysis: Guide to Rate Monotonic Analysis for Real-Time Systems.

Find Us Here

Find us on Youtube  Find us on LinkedIn  Find us on twitter  Find us on Facebook

Share This Page

Share on Facebook  Send to your Twitter page  Save to del.ico.us  Save to LinkedIn  Digg this  Stumble this page.  Add to Technorati favorites  Save this page on your Google Home Page 

For more information

Contact Us

info@sei.cmu.edu

412-268-5800

Help us improve

Visitor feedback helps us continually improve our site.

Please tell us what you
think with this short
(< 5 minute) survey.