Evaluating System Architecture
One recurring theme in defense challenge problems is the need to predict problems before a system has been built. Maintenance and improvement costs represent more than half the total cost of a system, a percentage that has grown steadily since 1960. A problem for the DoD is to predict problems with modifiability before the system is constructed and before these problems occur.
The SEI pioneered the use of scenario-based methods in the evaluation of software architectures for modifiability and other qualities. The first SEI-developed architecture analysis method, the Software Architecture Analysis Method (SAAM), introduced the concept of a quality attribute scenario, giving specific modifications against which the system is to be tested. The SAAM led directly to the Architectural Tradeoff Analysis Method (ATAM), which evaluates a system for a collection of quality attributes.
Major defense contractors, such as Boeing and Raytheon, have architecture evaluation teams and architecture evaluation as a portion of their architect certification process. U.S. Army staff have reported that using scenario-based architecture evaluation methods reduced risk in schedule and cost, improved documentation, and resulted in a higher quality product.