Managing Technical Debt with Data-Driven Analysis

Technical Debt: A Compounding Problem

Technical debt is a term that conceptualizes the tradeoff between the short-term benefits of rapid delivery and the long-term value of developing a software system that is easy to evolve, modify, repair, and sustain. Like financial debt, technical debt can be a burden or an investment. Technical debt can be a burden when it is taken on unintentionally without a solid plan to manage it. Technical debt can also be part of an intentional investment strategy that speeds up development … as long as you have a plan to pay it back before the interest swamps your principal.

The U.S. Department of Defense (DoD) must constantly manage the dual challenges of budget constraints and the need to accelerate capability delivery. These challenges have encouraged the DoD to adopt incremental approaches to software development and to shift from the acquisition of new systems to the more cost-effective evolution and sustainment of existing systems. In this context, accumulated design and implementation decisions made for expediency, in the absence of structural quality requirements or without due consideration for sustainment and evolution, often result in systems that become prohibitively expensive to maintain or extend. In other words, these systems have a lot of technical debt.

Current tools for quantifying debt focus primarily on code quality and implementation decisions, such as complexity and cyclicity metrics. Our research has shown that key contributors to technical debt are design and architectural decisions that are not managed in terms of their consequences for system quality and rework. Existing tools cannot detect technical debt that results from design and architectural decisions. Moreover, the DoD and industry have started to invest heavily in model-driven and compositional software development approaches that existing technical debt identification techniques do not address. These types of debt are harder to recognize and occur earlier in system development, so the consequences can accrue for longer periods of time before they become visible.

Our Solution: Measure It and Manage It

The SEI is making software more sustainable through a comprehensive approach to managing technical debt. We are developing tools and techniques for uncovering technical debt that integrate data from multiple commonly available sources to reveal problematic decisions and quantify their consequences in a repeatable and reliable way. Together, the tools and techniques provide a complete view of the technical debt that you need to manage.

Our tooling approach combines existing code-oriented analysis tools with techniques that uncover and analyze architectural abstractions to reveal the structural causes of technical debt. It provides a more accurate scope for technical debt that exists in the system and assists both ongoing development and difficult sustainment decisions such as how to balance system improvements, early delivery, and upgrades.

Our Approach: Automating Technical Debt Analysis Through Software Analytics

We combine techniques from machine learning, refactoring and code analysis, and data mining with multiple data sources to describe technical debt items that identify problematic design issues.

Create a technical debt classifier: Apply topic-modeling algorithms to issue tracker data to extract topics related to rework; extract categories of design related to technical debt.
Correlate analysis rules with technical debt topics: Identify recurring design concepts and their mappings to code analysis rules; run code analyzers to detect quality violations to identify candidate technical debt items.
Consolidate technical debt items: Run criteria for consolidations, and extract additional affected files with related violations.
Rank technical debt items: Identify defects, change, and bug churn and locations in the code base that require changes; run causal analysis, and create an initial ranking.

This project will deliver a suite of tools and techniques for technical debt analytics, including

a prototype tool that demonstrates clustering and ranking of technical debt items
an extraction technique and a description template for technical debt items
data set exemplars for organizations and researchers

Software Engineering Institute