Software Engineering Institute | Carnegie Mellon University
Software Engineering Institute | Carnegie Mellon University

Evaluation Process

For each nomination, the following steps are completed:

  • The Humphrey Award Selection Subcommittee evaluates the degree to which the nominee's work satisfies the award criteria for significant, measured, sustained, and shared improvement.
    • To evaluate whether the work is significant, the subcommittee considers as factors not only the breadth and depth of the work's deployment throughout the target organizations (e.g., the number of projects that have benefited) but also the certainty of the connection between the software engineering process improvement work and achievement of the target organizations' performance goals (e.g., the strength of the link between the work and the improvements in productivity and quality). If a heterogeneous set of organizations (or projects) have different business goals (or different improvement starting points), it may not be sensible to aggregate deployment or impact data across organizations. In this case, the nominator should select a small set of individual organizations (possibly three to five) and report specifically the work for these.
    • To evaluate whether the work is measured, the subcommittee considers factors such as the breadth to which the data demonstrate the improvement (e.g., the measurement of customer satisfaction factors as well as workforce capability-related factors), the depth of the data regarding the improvement (e.g., the consideration of defect containment as well as defect incidence data), and the use of well-defined measures to demonstrate the improvement (e.g., the definition of a concrete measure relating planned vs. actual  project effort).
    • To evaluate whether the work is sustained, the subcommittee considers factors such as the work's level of documentation (i.e., whether the results of the work are described in informal memoranda, internal reports, or organizational standards), the likelihood that the results achieved to date are permanent (e.g., as indicated by the support of senior executives), and the likelihood that there will be additional improvements in the future (e.g., as indicated by establishing procedures for collecting, considering, and implementing process improvement suggestions).
    • To evaluate whether the work is shared, the subcommittee considers factors such as the degree to which the work has directly affected other organizations (e.g., the number of other organizations using methods influenced by the nominee's work) and the extent to which the nominee's work has impacted (or will impact) the nominee's local and global software engineering process improvement communities (e.g., in terms of presentations in professional society meetings).
  • The Selection Subcommittee decides whether the nominee's work satisfies all of the criteria to at least a minimal degree. If this is not the case, a detailed list of deficiencies is sent to the nominator and nominee accompanied by advice regarding a resubmission in the future. If the work at least minimally satisfies all of the criteria, then a detailed list of questions and issues is sent to the nominee.
  • The Selection Subcommittee interacts with the nominee to gather information addressing the questions and issues. If the Selection Subcommittee feels it is necessary (and, recently, this has been the exception rather than the norm), they may conduct an on-site visit to interact with the nominee and personnel from the target organizations.


Nominate Today

Do you know a person or team that deserves recognition for their process improvement activities?