Modern data analytic methods and tools, including Artificial Intelligence (AI) and Machine Learning (ML) models, depend on correlations; however, such approaches fail to account for confounding in the data, which prevents accurate modeling of cause and effect and often leads to bias. Edge cases, drift in data/concept, and emerging phenomena undermine the significance of correlations relied upon by AI. New test and evaluation methods are therefore needed for ongoing evaluation. Carnegie Mellon University Software Engineering Institute (CMU SEI) has developed a new AI Robustness (AIR) tool that allows users to gauge AI and ML classifier performance with data-based confidence.

The SEI AIR tool offers a precedent-setting capability to improve the correctness of AI classifications and predictions, increasing confidence in the use of AI in development, testing, and operations decision making.

The AIR tool uses state-of-the art algorithms and techniques to:

• Build a causal graph (Step 1) that includes the treatment variable (X), outcome variable (Y), and intermediate variables (M); as well as other variables in the dataset.
• Determine adjustment sets (Step 2) Z1 and Z2, which are selected ancestors, respectively, of X and M, not on any direct path from X into Y; such that conditioning on either will remove the confounding effects that other variables have on X and Y.
• Finally, estimate the causal effect that changing X has on Y (Step 3) by calculating the average risk difference and associated 95% confidence intervals for each adjustment set. Then AIR compares the AI classifier’s predictions to these two confidence intervals to determine whether the classifier is significantly biased and should be retrained.

This project is sponsored and funded by the Office of the Under Secretary of Defense for Research and Engineering OUSD(R&E) to transition the AIR tool to users across the DoD to help identify when/where their AI classifiers are giving biased predictions. We are looking for DoD collaborators to use and provide feedback on our technology. As a participant, your AI and subject-matter experts will work with our team to identify known causal relationships and build out an initial causal graph. Contact us at to learn more.