Can You Rely on Your AI? Applying the AIR Tool to Improve Classifier Performance
• Webcast
Publisher
Software Engineering Institute
Watch
Abstract
Modern analytic methods, including artificial intelligence (AI) and machine learning (ML) classifiers, depend on correlations; however, such approaches fail to account for confounding in the data, which prevents accurate modeling of cause and effect and often leads to prediction bias. The Software Engineering Institute (SEI) has developed a new AI Robustness (AIR) tool that allows users to gauge AI and ML classifier performance with unprecedented confidence. This project is sponsored by the Office of the Under Secretary of Defense for Research and Engineering to transition use of our AIR tool to AI users across the Department of Defense. During the webcast, the research team will hold a panel discussion on the AIR tool and discuss opportunities for collaboration. Our team efforts focus strongly on transition and provide guidance, training, and software that put our transition collaborators on a path to successful adoption of this technology to meet their AI/ML evaluation needs.
What Attendees Will Learn:
- How AIR adds analytical capability that didn’t previously exist, enabling an analysis to characterize and measure the overall accuracy of the AI as the underlying environment changes
- Examples of the AIR process and results from causal discovery to causal identification to causal inference
- Opportunities for partnership and collaboration
About the Speaker
Linda Parker Gates
Linda Parker Gates is the principal investigator on the Software Engineering Institute’s (SEI's) Artificial Intelligence Robustness (AIR) research and transition project and leads the Software Acquisition Pathways Initiative in the SEI's Software Solutions Division. In both roles, she leverages her specialization in strategic planning, technology transition, change management, and performance …
Read more
Crisanne Nolan
Crisanne Nolan is an agile transformation engineer in the Software Solutions Division. She holds an MA in English & Literature from the University of Pittsburgh and an MS in Public Management and Policy from Carnegie Mellon University.
Read more
Michael D. Konrad
Dr. Michael D. Konrad is a principal researcher at the Software Engineering Institute (SEI) of Carnegie Mellon University. He applies causal discovery and inference, and more broadly, artificial intelligence/machine learning, to systems engineering and software engineering problems. From 1999-2013, Konrad was the model team lead for the CMMI for Development …
Read more
Suzanne Miller
Suzanne Miller is an SEI alumni employee.
Suzanne Miller is a principal researcher at the Software Engineering Institute of Carnegie Mellon University in the Continuous Deployment of Capability Directorate. Miller actively supports multiple large DoD cyber-physical programs in their Agile/Lean adoption efforts, in addition to designing and teaching Agile courses …
Read more
Nicholas Testa
Nick Testa is a Senior Data Scientist on the SEMA team within the Software Solutions Division (SSD). Since joining the SEI in August 2022, Nick has been involved in several research projects that involve applying tools like anomaly detection, causal discovery and inference, and experimental design and metrics.
Before joining …
Read more
David James Shepard
David Shepard has made a career working in many different areas of the information-technology field, but since 2010 he has worked as a software developer within the SEI’s Software Solutions Division. Shepard has spent time building networks, administering servers, designing software, writing and debugging software, working on process-improvement initiatives, auditing …
Read more