Our Research
Artificial Intelligence Engineering
AI engineering is an emergent discipline focused on developing tools, systems, and processes to enable the application of artificial intelligence in real-world contexts.
In contrast to the prevalent rush to develop capabilities and progress individual tools, AI Engineering asks a different set of questions: How can AI help humans achieve mission outcomes? What are the limits of AI systems in practice today? How can we ensure that ethical standards are upheld as AI systems are deployed?
The rise in availability of computing power and massive datasets have led to the creation of new AI, models, and algorithms encompassing thousands of variables and capable of making rapid and impactful decisions. Too often, though, these capabilities work only in controlled environments and are difficult to replicate, verify, and validate in the real world.
The need for an engineering discipline to guide the development and deployment of AI capabilities is urgent. For example, while an autonomous vehicle functions well cruising down an empty race track on a sunny day, how can it be designed to function just as effectively during a hail storm in New York City? AI engineering aims to provide a framework and tools to proactively design AI systems to function in environments characterized by high degrees of complexity, ambiguity, and dynamism. The discipline of AI engineering aims to equip practitioners to develop systems across the enterprise-to-edge spectrum, to anticipate requirements in changing operational environments and conditions, and to ensure human needs are translated into understandable, ethical, and thus trustworthy AI.
Developing the Discipline of AI Engineering
AI Engineering is taking shape as a discipline already across different organizations and institutions. We at the SEI see ourselves not only a source of AI Engineering expertise, but also as conveners and catalysts, bringing together people and ideas to share the lessons learned, the techniques developed, and the discoveries made.
With funding and guidance from the U.S. Office of the Director of National Intelligence (ODNI), the SEI is leading a national initiative to advance the discipline of AI engineering that aligns with the DoD’s vision of creating viable, trusted, and extensible AI systems.
Our Pillars to AI Engineering
AI Engineering is a field of research and practice that combines the principles of systems engineering, software engineering, computer science, and human-centered design to create AI systems in accordance with human needs for mission outcomes. Through conversations with partners, we’ve developed three pillars to guide our approach to AI Engineering.
Human-centered AI
Scalable AI
Robust and Secure AI
AI Engineering Today
The SEI works to publish information to advance the field of AI, and to bring other leading work to researchers and other partners who are trying to develop secure and robust AI. Seminal resources we look to in advancing the discipline of AI Engineering include the following:
- In the 2014 paper Machine Learning: The High Interest Credit Card of Technical Debt, researchers from Google explain the maintenance challenges unique to machine learning and offer recommendations for avoiding risks.
- The SEI short paper AI Engineering: 11 Foundational Practices offers recommendations to help organizations build, acquire, and integrate artificial intelligence capabilities into business and mission systems.
- In Software Engineering for Machine Learning: A Case Study, researchers from Microsoft offer observations include the importance of data, ML model development is fundamentally different from traditional software engineering, and AI systems are harder to modularize.
What We Offer
The Latest from the SEI Blog
Beyond Capable: Accuracy, Calibration, and Robustness in Large Language Models
December 03, 2024 • Blog Post
Matthew Walsh, David Schulker, Shing-hon Lau
For any organization seeking to responsibly harness the potential of large language models, we present a holistic approach to LLM evaluation that goes beyond...
read
GenAI for Code Review of C++ and Java
November 18, 2024 • Blog Post
David Schulker
Would ChatGPT-3.5 and ChatGPT-4o correctly identify errors in noncompliant code and correctly recognize compliant code as...
read
Our Vision for the Future Of Artificial Intelligence Engineering
Bolstered by our expertise in developing applications for AI, the SEI is leading a movement to cultivate and mature the professional discipline of AI engineering. This discipline will lay the groundwork for developing scalable, robust and secure, and human-centered AI systems as well as the planning and commitment it takes to support, expand, and evolve those systems for the coming decades.
Join the movement to establish and mature the AI Engineering discipline.