http://sei.cmu.edu/feeds/tag/Updates on changes and additions to the SEI Blog for posts matching Machine Learningen-usMon, 27 Apr 2026 00:00:00 -0400https://www.sei.cmu.edu/blog/data-poisoning-in-ai-models-the-case-for-chain-of-custody-controls/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesThis post explores data poisoning, which occurs when training data is modified to influence the performance of a model, and proposes cryptographic chain of custody as a mitigation.Renae Metcalf, Matt ChurillaMon, 27 Apr 2026 00:00:00 -0400https://www.sei.cmu.edu/blog/data-poisoning-in-ai-models-the-case-for-chain-of-custody-controls/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesMachine LearningAISIRThttps://www.sei.cmu.edu/blog/dataops-towards-more-reliable-machine-learning-systems/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesDecisions based on ML models can have significant consequences, and managing the raw material—data—in ML systems is a challenge. This post explains DataOps, an area that focuses on the management and optimization of data throughout its lifecycle.Daniel DeCapriaMon, 21 Apr 2025 00:00:00 -0400https://www.sei.cmu.edu/blog/dataops-towards-more-reliable-machine-learning-systems/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesArtificial Intelligence EngineeringMachine Learninghttps://www.sei.cmu.edu/blog/evaluating-llms-for-text-summarization-introduction/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesDeploying LLMs without human supervision and evaluation can lead to significant errors. This post outlines the fundamentals of LLM evaluation for text summarization in high-stakes applications.Shannon Gallagher, Swati Rallapalli, Tyler BrooksMon, 07 Apr 2025 00:00:00 -0400https://www.sei.cmu.edu/blog/evaluating-llms-for-text-summarization-introduction/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesMachine Learninghttps://www.sei.cmu.edu/blog/enhancing-machine-learning-assurance-with-portend/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesThis post introduces Portend, a new open source toolset that simulates data drift in machine learning models and identifies the proper metrics to detect drift in production environments.Jeffrey Hansen, Sebastián Echeverría, Lena Pons, Gabriel Moreno, Grace Lewis, Lihan ZhanMon, 24 Mar 2025 00:00:00 -0400https://www.sei.cmu.edu/blog/enhancing-machine-learning-assurance-with-portend/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesSoftware AssuranceMachine Learninghttps://www.sei.cmu.edu/blog/introducing-mlte-systems-approach-to-machine-learning-test-and-evaluation/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesMachine learning systems are notoriously difficult to test. This post introduces Machine Learning Test and Evaluation (MLTE), a new process and tool to mitigate this problem and create safer, more reliable systems.Alex Derr, Sebastián Echeverría, Katherine Maffey, Grace LewisMon, 17 Feb 2025 00:00:00 -0500https://www.sei.cmu.edu/blog/introducing-mlte-systems-approach-to-machine-learning-test-and-evaluation/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesTestingMachine Learninghttps://www.sei.cmu.edu/blog/cyber-informed-machine-learning/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesThis blog post proposes cyber-informed machine learning as a conceptual framework for emphasizing three types of explainability when ML is used for cybersecurity.Jeffrey Mellon, Clarence WorrellMon, 10 Feb 2025 00:00:00 -0500https://www.sei.cmu.edu/blog/cyber-informed-machine-learning/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesCybersecurityMachine LearningCybersecurity EngineeringAI Engineering and Machine Learninghttps://www.sei.cmu.edu/blog/the-myth-of-machine-learning-reproducibility-and-randomness-for-acquisitions-and-testing-evaluation-verification-and-validation/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesA reproducibility challenge faces machine learning (ML) systems today. This post explores configurations that increase reproducibility and provides recommendations for these challenges.Andrew Mellinger, Daniel Justice, Marissa Connor, Shannon Gallagher, Tyler BrooksMon, 13 Jan 2025 00:00:00 -0500https://www.sei.cmu.edu/blog/the-myth-of-machine-learning-reproducibility-and-randomness-for-acquisitions-and-testing-evaluation-verification-and-validation/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesAcquisition TransformationTestingMachine LearningVerificationhttps://www.sei.cmu.edu/blog/the-top-10-blog-posts-of-2024/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesThis post presents the top 10 most-visited posts of 2024, highlighting our work in software acquisition, artificial intelligence, large language models, secure coding, and more.Bill ScherlisMon, 06 Jan 2025 00:00:00 -0500https://www.sei.cmu.edu/blog/the-top-10-blog-posts-of-2024/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesSecure CodingInsider ThreatArtificial Intelligence EngineeringMachine LearningAI Engineering and Machine LearningAcquisition Transformationhttps://www.sei.cmu.edu/blog/introduction-to-mlops-bridging-machine-learning-and-operations/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesMachine learning operations (MLOps) has emerged as a critical discipline in artificial intelligence and data science. This post introduces MLOps and its applications.Daniel DeCapriaMon, 04 Nov 2024 00:00:00 -0500https://www.sei.cmu.edu/blog/introduction-to-mlops-bridging-machine-learning-and-operations/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesArtificial Intelligence EngineeringMachine LearningEdge Computinghttps://www.sei.cmu.edu/blog/measuring-ai-accuracy-with-the-ai-robustness-air-tool/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesUnderstanding your artificial intelligence (AI) system’s predictions can be challenging. In this post, SEI researchers discuss a new tool to help improve AI classifier performance.Michael Konrad, Nicholas Testa, Linda Parker Gates, Crisanne Nolan, David Shepard, Julie Cohen, Andrew Mellinger, Suzanne Miller, Melissa LudwickMon, 30 Sep 2024 00:00:00 -0400https://www.sei.cmu.edu/blog/measuring-ai-accuracy-with-the-ai-robustness-air-tool/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesMachine LearningArtificial Intelligencehttps://www.sei.cmu.edu/blog/the-challenge-of-adversarial-machine-learning/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesThis SEI Blog post examines how machine learning systems can be subverted through adversarial machine learning, the motivations of adversaries, and what researchers are doing to mitigate their attacks.Matt Churilla, Nathan VanHoudnos, Robert BeveridgeMon, 15 May 2023 00:00:00 -0400https://www.sei.cmu.edu/blog/the-challenge-of-adversarial-machine-learning/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesMachine Learninghttps://www.sei.cmu.edu/blog/tackling-collaboration-challenges-in-the-development-of-ml-enabled-systems/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesThis SEI blog post highlights research examining the collaboration challenges inherent in the development of machine-learning-enabled systems compared to traditional software development projects.Grace LewisMon, 27 Feb 2023 00:00:00 -0500https://www.sei.cmu.edu/blog/tackling-collaboration-challenges-in-the-development-of-ml-enabled-systems/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesMachine Learninghttps://www.sei.cmu.edu/blog/improving-automated-retraining-of-machine-learning-models/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesThis post describes how to improve representative MLOps pipelines by automating exploratory data-analysis tasks.Rachel Brower-SinningMon, 02 May 2022 00:00:00 -0400https://www.sei.cmu.edu/blog/improving-automated-retraining-of-machine-learning-models/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesArtificial Intelligence EngineeringMachine LearningArtificial Intelligencehttps://www.sei.cmu.edu/blog/six-dimensions-of-trust-in-autonomous-systems/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesThis post chronicles the adoption and growth of autonomous systems and provides six considerations for establishing trust.Paul NielsenWed, 20 Apr 2022 00:00:00 -0400https://www.sei.cmu.edu/blog/six-dimensions-of-trust-in-autonomous-systems/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesVulnerability AnalysisSoftware AssuranceVulnerability DiscoveryDevopsArtificial Intelligence EngineeringMachine LearningCybersecurityAutonomy and Counter-AutonomySoftware and Information AssuranceHuman-Machine InteractionsArtificial IntelligenceDigital EngineeringCyber-Physical Systemshttps://www.sei.cmu.edu/blog/release-of-scaife-system-version-200-provides-support-for-continuous-integration-ci-systems/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesKey features in new release of SCAIFE System Version 2.0.0 including support for continuous-integration (CI) systems, and status of evolving SEI SCAIFE workLori FlynnMon, 25 Oct 2021 00:00:00 -0400https://www.sei.cmu.edu/blog/release-of-scaife-system-version-200-provides-support-for-continuous-integration-ci-systems/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesContinuous Deployment of CapabilitySCALE: A Static Analysis Auditing ToolSecure CodingMachine LearningStatic AnalysisStatic Analysis Classification and PrioritizationSecure DevelopmentArtificial IntelligenceSource Code Analysis Integrated Framework Environment (SCAIFE)https://www.sei.cmu.edu/blog/systems-engineering-and-software-engineering-collaborating-for-the-smart-systems-of-the-future/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesConvergence between systems engineering and software engineering is forging new practices for engineering the smart systems of the future.Paul NielsenMon, 20 Sep 2021 00:00:00 -0400https://www.sei.cmu.edu/blog/systems-engineering-and-software-engineering-collaborating-for-the-smart-systems-of-the-future/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesSoftware ArchitectureArtificial Intelligence EngineeringMachine LearningArtificial IntelligenceSystems EngineeringDevopsData Modeling and AnalyticsDigital EngineeringModel-Based Systems EngineeringContinuous Deployment of Capabilityhttps://www.sei.cmu.edu/blog/software-engineering-for-machine-learning-characterizing-and-detecting-mismatch-in-machine-learning-systems/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesThis post describes how we are creating and assessing empirically validated practices to guide the development of machine-learning-enabled systems.Grace Lewis, Ipek OzkayaMon, 17 May 2021 00:00:00 -0400https://www.sei.cmu.edu/blog/software-engineering-for-machine-learning-characterizing-and-detecting-mismatch-in-machine-learning-systems/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesArtificial Intelligence EngineeringMachine LearningArtificial Intelligencehttps://www.sei.cmu.edu/blog/aligning-devsecops-and-machine-learning/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesLuiz Antunes explores the machine learning (ML) and DevSecOps domains and proposes ways to use them in collaboration for increased performance.Luiz AntunesMon, 03 May 2021 00:00:00 -0400https://www.sei.cmu.edu/blog/aligning-devsecops-and-machine-learning/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesMachine Learninghttps://www.sei.cmu.edu/blog/a-game-to-assess-human-decision-making-with-ai-support/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesIn decision-support systems based on AI, humans often make poor choices causing the systems to be abandoned. Rotem Guttman introduces a game that collects data on actual human decision making to determine effective designs for AI-system interfaces.Rotem GuttmanMon, 22 Mar 2021 00:00:00 -0400https://www.sei.cmu.edu/blog/a-game-to-assess-human-decision-making-with-ai-support/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesMachine LearningArtificial Intelligencehttps://www.sei.cmu.edu/blog/managing-the-risks-of-adopting-ai-engineering/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesThis SEI Blog post discusses how organizations can manage the risks associated with adopting AI engineering, including developing a risk management framework.Brett TuckerMon, 17 Aug 2020 00:00:00 -0400https://www.sei.cmu.edu/blog/managing-the-risks-of-adopting-ai-engineering/?utm_source=blog&utm_medium=rss&utm_campaign=my_site_updatesMachine LearningResilience Management Model (RMM)Artificial Intelligence