DoD applications at the tactical edge will increasingly involve the use of sensors and other devices across a range of edge-based capabilities. To support this capability, an emerging branch of ultra-low-power machine learning (ML) technology has arisen, in part, to meet the needs of this technology. To make it work, the DoD needs access to high-performing software support for the wide variety of hardware architectures encountered in the embedded space. However, the software ecosystems for low-power embedded devices remain rudimentary, highly varied or non-existent, and dependent on users to develop their own software stack.

To address these challenges, our project, Portable High-Performance Inference at the Tactical Edge (PHITE), a collaboration with experts from the Department of Electrical and Computer Engineering at Carnegie Mellon University, applies performance engineering processes to the analysis of existing open source ML frameworks for embedded systems, to inform the development and optimization of a portable software library that can achieve significantly higher performance (10–100x power efficiency) for a set of ML applications across a range of targeted embedded devices.

Our approach proceeds in three phases:

1. selection and characterization of baseline applications and hardware platforms
2. software design and development
3. integration, evaluation, benchmarking, and demonstration of open source benchmark applications that have DoD relevance

In the first year of the project, we analyzed application workloads and performance modelling of the hardware, specified the first version of the API specification for the portable inference interface, and completed the initial implementations of a library of ML algorithms for each ultra-low power hardware platform. In year two, we will continue to optimize the portable library of ML algorithms (targeting 10x—100x performance gains over the baseline) for embedded devices and address any additional needs to support a live data stream.

In Context

This FY2022–23 project

• aligns with the SEI technical objectives to bring capabilities that make new missions possible or improve the likelihood of success of existing ones
• aligns with the SEI technical objective to be timely so that the cadence of acquisition, delivery, and fielding is responsive to and anticipatory of the operational tempo of DoD warfighters and that the DoD is able to field these new software-enabled systems and their upgrades faster than our adversaries
• aligns with the DoD software strategy to realize computational and algorithmic advantage