Harvesting AI Computation at the Edge via Generic Approximation

AIoT processors often integrate specialized AI accelerators (DLAs) alongside general-purpose CPUs. While these DLAs are designed for peak neural network workloads, they frequently suffer from significant underutilization due to fluctuating demands. Conversely, CPUs often struggle with compute-intensive signal processing or numerical tasks. This paper investigates whether these idle AI resources can be harvested to accelerate non-AI tasks.

The authors introduce a framework that treats neural networks as universal function approximators. The process involves two main stages:

1. Offline Approximation: A Neural Architecture Search (NAS) method, specifically a modified DARTS framework, automatically designs compact neural networks to approximate traditional mathematical functions (e.g., FFT, trigonometric functions). The authors also implement a difficulty-aware piecewise approximation, which uses larger models for complex regions of a function and smaller ones for simpler regions to optimize efficiency.
2. Online Scheduling: A runtime scheduler uses an Earliest Finish Time (EFT) strategy to dynamically offload these approximate tasks to the DLA during its idle periods, ensuring that primary AI workloads are not compromised.

Experiments on the MAX78000 AIoT processor demonstrate that the proposed harvesting strategy effectively recycles idle DLA compute capacity. By offloading compute-intensive kernels to the DLA, the framework achieves up to 60.5% higher performance in synthetic application benchmarks and significant speedups in realistic tasks like motor control (FOC) and sensor fusion (EKF). The results show that the strategy successfully balances the trade-off between approximation accuracy and computational throughput, with performance gains scaling as more tasks are offloaded to the DLA.