HiFi-LLP: High-Fidelity, Low-Cost Latency Predictors with Confidence for Robust HW-NAS

As deep neural networks (DNNs) are increasingly deployed on edge devices, hardware-aware neural architecture search (HW-NAS) has become critical for optimizing performance. However, traditional hardware-in-the-loop (HIL) latency measurements are slow and sequential, creating a bottleneck. While existing latency predictors aim to replace HIL, they often struggle with low accuracy in data-scarce scenarios or lack the reliability required for robust optimization. This paper addresses these challenges by introducing HiFi-LLP, a predictor designed to be both sample-efficient and confidence-aware.

HiFi-LLP utilizes a Graph Attention Network (GATv2) architecture to model DNNs as graphs, where nodes represent layers and edges represent dependencies. To improve reliability, the authors incorporate a Gaussian Process (GP) regression head that provides both a predicted latency value and an associated uncertainty (variance). This uncertainty serves as a confidence metric. The framework also employs a Bayesian optimization-based data sampler to select informative training samples and a hybrid NAS pipeline that routes low-confidence predictions to actual HIL measurements, ensuring that the search process remains accurate even when the predictor is uncertain.

HiFi-LLP demonstrates superior performance compared to state-of-the-art predictors across six diverse hardware devices in the LatBench dataset. It achieves a Spearman’s rank correlation of up to 0.996 and outperforms existing methods by up to 9 percentage points in the 10% accuracy bound. By leveraging the confidence metric, the proposed hybrid NAS framework achieves up to 8.6x speedup compared to standard NAS while maintaining a competitive Pareto front of architectures. Furthermore, the model exhibits significantly lower variance across different training seeds, indicating higher robustness in real-world deployment scenarios.