EnerInfer: Energy-Aware On-Device LLM Inference

Research Context and Problem

On-device Large Language Model (LLM) inference is increasingly popular for privacy and latency reasons, but it is often bottlenecked by high energy consumption and thermal limits. Current systems typically prioritize maximum decoding speed, which often leads to inefficient hardware utilization. The authors identify that there is significant 'configuration slack'—opportunities to lower NPU and memory frequencies to save energy and reduce heat without noticeably degrading the user's quality of experience (QoE).

The EnerInfer Approach

EnerInfer addresses the challenge of finding the optimal hardware configuration (NPU and DDR frequency) for diverse models and platforms. Instead of relying on impractical per-model offline profiling or hardware-level power sensors, the authors propose a disaggregated, model-structure-aware prediction pipeline. This system predicts throughput and power for unseen models across various frequency settings. It uses an online, ranking-driven feedback controller to select configurations that satisfy QoE requirements while maximizing energy efficiency. Additionally, it incorporates a lightweight, limited-horizon thermal predictor to dynamically switch between energy-optimized and thermally constrained modes, ensuring the device remains within safe temperature limits.

Key Findings

Evaluations on smartphones, laptops, and development boards demonstrate that EnerInfer significantly improves energy efficiency—by up to 65% on phones—compared to default operating system governors. The framework successfully maintains throughput targets while reducing overall device energy consumption by up to 11%. The authors show that because the relationship between hardware frequency, model architecture, and energy efficiency is non-monotonic and model-dependent, their predictive approach is essential for production-ready deployment.

As LLMs become standard features on mobile devices, balancing battery life and thermal comfort is critical for user adoption. EnerInfer provides a practical, automated solution that bridges the gap between high-performance inference and sustainable mobile operation, allowing developers to deploy advanced models without causing excessive battery drain or overheating.