Hierarchical Reinforcement Learning for Neural Network Compression (HiReLC): Pruning and Quantization

Research Approach

HiReLC (Hierarchical Reinforcement Learning for Neural Network Compression) addresses the challenge of compressing deep neural networks by automating the joint selection of pruning ratios, bitwidths, quantization types, and granularity. Unlike traditional methods that use single-level controllers, HiReLC employs a two-tier hierarchy: High-Level Agents (HLAs) manage global budget allocation across the network, while Low-Level Agents (LLAs) operate independently on individual blocks to refine local compression configurations. To handle the high computational cost of evaluating many compression candidates, the authors integrate an iterative active learning loop that uses a lightweight MLP surrogate to estimate performance, reducing the need for expensive full-model fine-tuning during the search process.

Key Innovations

The framework is architecture-agnostic, meaning the controller design is decoupled from the specific topology of the network being compressed. A critical component is the use of Fisher Information-based sensitivity estimates, which guide both the HLA budget allocation and the LLA reward shaping. This ensures that computationally or accuracy-sensitive blocks are prioritized for lighter compression. Furthermore, the use of an ensemble of agents—rather than a single controller—helps stabilize the search process, with the final compression policy determined by a voting mechanism that balances accuracy retention against storage constraints.

As deep learning models grow in size, deploying them on resource-constrained edge devices requires aggressive compression. HiReLC provides a practical, automated path to achieving high compression ratios (5.99–6.72x) without requiring manual, layer-by-layer tuning. By combining hierarchical policy decomposition with sensitivity-aware guidance, the framework offers a robust alternative to static heuristics and computationally expensive Hessian-based methods, making it a viable tool for optimizing both Vision Transformers and CNNs.