Hybrid Compression: Integrating Pruning and Quantization for Optimized Neural Networks

As deep neural networks grow in size and complexity, their deployment on resource-constrained edge and mobile devices becomes increasingly difficult. This paper investigates how to effectively compress Convolutional Neural Networks (CNNs) to reduce memory and computational requirements without sacrificing performance.

The authors introduce a multi-stage compression framework consisting of three distinct phases:

1. Pruning: Using Automated Gradual Pruning (AGP), the method systematically removes redundant neural connections to reduce model complexity while avoiding layer collapse.
2. Quantization: The team employs Quantization-Aware Training (QAT) to convert high-precision floating-point weights into lower-bitwidth integers, further reducing memory usage.
3. Mixture of Experts (MoE): Finally, the compressed models are integrated into an MoE architecture. A gating network routes inputs to specific expert models, allowing the system to maintain high accuracy by leveraging the diverse representations of the compressed experts.

The proposed method demonstrates a 10x to 11x reduction in both FLOPs and total parameters across various CNN architectures (VGG16, ResNet18, InceptionV3, and DenseNet121). While individual pruning and quantization can lead to accuracy drops, the integration of the MoE paradigm successfully recovers performance, achieving accuracy levels comparable to or better than the original, uncompressed models. This suggests that the hybrid approach is highly effective for balancing efficiency and accuracy in resource-limited environments.