P-PatchDiff: Progressive Patch Diffusion Models for Low-light Image Enhancement

Low-light image enhancement requires balancing local detail recovery with global brightness consistency. While diffusion models produce high-quality results, they are computationally expensive. Existing patch-based diffusion models often use fixed patch sizes, which fail to capture global context, or multi-scale approaches that significantly increase inference time. This paper asks: can a progressive patch-based strategy effectively balance local and global information while maintaining high efficiency and scalability across varying image resolutions?

The authors propose P-PatchDiff, a framework that dynamically adjusts patch size and stride throughout the denoising process. By dividing the total diffusion time steps into subsets, the model gradually shifts from local to global views. To ensure visual coherence, the authors introduce a Multi-Patch Alignment strategy that uses a lightweight U-Net to estimate a global brightness proxy from the input image. This proxy is then used to normalize features across different patch scales, ensuring that locally enhanced regions are consistent with the overall image illumination.

P-PatchDiff demonstrates significant improvements in both efficiency and performance. It is empirically shown to be 80 times faster than existing patch diffusion models while maintaining high-quality outputs on images ranging from 400x600 to 4K resolution. The progressive strategy allows the model to capture multi-level information without the computational overhead of multi-scale cropping. Furthermore, the multi-patch alignment strategy effectively mitigates the color and brightness inconsistencies typically found in patch-based restoration, achieving competitive results on both in-domain and cross-domain benchmarks.

This work provides a scalable solution for high-resolution image restoration using diffusion models. By addressing the trade-off between receptive field size and computational cost, P-PatchDiff enables the deployment of generative models on devices with limited memory. The framework is also shown to generalize well to other restoration tasks, such as deblurring and deraining, suggesting that the progressive patch strategy is a robust approach for various image enhancement challenges.