Jingbo Lin, Zhilu Zhang, Fan Li, Renjing Pei, Wenbo Li, Hang Xu, Hongzhi Zhang, Wangmeng Zuo
7 min
Abstract
Universal image restoration aims to handle diverse degradations within a single model, but a reliable scaling strategy remains underexplored. Increasing backbone depth and width can lead to performance saturation, while degradation-aware specialization depends on accurate degradation estimation and is vulnerable to errors on out-of-distribution and real-world inputs. We propose UniRestorer++, a scalable framework based on multi-granularity degradation modeling. It progressively refines degradation groups and introduces group-specific experts, coupling capacity growth with specialization. Degradation-estimation-based routing selects a root-to-leaf path, while granularity-estimation-based routing chooses a level along that path to balance specialization and robustness. Spatially dense degradation representation maps further enable region-aware routing for spatially non-uniform corruptions. We develop CDD-50, a benchmark covering 50 degradation categories from single to penta-degradation. Experiments show approximately log-linear PSNR trends over the evaluated model-size range and stronger scaling gains on out-of-distribution and real-world data than vanilla MoE scaling. UniRestorer++ achieves state-of-the-art performance across benchmarks and demonstrates zero-shot generalization and few-shot transfer on real-world images. Code: https://github.com/mrluin/UniRestorer_plusplus
Sam: They learn a degradation representation, meaning a description of the image damage. Training aligns local image descriptions with text describing damage type and severity. They then cluster those learned descriptions recursively to construct the hierarchy.
Alex: But a photograph can be blurry in one region and relatively sharp elsewhere. Does the whole image receive the same expert?
Sam: No. They retain a spatial map of the damage descriptions rather than only an image-wide summary. Different locations can select different experts. Routing also uses the current restoration features, so those decisions can adapt throughout the network.
Alex: That sounds expensive if every region has a large menu of experts. What actually runs?
Sam: A shared restoration backbone runs, with at most one additional expert contribution per location in each block. The experts use low-rank adaptation, compact weight updates added to the shared model. So total stored capacity can grow without activating every expert.
Alex: Let's turn to the evidence. What would distinguish a useful scaling strategy from a method that merely beats a smaller baseline?
Sam: They compare enlarging the shared network, adding finest-level experts, and retaining experts across hierarchy levels. Their hierarchical variants span about 154 million to 2.9 billion parameters. The comparison asks whether additional capacity keeps improving measured restoration.
Alex: What kinds of damage did they test? A broad claim needs more than separate tests of blur and noise.
Sam: They introduce CDD-50, a benchmark containing fifty degradation categories. It includes individual corruptions and mixtures with several operations applied sequentially. They test both training-range parameters and unseen parameter ranges, then evaluate on real-world datasets too.
Alex: So their synthetic out-of-distribution test changes damage parameters beyond the training range. The real-world tests provide a different check on generalization.
Sam: Yes. For accuracy they report peak signal-to-noise ratio, or PSNR, where higher values mean lower pixel reconstruction error. Hierarchical scaling improves that score across the evaluated sizes. Enlarging the shared network alone gives a non-monotonic pattern.
Alex: And against ordinary expert scaling? That seems the more demanding comparison, since it also reduces interference between tasks.
Sam: Both expert strategies show approximately log-linear trends: gains track the logarithm of model size over the tested range. Their fitted gains are similar on familiar synthetic damage. The hierarchy has steeper gains on unfamiliar synthetic parameters and the real-world FoundIR benchmark.
Alex: Can you anchor the real-world improvement with one number?
Sam: Across the hierarchical size range, FoundIR improves by roughly two decibels. That result comes from models trained on the complete synthetic degradation space, not fine-tuned on FoundIR. It is evidence that additional capacity can help beyond familiar synthetic inputs.
Alex: But a fitted line across these models is not a promise that scaling continues indefinitely.
Sam: The paper explicitly scopes the trend to the evaluated range. It also distinguishes its empirical fit from a theoretical scaling relationship. I would read this as evidence for a better scaling design, not a universal law.
Alex: Do they show that uncertainty-driven fallback matters, rather than simply having more experts available?
Sam: They compare the full system with a variant forced to choose the finest level. As damage parameters move away from training conditions, the full system increasingly selects broader experts. That adaptation preserves familiar-data accuracy while improving robustness under larger shifts.
Alex: And the spatial component needs its own test. Otherwise the hierarchy could explain the gains attributed to local routing.
Sam: They test that separately. Learning dense damage descriptions helps even when those descriptions are averaged before routing. Preserving their spatial information improves the result further, with gains on dehazing and deblurring, both representative spatially varying degradations.
Alex: How far does this get toward specialist systems trained for a particular real-world task?
Sam: They adapt the largest model using only a few whole training images, updating the routers and experts while freezing the shared backbone. With five-image adaptation, it exceeds the compared specialists on fourteen of seventeen test sets. That is an adaptation result, not a blanket zero-shot victory.
Alex: What is the limitation someone considering this architecture should keep foremost?
Sam: The hierarchy and expert pool are built offline and stay fixed during inference. Routing can select among existing capabilities; it cannot create an expert for new damage. The authors propose a self-improving restoration agent as future work, not something demonstrated here.
Alex: So who should read the full document, and where should they begin?
Sam: Researchers in universal restoration or expert routing should start with the scaling comparisons in Section Four D. Then read the routing ablations in Section Five to see what supports the mechanism. Building it also requires substantial training: the largest model took weeks on dozens of graphics processors.
Alex: And for someone who only needs the principle to carry back to their own research?
Sam: In these experiments, more specialists helped because uncertainty could change the level of expertise selected. The line to remember is this: grow the specialist pool, but keep a route back to broader expertise.
Alex: Thanks for listening.