Research Question

Standard Image Aesthetic Assessment (IAA) models typically use a single, shared parameter set to predict an overall aesthetic score. However, aesthetic quality is a multi-faceted concept influenced by distinct attributes like brightness, contrast, and blur. The authors investigate whether this shared-parameter approach leads to optimization interference, where updates beneficial for one attribute-dominant subset of images conflict with those required for another, resulting in systematic prediction biases.

Approach

To address this, the authors provide a theoretical analysis of gradient conflict in IAA. They define gradient conflict as the scenario where subset-specific gradients point in opposing directions, leading to cancellation and stagnation. To mitigate this, they introduce AGREE (Attribute-guided Gradient Routing for Establishing Agreement), a plug-and-play framework consisting of four mechanisms:

1. Attribute-aware Decoupling: Decomposes the model into separate parameter branches for different attributes to eliminate parameter-level conflict.
2. Multimodal Fusion: Injects frozen text-based semantic anchors to enhance feature-level separation across attributes.
3. Sensitivity-guided Routing: Uses offline perturbation analysis to estimate sample-wise attribute sensitivity, routing features based on these signals.
4. Error-aware Reweighting: Dynamically upweights persistently hard samples during training to ensure the model focuses on difficult, under-represented cases.

Main Findings

Experiments across five diverse IAA benchmarks (AVA, LAPIS, AADB, TAD66K, and PARA) demonstrate that AGREE consistently improves the performance of six representative IAA baselines. The method achieves state-of-the-art results on all datasets, with significant gains in correlation metrics (SRCC/PLCC) and substantial reductions in prediction errors. Notably, the improvements are most pronounced on hard samples—images that consistently fail across multiple baseline models—suggesting that AGREE successfully corrects systematic biases caused by attribute-level optimization imbalance.

Why It Matters

This work shifts the focus of IAA research from architectural innovation to optimization coordination. By identifying that common model failures are not just isolated outliers but are rooted in attribute-level gradient conflict, the authors provide a robust, model-agnostic solution. This approach is particularly valuable for applications like content recommendation and photo curation, where consistent performance across diverse image styles and aesthetic attributes is critical.

Key Terms and Definitions

• Gradient Conflict — A condition where gradients from different attribute-dominant subsets of data point in opposing directions, causing cancellation and hindering convergence in shared-parameter models.
• Attribute-aware Decoupling — The strategy of using separate parameter branches for different aesthetic attributes to prevent cross-attribute interference during training.
• Sensitivity-guided Routing — A mechanism that uses per-sample attribute sensitivity estimates to dynamically weight and combine features from different attribute branches.
• Dominant-attribute Marginalization — A metric measuring how rare the dominant attribute of a specific image is within the training set, used to identify samples that are harder to optimize.
• Error-aware Reweighting — A training technique that dynamically increases the loss weight for samples that exhibit consistently high prediction errors, forcing the model to prioritize difficult cases.