Reviving Ancient Paintings via Poem: A Colorization Framework for Aligning Cultural Semantics

Restoring the original colors of ancient paintings is a highly ambiguous task because faded pigments offer little guidance, and modern generative models often introduce anachronistic colors or semantic biases. This paper addresses how to leverage the traditional aesthetic principle of "congruence between poems and paintings" to guide the restoration process, ensuring that the recovered colors are both historically faithful and poetically resonant.

The authors introduce PoemColor, a framework that integrates two primary innovations into a diffusion-based restoration model:

1. Poetic Painting Projector (P3): Instead of using raw text embeddings, which are often dominated by modern concepts, P3 maps poetic context into a compact, learnable "classical color manifold." This ensures the model operates within a historically plausible color space rather than an unconstrained one.

2. Structure-Aware Semantic Attention (SASA): To prevent the "color leakage" common in standard diffusion models, SASA uses the painting's original ink contours to regulate color propagation. It estimates boundary-normal fields and uses a dynamic spatial gate to ensure that colors are injected only within appropriate brushstroke regions, preserving the integrity of the original ink topology.

To overcome the scarcity of paired training data, the authors constructed a hybrid dataset that combines synthetic degradation with high-quality, expert-restored artifacts, providing both scale and historical accuracy.

Extensive experiments demonstrate that PoemColor significantly outperforms state-of-the-art GAN, Transformer, and diffusion-based baselines. Quantitative metrics show a 54% reduction in color distribution error compared to existing methods, with superior LPIPS and FID scores. Qualitatively, the model successfully revives the intended mood and seasonal colors described in associated classical poems while maintaining the structural precision of the original ink work.