MakeupMirror: Improving Facial Attribute Preservation in Diffusion Models for Makeup Transfer

Traditional makeup transfer models often struggle to balance the accurate application of makeup with the preservation of a user's original facial identity and skin tone. Existing diffusion-based solutions frequently alter facial features or unintentionally shift skin tones when transferring makeup between subjects with different characteristics. This paper introduces MakeupMirror, a system designed to achieve high-fidelity, photorealistic virtual try-on (VTO) suitable for e-commerce applications.

MakeupMirror builds upon the Stable-Makeup architecture, introducing four key technical innovations to enhance control and fidelity:

• Geometric Conditioning: The model integrates ControlNets using Depth-Anything and Canny edge detection to explicitly constrain the diffusion process, ensuring that facial structure and anatomical landmarks are maintained.
• Region-Specific Strength Control: By using warped facial segmentation masks, the model dynamically adjusts the classifier-free guidance scale and diffusion time-steps. This allows for strong makeup application on eyes and lips while minimizing interference with the skin.
• Adaptive Skin Tone Preservation: The system calculates the perceptual skin tone difference between the source and reference images using the Monk scale. If a significant difference is detected, it automatically modulates the transfer intensity to prevent unwanted skin tone shifts.
• Inference Acceleration: The authors integrate a Levenberg-Marquardt Langevin sampler, which reduces the number of required diffusion steps, achieving a 2.8x speed-up with a total latency of 0.7s.

Experiments conducted on public datasets (CPM-Real, Makeup Wild) and a newly collected, diverse dataset (MakeupSelfies) demonstrate that MakeupMirror outperforms the state-of-the-art. The model achieves a 60% improvement in relative facial recognition similarity and a 50% reduction in relative skin tone difference compared to Stable-Makeup. Furthermore, the system attained a 94% acceptance rate in expert audits, confirming its viability for professional-grade virtual try-on scenarios.