In-context Region-based Drag: Drag Any Region to Any Shape

Traditional drag-style image editing often relies on point-based inputs, which are inherently ambiguous and lack the precision required for complex geometric transformations. This paper addresses the need for more robust, region-based control by proposing a framework that treats source and target region masks as explicit conditioning inputs, allowing for fine-grained manipulation of object shapes and poses.

ICRDrag is built upon a Diffusion Transformer (DiT) architecture and operates under an in-context learning paradigm. The model takes a source image, a source mask, and a target mask as inputs to synthesize the edited image in a single forward pass. To ensure high-quality results, the authors introduce two key innovations:

• Image-Mask Attention Consistency (IMAC): This regularization forces the model to align visual attention with spatial structure, ensuring that the generation process is grounded in the regions defined by the masks.
• Source-Target Attention Correspondence (STAC): This mechanism enforces mutual correspondence between source and target regions, ensuring that the model maintains structural integrity during the transformation.

Additionally, the authors employ a two-stage curriculum training strategy, starting with complete masks and progressing to incomplete, sparse masks to better simulate real-world user inputs. They also introduce the Paired Region Dataset (PRD), a large-scale collection of over 280,000 paired samples to facilitate training.

By shifting from ambiguous point-based interactions to dense, region-based control, ICRDrag provides a more intuitive and accurate way for users to perform complex image edits. The use of attention-based regularization and a large-scale, purpose-built dataset demonstrates a significant step forward in achieving high-fidelity, controllable image generation that respects the underlying geometry of the objects being edited.