MultiAnimate: A Unified Framework for Controllable Multi-Character Animation

Existing character animation models are primarily designed for single-character scenarios or require all characters to be present within a single reference image. This limitation prevents users from animating multiple distinct characters independently. MultiAnimate addresses this by asking: How can we enable the concurrent animation of multiple characters using separate reference images and individual pose sequences within a shared environment?

The authors introduce three primary architectural innovations to solve the challenges of identity preservation and spatial interaction:

1. Identity-Specific Reference Net: Unlike traditional reference nets that struggle with multiple inputs, this module adds learnable positional encodings to features extracted from separate reference images. This allows the model to distinguish between characters and prevent feature attribution errors, such as incorrect clothing assignment.

2. Identity-Aware Pose Encoder: To solve the 'character-pose binding' problem, the framework uses a cross-attention mechanism. This binds specific identity information from a reference image to its corresponding pose sequence, ensuring the model knows which character should follow which motion.

3. Interaction Guider: This optional module uses character-specific masks (generated by tools like SAM2) to refine pose sequences. It explicitly handles occlusion by rendering occluded parts of a skeleton in a specific way, helping the model maintain correct spatial relationships when characters move in front of or behind one another.

MultiAnimate significantly expands the practical utility of generative video models for professional applications like animation production and advertising. By allowing users to mix and match characters from different sources and control their interactions through independent pose sequences, it provides a level of creative flexibility that was previously unavailable in diffusion-based animation frameworks.