MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Control

Existing video generation models struggle to produce coherent, multi-shot audio-visual content that adheres to professional filmmaking standards. Specifically, these models often fail to maintain temporal alignment between audio and visual events, lack fine-grained control over character identity and vocal timbre in multi-subject scenes, and rely on incomplete scripting that ignores the complex narrative timelines required for immersive storytelling.

MAVIN introduces a novel framework designed to address these challenges through three primary innovations:

1. Boundary-Aware Attention: This mechanism partitions latent spaces and routes tokens to ensure that audio and visual elements are rendered strictly within their designated temporal boundaries, preventing semantic leakage across shots.
2. ID-Aware Propagation: To maintain identity consistency, the model uses identity embeddings and masks to bind specific visual appearances and vocal timbres to characters, ensuring they remain consistent even in complex multi-subject scenes.
3. Multi-Agent Scripting Pipeline: This pipeline translates free-form user inputs into hierarchical captions (global, shot, and role-level), providing the necessary structural and temporal cues to guide the joint audio-visual generation process.

Additionally, the authors introduce MAVINSet, a large-scale dataset containing 800K samples with hierarchical annotations, and a manually verified 1K-sample benchmark to rigorously evaluate performance.

Extensive experiments demonstrate that MAVIN achieves state-of-the-art performance across 13 metrics, significantly outperforming both cascaded (T2V + V2A) and existing joint (T2AV) generation models. The framework excels in shot transition accuracy (STA) and character inter-shot consistency (CISC), effectively ensuring that characters speak and act strictly within their planned narrative intervals. Qualitative results confirm that MAVIN successfully preserves visual appearance and vocal timbre across cinematic cuts, offering a more practical solution for professional creative workflows.