LCG: Long-Context Consistent Image Generation with Sparse Relational Attention

Overview

Long-Context Generation (LCG) is a diffusion-based framework designed to address the challenge of maintaining visual and semantic consistency across long sequences of images, such as those required for comics, storyboards, or visual narratives. While existing text-to-image models excel at single-image synthesis, they often struggle with identity drift or role confusion when generating multiple related images. LCG solves this by processing a sequence of prompts in parallel and enabling controlled information exchange between these generation branches.

The core of the LCG framework consists of two primary innovations:

• Sparse Relational Attention (SRA): Instead of performing computationally expensive dense attention across all images, SRA compresses visual context into block-level summaries. It then allows each generation branch to selectively attend only to the most relevant tokens from other branches, ensuring that character identity and scene layout remain consistent without exceeding memory budgets.
• Routing Consistency Constraint (RCC): To prevent the model from attending to semantically irrelevant regions, RCC uses identity-aware masks to guide the attention mechanism. By penalizing deviations from these mask-derived correspondences, the model learns to maintain stable, semantically grounded links between characters across different panels.

Dataset and Evaluation

To facilitate research in this area, the authors introduced the Long-Context Consistency Dataset (LCCD), which contains 600,000 training sequences and a 1,000-sequence test set. Each sequence spans 6 to 20 images, providing a significantly larger and more complex benchmark than previous datasets. Experimental results demonstrate that LCG outperforms existing baselines in prompt alignment, character consistency, and overall visual fidelity, particularly in complex multi-character scenes.