Persistent Object Narratives for Token-Efficient Video Language Models

Video Large Language Models (Video-LLMs) typically rely on dense, frame-wise visual tokens, which are computationally expensive and fail to explicitly track objects across time. The authors investigate how to create a more compact, structured visual interface that represents video content through persistent object identities rather than raw frame sequences.

SlotNarrative introduces a three-stage pipeline to compress video data:

1. Object-Centric Encoding: A recurrent slot encoder groups visual features into object-like slots, providing local temporal continuity.
2. Persistent Memory: A lightweight, parameter-free memory module associates recurring observations across frames, even during temporary occlusions or scene cuts, to maintain consistent object identities.
3. Typed Tokenization: The system serializes these objects into a fixed inventory of 144 tokens, consisting of one 'identity token' (summarizing appearance) and multiple 'state tokens' (encoding geometry, trajectory, and segment-level changes).

This approach allows the system to remain independent of the number of sampled frames, as the token budget is fixed regardless of video length.

SlotNarrative demonstrates a superior accuracy-to-token-count trade-off compared to existing compact interfaces. By explicitly linking recurring object evidence, the model achieves strong performance on standard VideoQA benchmarks (MSVD-QA, MSRVTT-QA, and ActivityNet-QA) while using significantly fewer tokens than dense baselines. Ablation studies confirm that both the trajectory-tracking and persistent memory components are essential for maintaining accuracy, particularly in longer, untrimmed video sequences.

This work addresses the "token-intensity" bottleneck in video-language modeling. By moving away from dense frame-wise representations toward structured, object-centric narratives, researchers can build more efficient models that better capture the temporal evolution of objects, potentially enabling longer video context windows without the prohibitive computational costs of standard visual tokenization.