Alizishaan Khatri, Chiquita Prabhu
5 min
As AI-powered video generation models become more capable, the risk of generating non-consensual or illegal adult content increases. Current moderation methods typically operate at the interface level—either by filtering text prompts or by decoding generated video into pixels for classification. These methods are computationally expensive and ignore the rich semantic information already present in the model's internal states. This paper asks: can we perform accurate, real-time safety moderation by probing the internal latent representations of a video diffusion model?
The authors propose a latent-space probing framework that attaches a lightweight classifier directly to the denoised latent tensors produced by the CogVideoX model. By intercepting these latents before they are passed to the autoencoder decoder, the probe can identify harmful content without the latency penalty of a full pixel-space decode. To train these probes, the authors constructed a large-scale dataset of 11,039 ten-second video clips, balanced between adult and non-adult content. They evaluated two probe architectures: a simple 3D CNN and a more complex CNN-Transformer hybrid that models temporal dependencies across frames.
The latent-space probing approach is highly effective and efficient. The CNN-Transformer probe achieved an F1 score of 97.29% on a held-out test set. Because the probe operates on compressed latent tensors rather than high-resolution pixel data, it adds only 4-6ms of overhead per inference request, compared to 3-5 seconds for traditional pixel-space classifiers. The authors also found that these probes are modality-invariant, meaning a probe trained on text-to-video latents performs well on image-to-video and video-to-video tasks without modification.
This work demonstrates that internal model representations contain highly discriminative semantic signals that can be leveraged for safety at a fraction of the cost of traditional methods. By shifting the moderation paradigm from post-generation pixel analysis to in-generation latent analysis, developers can implement more scalable and responsive safety guardrails for generative video systems.
Sam: So the CNN gives you frame-level features, and the Transformer integrates them into a clip-level judgment.
Alex: That's the right way to think about it. And they trained this on a purpose-built dataset of over eleven thousand clips, balanced across categories, to give the probe enough signal to generalize.
Sam: Where does the paper's confidence break down? There has to be a constraint on what these results actually support.
Alex: The primary one is distribution shift. The probes are trained on encoded real-world videos, but at inference they're operating on model-generated latents — which may have different statistical properties. That domain gap is acknowledged but not fully characterized. The authors are also transparent that adversarial robustness hasn't been tested. A prompt engineered to evade detection could exploit the fact that the probe never sees pixel space at all.
Sam: Which raises a broader question about whether the latent space remains a reliable diagnostic as models evolve and their internal representations shift.
Alex: Right. The probe is essentially a learned mapping from one model's latent geometry to a safety label. If the generative model is retrained or fine-tuned, the probe likely needs to be retrained too. That's a maintenance cost the paper doesn't fully account for.
Sam: Still, the conceptual move here is notable — using the model's own internal representations as the substrate for safety, rather than treating the model as a black box and inspecting only its outputs.
Alex: And it has practical implications beyond video. If the same approach generalizes to other diffusion-based modalities — image, audio, eventually multimodal systems — you're looking at a modular safety layer that slots into the inference pipeline at near-zero latency cost. That's a meaningful architectural shift in how safety is operationalized.
Sam: The efficiency numbers are hard to argue with. The open question is how robust that 97% figure is once you move outside the training distribution.
Alex: Which is exactly where the next round of work needs to go — systematic evaluation under distribution shift, and some adversarial probing of the probe itself. Thanks for listening to ResearchPod.