GuangJian Team
8 min
Abstract
Real-world forgery is inherently heterogeneous: it manifests not only as digital manipulation but also as physical spoofing, and is carried not only by images but also by videos. Despite substantial progress, a unified anti-forgery foundation model covering such a broad threat spectrum remains absent. In this paper, we present Venus-DefakerOne2, which substantially extends the forensic capability boundary along both the forgery-form and media-type dimensions. Trained on tens of millions of open-source and in-house samples, it incorporates two key designs. Tide-Style-Tuning alternates LoRA and full-parameter fine-tuning to balance computational efficiency, generalization, and hard-sample fitting while mitigating catastrophic forgetting. Entropy-based Test Adaptation determines the optimal prediction modality by analyzing prediction entropy. Venus-DefakerOne2 achieves new state-of-the-art performance across a comprehensive suite of benchmarks, including DeepFake faces (96.7%), AI-generated images (93.3%), tampered documents (87.6%), manipulated natural images (89.6%), physical spoofs (93.1%), and AI-generated videos (96.0%). It significantly outperforms both specialized expert models and its predecessor, DefakerOne, establishing itself as a powerful and versatile foundation model for binary-decision forensic tasks.
Alex: They call it Tide-Style Tuning. It begins with LoRA: small trainable adapters attached to a frozen backbone. That provides a restricted warm-up before the authors merge the adapters and open the backbone for broader training.
Sam: Then they restrict it again, rather than leaving everything trainable through the finish?
Alex: They freeze the backbone and attach a fresh adapter for new and difficult samples. They also increase training noise through the stages to discourage memorization. In their comparisons, this alternating schedule beats both adapter-only training and full-model training from the outset.
Sam: That addresses learning capacity. But a video still poses a different question: should the detector study the motion, or inspect the frames?
Alex: That’s the second main design. Motion can reveal flickering or broken continuity. Individual frames can reveal subtle textures or editing boundaries. The authors train the same backbone through both pathways, with a separate detection loss for each.
Sam: At deployment, do they average those answers? That could dilute strong evidence with a weaker prediction.
Alex: Instead, they compare prediction entropy—a measure of uncertainty. Each pathway produces a real-or-fake distribution. The system takes the answer from the pathway with lower entropy, meaning the more decisive prediction. There is no separately learned routing model.
Sam: More decisive does not necessarily mean more correct. Does their evidence show that confidence is a useful way to choose?
Alex: On average, yes. Their ablation holds the trained model fixed and changes only the input pathway or selection rule. Routing performs better overall than always choosing frames or always choosing native video. But that advantage does not hold on every benchmark.
Sam: Give us an example where the distinction matters.
Alex: On their new rumor-video benchmark, frames alone reach about ninety-eight percent. Routing reaches about ninety-five percent on the same evaluation. So the general selection rule improves the overall average, but it can lose to a fixed choice on a particular dataset.
Sam: That makes the claim more useful: the pathways complement each other, but uncertainty is not an oracle. How broad is the main evaluation?
Alex: They test detection on fifty-three benchmarks, plus separate tests of where tampering occurred. The paper reports the best domain-level scores across its evaluated categories. Those categories cover digitally altered faces, generated images, document edits, natural-image manipulations, physical spoofs, and videos.
Sam: “Best domain-level scores” is narrower than winning every test. Does a specialist ever beat it?
Alex: Yes. On a classic face presentation-attack benchmark, the dedicated detector FLIP remains ahead. Some individual image benchmarks also favor the predecessor or another method. The strongest supported reading is broad average performance, not universal superiority on every dataset.
Sam: The new rumor-video benchmark seems central to their generalization claim. What makes that test different from older video evaluations?
Alex: The authors argue that public video-forensics datasets lag newer generators. Their Realistic Rumor Video Benchmark uses contemporary open- and closed-source generators. Its subjects include military actions, political events, unrest, and public-health emergencies.
Sam: Those topics could themselves become shortcuts. How do they stop the detector from learning that one kind of scene means fake?
Alex: They describe authentic videos in detail, then use those descriptions to prompt the generators. That aims to align real and synthetic content. They evaluate each generator against the same authentic set, using balanced accuracy, which gives equal weight to recognizing real and fake videos.
Sam: And what is the comparison that matters most?
Alex: DefakerOne2 reaches roughly ninety-five percent balanced accuracy on this benchmark. The strongest competing forensic language-and-vision model reaches about seventy percent. The authors report this as a zero-shot evaluation: the test generators were unseen during training.
Sam: That is substantial evidence for this benchmark. Does it establish that the training schedule or the routing rule caused the entire improvement?
Alex: No. The model also has broader training coverage and a large data-construction effort. The ablations support benefits from the proposed components. But the headline comparison does not isolate every contribution to the new model’s advantage.
Sam: Detection is only part of forensic work. Can the system show where an image was changed?
Alex: It adds localization: marking pixels suspected of tampering. An auxiliary encoder, called the Anomaly Magnifier, supplies fine-grained forensic features. The global fake probability also adjusts how readily the system includes pixels in the predicted mask.
Sam: That sounds helpful, but could a convincing mask make a wrong verdict look better supported than it is?
Alex: The limitations section gives a reason for caution. The segmentation module trained only on regionally tampered examples, not authentic or fully generated ones. Its mask can therefore disagree with classification. The authors explicitly say classification should take precedence; the mask highlights cues in already-suspected images.
Sam: So a mask is not an independent certificate of authenticity. What else should temper confidence before deployment?
Alex: The authors acknowledge failures even when anomalies look obvious to humans. I’d treat the extensive benchmarks as evidence of broad capability, not proof of reliability on every incoming sample. The model is presented as a foundation for further task-specific adaptation.
Sam: Who should read the full document, and where should they start?
Alex: Researchers building forensic systems should start with Modality Synergism and its ablation. Check both the average benefit and the cases where routing loses. Then read the rumor-video benchmark protocol and Limitations, especially before using localization as evidence.
Sam: And for researchers who only need the main idea?
Alex: One detector can cover many kinds of forgery without becoming equally reliable on every case. This work makes broad coverage credible on benchmarks; it does not make verification automatic.
Sam: Thanks for listening.