Advances in radiance fields have enabled photorealistic novel view synthesis. In several domains, large-scale real-world datasets have been developed to support comprehensive benchmarking and to facilitate progress beyond scene-specific reconstruction. However, for distractor-free radiance fields, a large-scale dataset with clean and cluttered images per scene remains lacking, limiting the development. To address this gap, we introduce DF3DV-1K, a large-scale real-world dataset comprising 1,048 scenes, each providing clean and cluttered image sets for benchmarking. In total, the dataset contains 89,924 images captured using consumer cameras to mimic casual capture, spanning 128 distractor types and 161 scene themes across indoor and outdoor environments. A curated subset of 41 scenes, DF3DV-41, is systematically designed to evaluate the robustness of distractor-free radiance field methods under challenging scenarios. Using DF3DV-1K, we benchmark nine recent distractor-free radiance field methods and 3D Gaussian Splatting, identifying the most robust methods and the most challenging scenarios. Beyond benchmarking, we demonstrate an application of DF3DV-1K by fine-tuning a diffusion-based 2D enhancer to improve radiance field methods, achieving average improvements of 0.96 dB PSNR and 0.057 LPIPS on the held-out set (e.g., DF3DV-41) and the On-the-go dataset. We hope DF3DV-1K facilitates the development of distractor-free vision and promotes progress beyond scene-specific approaches. The dataset and leaderboard are available at https://johnnylu305.github.io/df3dv1k_web/.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at why 3D models of real-world places often look haunted by blurry, ghost-like artifacts. Sam, what's the problem?
Sam: We're discussing a paper about what researchers call "distractor-free radiance fields." Imagine you're trying to build a perfect 3D model of your favourite café. You take dozens of photos from different angles, but in every shot, there are different customers walking by, shifting shadows, or reflections in the window. These temporary, unwanted objects are what the researchers call "distractors." The computer building the 3D model doesn't know they're temporary — it treats them as part of the building itself. The result is a messy, ghost-filled model where people seem to be frozen into the walls.
Alex: So this paper is asking how we teach AI to ignore those distractions and focus only on the permanent parts of a scene?
Sam: Exactly. And the core problem has been that until now, researchers haven't had a large enough training ground for these systems. Most approaches optimised a model for one specific scene at a time — slow work that doesn't help the AI learn general rules it can apply to any new photo. This paper introduces a large new dataset called DF3DV-1K, which gives AI thousands of paired examples of both "clean" and "cluttered" versions of the same scenes.
Alex: So it's like a "spot the difference" game for computers?
Sam: That's a good way to put it. By showing the AI the clean version — the ground truth — alongside the cluttered version, the model learns to identify and discard the temporary objects that don't belong. The research team used this to build a tool called DI²FIX, which acts like a specialised filter. It takes an existing 3D reconstruction and removes those transient artifacts, improving the final quality of the model.
Alex: So the key contribution isn't a new way to render images — it's the dataset itself that lets the AI actually learn what a distractor looks like?
Sam: Precisely. They captured over 1,000 scenes specifically to test how well different methods handle a wide variety of distractors — moving people, shadows, reflections, and more. That breadth allows for a much more rigorous evaluation than the field has had before, and it moves research away from scene-specific fixes toward something more general.
Alex: Wait — if the data selection is that aggressive, doesn't it risk the model becoming too specialised? Like, only learning to handle the exact types of distractors in the dataset?
Sam: That's a critical question, and the authors addressed it directly. They trained on a very large pool of image pairs, but applied quality filtering to remove cases where the distortion was so severe the model couldn't reasonably learn from it. Think of it like a student studying for an exam — you want practice problems that are challenging but solvable. Problems that are completely broken just teach you bad habits. By keeping only the realistic, fixable examples, the resulting tool can generalise to reconstruction methods it was never explicitly trained on.
Alex: So it's quality control in the training process itself.
Sam: Exactly. And that's what makes the benchmark meaningful. When the authors tested nine different existing methods against this dataset, they found that even the best systems still struggled with certain scenarios — nighttime scenes and fluid or unpredictable motion were particular weak points. That's a useful finding. It tells the field precisely where the gaps are, rather than just confirming that progress has been made.
Alex: So the paper is as much about mapping the problem clearly as it is about solving it.
Sam: That's a fair reading. By establishing this benchmark, the authors give future researchers a shared measuring stick — a way to compare approaches honestly and identify where the next improvements need to come from. It's a meaningful contribution, not because it closes the problem, but because it defines it more precisely than before.
Alex: Thanks for listening to ResearchPod.