The rapid growth of 3D Gaussian Splatting (3DGS) research demands significant effort to reimplement papers before building on them. We introduce SPLATIFY, a multi-agent framework that converts 3DGS papers into trainable gsplat-based implementations, where generic paper-to-code methods and frontier models fail. SPLATIFY achieves this through five innovations: (1) A context-free grammar for gsplat over a modular method template with extension points for losses, densification, rendering, and optimization, constraining synthesis so generated code satisfies gsplat's architectural invariants by construction. (2) Architectural elements for faithful reproduction: fork-aware citation recovery retrieving component-level code at function-level granularity, Graph-of-Thought synthesis in topological dependency order, RAG-guided in-context example selection from over 20 verified implementations, and visual feedback combining PSNR-guided regeneration, Gaussian-level structural checks, and VLM-driven patching. (3) Knowledge-driven compositional improvement that autonomously finds weaknesses and composes complementary regularizers, losses, and densification strategies to improve upon original results. (4) Interdisciplinary method discovery where agents retrieve physical priors from outside the 3DGS literature and compose them with rendering knowledge to produce methods for previously unaddressed scene types. (5) SPLATIFY-Bench, an evaluation framework across 30 diverse 3DGS papers. On papers without public code, SPLATIFY matches expert implementations while reducing development time from weeks to minutes, and through compositional discovery further improves PSNR by up to 2.4 dB. We additionally demonstrate novel methods for volumetric nebula rendering and other scientific domains, synthesized entirely by SPLATIFY.
Alex: Rebuilding a rendering paper can take weeks; these authors report doing it in minutes. Their system turns papers into working code, then searches for ways to improve it.
Sam: Working code is a stronger claim than a plausible implementation. Whose work is this, and what did they test?
Alex: It’s SPLATIFY, an arXiv method preprint from Seemandhar Jain, Keshav Gupta, and Manmohan Chandraker at UC San Diego. They introduce a team of software agents and a benchmark for reproducing rendering research.
Sam: So this targets researchers who spend their time rebuilding someone else’s method before they can ask their own question.
Alex: That’s the bottleneck. The authors report expert-like rendering results, alongside experiments combining techniques and adapting them to new scenes. We’ll unpack what makes the code work, what the comparisons show, and where the claims need caution.
Sam: Before we get into the agents, what kind of rendering are they working with?
Alex: Three-dimensional Gaussian Splatting represents a scene using collections of soft blobs. Their shapes, colours, and transparency are adjusted to reproduce photographs from different viewpoints. SPLATIFY generates implementations for gsplat, a rendering framework within the Nerfstudio ecosystem.
Sam: Why is this particularly difficult for a paper-to-code system? Is the obstacle missing instructions, or incompatible software?
Alex: Both. Papers borrow rendering, loss functions, and rules for adding or removing blobs from earlier work. Much of the released code lives in repository forks: modified copies of an existing codebase. Those copies can disagree about data layouts and training conventions.
Sam: So reading a paper’s references doesn’t necessarily tell you which pieces of software its method actually depends on.
Alex: That distinction drives the design. The closest prior system, NERFIFY, targeted neural radiance fields, a different scene representation. The authors argue that Gaussian methods change more tightly coupled components, spread across forks. SPLATIFY follows those code changes and adds checks specific to Gaussian training.
Sam: What keeps it from assembling individually sensible components that don’t work together?
Alex: The central idea is to constrain assembly before asking agents to write code. A context-free grammar supplies formal rules for allowed program structures. A runnable template already contains the training loop and rendering machinery, so agents fill designated hooks instead of inventing the whole repository.
Sam: That sounds like giving the model a stable chassis. It can change the parts relevant to the paper without rebuilding every connection.
Alex: For example, a new regularizer goes into the custom-loss hook. A new attribute attached to each blob also needs the relevant creation and splitting hooks. The template preserves the surrounding optimizer and bookkeeping structure, though that doesn’t guarantee the scientific contribution is implemented correctly.
Sam: Can you make the dependency recovery concrete? What does it find that an ordinary citation search might miss?
Alex: The paper uses Trick-GS as an example. It combines techniques for pruning blobs, masking colour components, progressive training, and accelerated rendering. SPLATIFY compares each fork with its parent repository, labels changed functions by their role, and follows inherited changes through the chain.
Sam: So it retrieves the actual modification rather than treating the entire borrowed repository as the contribution. What happens when there’s no code?
Alex: It falls back to extracting the method from the PDF. It also retrieves similar verified implementations as examples. Then a coordinating agent builds files in dependency order, checks interfaces, and runs short training tests.
Sam: A short training test can show that a program runs. It can’t show that the method survives the rest of training.
Alex: That’s why feedback continues beyond the initial test. Agents inspect image errors, look for floating blobs and inconsistent artifacts across views, and propose patches. They also monitor peak signal-to-noise ratio, or PSNR: an image-reconstruction score where higher means less pixel error.
Sam: What does the training curve reveal that looking at an early rendered image would miss?
Alex: Gaussian training has a phase where blobs are added, split, or pruned. A faulty schedule can look acceptable early, then collapse after that phase ends. SPLATIFY checks for late drops and stalled improvement, and targets repairs at the responsible hook.
Sam: Let’s turn to the evidence. How broad was the reproduction test?
Alex: The benchmark contains thirty papers, separated by evaluation purpose. One group has no public code, another has released implementations, and another tests whether distinct technical contributions survive translation. The comparisons use matched training conditions and held-out viewpoints.
Sam: The papers without code seem especially informative. Otherwise, a model might have seen the implementation already.
Alex: That group contains ten papers, paired with commissioned expert implementations. It avoids direct exposure to corresponding public code, which strengthens the paper-driven synthesis test. It doesn’t mean the models lack relevant knowledge of earlier methods.
Sam: How close did the generated implementations get to the experts?
Alex: The authors report average visual quality within half a decibel of expert implementations on PSNR. That’s an average comparison, not a guarantee for every paper. They also compare structural image similarity and perceptual image error.
Sam: And the time saving means what in practice? Are they reporting fast code generation or a complete experimental campaign?
Alex: They report completion in ten to thirty minutes per paper, versus weeks for expert implementations. Full evaluations still involve training on GPUs. I’d treat that as their reported development-time comparison, not a promise that every downstream research campaign finishes that quickly.
Sam: How did the competing systems fare? Some generated code might compile but never produce useful training.
Alex: The authors report that generic paper-to-code systems and frontier models failed to produce stable, paper-level training results in this setup. Their executability breakdown makes distinctions: NERFIFY could produce trainable code, but failed training stability. “Runs” and “reproduces” are different thresholds here.
Sam: There’s another possible shortcut: dropping a difficult contribution while keeping enough of the baseline to score well.
Alex: They test that separately, checking losses, attributes, modules, and schedules against the source papers. An automated judge scores the components, and experienced developers independently inspect them blind to system identity. That supports fidelity beyond aggregate image quality, although it remains rubric-based evaluation.
Sam: Once reproduction works, what does the improvement stage actually search?
Alex: It searches combinations within the same extension points. Loss terms can be added together; competing split-and-prune strategies cannot simply coexist. Explorer agents propose variants, and progressively longer training runs eliminate weak candidates.
Sam: Does that produce a measurable gain, rather than just a collection of alternative implementations?
Alex: The best reported improvement is up to two-point-four decibels over reproduction alone. In one example, agents add patch-structure and geometry-related penalties to DWTGS. That’s a best-case gain, not the average benefit across methods.
Sam: And the new-domain experiments go further: they retrieve physics rather than another rendering technique. What’s the clearest example?
Alex: Nebula rendering. The authors describe nebulae as optically thin emission media, where light adds along the viewing path. Standard transparency compositing can instead produce opaque shells. Agents retrieve radiative-transfer and turbulence ideas, then add emission parameters and penalties addressing opacity, spatial detail, and blob shape.
Sam: Does that mean the system has implemented the correct physical renderer?
Alex: The excerpt supports a narrower conclusion. NebulaGS improves PSNR by about one-point-three decibels over standard Gaussian Splatting, with better cross-view consistency. But it does not change the rasterizer—the machinery projecting blobs into images—so this demonstrates a useful physics-informed adaptation, not comprehensive physical validation.
Sam: That also seems like the boundary on generalization. The system succeeds inside a carefully prepared framework, not an arbitrary software ecosystem.
Alex: That’s my main caution. It uses a specialized template and verified examples. The authors say another domain would require its own grammar and seed implementations. Their ablations also show that removing dependency recovery or training tests substantially weakens results; the infrastructure is doing real work.
Sam: Who should read the full document, and where should they start?
Alex: Rendering researchers should start with Comparison to Expert Implementations, then Novelty Coverage Analysis. Those separate useful images from faithful methods. Researchers building scientific coding agents should read the method template, fork-aware dependency resolution, and ablations before focusing on the invention examples.
Sam: And what should everyone else carry away?
Alex: The strongest lesson is not that a model can write anything from a paper. It’s that a fixed framework, recovered dependencies, and training feedback can make paper-to-code generation reliable in a defined domain.
Sam: Keep that boundary in mind when you read the claims.