ResearchPod Summary
Foundation models are frequently released without disclosure of their training data recipes, specifically the domain mixture weights used during training. This creates an access asymmetry where researchers cannot audit the data composition of models they study. The authors investigate whether it is possible to reverse-engineer these domain proportions by analyzing only the released model weights, without access to the original training data or intermediate checkpoints.
The authors introduce WARP (Weight-space Analysis for Recovering Training Data Portfolios). Since the true training trajectory is typically unavailable, WARP uses model merging techniques (such as linear interpolation or TIES) to create a sequence of pseudo-checkpoints between the base model and the final fine-tuned model. By calculating the Mimic Score—a measure of how much a sample's gradient aligns with the direction of the fine-tuning path—the authors construct a geometric footprint of the training data. This footprint is then mapped to domain proportions using either a simple parameter-free softmax readout or a supervised MLP projector trained on synthetic mixtures.
WARP demonstrates that weight-space geometry contains a strong, recoverable signal regarding training data composition. In controlled experiments with BERT and GPT-2, WARP significantly outperformed sample-level membership inference baselines, achieving a mean absolute error (MAE) as low as 0.046 for BERT and 0.104 for GPT-2. Notably, the simulated trajectories created via model merging often outperformed the use of actual training checkpoints, likely because the simulated paths are smoother and less affected by the stochastic noise of real training processes.
This work provides a practical tool for researchers to audit foundation models, enabling better understanding of data contamination, capability drift, and the impact of data curation on model performance. By breaking the access asymmetry between model providers and the research community, WARP facilitates more transparent and reproducible AI research.
Alex: Welcome to another episode of ResearchPod.
Sam: Today we're looking at a paper called WARP — Weight-space Analysis for Recovering Training Data Portfolios. The central puzzle is this: AI companies release powerful models to the public, but they almost never reveal exactly what data went into building them. WARP claims to recover that hidden recipe just by examining the finished model — no access to training logs required.
Alex: So it's like being handed a finished cake and trying to figure out the exact recipe — how much flour, how much sugar — just from the taste and texture, without ever seeing the recipe card.
Sam: That's a useful way to put it. And the reason this matters is something the researchers call "access asymmetry." You can use these models, but you can't see inside their construction. Without knowing what a model learned from — how much web text, how much code, how many scientific papers — it's very difficult to audit it for bias, or understand why it behaves the way it does.
Alex: And if you don't know what a model learned from, it's hard to trust it for any specific task. So how does WARP actually get around the fact that those training logs are hidden?
Sam: The core mechanism is something the researchers call Simulated Trajectory Reconstruction. Here's the problem it solves: when a model trains, it passes through thousands of intermediate states — like a hiker moving through a forest. Those intermediate snapshots are almost never saved. So you're left with only two things: the model before training started, and the model after it finished.
Alex: And you need the journey, not just the destination.
Sam: Right. So the researchers draw a straight line between those two endpoints. They take the base model and the final trained model, and mathematically blend their internal numbers in small, careful steps — like mixing two paint colours incrementally. Each step produces what they call a "pseudo-checkpoint," an artificial snapshot that stands in for the missing training history.
Alex: So you're constructing a plausible path to substitute for the real one. But once you have those artificial checkpoints, how do you turn them into information about the actual data?
Sam: That's where a metric called the Mimic Score comes in. Take a specific piece of text — say, a sentence from a medical textbook. You measure how much that piece of text nudges the model in the same direction it actually moved during training. If medical texts consistently align with the direction the model evolved, you can conclude the model was heavily shaped by that kind of material. Do this across many domains and you build up a picture of the full data mixture.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.
Alex: So you're measuring which types of data pulled the model the hardest — like checking which magnets had the strongest grip.
Sam: Exactly. And this is meaningfully different from older approaches. Previous methods, called membership inference, could only tell you whether one specific sentence was in the training data — one data point at a time. WARP produces what the researchers call a "geometric footprint" — a holistic view of the entire data mixture at once. In their experiments with models like BERT and GPT-2, this approach recovered training proportions with significantly higher accuracy than those older, sample-by-sample methods.
Alex: You mentioned something that caught my attention — that this simulated path actually outperformed an approach that had access to the real training logs. Why would a constructed path beat the genuine article?
Sam: It comes down to noise. Real training runs are messy. Data gets shuffled randomly, learning rates shift, and all sorts of unpredictable factors leave marks on the model's internal numbers. The smooth, linear path created by model merging filters out that randomness. The resulting footprint is cleaner, and a cleaner signal makes it easier to identify the underlying data mixture. The constructed path is, in a sense, a more idealised version of the truth — stripped of the clutter.
Alex: That's a genuine surprise. Are there limits to what this can do?
Sam: There's an important constraint. The method requires a representative sample of text from each domain you're trying to investigate. If you want to know whether a model was trained on medical data, you need some medical texts to test against — something to measure the alignment against. Without that starting material, the system has nothing to calculate. So it's an audit tool, but one that requires some prior knowledge of what you're looking for.
Alex: So it's not a universal key — it's more like a targeted probe. But if you have the right material, you can force a degree of transparency that the developers never intended to provide.
Sam: That's the practical implication. The authors suggest this could lead to automated data audits — where open-source models are routinely tagged with their training composition, making it possible to hold developers accountable for the data choices behind their systems.
Alex: There's something worth sitting with there. The geometry of numbers — the way values are arranged inside a model — ends up carrying a record of the human decisions that shaped it. What looks like a sealed black box turns out to be measurable.
Sam: That's the core insight. By treating a model as a physical object with a history, rather than just a tool that produces outputs, we can start to examine the layers of opacity that have defined much of recent AI development. And it suggests that secrecy around training data may be harder to maintain than many developers currently assume.
Alex: Thanks for listening to ResearchPod.