Aman Singh Thakur, Rayan Khoury
6 min
Abstract
Open-weight language models are fine-tuned, quantized, pruned, and merged, yet their provenance is often undocumented. We study data-free white-box lineage verification: can weights alone reveal whether two compatible model checkpoints share ancestry? Residual training produces a shared identity-aligned component in branch products, so this structure alone cannot establish ancestry. We remove it and compare checkpoint-specific structure across residual blocks, yielding a symmetric lineage score calibrated against independent checkpoints. On residual-MLP and GPT-2 benchmarks, the score separates fine-tuned, LoRA-merged, pruned, and quantized descendants from independent and distilled models (AUROC=1.0), distinguishing weight ancestry from behavioral similarity. Under function-preserving checkpoint laundering experiments, weight-space baselines lose margin or fail; our score remains unchanged and runs 76x faster than the nearest robust baseline on GPT-2. The projection-pairing signal appears across six language-model families and beyond, and a case study correctly identifies 3 related and 7 unrelated LLaMA-2 public checkpoints. Collectively, these results establish a passive, data-free provenance signal for compatible open-weight language-model checkpoints
Sam: How do they actually compare those fingerprints between a reference model and a suspect model?
Alex: This is where it gets a bit like solving a puzzle. The two models might have their layers in a different order, so you can't just compare them one-to-one directly. They build a similarity matrix — essentially a table scoring how well each layer in one model matches each layer in the other — and then run an optimization algorithm to find the best possible pairing. The goal is to match up layers in a way that maximizes the total similarity across the whole model.
Sam: And once those layers are paired up?
Alex: They average the similarity scores across all the matched pairs to produce a single lineage score. Then they check that score against a baseline built from independently trained models — models that definitely don't share an ancestor. If the suspect's score is clearly above that baseline, the method flags a likely lineage relationship.
Sam: So the baseline is doing important work. It's what separates "these look similar because of shared architecture" from "these look similar because one came from the other."
Alex: Precisely. And in testing, the method clearly separates true descendants from independent models, including models trained from scratch on the same tasks. That distinction matters because two models can produce nearly identical outputs while having completely different origins under the hood. Weight-level lineage is fundamentally different from behavioral similarity.
Sam: That's a meaningful distinction. A plagiarized essay and an original essay on the same topic might read similarly, but the writing process was different.
Alex: A useful parallel. The method is looking at the process, not just the output.
Sam: What are the limits here? Where does it break down?
Alex: The authors are clear about several constraints. First, it only works when both models use the same type of architecture and have matching depth and internal dimensions. You can't use it to compare models built on fundamentally different designs.
Sam: And what about determining which model came first?
Alex: That's a second limitation. The lineage score is symmetric — it tells you that two models are related, but not which one is the ancestor. You'd need release dates or other external records to establish the direction of descent.
Sam: And if someone really goes to town modifying the model?
Alex: The signal does degrade under extreme modifications — heavy pruning or extensive continued training can narrow the margin significantly. The fingerprint becomes harder to detect, though the authors note it remains detectable even under aggressive alteration.
Sam: And what does a high score actually mean in practice? Is it proof of theft?
Alex: The authors are careful here. A high score produces a similarity metric, not a legal determination. It's an audit tool — a flag that warrants further investigation. Human review, release records, and metadata all need to be part of the picture before anyone draws conclusions about model origins.
Sam: So it's more like a forensic lead than a verdict.
Alex: That's a fair characterization. It gives organizations a way to audit the open-weight AI supply chain — to ask serious questions about where a model came from — without replacing the judgment that any real investigation requires. Thanks for listening to ResearchPod.