Scientific ideas rarely start from a blank page. They inherit mechanisms, repair known limitations, and recombine pieces of earlier work, much like biological genomes. Current benchmarks still say little about whether AI systems can follow this inheritance structure. We present IdeaGene-Bench (IG-Bench), a benchmark for scientific lineage reasoning and lineage-grounded idea generation. IG-Bench is organized around the IdeaGene framework: each paper or proposal is represented as a set of minimal, typed, evidence-grounded Idea Genome objects, and a GenomeDiff aligns these objects to record inheritance, mutation, loss, external import, and novel insertion under six operational evolutionary dynamics. The benchmark contains 1,961 golden lineage traces, 1,085 curated Idea Genome objects, and 920 pairwise GenomeDiff records across 10 scientific domains. It supports two evaluations. IG-Exam (42 task types, 1,029 instances) tests closed-form lineage reasoning across Idea Genome abstraction, inheritance tracing, evolutionary reasoning, and lineage verification. IG-Arena evaluates generation with a lineage-conditioned Population-Evolution Score(PES), asking whether a proposal can be inserted as a coherent descendant of a given lineage population: it should inherit the right Idea Genome objects, vary meaningfully from nearby work, and offer selection value for future research. Experiments on 14 LLM-based scientists expose a compositional bottleneck. The strongest system reaches only 27.3% exact accuracy on lineage reasoning, and structured lineage context reshuffles system rankings rather than helping every participant uniformly.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a study that asks a fundamental question: can AI actually do science, or is it just mimicking it?
Sam: We're discussing a paper that challenges how we evaluate AI research agents. The argument is that current systems can write fluent-sounding papers, but they often fail to understand the actual evolutionary history of the scientific ideas they're working with.
Alex: So it's asking whether AI can do real science, rather than just sounding like it can?
Sam: Exactly. Most AI systems look at research papers as whole documents. They might find papers on the same topic, but they struggle to track how specific ideas—like a particular way of solving a problem—get passed down or changed from one study to the next.
Alex: So it's like the difference between knowing two people are in the same room versus understanding their family tree?
Sam: That's a useful way to think about it. In science, ideas move like genes in a population. One paper might take a mechanism from an earlier study, fix a known limitation, and pass that improved version forward to the next. Researchers call this ability to trace the actual ancestry of an idea "scientific lineage competence."
Alex: And current AI models aren't doing that?
Sam: They aren't. Because they look at entire papers, they get distracted by what the paper calls "topical proximity." A system might group two papers together just because they both discuss "object detection," even if they use completely different methods that share no real evolutionary link.
Alex: So if the AI doesn't understand the lineage, it might propose a "new" idea that's actually just a broken version of something that was already solved?
Sam: Precisely. And that's the core problem the authors are trying to fix. Their framework is called "IdeaGene." The idea is to break a research paper down into its smallest, most essential building blocks—the specific mechanisms it introduces, the limitations it addresses, the goals it sets out to achieve. They call these pieces "Idea Genome" objects.
Alex: So it's like DNA sequencing for research papers?
Sam: That's a good analogy. Once you have these individual "genes," you can use a tool called "GenomeDiff" to compare them across different papers. It records the fate of every idea—was it kept, modified, or dropped entirely? This makes the hidden history of scientific progress visible, and more importantly, testable.
Alex: That shifts the evaluation from "does this look like a paper?" to "does this actually fit into the history of this field?"
Sam: Exactly. And when they tested current AI systems against this standard, using a benchmark they built called IG-Bench, even the strongest models scored around 27% accuracy on these lineage reasoning tasks. The paper describes this as a "compositional bottleneck"—the models can handle local details within a single paper, but they fail to maintain the big-picture consistency needed to build a coherent scientific argument across many studies.
Alex: That's a striking gap. So how does the framework actually establish a real lineage, rather than just a topical connection?
Sam: The key is that GenomeDiff doesn't just list what's in each paper—it maps the objects from one paper to the next and labels what happened to each one. Was a mechanism carried forward unchanged? Was it adapted for a new context? Was it abandoned because something better came along?
Alex: And if two papers are just working in the same area but don't actually share any mechanisms?
Sam: Then they don't have a true lineage, even if they cite each other. The framework distinguishes between what it calls "EcologyContext"—things like the datasets or benchmarks a field happens to share—and genuine inheritance of ideas. Two papers might compete for attention in the same space without one actually descending from the other.
Alex: So sharing a dataset isn't enough to claim a family connection?
Sam: Right. A true lineage requires continuity of the actual mechanisms. The framework also labels the "driver" of each new paper—the central reason it exists. If a successor paper doesn't share a driver with its supposed predecessor, the claimed lineage doesn't hold up.
Alex: How do they classify the different ways ideas can change as they move forward?
Sam: They use what they call "Evolutionary Dynamics." If a mechanism persists but gets applied to a new task, that's labeled "Adaptive Radiation"—like a tool being repurposed. If a paper pulls ideas together from two separate research threads, that's "Hybridization." These labels turn the messy, tangled history of a research field into a structured map you can actually reason about.
Alex: And this is where the evaluation of AI comes back in?
Sam: Exactly. They built a metric called the Population-Evolution Score. Think of it as a report card that checks whether an AI's proposed new idea is a coherent descendant of the work that came before it. If the model can't explain how its "new" idea connects to the history of its field—which mechanisms it inherited, which limitations it's addressing, which driver it shares—the score drops.
Alex: So the AI has to prove it understands the why behind a change, not just the what.
Sam: That's the core shift. It's the difference between a student who can copy a solution from a textbook and one who can explain why that solution works and what problem it was originally designed to solve. The second student is actually learning. The first is just pattern-matching.
Alex: It sounds like the whole framework is moving from a "paper-centric" view—where you judge AI by the documents it produces—to something more like a "genome-centric" view, where you judge it by whether it understands the living history of ideas.
Sam: That's well put. The authors' argument is that science isn't just a collection of documents. It's a process—a continuous, evolving conversation where each contribution only makes sense in relation to what came before. If an AI agent can't participate in that conversation authentically, then it isn't really doing science, regardless of how polished its output looks.
Alex: And right now, the evidence suggests most systems are still at the polished-output stage.
Sam: That's what the benchmark results point to. The paper doesn't claim this problem is unsolvable—it's proposing a framework and a test so that future work has a clearer target. The hope is that by making lineage reasoning measurable, the field can actually start building systems that improve at it.
Alex: That feels like a meaningful contribution on its own—not just identifying the gap, but giving researchers the tools to close it. Thanks for walking us through it.
Sam: Thanks for having me.
Alex: And thank you for listening to ResearchPod.