Andrej Leban, Yuekai Sun
5 min
CausalDS is a comprehensive benchmark designed to evaluate how well LLM-based data-science agents perform causal reasoning. Unlike existing benchmarks that often isolate symbolic causal logic from practical data analysis, CausalDS integrates both. It presents agents with a realistic narrative, a corresponding structural causal model (SCM), and a tabular dataset, requiring the agent to use code and analysis tools to answer questions spanning Pearl's three rungs of causality: association (Rung 1), intervention (Rung 2), and counterfactuals (Rung 3).
Each benchmark instance is a 'scene' generated through a multi-stage pipeline. First, a directed acyclic graph (DAG) is sampled and populated with structural equations to create an SCM. This SCM is then mapped to a domain-specific story, ensuring the causal structure is grounded in realistic variables. A key feature is the separation of the conceptual causal model from the observed data; the benchmark introduces an 'observation layer' that provides the agent with noisy measurements of the conceptual variables. This allows the researchers to vary the difficulty of the data-science task without altering the underlying causal identifiability.
As LLMs are increasingly used as autonomous data-science agents, it is critical to determine if they are truly reasoning about causal structures or merely pattern-matching. CausalDS forces agents to demonstrate a range of skills: interpreting causal graphs, performing statistical estimation, quantifying uncertainty, and—crucially—recognizing when a causal question cannot be answered given the available data. By treating abstention as a first-class outcome, the benchmark provides a rigorous way to measure whether agents can distinguish between identifiable causal claims and those that are not supported by the data.
Large language models (LLMs) increasingly act as integrated data-science agents, combining abstract reasoning with advanced tool use. Yet the relevant benchmark landscape largely divides into symbolic causal reasoning benchmarks without realistic data analysis or data analysis benchmarks without a principled causal data-generating structure. Furthermore, existing causal evaluation datasets are often restricted to curated examples from existing sources, with diversity coming from limited templatized variations rather than from systematic generation of novel synthetic causal structures. We introduce CausalDS, a benchmark for evaluating causal reasoning in agentic data-science workflows. Each benchmark instance is a scene consisting of a sampled structural causal model (SCM) with generated observational data and an accompanying synthetic natural-language story grounded in a realistic domain. We optionally ground the composition of the benchmark components in empirical distributions obtained from real-world datasets, thus retaining empirical structure while reducing the "causal parrot" risk through completely synthetic generation. From each scene, we then derive tasks spanning all three of Pearl's rungs, with typical data-science prediction tasks appearing as Rung 1. Most tasks include a data science coding component, where the model typically needs to use several tools to arrive at the final answer due to the frequent presence of imperfect observations, which are generated by an observation model. Additionally, recognizing when a question admits no warranted answer and abstaining is treated as a first-class scored outcome. The benchmark thus jointly evaluates symbolic causal reasoning, data science, uncertainty quantification, abstention, and tool use/coding.
Sam: So how do the models actually do? Are the better ones stronger at the logic itself, or just better at using the tools?
Alex: A bit of both, according to the study. Most models can correctly map out the causal structure when the data is clean enough. Where they diverge is in knowing when to stop. Some models are prone to what the researchers call "over-claiming"—they insist they have an answer even when the data is too noisy or incomplete to support one. The top-performing models are the ones that resist that temptation.
Sam: So the models that know when to abstain are actually the most reliable ones overall?
Alex: That's the central finding. Getting the right answer matters, but knowing when an answer isn't possible is equally important—maybe more so, if you want an AI you can actually trust in a scientific context.
Sam: It makes sense. A doctor who confidently gives a wrong diagnosis is more dangerous than one who says "I need more tests."
Alex: That's a good parallel. The benchmark also draws on something called Pearl's Hierarchy—a framework developed by computer scientist Judea Pearl that ranks causal reasoning into three levels. At the bottom is simple association: two things tend to happen together. In the middle is intervention: if I change this, what happens to that? At the top is counterfactual reasoning: what would have happened if things had been different? CausalDS tests AI across all three levels, so it's not just checking whether a model can spot a correlation—it's checking whether it understands the full depth of what "cause" actually means.
Sam: So it's not just "did you get it right?" but "did you recognise what level of proof the question actually required?"
Alex: Exactly. And to make sure models aren't cheating by recognising a familiar story, the researchers use what they call a "matched verbalization-swap." They keep the formal structure of the problem identical but change the narrative around it—swapping a farm irrigation study for a medical trial, for instance, while the underlying numbers and logic stay exactly the same.
Sam: Oh, that's clever. If the AI is genuinely solving the logic, its answer shouldn't change just because the story changed.
Alex: Precisely. And that's where weaker models tend to fail. They're sensitive to the prose—they pick up on details in the story and import assumptions that aren't actually in the data. Stronger models stay anchored to the structure and ignore the surface-level narrative.
Sam: Like a good student who ignores the flavour text in a word problem and just focuses on the actual numbers.
Alex: That's a fair way to put it. What CausalDS ultimately tests is whether an AI can behave like a careful scientist: follow the evidence, use the right tools, and—when the evidence runs out—have the discipline to stop rather than guess. That's a harder standard than most current benchmarks set, and the study suggests that even the best models today still have meaningful room to improve.
Sam: It's a useful reminder that confidence and competence aren't the same thing—in AI or anywhere else.
Alex: Well said. Thanks for listening to ResearchPod.