Bruno Mlodozeniec, David S. Krueger, Richard E. Turner
9 min
There is a prevailing belief in modern machine learning that answering causal questions requires specialized frameworks, such as structural causal models or do-calculus. This paper challenges this view, demonstrating that these tools—while useful—are essentially syntactic sugar for standard probabilistic reasoning. The authors argue that the entire field of causal inference can be unified under the rule: always write down the joint probability of everything.
The central proposal of the paper is the "twin model" approach. Instead of treating causal inference as a separate discipline, the authors demonstrate how to explicitly model multiple settings—such as an observed world, an interventional world, and a counterfactual world—within a single joint distribution. By defining how these worlds are related, one can use standard Bayesian inference to answer causal queries. The authors illustrate this using a concrete medical example (aspirin efficacy), showing how to calculate expected outcomes under specific interventions or counterfactual conditions by marginalizing over latent variables or conditioning on observed data, all without departing from the axioms of probability.
The authors highlight several benefits of this unified perspective. First, it makes causal inference accessible to the broader machine learning community by leveraging existing knowledge of probabilistic programming and Bayesian networks. Second, it provides a flexible abstraction that handles scenarios where standard causal graphical models struggle, such as settings where the number of variables changes or where dependencies are non-standard. By stripping away the need for a specialized "causal" toolkit, the approach simplifies the entry barrier for new researchers and clarifies the relationship between associative statistics and causal reasoning.
A recurring debate in the literature, notably between Judea Pearl and Andrew Gelman, centers on whether causal and statistical inference are fundamentally distinct. The authors argue that this debate is primarily semantic. They contend that the claim of a "causal-statistical dichotomy" often relies on an overly narrow definition of statistics as merely the study of static correlations. By adopting a more robust definition of statistics that encompasses modeling domain shifts and interventions, this artificial barrier disappears, paving the way for more integrated future research.
Sam: Add a hypothetical setting alongside the observed one. In that setting, the dose is assigned independently of initial severity. Then state what stays unchanged: the distribution of initial severity, and the relationship linking severity and dose to headache duration.
Alex: That unchanged relationship seems to carry the argument. Are we assuming the intervention changes how treatment is chosen, but not how the body responds?
Sam: Yes, for this example. The authors explicitly assume the physiological mechanism stays the same. They share model parameters across the observed and hypothetical settings, so survey observations inform predictions under assignment. They call this the twin model approach: related settings represented together.
Alex: And once those connections are specified, what does the calculation do?
Sam: It averages predicted headache duration over the initial-severity distribution in the hypothetical setting. It also accounts for uncertainty about the shared parameters, using what the observations tell us. In their chosen model, that average can be calculated analytically. Under the model’s remedial-effect assumption, higher assigned doses shorten expected headache duration.
Alex: We should separate that from medical evidence. Did they estimate aspirin’s effect in a real study?
Sam: This is a worked modelling example, not a clinical efficacy study. The survey dataset is described symbolically, without a reported numerical sample size. The evidence is the mathematical construction and calculation, not a measured treatment benefit. Their examples address interventions and individual hypothetical alternatives, rather than comparing methods across a benchmark.
Alex: What is actually new, then? Representing related hypothetical worlds sounds close to existing causal methods.
Sam: The authors make that connection themselves. They explicitly say they are not proposing a new framework. Their contribution is an exposition and argument about sufficiency. They connect their construction to Pearl’s earlier twin-network method, and show how familiar causal tools fit inside probabilistic modelling.
Alex: Where does that leave the do-operator, the notation that distinguishes assigning treatment from observing it?
Sam: They treat it as a concise way to specify the changed model. Replace the mechanism assigning treatment, and leave the other specified mechanisms unchanged. The notation packages those assumptions. They also recognise that do-calculus offers useful rules for finding intervention quantities from observed distributions.
Alex: So calling it shorthand does not mean its insights are dispensable. Does the paper explain where the explicit modelling approach can still go wrong?
Sam: Appendix F does. Suppose a measured variable is a consequence of aspirin, rather than a factor influencing dose choice. Making it independent of treatment could accidentally block the mechanism you wanted to study. The authors use post-ingestion prostaglandin levels to illustrate that mistake.
Alex: And an unmeasured factor could still influence both treatment choice and outcome. Writing a larger model would not automatically reveal it.
Sam: The authors discuss that too. They suggest fever as a possible overlooked factor connected to dose choice and headache type. Their central warning is that no universal rule converts an observed distribution into the intended intervention distribution. You must specify how the action changes the data-generating process.
Alex: Let’s turn to the individual question. What changes when someone asks what would have happened to their own headache?
Sam: Their friend took aspirin before an exam, but the headache disrupted her performance. She remembers the dose and duration, not the initial severity. The counterfactual question—an alternative conditioned on what actually happened—is whether a larger dose would have cleared the headache before the exam.
Alex: For that question, a fresh person drawn from the population would be wrong. What connects the actual friend to the hypothetical one?
Sam: In their first construction, the initial headache severity is shared. Her observed dose and duration update beliefs about that hidden severity. Those updated beliefs then inform the hypothetical outcome under a different dose. Again, the question becomes conditioning within a joint probability model.
Alex: But severity cannot describe everything about her. Shouldn’t other person-specific factors carry across as well?
Sam: The authors say it depends on what those factors mean. A hidden health attribute might reasonably be shared. Random variation in a tablet’s active content might not. They contrast this flexibility with standard structural causal models, which describe outcomes through functions and hidden noise, and whose counterfactual construction shares that noise.
Alex: Does choosing what to share introduce assumptions that observations cannot settle?
Sam: It can. They construct different hidden-variable descriptions with the same observed distribution but different counterfactual answers. That is an identifiability problem: observed data cannot uniquely determine the target. A fully specified model permits inference, but it does not make those extra assumptions empirically established.
Alex: That seems the key boundary: a demonstration that probability can express the question, not a shortcut around learning the answer. Who should read the full document?
Sam: Researchers using probabilistic models who want to tackle causal questions should start with Section 2.2, on modelling interventions. Read Appendix F alongside it for the assumptions that can fail. For individual counterfactuals, Section 3.1 and Appendix O are the sharper test: what should be shared, and what the observations cannot determine. Everyone else should remember: specialised causal notation is optional; assumptions about what changes and what stays fixed are not.
Alex: Keep that distinction in mind when you choose your model.