Salah-Eddine El Basyr
5 min
This paper investigates whether econometric coefficients and machine-learning-based SHAP explanations faithfully recover true economic relationships, or if they simply share blind spots. While machine learning improves predictive accuracy in macroeconomics, its outputs are typically black boxes. Explainable AI methods like SHAP attempt to open these models, but their explanatory validity remains untested against a known ground truth because real-world economic relationships are never directly observable.
To bridge this gap, the author combines a real macroeconomic panel dataset spanning 163 countries from 2000 to 2022 with seven controlled Monte Carlo simulation experiments. In the simulations, the data-generating process is fully known by construction, allowing a direct comparison between true parameters, econometric fixed-effects estimates, and XGBoost-derived SHAP feature attributions under various data pathologies such as non-linearity, multicollinearity, heteroskedasticity, and omitted-variable bias.
The simulation experiments reveal several critical insights into how econometrics and SHAP interact. Under ideal linear conditions, both methods agree closely and remain efficient. When genuine non-linearity is introduced, SHAP successfully detects curvature that a standard linear econometric specification misses. However, under multicollinearity and small samples, SHAP loses precision faster than econometrics, whereas econometric inference is fragile under uncorrected heteroskedasticity and autocorrelation.
Most centrally, the sixth simulation experiment demonstrates that naive Fixed Effects and naive SHAP are biased by nearly identical magnitudes when an omitted variable confounds the relationship. Neither approach can distinguish correlation from causation without an explicit identification strategy. Applying these insights to real-world panel data shows that an apparent Phillips Curve anomaly was merely a fragile artifact of uncorrected error structures, while a non-linear relationship between money supply and inflation was independently flagged by both methods.
The central takeaway is that convergence between an econometric coefficient and a machine-learning SHAP value does not validate the economic mechanism. Because both tools can be simultaneously misled by the same confounding factors, researchers cannot treat model agreement as a substitute for rigorous causal identification. Both econometric and machine-learning explanations require matched diagnostic scrutiny before their outputs can be trusted for policy design.
Sam: Why would the traditional formula win in a simple setup? Isn't the computer model supposed to be more powerful?
Alex: That's exactly the right question. A flexible computer model has to learn the shape of the relationship from scratch, purely from the data. That flexibility makes it noisier — it's searching through many possible shapes when the answer is actually just a straight line. The traditional formula, which already assumes a straight line, doesn't waste any effort on that search. So in a simple world, its built-in assumption is an advantage.
Sam: So the computer model's flexibility becomes a liability when the problem is actually straightforward.
Alex: Exactly. But things shift when the researchers introduced a curved relationship instead of a straight line. Now the traditional formula's built-in assumption of a straight line becomes a handicap, and the computer model's flexibility starts to pay off — it can detect that the curve exists in the first place.
Sam: So each tool has a home turf. The traditional formula is sharper when the world is simple, and the computer model is better at spotting complexity.
Alex: That's a good way to put it. And that insight shapes the practical recommendation that comes out of the thesis. The authors suggest using the computer model as a screening tool first — let it scan the data and flag where curves or non-linearities might be hiding. Then, once you know where to look, bring in the traditional economic formula to test those specific relationships rigorously.
Sam: So the computer model does the exploration, and the traditional math does the confirmation.
Alex: Exactly. They work better in sequence than in competition. But the authors are careful to flag the limits of what they've shown. The real-data portion of the study only used two predictor variables, which is quite simple compared to a real global economy with dozens of interacting forces.
Sam: And the simulations tested each type of problem separately — one challenge at a time — whereas real economic data throws curves, noise, and hidden variables at you all simultaneously.
Alex: Right. So the natural next step for the field is to test more advanced causal machine learning tools against combinations of these problems, not just one at a time. The simulation framework this thesis builds is actually a useful foundation for that kind of future work.
Sam: It's a more honest picture of what these tools can do than you usually get. Agreement between methods sounds reassuring, but it turns out that's not sufficient evidence on its own.
Alex: That's the takeaway worth holding onto. Consistency between two models is a starting point for investigation, not a conclusion. Thanks for listening to ResearchPod.