Kyle Schindl, Larry Wasserman
8 min
Abstract
Regression discontinuity and kink designs are typically analyzed through mean effects, even when treatment changes the shape of the entire outcome distribution. To address this, we introduce distributional discontinuity designs, a framework for estimating causal effects for a scalar outcome at the boundary of a discontinuity in treatment assignment. Our estimand is the Wasserstein distance between limiting conditional outcome distributions; a single scale-interpretable measure of distribution shift. We show that this weakly bounds the average treatment effect, where equality holds if and only if the treatment effect is purely additive; thus, departure from equality measures effect heterogeneity. To further encode effect heterogeneity we show that the Wasserstein distance admits an orthogonal decomposition into squared differences in $L$-moments, thereby quantifying the contribution from location, scale, skewness, and higher-order shape components to the overall distributional distance. Next, we extend this framework to distributional kink designs by evaluating the Wasserstein derivative at a policy kink; this describes the flow of probability mass through the kink. In the case of fuzzy kink designs, we derive new identification results. Finally, we apply our methods on real data by re-analyzing two natural experiments to compare our distributional effects to traditional causal estimands.
Alex: So once they've got these breakdowns, how do they actually pull the distributions from the data right at that cutoff?
Sam: They start by estimating the cumulative distribution function on each side separately—like plotting the proportion of outcomes below a certain value, using only data just left of the cutoff for the untreated limit, and just right for treated. To smooth it accurately near the edge, they fit a low-degree polynomial curve, weighted so points closer to the cutoff matter more; that's local polynomial regression. The constant term gives the limiting CDF value. From there, they invert it to get the quantile functions—what outcome sits at the 10th percentile, 50th, and so on. They add a bias correction to subtract off systematic error from the polynomial.
Alex: Weights fading away from the cutoff makes sense for local accuracy. But testing if the Wasserstein is truly zero—that quadratic nature complicates things?
Sam: Yes, near no effect, the usual methods break because it's like the square of a mean. They test by expanding the quantile process into orthogonal basis functions via Karhunen-Loève, turning the scaled squared Wasserstein into a weighted sum of chi-squares. They estimate eigenvalues of the covariance, simulate the null many times with truncation, and get critical values.
Alex: A conservative Monte-Carlo test grounded in that expansion. Practical for spotting hidden shifts averages miss.
Alex: That sounds solid for the main test. But what if samples are small—do they have a backup way to check if the Wasserstein is zero?
Sam: Yes, they offer a conservative test that errs on the safe side. It estimates just the mean and variance of the squared Wasserstein under no effect, using bounds from prior work, and sets a threshold that's guaranteed not to falsely reject too often.
Alex: So it's like building a fence around the no-effect zone that's a bit wide on purpose, to avoid calling a shift when there isn't one?
Sam: Precisely. For confidence intervals around the squared Wasserstein, they propose two conservative approaches. One uses bands around the quantile shifts, squaring the extremes to bound the integral safely. The other widens a normal-style interval with extra padding based on outcome spread. Both ensure coverage at least as good as claimed.
Alex: Okay, practical tools for sharp cases. But real policies often aren't all-or-nothing—sometimes crossing a cutoff just raises treatment odds?
Sam: That's the fuzzy case. Here, treatment probability jumps at the cutoff, but some below still get it, some above don't—like an incentive that sways most but not all. The key group is compliers: people who'd skip treatment below but take it above. They identify complier distributions by differencing weighted outcomes—treated below versus untreated above, normalized by propensity jumps—estimated via local polynomials near the cutoff. The fuzzy Wasserstein follows the same formula on those, bounding the complier average effect just like sharp designs.
Alex: So it extends cleanly, isolating true responders amid fuzziness.
Alex: That fuzzy extension handles partial compliance well. But what if there's no sharp jump in treatment probability—just a change in how steeply it rises past the cutoff?
Sam: That's a kink design. Imagine the chance of getting treatment as a line that bends at the cutoff, steeper on one side—like a road ramp that suddenly gets inclines instead of flat. Researchers measure effects from that slope change, focusing on compliers whose treatment shifts more because they're responsive to the kink. The paper defines a counterfactual where treatment nudges proportionally to each person's responsiveness, normalized so a unit shift matches the average kink size. For Wasserstein, that identifies the derivative of the distance itself at the kink—think a sensitivity measure for how it changes under small nudges.
Alex: So the counterfactual weights units by how much their treatment path bends at the cutoff, capturing who drives the observed slope jump?
Sam: Exactly. With smoothness assumptions, it decomposes into L-moment derivative differences, so you see instant changes in center, spread, or skew from the kink. Local polynomials estimate those slopes, with variance scaling accordingly. Simulations show conservative intervals narrower than bootstrap for modest samples.
Alex: So with all these tools in place—what does this look like in actual studies?
Sam: A good example comes from re-analyzing elections where the winner is decided by a tiny vote margin. The average vote share for the winner's party jumps in the next election. The Wasserstein measure comes in close, and the breakdown shows most of that from a simple location shift, with the rest from slight shape changes—plus a dominance score pointing to benefits across the outcome range.
Alex: Right, so means and full shift align there, suggesting even effects.
Sam: Contrast that with a study on Swedish grants that kick in more steeply past a certain out-migration rate. The average employment slope shows no change. Yet the Wasserstein derivative has a confidence interval touching zero, and the L-moment split reveals almost nothing from the mean—mostly higher-order terms, hinting at outliers where a few places used grants very differently.
Alex: Huh—that flags something means miss, like uneven impacts in the tails.
Sam: Exactly. Overall, this frames the Wasserstein as a benchmark that always bounds the average effect from above, equal only for pure additive shifts. When it's meaningfully larger, you know heterogeneity matters. The L-moment table acts like a dashboard, answering if effects come from center moves, wider spreads, tilts, or tails—practical for reports. While focused here on discontinuities, the logic fits randomized trials or differences over time. One limit is sticking to single outcomes; multivariate would need new identification.
Alex: Makes sense—this gives a fuller causal picture, especially for policies where spreads or skew matter as much as averages. Thanks, Sam—appreciate the clear breakdown. That's it for this look at distributional effects in discontinuity designs. Thanks for listening to ResearchPod.