Nils Lid Hjort
7 min
Abstract
The best known methods for estimating hazard rate functions in survival analysis models are either purely parametric or purely nonparametric. The parametric ones are sometimes too biased while the nonparametric ones are sometimes too variable. In the present paper a certain semiparametric approach to hazard rate estimation, proposed in Hjort (1991), is developed further, aiming to combine parametric and nonparametric features. It uses a dynamic local likelihood approach to fit the locally most suitable member in a given parametric class of hazard rates, and amounts to a version of nonparametric parameter smoothing within the parametric class. Thus the parametric hazard rate estimate at time $s$ inserts a parameter estimate that also depends on $s$. We study bias and variance properties of the resulting estimator and methods for choosing the local smoothing parameter. It is shown that dynamic likelihood estimation often leads to better performance than the purely nonparametric methods, while also having capacity for not losing much to the parametric methods in cases where the model being smoothed is adequate.
Alex: Least wrong guess makes sense for misspecified models. Does the paper give a sense of how the bias compares to a global parameter?
Sam: The bias stays small if the window shrinks slowly with more data, while variance matches nonparametric smoothing.
Alex: Okay, so bias shrinks faster than variance grows if you balance the window right. They mention kernels—how does smoothing with those change things?
Sam: Kernels are weighting functions, like a bell curve centered at s, giving more say to data right near s and tapering off—like weighting votes closer to your house higher in a local poll. Smoother kernels reduce jumps at edges and tweak variance, often lowering mean squared error versus plain windows.
Alex: Huh, so kernels make it even steadier without losing the local adaptivity.
Sam: Right—and for bandwidth, one idea is to start narrow and expand until a goodness-of-fit test starts rejecting. This adapts to where the parametric shape fits well. The paper notes these hold under random censoring, with i.i.d. lifetimes cut by independent censor times.
Alex: So it adapts the window based on fit tests, and works under standard random censoring. But how does this stack up against the usual nonparametric smoother, like Nelson-Aalen?
Sam: The classic nonparametric Nelson-Aalen tallies events divided by at-risk people, then smooths it with a kernel. The dynamic estimator matches its variance exactly. But the bias is smaller when the parametric family curves close to truth.
Alex: Huh, same wobble but potentially less systematic error if your local model guess is decent. Does the paper say outright when dynamic beats it?
Sam: Yes—it's always at least as good, and strictly better, when the model's local curvature tracks reality's bend reasonably well. If perfectly matched locally, bias shrinks even more.
Alex: So near a good parametric fit, you get extra wins beyond just smaller bias?
Sam: Precisely—in those spots, you pick larger windows for less variance, yielding a clear improvement over fixed-shape or pure nonparametric curves. For cancer survival, where risks might follow certain patterns partway, this means more reliable local estimates without global assumptions.
Alex: That adaptive widening where the parametric shape holds makes practical sense for uneven risks like in cancer data. But picking the window size seems tricky—how does the paper suggest choosing it in practice?
Sam: Squared bias makes up a small part of total error, variance the rest—so trimming bias lets you use wider windows for less overall error. The paper favors adaptive goodness-of-fit: start with narrow windows and expand until a local test rejects the parametric fit.
Alex: Goodness-of-fit test—how does that work locally?
Sam: They use martingale residuals—the surprise differences between observed events and what the local model predicts, like tracking deviations in a random walk. If the model fits, these behave like a tied-down random path. The test expands the window until the biggest scaled deviation hits a threshold, signaling poor fit.
Alex: Huh, so it self-adjusts: wide where parametric works, narrow where it doesn't.
Sam: Exactly—it computes efficiently and grabs parametric gains where possible, under standard random censoring. The paper notes it's conservative but practical.
Alex: So those tests give a practical way to pick windows automatically. But in real data like cancer survival, where events might cluster or thin out, are there spots where this still struggles?
Sam: Yes, the paper cautions that small windows risk instability with sparse events—too few cases mean the local fit wobbles wildly. It suggests minimums like at least ten observed events per window, and post-smoothing to iron out jumps.
Alex: Right, so sparse data forces wider windows or simpler models. Does it extend beyond plain survival, say to patients with different traits?
Sam: It does—the methods adapt to setups like Cox models, where risk scales by patient factors such as age or treatment, estimating a baseline hazard locally.
Alex: That balances the trade-offs well—flexible yet grounded. Makes sense for practical tools without overpromising.
Sam: Exactly. This approach refines hazard estimation incrementally, blending parametric stability with data-driven adaptation. Thanks for the discussion, Alex.
Alex: Pleasure as always, Sam. That's our look at dynamic likelihood in survival analysis. Thanks for listening to ResearchPod.