When forecasting hourly returns for 1,000 US equities, we observe an unexpected phenomenon: predictions become nearly flat and show poor stock ranking, as measured by cross-sectional correlation. We call this forecast collapse. Surprisingly, the phenomenon largely disappears when forecasting trading volume under the same setting. We investigate forecast collapse across time-series foundation models (TSFMs), twelve deep-learning forecasting models, and 97 public benchmark configurations, and find that it is closely tied to target predictability. We identify two distinct reasons behind it: low predictability limits the amplitude of calibrated point forecasts, while per-series objectives leave cross-series structure unidentified. These findings reveal a calibration-ranking tradeoff: optimizing squared error leads to flat predictions, whereas directly optimizing cross-sectional correlation improves ranking but can inflate forecast amplitude by more than an order of magnitude. To address this tradeoff, we introduce CalibRank, a simple objective that balances calibration and ranking. On Finance1K, CalibRank nearly triples cross-sectional correlation while keeping amplitude close to the target, and improves correlation on all tested models. Our results reveal a blind spot in conventional time-series evaluation: per-series metrics can hide failures in cross-series structure needed by downstream decisions.
Alex: Welcome to another episode of ResearchPod. Today we're looking at a study by Shu Wan and colleagues from Arizona State University and the University of Oxford. They're investigating something they call forecast collapse — a puzzling failure mode in modern prediction systems applied to financial markets.
Sam: So what exactly is forecast collapse? What does it look like in practice?
Alex: Imagine you ask a weather app to predict tomorrow's temperature for ten different cities. Instead of giving you different numbers for each city, it just says "72 degrees" for all of them. That's essentially what's happening here. When researchers applied large, sophisticated forecasting models to hourly stock returns, the predictions flattened out into near-zero lines — completely useless for deciding which stocks to buy or sell.
Sam: That seems like a fundamental problem. If everything looks the same to the model, you can't rank anything.
Alex: Exactly. And what makes this interesting is that it's not a bug in one specific model. The authors found it happening across many different systems. So they started asking: what is it about stock return data that causes this, and can we fix it at the training level?
Sam: How did they figure out what was causing it?
Alex: They ran a telling experiment. They kept everything identical — same models, same setup — but changed what they were predicting. When they switched from stock returns to trading volume, the collapse disappeared entirely. The models worked fine.
Sam: Same model, same architecture, but completely different behavior just by changing the target. What does that tell you?
Alex: It tells you the problem lives in the data itself, not the model design. Here's the key idea. Think of any dataset as having two parts: a meaningful pattern you can actually learn from, and random noise you can't predict no matter how smart your model is. Stock returns are almost entirely noise. The genuine predictable signal is extremely weak.
Sam: And when the signal is that weak, what happens during training?
Alex: This is where it gets subtle. When a model is trained to minimize its average error, and the target is mostly unpredictable noise, the mathematically safest strategy is to predict something close to zero for everything. You never make a catastrophically wrong call, but you also never say anything useful. The model learns to be cautiously wrong rather than boldly right.
Sam: So the training process itself is pushing the model toward flatness.
Alex: Right. And there's a second problem layered on top of that. Standard training scores each stock independently — it asks, "how wrong were you about Apple?" and "how wrong were you about Microsoft?" as separate questions. It never asks, "did you correctly rank Apple above Microsoft?"
Sam: So a model could score well on individual errors while completely failing at the thing investors actually care about — which stock will do better than another.
Alex: Precisely. That gap between individual accuracy and cross-sectional ranking is the hidden flaw. You can have a model that looks decent on standard benchmarks and is still useless for building a portfolio.
Sam: So how do the authors propose fixing this?
Alex: They introduce a composite training objective. Think of it like a report card with two grades instead of one. The first grade is the traditional one — how close were your individual predictions to the actual values? The second grade is new — did you correctly rank the stocks relative to each other at each point in time? The model has to do well on both to pass.
Sam: And does adding that ranking grade actually help?
Alex: Across twelve different model architectures, adding the ranking component meaningfully improved the models' ability to correctly order stocks. The individual prediction errors did increase slightly — there's a genuine tradeoff — but the improvement in ranking was substantial, and the error stayed far lower than if you'd optimized for ranking alone without any accuracy anchor.
Sam: So you're finding a middle ground between two competing goals.
Alex: That's a good way to put it. There's a frontier of possible operating points, and practitioners can choose where on that frontier they want to sit depending on what matters more for their specific use case.
Sam: Are there limitations to keep in mind here?
Alex: Several important ones. The empirical results come primarily from one specific financial dataset, so it's not yet clear how broadly these findings generalize to other markets or other types of time-series data. The ranking measure they use — Pearson correlation — captures linear relationships between stocks, but it doesn't capture more complex dependencies, like what happens during a market crash when everything moves together in unusual ways. And the authors are candid that future work will need to address those richer joint behaviors.
Sam: So this is a meaningful step forward, but with real boundaries on what it claims.
Alex: That's a fair summary. The paper identifies a structural reason why sophisticated models fail on low-signal data, and it offers a principled fix that works across a range of architectures. But the authors are careful not to overstate it. The field will likely need to extend these ideas to handle tail risks and more complex market dynamics before this becomes a complete solution.
Sam: It's a good reminder that even the most advanced models are only as useful as the objectives they're trained on.
Alex: Well put. The model learns exactly what you ask it to learn — and if you ask the wrong question, you get the wrong answer, no matter how powerful the system is. Thanks for listening to ResearchPod.