Continual learning (CL) seeks to adapt pretrained models while preserving prior capabilities. Yet the tasks supplied for preservation cover only part of what a model knows, while forgetting can extend beyond them. Recent work shows that language models can supplement this limited evidence by generating useful replay without access to their original training data. This raises a question about self-preservation: can a model help choose preservation targets that protect its own capabilities more broadly? To study this question, we organize task descriptions and their examples into Task Banks, whose entries can be selected and combined to guide replay generation by a frozen model. Continual instruction tuning with Qwen models of varying sizes yields three findings. First, the new task and learning rate shape which prior capabilities degrade, while tracking previously correct answers reveals losses that average accuracy can conceal. Second, generated replay reduces forgetting on tasks outside the bank. Combining partial banks strengthens this protection, improving mean retention of previously correct answers over either bank alone across the tested sizes. Third, the model can also help choose replay targets: selection informed by the new task improves mean held-out retention over random selection at 4B, with similar new-task gains, although smaller models show no corresponding advantage. Under a fixed example budget, however, replacing new-task examples with replay improves preservation at the cost of smaller new-task gains. Together, these findings show that models can support broader self-preservation by generating replay data and, in some settings, choosing which tasks to replay.
Alex: When you fine-tune a language model, it forgets things you never thought to protect. A new study asks whether the model itself can help guard those skills. The short answer is a qualified yes. A frozen copy of the model can write its own practice questions, and that practice protects even skills nobody targeted.
Sam: Before the qualifications, place it for me. Whose work is this, and what kind of paper?
Alex: It's an arXiv preprint from Dongkyu Cho and Rumi Chunara at New York University, with Ali Behrouz of Google Research and Cornell. It's a method paper with controlled experiments, all on Qwen models from 0.6 to 4 billion parameters.
Sam: Plenty of people fine-tune open models without the original training data. Why should they care?
Alex: Because the data you set aside to protect old skills only covers part of what the model knows. Forgetting can spread beyond it. This paper shows a cheap way to widen protection, and a better way to measure what you've lost. So we'll cover the problem, the one idea behind the method, what the numbers show, and where they stop.
Sam: Start with the problem. What's the standard fix, and where does it fall short?
Alex: The problem is catastrophic forgetting: fine-tuning on something new erodes abilities the model already had. The standard fix is replay, meaning you mix examples from old tasks into the new training. But replay only directly rehearses the tasks you supplied. Anything else is protected only if the benefit transfers.
Sam: Models generating their own replay isn't new, though. I've seen that idea before. What does this add?
Alex: Right, the authors cite earlier self-replay work. Those methods mostly add depth, more examples for the tasks you already named. This paper asks about breadth and choice. Does combining different targets protect untargeted skills better? And can the model help pick which targets to rehearse, given what it's about to learn?
Sam: So the question shifts from how much to rehearse to what to rehearse.
Alex: Yes, and they treat those as separate design decisions. The tool is what they call a Task Bank. It's a short list of entries, each a task name, a description of the behavior and format, and optionally a few demonstrations. A frozen copy of the model, taken before the update, reads a description and writes new questions with answers. Those get mixed into fine-tuning.
Sam: Make it concrete. What does a run actually look like?
Alex: Take Qwen3 learning MedQA, a medical exam question set. The bank holds four tasks: MMLU, a broad knowledge test, plus grade-school math, a commonsense sentence-completion task, and yes-or-no reading questions. Then there are four held-out tasks, including a science reasoning set and a math word-problem set. Nothing about those held-out tasks touches replay construction or target selection. So any protection there is protection beyond the bank.
Sam: Before results, how are they scoring this? Average accuracy can look fine while a lot happens underneath.
Alex: That's their first finding. In one Qwen3-4B MedQA run, about three and a half percent of held-out answers flipped from right to wrong. A nearly identical share flipped from wrong to right. Net change was basically zero. So they report Retention: the share of answers that were correct before training and are still correct after.
Sam: So a flat average can hide real churn in what the model knows. Which is the thing you'd actually care about.
Alex: And they found the incoming task and learning rate shape which skills degrade. In one sweep, a lower learning rate gave both a larger new-task gain and much better retention than a higher one. Choosing a gentler rate isn't automatically a trade-off.
Sam: Okay, now the replay itself. Does model-generated data actually protect the held-out tasks?
Alex: In one comparison, Qwen3-4B learns a stance-detection task in Chinese, called C-STANCE. Plain fine-tuning lost about four points on average across held-out tasks. Replaying a single old task barely helped. Replaying all four bank tasks turned that into a small gain, about one point, whether the replay was real stored data or model-generated. And new-task learning stayed essentially the same.
Sam: That's the broad-versus-narrow point. What happens when you combine partial banks?
Alex: They split the bank in half. One half is knowledge and math, the other is commonsense and reading. Then they compared each half against the two combined, on MedQA, at all three model sizes. The combined bank had the best mean held-out Retention every time, roughly three to four points above the better half. At 4B it kept about 96 percent of previously correct answers. New-task gains dropped by less than a point relative to plain fine-tuning.
Sam: You said mean. Did combining win on every held-out task?
Alex: No. The best bank varied by task. The combined bank wins on the average, not uniformly.
Sam: Now the part about the model choosing. If it picks two of the four tasks, does it beat random?
Alex: At 4B, yes. When the model saw a description and examples of the incoming task, its chosen pair kept about 96 percent of correct held-out answers. A random pair kept about 93. Seeing the bank alone, without the incoming task, did little better than random. And the informed pair came within a fraction of a point of using all four.
Sam: That's one model size. What about the smaller ones?
Alex: No advantage. At 0.6B the informed choice was slightly worse than random. At 1.7B, roughly a tie. The authors are explicit that the benefit is limited to the tested 4B setting. They also didn't vary the strength of the learner, the selector and the generator separately, so we can't say which one drives it.
Sam: Here's my practical worry. Replay costs something. If I have a fixed training budget, what do I give up?
Alex: They tested that directly. With the number of examples held fixed, replay replaced some new-task examples. Generated replay cut the worst held-out loss by more than half. But the MedQA gain also fell by more than half. The authors are careful here. The benefit can't be credited to replay content alone, because replay also reduced exposure to the new task. Token counts and generation cost weren't matched either.
Sam: So under a tight budget, it's a trade, not a free lunch.
Alex: That's how they frame it. They don't claim a better trade-off curve.
Sam: What about letting the model propose new tasks for the bank? That seems like the natural next step.
Alex: They tried it as a single pilot, and it went badly. Accuracy dropped on the new task and all four held-out tasks. Part of that was output formatting, the math answers stopped parsing. But it's one run at one setting, so it says nothing either way about discovery's value. A two-step sequential pilot was also mixed.
Sam: Then how far should I trust the main results overall?
Alex: I'd call it moderate. The direction is consistent across sizes. But it's one model family, two incoming tasks, eight benchmarks, mostly a single update. Main studies used five seeds, some only two. Generated answers aren't verified, so replay can reproduce the model's own errors. And held-out transfer doesn't show the full pretrained repertoire survived.
Sam: So who should actually read the full paper, and where should they start?
Alex: People working on continual learning or post-training. Start with the bank-composition and target-selection results, then the fixed-allocation trade-off. Read the limitations appendix carefully. It's candid about what wasn't controlled.
Sam: And for everyone else, the one thing to carry around?
Alex: Count the right answers you lose, not just the average. And when you rehearse, rehearse broadly, because protection spills beyond what you targeted.
Sam: Worth keeping in mind before the next fine-tune.