Industrial recommenders increasingly adopt the pretrain-then-transfer paradigm, yet behavioral distribution drift raises two questions: what to learn from behavior sequences, and how to transfer the learned knowledge while the pretrained model is continually refreshed. To resolve them, we propose Knowledge-Geometry Decoupling (KGD). For what to learn, conventional next-token prediction treats adjacency as dependency and may encode spurious transitions across unrelated sessions. We introduce Behavioral Multi-Token Prediction (BMTP) to retain only collaboratively or semantically related future items as supervision, yielding cleaner and more transferable behavioral knowledge. For how to transfer, pretrained knowledge and task-specific geometry impose conflicting optimization demands on shared parameters. To handle it, KGD assigns them to separate parameter sets: a refreshable encoder owns behavioral knowledge, while a task learner reads contextualized encoder states through read-only cross-attention and writes task-specific geometry through Anchored Calibration Residual (ACR) orthogonal to the pretrained embedding. The decoupled ownership enables continual knowledge refresh without task-gradient interference or invalidating downstream adaptation. KGD improves over strong pretrain-transfer baselines by 4-12% on eight public benchmarks and sustains its advantage over a 90-day production stream where baselines show no gains. KGD has been fully deployed in Shopee. In a live A/B test on Shopee Homepage Search, it increases GMV per user by 1.75% and advertising revenue by 1.53%, demonstrating its high practical value. We provide the core implementation of KGD at https://github.com/FuCongResearchSquad/KGD4REC.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a common challenge in how modern recommendation systems—like the ones that suggest videos or products to you—actually learn.
Sam: We're discussing a paper about a framework called Knowledge–Geometry Decoupling, or KGD. It addresses a real paradox: to keep a system accurate, you need to update it with new data, but those updates often accidentally erase the useful patterns the system already learned.
Alex: So this paper is basically asking how we can keep a system fresh without breaking the foundation it's built on?
Sam: Exactly. Think about what happens when new and old information have to share the same space inside a model. The new stuff doesn't politely squeeze in—it overwrites what was already there. Researchers call this "catastrophic forgetting," and it's a genuine obstacle for any system that needs to keep learning over time.
Alex: That's a useful way to put it. It's like trying to update a map while someone is still drawing the original streets. How does KGD actually solve that?
Sam: It uses what the paper calls "decoupling"—keeping two things separate that would otherwise interfere. Think of the original model as a permanent ink map. Instead of drawing over it, KGD adds a transparent overlay on top, which the paper calls an "Anchored Calibration Residual." New information goes on the overlay. The ink underneath stays untouched.
Alex: So the "ink" stays safe, but the "overlay" can be updated whenever the data changes?
Sam: Right. The system separates the "what" from the "how." The foundational knowledge—all the patterns it has learned about products, users, and their relationships—lives in the permanent ink. The specific task it's trying to do right now, like predicting what you might click next, lives in that refreshable overlay.
Alex: And for someone without a math background, what does "geometry" mean in this context?
Sam: Think of it as the shape of the data. Imagine plotting every product on a giant graph, where similar things end up close together. Two brands of running shoes would be near each other; a cookbook and a yoga mat might be further apart. That arrangement—the relative positions of everything—is the "geometry." When a task updates, it often tries to warp that shape to suit its new goal, and that warping is exactly where the conflict with older knowledge happens.
Alex: So by keeping the task's adjustments on that separate overlay, the original relationships stay intact.
Sam: Precisely. The two layers are mathematically independent—they don't pull at each other. The system can refresh its understanding of current trends without losing the deeper, more stable knowledge it built up over time.
Alex: You also mentioned "denoising" earlier. How does the system know what's genuinely useful versus just noise?
Sam: That's handled by something the paper calls "Behavioral Multi-Token Prediction." Here's the problem it's solving: standard training assumes that every item a user clicks is meaningfully connected to the next one. But real shopping behavior is messy. You might buy a bike part, then immediately buy a pair of trousers. Those two actions aren't really related—you just needed both things that day.
Alex: Oh—so the model was learning false patterns because it assumed everything in a session was part of one coherent story?
Sam: Exactly. If the model treats every click as a meaningful signal, it ends up learning a lot of noise alongside the genuine patterns. By filtering out transitions between items that aren't truly related, the system only trains on the connections that actually mean something. It's a bit like watching a skilled teacher—you'd learn more by focusing on the moments when they're actually explaining a concept, rather than treating every single thing they do in the room as a lesson.
Alex: And when they tested this in practice?
Sam: The method outperformed existing approaches by a meaningful margin in controlled tests. More tellingly, in a real-world deployment at Shopee—a major e-commerce platform—it produced a measurable lift in both sales and ad revenue. That's a useful signal, because lab results and live results don't always agree. Here they did.
Alex: That's worth noting. But this sounds like it adds complexity. Is there a cost to running two separate layers instead of one?
Sam: There is. Because you're maintaining two distinct components—the foundational encoder and the task-specific overlay—the system carries roughly double the memory of a standard model. The daily training time also increases, from around one hour to two. Those are real infrastructure costs that any team adopting this would need to plan for.
Alex: So the trade-off is: more compute and memory upfront, but a system that actually holds up over time instead of quietly degrading.
Sam: That's the argument the paper makes. The data they present tracks performance over a 90-day window. Other methods might look competitive in the short term, but they erode as time passes. KGD maintains its performance because it can keep refreshing the overlay without ever corrupting the foundation.
Alex: Looking ahead, does this point toward a world where one large model could serve many different tasks simultaneously, just by swapping out its overlay?
Sam: That's the direction this architecture suggests. The paper points toward what researchers describe as a "Universal Recommender Backbone"—a single foundation that could serve many different downstream tasks at once, each with its own overlay being refreshed independently, without any of them interfering with the others. It would remove the need to retrain from scratch every time a new business goal emerges.
Alex: So the insight isn't just about having more data—it's about being more careful with how you use and preserve what you already know.
Sam: That's a good way to put it. The study suggests that how you manage your knowledge is just as important as how much of it you have. By keeping the geometry separate from the foundation, you stop the system from fighting itself every time it learns something new.
Alex: It's a meaningful step toward making these systems more stable in practice. Thanks for walking me through this, Sam.
Sam: Happy to. The open question for the field is probably data efficiency—whether you can get these benefits without the added infrastructure cost. But as a solution to catastrophic forgetting in live recommendation systems, this is a significant step forward. Thanks for listening to ResearchPod.