Intern-S2-Mobius Team
5 min
Abstract
We introduce Mobius-v0, an architecture that comprises a globally shared Memory (FFN) that stores knowledge vectors and multiple Reasoners (Self-Attn) that iteratively achieve compositional reasoning. Using hidden states as cache and carrier, reasoners repeatedly query memory for required knowledge-vectors, while the knowledge is transmitted back to reasoning operators. Through this knowledge-reasoning-separation architecture, Mobius achieves better knowledge compression and reasoning efficiency. Built upon Mobius-v0 architecture: 1) Our 7B model trained-from-scratch achieves similar downstream score as a 7B Transformer baseline with 62.6% of baseline's training data. 2) Our Intern-S2-Mobius, continually-pretrained from Qwen3.5-35B, achieves similar downstream score while delivering nearly 4x end-to-end inference speedup.
Sam: That's precisely it. And the paper provides a concrete example to show how much difference that makes. On a linear algebra multiple-choice question, a standard model generates well over two thousand tokens of text—long paragraphs of trial and error—before arriving at an answer. Mobius completes the same reasoning using roughly a quarter of that, without skipping any of the core mathematical steps.
Alex: That's a meaningful reduction. But I'd want to know whether the model is actually reasoning more carefully, or just cutting corners.
Sam: The authors asked exactly that question, and they were explicit about investigating it. The detailed traces they examined confirm that the shorter output comes from genuine efficiency improvements—the underlying logic remains intact. The system isn't guessing; it's just not writing out the redundant parts.
Alex: Which brings up a longer-term question. If you cleanly separate knowledge from reasoning, does that help with one of the persistent problems in AI—where updating a model's facts tends to corrupt what it already knows how to do?
Sam: That's one of the more significant implications the paper points to. When facts and reasoning rules are stored together, updating one tends to interfere with the other. A clean separation offers a pathway around that constraint. The authors suggest it could support systems that continuously absorb new information without forgetting past training—though they're careful to note that real-world deployment still needs to be validated.
Alex: And presumably the hardware picture changes too, if you're not keeping everything loaded into fast memory at once.
Sam: That's the practical upshot. If bulk knowledge lives on external storage while only the active reasoning components sit in fast memory, future systems could be deployed at significantly lower cost. The authors also position this latent approach as a stronger foundation for modeling continuous physical environments and generating scientific hypotheses—but again, those remain directions rather than demonstrated results.
Alex: So Mobius is less a finished product and more a proof of concept for a different way of thinking about how AI systems should be built.
Sam: That's a fair characterization. The architecture itself is the contribution. The claim is that by rethinking where knowledge lives and how reasoning operates, you can build systems that are more efficient, more updatable, and potentially more capable—without simply making them larger.
Alex: That's a worthwhile set of ideas to sit with. Thanks for listening to ResearchPod.