We present Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English. Its hybrid design activates only 3B of 30B parameters per token and keeps the inference cache near-constant as context grows, giving it a decisive throughput advantage over dense models for long-context, high-concurrency deployment. Pretrained on roughly 27 trillion tokens with deliberately up-weighted German, Soofi S matches dense 14 to 27B models on aggregate English and German benchmarks while achieving the best code aggregates in both languages among 17 open base models, and outperforms every European sovereign baseline in our comparison, including ones far larger in active parameters. Among fully open models, Soofi S obtains the highest English and German evaluation scores, ahead of Olmo 3 32B and Apertus 70B. Soofi S was built end-to-end on the German Industrial AI Cloud, a sovereign HPC scale AI infrastructure operated by Deutsche Telekom in Munich. Soofi S will be released under highly permissive, open-access terms: weights, selected intermediate checkpoints, full per-source data accounting, hyperparameters, and training and evaluation code. Where source licenses permit, data-construction artifacts are released under permissive licenses; commercially licensed sources are documented with aggregate statistics and exact mixture accounting.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at Soofi S, a new AI language model built for both German and English. What makes it interesting isn't just what it can do—it's how it's built to do it cheaply.
Sam: That's the core puzzle. Usually, to get high performance from an AI, you need a massive, expensive model running on powerful hardware. Soofi S tries to break that trade-off by using a different underlying structure—one that keeps costs low even when processing very long documents.
Alex: So the question this paper is really asking is: can you get top-tier results without the massive price tag?
Sam: Exactly. For a company or government agency processing huge files in real time, the cost of running a standard large model can be prohibitive. This research argues that by rethinking the architecture from the ground up, you can maintain strong performance while dramatically cutting memory usage.
Alex: How does changing the structure actually save that much memory?
Sam: Think of it like a smart librarian. A standard AI model reads the entire library every time you ask it a question. Soofi S takes a different approach—it has a large total number of internal components, around 30 billion, but for any single word it processes, it only wakes up a small fraction of them, roughly 3 billion. The rest stay dormant. This design is called a Mixture-of-Experts approach, because different specialist groups handle different kinds of tasks.
Alex: So instead of using the whole brain for every word, it only activates the parts it actually needs for that specific word?
Sam: Precisely. But the more significant innovation is how it handles long documents—because that's where traditional models genuinely struggle. When a standard model reads a long text, it keeps a running record of everything it's seen, and that record grows with every new word. Eventually it gets so large it clogs the system. Soofi S replaces that growing record with something called a Mamba-2 layer.
Alex: What does Mamba-2 actually do differently?
Sam: Instead of keeping a full transcript of everything it's read, it maintains a compact, fixed-size summary—like a student who takes condensed notes rather than copying out the entire textbook. That summary stays the same size no matter how long the document gets. Combine that with the specialist-expert structure, and the model stays fast even when reading very large files.
Alex: Oh—so the speed stays flat because it isn't carrying a massive, growing backpack of old data. The summary just gets updated in place.
Sam: That's exactly it. The paper reports that at the length of a long legal or technical document—around 40,000 tokens, which is roughly 30,000 words—this model runs about eight to nine times faster than comparable models built the traditional way.
Alex: That kind of efficiency seems particularly useful for organizations that want to run AI on their own servers rather than sending data to a cloud provider. Is that what the paper means by "sovereign" AI?
Sam: Yes. The idea of sovereign AI is that an organization hosts and controls its own system, so sensitive data never leaves its infrastructure. Making the model efficient enough to run on standard hardware—rather than requiring a data center full of specialized chips—is what makes that practical. The paper positions this as a meaningful step toward that goal, particularly for German-speaking institutions with strict data privacy requirements.
Alex: And they're open about how they built it?
Sam: They are. The researchers have released their full data accounting and training recipes—meaning they've documented exactly what data was used and how the training process was structured. That level of transparency is relatively uncommon in this field.
Alex: How do they actually manage the training process for something this complex?
Sam: They use what they call a Warmup–Stable–Decay schedule. Think of it like training for a long-distance race: you start at a gentle pace to let your body adjust, settle into a steady rhythm for the bulk of the work, and then in the final stretch, you sharpen your focus. In the model's case, that final phase means switching exclusively to the highest-quality data available, to refine the skills the model has already developed.
Alex: So it's a very deliberate, staged process—not just throwing data at the model and hoping for the best.
Sam: Exactly. The overall picture is of a carefully engineered system designed with practical deployment in mind from the start. The architecture choices, the training schedule, the transparency about data—they all point toward the same goal: a model that organizations can actually run themselves, on their own terms.
Alex: That's a useful frame. It's less about chasing benchmark scores and more about making capable AI genuinely usable outside of large tech companies. Thanks for walking through it, Sam.
Sam: My pleasure. And thanks to everyone listening—this is ResearchPod.