We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemma 4 model suite features dense and Mixture-of-Experts architectures, ranging from 2.3B to 31B parameters. Alongside improved vision and audio encoders for all model sizes, we propose a unified, encoder-free architecture for our 12B model, which ingests raw audio and image patches. Furthermore, we integrate a thinking mode, enabling Gemma models to generate reasoning traces prior to responding. We improve inference speed, memory, and compute efficiency, as well as long-context abilities through critical design choices. Gemma 4 establishes a leap in performance across STEM, multimodal, and long-context benchmarks, and rivals larger, frontier open models in human-rated tasks.
Alex: Welcome to another episode of ResearchPod. Today, we are looking at the technical report for Gemma 4, a new suite of open-weight, multimodal AI models from Google DeepMind.
Sam: So the core question this paper is trying to answer is: how do you build a powerful AI that can handle images, audio, and text—without needing a supercomputer to run it?
Alex: That is exactly it. And to understand why that is hard, you need to know how most of these systems are built today. When an AI needs to process an image, it typically runs that image through a separate, specialised piece of software first—a kind of translator that converts the picture into a format the main AI brain can understand. These translators are called encoders.
Sam: And those translators are expensive in terms of memory and processing power?
Alex: They are. Each one takes up space, and when you have several of them—one for images, one for audio—they create a kind of traffic jam in the system's memory. The researchers asked a direct question: what if we got rid of those translators entirely?
Sam: So instead of having a specialist for every type of input, the main model just learns to read everything directly?
Alex: Right. The model receives raw image data—just blocks of pixels—and raw audio data, and treats them the same way it treats words. It learns to make sense of all of it in one unified process. They call this an encoder-free architecture. The practical result is a simpler system with fewer moving parts competing for the same hardware resources.
Sam: Does it actually work as well as the traditional approach?
Alex: The data suggests it does. Their largest compact model performs at a level that rivals much larger models still using the older, heavier design. The key insight is that you do not necessarily need a pre-trained specialist if you give the main model enough training to develop that understanding on its own.
Sam: Though I imagine that puts a lot of pressure on the training phase?
Alex: It does. The authors note that the model needs more extensive training upfront to learn what a specialised encoder would have already known. It is a bit like learning to swim by jumping into the deep end rather than taking lessons first—harder initially, but you end up with a more unified skill set.
Sam: That trade-off seems worth it if the end result is a lighter, more capable system. What about the reasoning side of things? The report mentions a kind of "thinking mode."
Alex: Think of it like giving the model a scratchpad. Before it answers a difficult question, it works through the problem step by step, generating what the researchers call a reasoning trace. Only after that internal process does it produce a final answer. It is similar to teaching a student to show their work rather than just guessing.
Sam: And that helps with things like maths or coding problems?
Alex: Notably so. The performance gains on those kinds of tasks are meaningful. It is a post-training addition—meaning it is layered on top of the base model's existing capabilities rather than baked in from scratch.
Sam: Longer reasoning traces must use more memory, though. How do they manage that?
Alex: They optimised something called a KV cache. Here is the simplest way to think about it: imagine you are reading a long novel. You do not re-read every previous chapter before starting a new page—you just remember what happened. The cache works the same way, storing a running summary of everything the model has already processed so it does not have to recalculate from the beginning each time. By sharing parts of that cache across different layers of the model, they reduced its memory footprint significantly.
Sam: So the model is working smarter about what it keeps in short-term memory.
Alex: Exactly. And there is a related technique for speeding up how quickly the model produces text. A smaller, faster companion model drafts a few words ahead of time, and the main model rapidly checks and corrects that draft. It is a bit like having an assistant write a rough version that you quickly proofread. The report calls this speculative decoding, and the suggestion is that the impact on output quality is minimal—the main model still verifies every word.
Sam: So you get speed without sacrificing accuracy. What about making these models small enough to run on a phone or a laptop?
Alex: That is where quantization comes in. Think of it like compressing a high-resolution photograph into a smaller file. You lose a tiny amount of detail, but the file takes up far less space. Quantization does the same thing to the model's internal settings—it stores them using less precise numbers, which shrinks the overall size. The problem is that compressing a finished model usually degrades its performance noticeably.
Sam: So how do they avoid that?
Alex: They train the model with that compression in mind from the very beginning. Rather than building a full model and then squeezing it down, the model learns to be resilient to lower precision throughout training. The result is a much smaller model that still performs close to its full version—which is a meaningful step toward making these systems genuinely accessible on everyday hardware.
Sam: The report also covers a larger model with a different architecture entirely. How does that one work?
Alex: This one uses a different design philosophy. Rather than one large, unified network that activates fully for every question, it is divided into smaller specialised sub-networks. For any given input, only the most relevant sub-networks switch on. The rest stay idle.
Sam: Like having a team of specialists rather than one generalist doing everything at once.
Alex: That is a good way to put it. The total model is large, but at any given moment, only a fraction of it is active. You get the knowledge base of a large system with the running costs of a much smaller one.
Sam: And safety? These models are being released openly, which means anyone can use them.
Alex: The report describes a layered approach. Training data was filtered to remove sensitive personal information. Post-training techniques were applied to encourage the model to decline requests that could lead to harmful or private disclosures. And the reasoning trace we discussed earlier has a useful side effect here—by thinking through a request before answering, the model has an opportunity to recognise when something crosses a line before it commits to a response.
Sam: So the same mechanism that helps with maths problems also helps with safety judgements.
Alex: That is the suggestion, yes. Though it is worth noting the authors frame this carefully—it is a design feature that supports safety, not a guarantee of it.
Sam: Stepping back, what is the most significant takeaway from this report for someone following AI development?
Alex: The clearest signal is a shift in priorities. For years, the dominant approach was simply to build larger models. This report reflects a different question: can we make models that are architecturally smarter, so that size becomes less important? The encoder-free design, the cache optimisation, the speculative decoding—these are all answers to that question. The direction of travel is away from "how big can we make it" and toward "how efficiently can we make it work on hardware people actually have."
Sam: And if that trend continues, what does it look like?
Alex: The next generation of these models may treat all sensory input—text, images, audio—as a single unified stream, with no specialist translators required at all. That would make building things like on-device voice assistants considerably more straightforward. What this report demonstrates is that careful architectural choices can translate into real, practical improvements—not just on benchmarks, but in where and how these models can actually be used.
Sam: That is a useful frame for thinking about where this field is heading. Thanks for walking me through it, Alex.
Alex: It was a pleasure. And thank you for listening to ResearchPod.