Audio intelligence involves understanding, reasoning about, and generating both audio and speech. In this work, we introduce Nemotron-Labs-Audex-30B-A3B (Audex), a unified audio-text LLM built on Nemotron-Cascade-2-30B-A3B, a strong text-only MoE LLM. Audex adopts a simple unified design with a single Transformer decoder: audio inputs are encoded and projected into the text embedding space, while text tokens and quantized audio output tokens are treated uniformly during generation. This architecture enables strong audio-text fusion, seamless multimodal generation, and compatibility with standard LLM training and inference infrastructure. For training, we meticulously curate audio-text datasets comprising 157.4B audio tokens and 320.5B text tokens. We apply multi-stage supervised training on these datasets, followed by text-only Cascade RL and multi-domain on-policy distillation. Audex delivers state-of-the-art audio understanding, speech recognition and translation, text-to-speech, audio generation, and speech-to-speech generation, while preserving very compelling reasoning, alignment, knowledge, long-context, and agentic capabilities of its text-only LLM backbone with marginal or no regression. We release the model checkpoints to facilitate open research.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a new model called Audex.
Sam: That's right. The paper introduces a system called Nemotron-Labs-Audex-30B-A3B. The central claim is that it can listen to, understand, and generate audio while remaining just as capable at reasoning as its text-only version.
Alex: So this paper is basically asking: why do most AI models get worse at thinking when you teach them to listen?
Sam: Exactly. There's a well-known problem in AI research where adding new abilities—like processing sound—tends to degrade the skills the model already had. Researchers call this "multimodal regression." It's a bit like teaching a brilliant mathematician to play the violin. If you're not careful, all that practice on the instrument starts to crowd out the calculus. The authors wanted to show you don't have to sacrifice one for the other.
Alex: That's a significant goal. How do they actually keep the reasoning intact while adding audio skills?
Sam: The key is what they call a unified architecture. Rather than bolting on a separate "audio brain" alongside the existing text system, they teach the model to interpret sound using the same machinery it already uses for language. Think of it this way: the model already has a very powerful engine for understanding words. Instead of building a second engine for audio, they convert sound into a form that first engine can already read.
Alex: So the audio isn't handled separately? It just becomes part of the same vocabulary?
Sam: Precisely. They convert audio waves into discrete units called "tokens"—tiny, standardized building blocks, like letters in an alphabet. The model already processes text as tokens, so when audio arrives in the same format, the model treats a spoken word and a written word the same way. The reasoning engine doesn't even know the difference.
Alex: And this is what they call a "Mixture-of-Experts" model. How does that fit in?
Sam: Right. Imagine a library staffed not by one generalist librarian, but by a team of deep specialists—one for history, one for science, one for literature. When you ask a question, only the relevant specialists are called over. Everyone else stays at their desk. That's essentially how this model works. It's large and capable overall, but for any single task, only a small fraction of its total capacity is actually switched on. That keeps it efficient without sacrificing depth.
Alex: So the architecture handles both the sound-and-logic problem and the efficiency problem at the same time.
Sam: That's the idea. And the training process is designed just as carefully. They use what they call a multi-stage curriculum—a structured sequence, a bit like a school term. First, the model focuses purely on text reasoning to lock in its core intelligence. Then comes an "audio warmup" phase, where it learns to handle sound tokens without disturbing what it already knows. Only after that do they introduce more complex tasks, like generating audio from scratch.
Alex: And what happens at the very end of training?
Sam: They finish with a round of reinforcement learning aimed specifically at text reasoning. Think of it as a final exam in logic, taken after all the audio training is done. By sharpening the model's reasoning skills at the end, they effectively re-anchor its intelligence—making sure the audio work hasn't quietly eroded what was there before.
Alex: And does that actually hold up? Does the model stay sharp at reasoning?
Sam: That is the most notable finding the paper reports. On complex reasoning benchmarks, the audio-capable version performs almost identically to the text-only version. The authors argue this supports their central claim: if you treat audio as a language rather than a foreign add-on, you don't have to compromise the model's core intelligence to make it listen.
Alex: What can it actually do, in practical terms?
Sam: The paper describes a model that can transcribe speech, answer questions about audio content, generate sound effects, and work through complex coding or logic problems—all within the same system. The point isn't that any one of those tasks is new. Other models can do individual pieces. The claim is that this one does all of them without the usual trade-off.
Alex: It's a bit like the difference between a Swiss Army knife where every tool works properly, versus one where adding the scissors means the blade gets duller.
Sam: That's a fair way to put it. And the paper is careful not to overstate things. The authors acknowledge this is an early step. The approach works within the constraints they tested, but there's no suggestion it solves every challenge in multimodal AI. It's a meaningful result, not a final answer.
Alex: So the contribution here is really about the method—showing that the way you integrate audio matters enormously, and that a unified approach can preserve what makes a model useful in the first place.
Sam: Exactly. The architecture, the training sequence, the final reasoning refinement—each piece is designed to solve the same underlying problem: how do you add a new sense without losing your mind? Based on what this paper reports, treating sound as language rather than as something separate is a promising answer to that question.
Alex: Thanks for listening to ResearchPod.