Diffusion transformers (DiTs) achieve state-of-the-art image and video generation, but their multi-step sampling and growing parameter count make inference expensive. Post-training quantization (PTQ) is the natural remedy, yet DiT activations shift across timesteps, prompts, and guidance branches, forcing prior methods to re-fit calibration data for every new checkpoint or modality. We present OrbitQuant, a data-agnostic weight-activation quantizer that bypasses range estimation by quantizing in a normalized, rotated basis. In this basis, a randomized permuted block-Hadamard (RPBH) rotation concentrates each coordinate around one fixed, known marginal regardless of the input, so a single Lloyd-Max codebook serves all timesteps, prompts, and layers of a given input dimension. We extend the same quantizer to weight rows offline, absorbing the rotation into the weights so that it cancels inside each linear layer and only a forward rotation on the activations remains at runtime. The same recipe transfers from image to video with no per-modality tuning. Across FLUX.1, Z-Image-Turbo, Wan 2.1, and CogVideoX, it sets the state of the art for PTQ at several low-bit settings. It also pushes PTQ of image diffusion transformers to W2A4 with usable generation quality.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a paper called "OrbitQuant," which tackles a persistent problem in modern AI: how to make massive image and video generation models run faster and cheaper, without losing quality.
Sam: That's right. To understand the problem, think about how these AI image generators actually work. They use a type of model called a Diffusion Transformer — a "DiT" — which builds an image gradually, step by step, like developing a photograph in a darkroom. The catch is that these models are enormously demanding to run. They require huge amounts of computing power, which makes them expensive and slow, especially on everyday hardware.
Alex: And there's a standard trick for making AI models smaller and faster, right? Something called quantization?
Sam: Exactly. Quantization is essentially a compression technique. Imagine you're storing music. You could save every tiny detail of a recording in a massive file, or you could compress it into an MP3 — smaller, faster to load, and for most listeners, good enough. Quantization does the same thing to the numbers inside an AI model. Instead of storing each value with high precision, you round it to a simpler number, which takes up less space and runs faster.
Alex: So why doesn't that just work here? Compress the model, done?
Sam: The problem is that DiTs are unusually unstable on the inside. The numbers flowing through the model — called activations — don't stay in a predictable range. Every time you give the model a new prompt, or every time it takes another step in the generation process, those internal numbers shift dramatically. Standard compression assumes the numbers will behave consistently. When they don't, the compressed model produces garbled, noisy images.
Alex: So previous methods had to keep re-calibrating themselves to track those shifting numbers?
Sam: Right. They were constantly chasing a moving target. OrbitQuant's insight is: stop chasing. Instead of adapting the compression to fit the data, change the data so it always fits the compression.
Alex: How do you actually do that?
Sam: Think about packing oddly shaped luggage. If every bag is a different shape, you need a custom box for each one — slow and expensive. But if you put every bag through a machine that reshapes it into a uniform cube, suddenly one standard box works for everything. OrbitQuant applies a mathematical transformation to the model's internal numbers that reshapes them into a consistent, predictable form, regardless of what prompt the user typed.
Alex: And once the numbers are in that predictable form, you can use the same compression table every time?
Sam: Precisely. They pre-calculate a lookup table — called a Lloyd-Max codebook — that's perfectly tuned for that predictable distribution. Because the rotation guarantees the numbers always land in the same range, this one table works for any input, any prompt, any generation step. No re-calibration needed.
Alex: That's why they call it "data-agnostic." It doesn't need to see your data to work. What's the specific transformation they use?
Sam: It's called a Randomized Permuted Block-Hadamard rotation — RPBH for short. The name sounds intimidating, but the core idea is elegant: it shuffles and spreads the information across the model's activations so that no single value is unusually large or small. Think of it like shuffling a deck of cards before dealing — you go from a clumped, uneven distribution to something balanced and predictable. And crucially, they fold this rotation mathematically into the model's existing weights, so at runtime, the overhead is minimal. Just one lightweight operation on the activations.
Alex: So the model doesn't even slow down much from the rotation itself?
Sam: That's one of the practical strengths of the approach. The heavy lifting is done once, offline, when you set up the codebook. At runtime, the cost is low.
Alex: What happens when you really push this — compress the model as aggressively as possible?
Sam: This is where the results become particularly notable. There's a compression level the paper calls W2A4 — meaning the model's weights are stored at 2-bit precision and its activations at 4-bit. That's extremely aggressive. At that level, every other method the researchers tested collapsed into visual noise. OrbitQuant was the only approach that still produced usable images.
Alex: So it's not just marginally better — it's the difference between working and not working.
Sam: At that extreme, yes. And because the approach doesn't depend on any specific training data, the same technique transfers directly from image models to video models without any additional tuning. That generality is significant.
Alex: The paper also mentions something called AdaLN. Why couldn't they compress that part as aggressively?
Sam: Good question. AdaLN — Adaptive Layer Normalization — is a component that adjusts the model's behavior based on which step of the generation process it's currently on. Unlike most of the model's weights, which are fixed after training, these adjustments are dynamic. They change with every step. That means you can't fold a static rotation into them the way you can with the rest of the model.
Alex: So compressing those parts too hard would break the timing of the whole generation process?
Sam: Exactly. The researchers keep those components at a higher precision — INT4 rather than 2-bit — to preserve that dynamic behavior. It's a deliberate trade-off: apply aggressive compression where the math is stable, and hold back where it isn't.
Alex: Choosing your battles. Get the bulk of the model down to 2-bit, but protect the parts that actually need to move.
Sam: That's a precise way to put it. The insight is knowing which parts of the system can tolerate compression and which parts are load-bearing. By isolating the dynamic components, they avoid the failure modes that plagued earlier methods.
Alex: So the paper's contribution isn't just a new compression technique — it's a more principled way of thinking about where instability actually comes from in these models, and how to route around it.
Sam: That's the heart of it. By transforming the problem rather than fighting it directly, OrbitQuant offers a path toward running these powerful models on more modest hardware — and the approach is clean enough that it generalizes well beyond the specific models tested. It's a measured but meaningful step forward.
Alex: Thanks for listening to ResearchPod.