FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling

Large language models (LLMs) are increasingly deployed using 4-bit floating-point (FP4) formats like MXFP4 and NVFP4 to reduce memory and computational costs. However, maintaining model accuracy at this precision is difficult because existing methods force both quantization and dequantization scales to conform to rigid, hardware-mandated discrete formats. This paper investigates whether relaxing these constraints on the quantization scale—which is only used during training—can improve model performance without introducing inference overhead.

The authors propose FOCUS, a post-training quantization (PTQ) framework that treats the quantization scale as a learnable parameter. The framework introduces two key innovations:

1. Coupled-Relaxation Scaling (CRS): Instead of forcing the quantization scale to match the discrete dequantization scale, CRS introduces a learnable full-precision coefficient. This lifts the quantization scale optimization from a discrete, hardware-constrained space into a continuous space, enabling gradient-based learning.
2. Dual-Granularity Scaling (DGS): This technique further refines the quantization scale by partitioning each hardware-defined block into smaller sub-blocks. Each sub-block is assigned an independent coefficient, allowing the quantization process to adapt more precisely to local weight distributions within a block.

Because these coefficients are only used during the offline quantization phase, they are discarded after training. The resulting model remains fully compatible with standard hardware, ensuring no additional latency or computational overhead during inference.

FOCUS achieves state-of-the-art accuracy across multiple LLM families and benchmarks for both MXFP4 and NVFP4 formats. For instance, on the Qwen3-4B model, FOCUS recovers 98.2% of the original FP16 zero-shot accuracy under the NVFP4 format, consistently outperforming existing baselines. The authors demonstrate that by exploiting the freedom of the quantization scale, the model can better approximate the original weight distributions without requiring complex, overhead-heavy transformations like Hadamard rotations.