UE5M3 FP4 Block Scaling for Stable Language Model Pretraining

Stable 4-bit floating-point (FP4) pretraining is challenging due to the limited dynamic range of the standard E2M1 payload. Existing recipes, such as NVIDIA's Transformer Engine (TE) NVFP4, rely on complex techniques like randomized Hadamard transforms (RHT), current-tensor scaling, and bfloat16 (BF16) exemptions for final layers to maintain stability. This paper investigates whether using a wider, unsigned E5M3 block-scale format can provide sufficient dynamic range to simplify the pretraining recipe while maintaining or improving model quality.

Instead of the standard E4M3 block scales, the authors use unsigned E5M3 (UE5M3) block scales, which reallocate the unused sign bit to provide a significantly wider exponent range. The proposed recipe uses periodic sample-and-hold tensor scaling (refreshed every 50 steps), 2D weight scaling, and selective stochastic rounding for backward gradients. Crucially, the authors omit RHT and apply FP4 quantization to all internal linear layers, including those typically exempted in the TE recipe. To validate this, they pretrain a Nemotron-H 8B model for 188.7 billion tokens and develop a probe-matched software emulator to model native hardware GEMM outputs.

• The UE5M3-based recipe achieves lower final-window training loss and improved held-out negative log-likelihood (NLL) compared to the baseline TE NVFP4 recipe.
• The proposed method remains stable even when all 112 eligible internal linear layers are quantized to FP4, whereas a native NVFP4 ablation removing RHT and BF16 exemptions exhibits significant training instability.
• A native execution ablation demonstrates that removing RHT and quantizing all eligible layers increases model-body token throughput by 21.2%.
• The authors' probe-matched emulator successfully reproduces native hardware behavior, providing a reliable tool for software-based FP4 research.