Efficient foundation decoders for fault-tolerant quantum computing

As quantum error correction (QEC) scales toward fault-tolerant regimes, the demand for high-capacity neural decoders grows. However, training these 'foundation decoders' from scratch for each new, larger code distance is computationally prohibitive, often requiring thousands of GPU-hours. This paper asks whether it is possible to leverage knowledge learned from smaller, tractable code distances to accelerate and improve the training of decoders for larger, more complex codes.

The authors propose Neural Transfer Unification (NTU), a framework that treats decoder construction as a transfer-learning problem. By identifying scale-invariant algebraic structures—where the relative neighborhood of syndrome detectors remains consistent as the code distance increases—NTU aligns decoding tasks across different sizes. The authors instantiate this framework as NTU-Transformer, which utilizes a geometry-aware rotary positional encoding (RoPE) to ensure that syndrome data is embedded into a shared latent space, regardless of the specific code distance. This allows a model trained on a smaller code to serve as a high-quality initialization for a larger code, which is then fine-tuned to adapt to the expanded boundaries.

NTU-Transformer demonstrates superior scalability and accuracy compared to traditional baselines. For planar surface codes, the model outperforms correlation-aware matching on the [[361, 1, 19]] code and successfully scales to the [[625, 1, 25]] code. Crucially, the transfer paradigm resolves the 'cold-start' problem: while models trained from scratch exhibit stagnant performance during early training, NTU-initialized models show near-instantaneous convergence. Additionally, the framework proves effective for bivariate bicycle (BB) codes, outperforming Relay-BP in low-physical-error regimes. By reducing the computational scaling exponent, NTU provides a sustainable path for generating foundation decoders for increasingly large quantum processors.