When Close Enough Is Not Enough: Autoregressive Drift in Quantum Circuit Synthesis

This paper investigates whether a compact encoder-decoder transformer (44.8M parameters) can learn to optimize quantum circuits by mapping unoptimized input circuits to equivalent, lower-cost targets. The authors utilize a structured tokenization scheme that represents gates as semantic tokens rather than raw text. They evaluate the model on two distinct tasks: parameterized circuits, where continuous rotation angles are involved, and Clifford+T circuits, which consist entirely of discrete gates.

The researchers identify a sharp contrast in model performance based on the output requirements. For parameterized circuits, the model predicts the circuit structure while a separate regression head estimates rotation angles. Because the transformer reliably predicts the structure, a classical Nelder-Mead optimizer can refine the angles to achieve near-perfect fidelity. In contrast, Clifford+T circuits require exact discrete correctness. While the model learns valid syntax and accurate T-count statistics, it struggles with exact functional equivalence. The authors demonstrate that performance degrades significantly as the target circuit length increases, with exact-match rates dropping from 88% for short circuits (≤9 gates) to near zero for longer ones (≥26 gates).

The authors trace the failure in Clifford+T synthesis to a phenomenon they term autoregressive drift. Because the model generates tokens left-to-right, an error in an early token propagates through the entire sequence, making it impossible to recover the correct unitary. While inference-time strategies—such as generating multiple candidates and selecting the best via equivalence verification—and scaling the training data by 2.5x provide measurable improvements, they do not solve the fundamental issue: the model remains unable to guarantee exact equivalence for longer circuits. The authors conclude that while transformers are effective for circuit optimization when approximate outputs can be corrected post-hoc, they are currently limited for tasks requiring strict, discrete correctness.