A Quantum Roadmap for Softmax Attention: Exact Born-Rule Analogs for Softmax Attention on the Probability Simplex

This paper investigates the structural alignment between classical transformer attention and quantum amplitude encoding. Specifically, the authors address whether the attention mechanism—when inputs and outputs are constrained to the probability simplex—admits an exact quantum realization. The researchers decompose the single-head attention layer into six quantum primitives: amplitude encoding, block-encoded query/key projections, Hadamard-test-based inner product computation, a Born-rule-based softmax map, a column-loading channel for value aggregation, and a gated single-ancilla residual connection.

The authors establish a formal dictionary between classical transformer components and quantum circuit operations. Key results include:

• Softmax Bijection: The exponential softmax used in classical models is shown to be isomorphic to a cosine-squared family generated by Born-rule measurements. This quantum version allows for sparse attention (exact zeros) at finite parameter values, which the classical exponential softmax cannot achieve.
• Temperature as Repetition: The softmax temperature, typically a hyperparameter in classical models, is realized physically as the number of post-selected measurement rounds.
• Residual Gating: The additive residual connection is identified as a specific slice of a parameterized single-ancilla circuit, allowing for a unified representation of no-residual, additive, and full-sublayer cases.
• Formal Verification: The algebraic core of these equivalences, including the angle-score bijection and the identity of the variational circuit at the origin, is machine-checked using the Lean 4 theorem prover.

This work provides a rigorous theoretical foundation for implementing transformer-style attention on quantum hardware. By proving that these mechanisms can be mapped exactly to quantum primitives, the paper offers a roadmap for designing quantum-native architectures that maintain the functional properties of classical transformers while operating natively on the probability simplex. The use of Lean 4 to verify the algebraic core ensures the mathematical correctness of the proposed quantum-classical dictionary.