Soft Token Alignment for Cross-Lingual Reasoning

Multilingual large language models (LLMs) often struggle with cross-lingual inconsistency, where the model provides different reasoning paths for the same prompt depending on the language. While intermediate Transformer layers are generally language-agnostic, the final layers often diverge as the model commits to language-specific discrete tokens. The authors propose SOLAR (Soft Token Alignment for Cross-Lingual Reasoning), a training-time objective that aligns soft-token representations—probability-weighted mixtures over the vocabulary—across languages using English as a pivot. By minimizing the cosine distance between English and non-English soft-token summaries during supervised fine-tuning, the model is encouraged to maintain a shared semantic space throughout the generation process.

SOLAR consistently outperforms standard supervised fine-tuning and other alignment baselines across four multilingual reasoning benchmarks (MGSM, AIME 2024, AIME 2025, and GPQA). For the Qwen3-4B model, SOLAR improved accuracy by up to 3.8 points and cross-lingual consistency by up to 4.5 points. Notably, the gains were most pronounced in low-resource languages like Swahili, where accuracy nearly doubled. Representation analysis confirms that SOLAR successfully reduces language-cluster separability in the final layers, suggesting that the auxiliary objective effectively preserves shared semantic structure that is otherwise lost during discrete token selection.

This work demonstrates that the "bottleneck" of discrete token generation is a primary driver of cross-lingual reasoning divergence. By repurposing soft tokens—originally an inference-time technique—as a training-time alignment signal, the authors provide a scalable way to improve multilingual model performance without sacrificing native-language generation quality. This approach offers a robust alternative to complex contrastive learning methods, providing a more direct way to anchor multilingual reasoning in a shared semantic space.