Curriculum Reinforcement Learning Can Incentivize Reasoning Capacity in LLMs Beyond the Base Model

Standard Reinforcement Learning with Verifiable Rewards (RLVR) often improves a model's efficiency at sampling existing correct trajectories but frequently fails to expand the model's fundamental reasoning capacity. The authors investigate whether this limitation arises because difficult problems—those the model cannot solve even with many attempts—fail to provide a meaningful reward signal during training, effectively creating a 'zero-advantage' dead zone.

The authors propose a three-stage 'Boundary-Aware Curriculum RL' framework designed to push beyond the base model's reasoning capacity boundary:

1. Boundary Exploration: The model uses large-sample pass@k decoding (up to 256 samples) to categorize problems by difficulty, specifically identifying 'Frontier' problems (barely reachable) and 'Beyond' problems (unreachable).
2. Targeted Teacher Guidance: For problems near the reasoning boundary, the authors provide small amounts of structured teacher guidance (e.g., logical backbones, self-reflection) to make these previously 'zero-advantage' examples reward-reachable.
3. RL Consolidation: The model undergoes standard GRPO training using a curriculum that balances reward-reachable examples with these newly structured, boundary-pushing examples to stabilize new reasoning patterns.

Across multiple base models (Qwen, Llama, and DeepSeek), the proposed curriculum approach consistently outperformed standard Vanilla RLVR. While Vanilla RLVR often failed to improve—or even decreased—performance at large sampling budgets (pass@256), the boundary-aware approach improved average pass@256 scores by 9.8 percentage points over base models and 10.3 percentage points over Vanilla RLVR. These results demonstrate that by explicitly targeting the reasoning boundary with structured guidance, models can learn to solve problems that were previously outside their reach.