Yinghui He, Ling Yang, Jiarui Liu, Yongjin Yang, Lechen Zhang, Yingcheng Wu, Zhenfei Yin, Mengdi Wang, Sanjeev Arora
5 min
Modern LLMs often excel at single-domain tasks but struggle when a problem requires fluidly switching between different reasoning styles—such as moving from a mathematical derivation to a logical planning step. This paper formalizes these as cross-skill long-horizon tasks. The authors argue that current benchmarks fail to capture the difficulty of these transitions, leading to a "skill-switching gap" where models perform well on isolated skills but fail when those skills must be composed in a single reasoning chain.
To address this, the authors introduce Skill Entropy, a directed, pairwise measure of how difficult it is to transition from one skill to another. By evaluating a reference model on both single-skill and cross-skill sequences, they calculate a scalar score for any given task. They then release Skill^2-Bench, a comprehensive benchmark covering 558 skills across 9 domains (including math, coding, and creative writing). Their evaluation of 12 models reveals that accuracy drops consistently as task-level skill entropy increases, confirming that switching between skills is an orthogonal and significant challenge for current architectures.
Moving beyond evaluation, the authors propose Skill-Entropy RL, a reinforcement learning framework designed to make models "skill-native." In this framework, the model is trained to predict the specific skill it is using at each step of the reasoning process. The reward function is twofold: it rewards both the correctness of the final answer and the alignment between the model's predicted skill sequence and a ground-truth "gold" sequence. This approach yielded substantial performance improvements on the Skill^2-Bench, with the Qwen3-4B-Instruct model seeing its score jump from 34.4% to 68.4%. The authors demonstrate that this signal is reusable, as it can be applied to existing, off-the-shelf training datasets.
Long-horizon reasoning in recent LLMs demands that the model switch between distinct skills inside a reasoning chain, such as first doing a math derivation, then using the result to plan a schedule. We call such problems cross-skill long-horizon tasks: multi-step tasks whose steps require different reasoning skills and depend on earlier outputs. Existing benchmarks often evaluate individual skills, lacking a principled way to measure how well a model switches between skills. We address this gap from both the evaluation and training sides. We introduce Skill Entropy, a measure of the difficulty of switching from one skill to another. We then propose Skill^2-Bench, a benchmark of cross-skill long-horizon tasks built over 558 skills across 9 verifiable and open-ended domains. Each task is assigned a task-level skill-entropy score and grouped into three difficulty levels. Evaluating 8 frontier and 4 open-source models on Skill^2-Bench reveals a skill-switching gap: accuracy decreases on higher-entropy tasks. We then turn skill entropy from a benchmark scale into a training signal. We propose Skill-Entropy RL, an RL framework where the model predicts not only the answer at each step but also the skill used to produce it. The reward combines step-level correctness with a skill-entropy reward that measures the alignment between the model-predicted skill sequence and the gold skill sequence. On Qwen3-4B-Instruct and Qwen3-1.7B, Skill-Entropy RL improves the Skill^2-Bench score from 34.4% to 68.4% and from 14.6% to 40.1%, respectively, outperforming competitive baselines. The same pipeline can be applied to off-the-shelf training data such as OpenR1-Math, indicating that skill entropy is a reusable training signal. Code available at: https://github.com/Gen-Verse/Skill-Entropy-RL
Sam: So you're not just training it to get the right answer — you're training it to be aware of its own reasoning process as it goes.
Alex: Precisely. And that awareness is what makes the difference. The model learns to plan its own gear-shifts rather than stumbling through them. The results showed a meaningful increase in accuracy on complex, multi-step tasks.
Sam: Though I'd imagine there's a risk there — if you're rewarding a very specific kind of structured planning, could the model just learn to game that structure without actually improving its reasoning?
Alex: That's a legitimate concern. What the study found, though, is that because the reward is tied to the structure of the reasoning process rather than to specific correct answers, the model tends to generalise. It learns the strategy of switching — how to manage transitions between different types of thinking — rather than memorising solutions to particular problems. That's why it appears to transfer to tasks it wasn't explicitly trained on.
Sam: So it's learning a flexible habit of mind, not a fixed set of moves.
Alex: That's the suggestion, yes. And it points to something broader about where this field may be heading. Current models are largely trained to predict the next word — they're pattern-matchers operating at the level of language. What this research points toward is something more structured: systems that actively track what kind of thinking a task requires, and adjust accordingly.
Sam: So the gap isn't just about knowledge or raw capability. It's about self-awareness in the reasoning process.
Alex: That's the core idea. The researchers describe this as moving toward what they call "skill-native" models — systems that know which tool they're reaching for at every step. Whether that framing holds up as these systems grow more complex remains to be seen, but as a direction of research, it's a meaningful one. Making the reasoning process explicit, rather than leaving it implicit, appears to be a genuine step toward AI that can reliably handle long, multi-stage work.
Sam: So the lesson isn't just "train on more data." It's "train on better structure."
Alex: That's a fair summary. Thanks for listening to ResearchPod.