Zelin Tan, Yiqun Zhang, Hao Li, Zhiyao Cui, Hejia Geng, Shao Zhang, Hangfan Zhang, Yang Chen, Xiaosong Wang, Lilong Wang, Zhenfei Yin, Shuyue Hu, Chen Zhang, Lei Bai
5 min
Agent skills have become an important mechanism for equipping language-model agents with reusable procedural knowledge. However, providing skills alone does not guarantee that current models can effectively identify, apply, and coordinate them. To improve skill-use capabilities, we introduce SKT, a verified data synthesis pipeline that constructs skill-grounded tasks and executable trajectories from large collections of agent skills. SKT selects suitable single-skill and multi-skill configurations, synthesizes tasks through rule-based and agent-based verification with feedback-guided repair, and retains only successful trajectories that substantially use every required skill. Using 2,000 public skills, SKT produces 4,000 task packages and 27,164 verified trajectories. Based on the same pipeline and a disjoint test pool, we further construct SkillEval, a held-out executable benchmark for evaluating skill use. Experiments across diverse models, benchmarks, and agent harnesses show that supervised fine-tuning on SKT-generated trajectories consistently improves skill-use performance. Verification ablations, cross-harness evaluation, and scaling experiments further demonstrate that these gains depend on high-quality supervision, extend beyond a single agent interface, and increase with broader skill coverage. Together, these results establish verified data synthesis as an effective and scalable approach for skill-use training.
As language model agents increasingly rely on external 'skills'—reusable procedural knowledge packages—a critical gap has emerged: simply providing these skills does not guarantee that a model knows how to identify, apply, or coordinate them effectively. This paper investigates how to train models to master the use of these skills, moving beyond mere retrieval to active, successful task execution.
To address this, the authors introduce SKT (Skill-use Training), a data synthesis pipeline that generates high-quality, verified training data. The process involves three main stages:
These verified trajectories are then used for supervised fine-tuning (SFT) of smaller models (Qwen3.5-9B and Gemma 4 E4B-IT). The authors also introduce SkillEval, a held-out benchmark constructed using the same pipeline to evaluate skill-use capabilities on unseen tasks.
Training on SKT-generated trajectories consistently improves skill-use performance across all tested models, benchmarks, and agent harnesses. In every comparison, the fine-tuned models outperformed their original, off-the-shelf counterparts, with gains ranging from 3.2 to nearly 19 points on a 100-point scale. The results demonstrate that the improvements are robust, scaling with broader skill coverage and generalizing across different agent interfaces.
This work establishes that verified data synthesis is a scalable and effective strategy for teaching agents how to utilize procedural knowledge. By automating the creation of high-quality, skill-grounded training data, the SKT framework provides a path to improve agentic capabilities without requiring massive amounts of human-annotated demonstrations, which are often expensive and difficult to scale.
Alex: That's a meaningful distinction. So the verification step isn't just a quality check—it's the whole point.
Sam: Exactly. And it also addresses a concern you might naturally have: if the selection is that aggressive, does the model just memorize a narrow set of patterns? The answer is no, because the skills are so diverse. The model is forced to learn the process of tool use—how to read instructions, select the right tool, and execute the steps—rather than just memorizing specific answers.
Alex: Does it actually work across different kinds of tasks? What if the agent needs to juggle several tools at once?
Sam: The evidence suggests it holds up well. They tested scenarios requiring one tool, two tools, and three or more tools working together. The model showed consistent gains across all of them. It's not just learning "how to use a hammer"—it's learning how to build something using a hammer, a saw, and a level, in the right order.
Alex: And what about the teacher model itself? If the system relies on a more capable model to generate the training data, what happens if that teacher makes mistakes?
Sam: That's a genuine limitation the paper acknowledges. The quality of the training data depends entirely on the quality of the teacher. If the teacher fails to use a tool correctly, it produces flawed examples, and those can degrade the student model. It's a real bottleneck—the system is only as reliable as the model generating its practice problems.
Alex: So there's a ceiling built into the approach.
Sam: For now, yes. But the researchers point toward a more self-sufficient future. The next phase involves agents that can identify gaps in their own skill libraries and generate their own practice problems to fill them—essentially creating their own certification courses on the fly.
Alex: That would make these assistants considerably more adaptable. Rather than waiting for humans to build new training data, they'd be able to teach themselves new procedures as they encounter them.
Sam: That's the direction. And what this paper establishes is a solid foundation for that: a principled method for turning a library of tools into genuine, verified competence. The work suggests that with the right verification pipeline, we can automate the creation of high-quality training material at scale.
Alex: Thanks for walking us through this, Sam. It's a clear look at a practical problem—and a practical approach to solving it.
Sam: It is. Thanks for listening to ResearchPod.