What Is a Skill Worth? Structure-Aware Shapley Valuation of Agent Skills

As automated systems increasingly generate complex, multi-part agent skills (such as rules, scripts, and examples), it has become difficult to determine which specific parts of these artifacts actually contribute to performance. The authors address the problem of intra-skill valuation: how to assign credit to internal units of a skill to guide debugging, pruning, and optimization.

The authors introduce SkillSV, a framework that treats skill valuation as a structure-constrained cooperative game. Unlike standard Shapley value applications that treat data as flat, independent points, SkillSV recognizes that skill units are interdependent. It uses a compiler to map a skill into a graph of units, dependencies, and document hierarchies. This ensures that only 'feasible' counterfactuals—those that remain syntactically and logically valid—are evaluated. To manage the high cost of agent rollouts, the authors employ a chain-coupled estimator that uses paired deletion and length-neutral padding to isolate the content value of a unit from the cost of the context it occupies.

SkillSV successfully recovers the interactions between skill units, allowing researchers to distinguish between units that provide genuine performance gains and those that merely add unnecessary length to the prompt. The framework satisfies value closure and provides actionable insights, such as identifying units that are candidates for compression versus those that should be deleted entirely. Across four agentic benchmarks, the authors demonstrate that their method guides safe pruning and compression with negligible impact on overall skill performance.

This work provides a principled way to interpret the 'black box' of automated agent skill generation. By moving beyond aggregate performance metrics, developers can now understand the internal composition of their agents, leading to more efficient, maintainable, and reliable skill artifacts.