SkillsBench: A Benchmark for Evaluating AI Agent Skills Across Diverse Tasks

Summary

SkillsBench is a new benchmark for evaluating how well AI agent skills work across diverse tasks. The benchmark includes 86 tasks across 11 domains with curated skills and deterministic verifiers. Testing 7 agent-model configurations over 7,308 trajectories showed that curated skills improve average pass rates by 16.2 percentage points, though effectiveness varies significantly by domain (from +4.5pp in Software Engineering to +51.9pp in Healthcare). Notably, 16 of 84 tasks showed negative performance impacts from skills. Self-generated skills provided no benefit on average, indicating models cannot reliably author the procedural knowledge they benefit from consuming. The research also found that focused skills with 2-3 modules outperform comprehensive documentation, and smaller models with skills can match larger models without them.