Skill Creator

What it does

Walks Claude through the full skill authoring loop: interviewing you about intent, drafting a SKILL.md with a triggering description and progressive-disclosure structure, generating realistic test prompts, running them (with and without the skill as a baseline) via subagents, and grading outputs against assertions. It also runs a description-optimization loop that measures how reliably the skill triggers on should-fire vs should-not-fire queries and rewrites the description to improve accuracy. Most useful when you want a rigorously evaluated skill rather than a one-shot draft.

Use cases

• Draft a new skill from a workflow you keep repeating — capture the tools, steps, and output format into a SKILL.md with bundled scripts under scripts/ and reference docs under references/.
• Benchmark an existing skill against its previous version — snapshot the old skill, rerun the same test prompts against both, and view a side-by-side pass-rate, token, and latency comparison in the eval viewer.
• Optimize a skill’s description for triggering accuracy — generate 20 realistic should-trigger and near-miss queries, then run the automated loop that proposes and tests new descriptions against a held-out split.

When to use it

• You are writing a new Claude Skill and want it structured according to Anthropic’s progressive-disclosure conventions.
• You have a draft skill and want quantitative evals (pass rate, variance, token cost) instead of vibes.
• Your existing skill fires inconsistently and you suspect the description field needs tuning.
• You want to package a skill as a .skill file for distribution.

When NOT to use it

• You just want a one-off prompt template — the eval loop is overhead you don’t need.
• The skill’s output is purely subjective (creative writing, visual design) where assertions don’t apply and human review alone is enough.
• You are on Claude.ai without subagents and don’t need benchmarking — a shorter workflow will do.
• You want to edit a skill that is already installed read-only without copying it to a writeable path first.

Installation

/plugin install skill-creator@anthropic-agent-skills

If you haven’t added the Anthropic marketplace yet, run this first:

/plugin marketplace add anthropics/skills

After install, just mention it in conversation:

Use the skill-creator to build a new skill that converts our internal meeting notes into structured action-item tickets.

Troubleshooting

The skill under-triggers — Claude ignores it when it should fire. Run the description-optimization loop (scripts/run_loop.py) with a fresh eval set that includes casual, misspelled, and indirect phrasings of the target task. The optimizer rewrites the description to be more assertive about when to invoke, since Claude tends toward under-triggering by default.

Eval viewer won’t open because the environment has no display (Cowork, remote shell). Pass --static <output_path> to generate_review.py so it writes a standalone HTML file instead of starting a server. Share the file path with the user; the viewer’s Submit button will download feedback.json which you then copy into the workspace directory.

Timing data (tokens, duration) is missing from benchmark.json. The total_tokens and duration_ms values only arrive in the subagent task-completion notification and are not persisted anywhere else. Write each run’s timing.json immediately when the notification arrives — don’t batch them at the end or the data is gone.

Grading results don’t render in the viewer. The viewer expects each entry in grading.json’s expectations array to use the exact field names text, passed, and evidence. Variants like name/met/details will be silently ignored. Regenerate the grading file with the correct schema (see references/schemas.md).