The skill-that-fires kit: make a Claude Code skill load on its own

TL;DR and verdict

A Claude Code skill can run perfectly when you type its name and still never load when you ask in plain words. Before Claude decides to load a skill, it reads one line per skill: the name and the description. It matches that line against your words. If the line does not use your words, the skill stays silent.

In our run, a release-notes skill with the description "Generates changelog documents from repository history." loaded in 0 of 5 fresh sessions on the request "what did we ship this week?". We rewrote that one line as what it does, when to use it and the words we type. The same request then loaded it in 5 of 5.

The verdict:

• A skill pays for a task you repeat with a fixed output: weekly notes, a pull request description, a release checklist.
• It does not pay for a one-off question, and it is not the tool for a rule that must always hold. That is a hook's job.
• A pile of skills nobody uses costs you, because past the listing budget Claude Code cuts descriptions, least used first.
• The one rule: write the description in the words you actually type when you ask for the task.

The kit below has our skill file before and after, a description template, the fresh-session checklist and the twenty-request test we used.

What the docs and the field say

Where a skill lives. A skill is a folder with a SKILL.md inside, either in ~/.claude/skills/<skill-name>/ for every project or in .claude/skills/<skill-name>/ so your team gets it too. A loose name.md file does not load. s1 s2

Claude reads the description, not the body. The listing of names and descriptions is in context every turn, and the body loads only when the skill is invoked. The docs say the description is what Claude uses to decide when to apply the skill. s1

The listing has a cap and a budget. The description and when_to_use text are truncated together at 1,536 characters. The whole listing is limited to 1% of the context window, and when it overflows, Claude Code drops descriptions starting with the skills you invoke least. The docs say this removes the keywords Claude needs to match your request. You see a warning in the debug log with --debug. s1

A malformed header is the trap that looks like success. If the YAML is malformed, the body loads with empty metadata, so /skill-name still works but Claude cannot match your description. Run with --debug to see the parse error, or run claude plugin validate .claude/skills. s1 s2

Check the badge. If a skill appears in /skills but Claude never invokes it, either disable-model-invocation: true is set or the description does not match how you phrase the request. A user-only label means Claude will not trigger it on its own. s2

What a good description contains. Anthropic's guide says the description should include both what the skill does and when to use it, written in third person, with the key terms users type. It notes that Claude uses it to choose the right skill from potentially 100+. s3

Claude tends to undertrigger. Anthropic's skill-creator says "currently Claude has a tendency to 'undertrigger' skills" and recommends a slightly "pushy" description. s18 Its trigger test uses 20 queries, a mix of should-trigger and should-not-trigger, with near misses the most valuable. The loop splits them 60% train and 40% held-out test, runs each query 3 times, and keeps the description with the best test score. s4 The eval script runs each query through claude -p and checks whether the skill was used. s19

Activation behaves like keyword matching. On the real binary, one third-party test (22 prompts, Sonnet 4.5) measured a baseline of 55% and 50% across two runs, and a forced-evaluation hook at 100%. A prompt with an explicit token was activated near 100% of the time, while indirect phrasing sat near 0%. The author's reading: closer to keyword matching than semantic matching. This is one blog, an older model and a different repo, so treat it as supporting evidence, not a rule. s5

The description is a trigger, not a summary. An audit of nine real skills reached the same conclusion: a description that lists concrete situations beats one that says what the skill is. s8

The opposite failure exists too. One developer watched a skill trigger on the wrong tasks, wrote trigger evals, rewrote the description as "Load when...", and noted that a skill worth keeping is usually born from a failure you are tired of seeing twice. s11

Others hit this too. The issue "Claude Code not aware of skills in .claude/skills" has 75 reactions and was still open on 2026-10-17. Workarounds in the thread include a CLAUDE.md pointer and a hook. s7

Where a hook beats a skill. A top answer in a 778-upvote thread says a skill is "basically just a prompt you save for later" that Claude decides to use or you call with a slash command, and that a hook is for when skills are too loose and you want determinism. s9 A different thread warns that reminder nudges can fade over a long session. s10

The kit

The skill before and after. Only the description line differs, the body is identical.

release-notes.before.SKILL.md

---
name: release-notes
description: Generates changelog documents from repository history.
---

# Release notes

1. Find the range: commits since last Monday (`git log --since="last monday" --no-merges --pretty="%h %s"`).
2. Drop `chore` and `docs` commits.
3. Group what is left by the conventional-commit scope (router, css, jsx, serve-static...), fixes and features apart.
4. Write one line per change, in plain English, present tense, no commit hashes, with the PR number at the end.
5. Output a markdown block titled "Release notes, week of <date>", with a two-line summary on top.

release-notes.before.SKILL.md on GitHub

release-notes.after.SKILL.md

---
name: release-notes
description: Writes this week's release notes from git log, grouped by scope (router, css, jsx). Use when the user asks what we shipped, what changed this week, for release notes, a changelog, or a summary of recent commits.
---

# Release notes

1. Find the range: commits since last Monday (`git log --since="last monday" --no-merges --pretty="%h %s"`).
2. Drop `chore` and `docs` commits.
3. Group what is left by the conventional-commit scope (router, css, jsx, serve-static...), fixes and features apart.
4. Write one line per change, in plain English, present tense, no commit hashes, with the PR number at the end.
5. Output a markdown block titled "Release notes, week of <date>", with a two-line summary on top.

release-notes.after.SKILL.md on GitHub

The description template, with our real description as the worked example.

description-template.md

# Skill description template

The description is the one line Claude reads before it decides to load your skill. Write it for the moment you ask, not for the file.

## Template

```
<Verb in third person> <what the skill produces>, <how it is shaped>. Use when the user asks <phrase 1>, <phrase 2>, <phrase 3>, or <phrase 4>.
```

Three parts, in this order:

1. What it does. Third person ("Writes", never "I can" or "You can use this"). Name the output.
2. When to use it. Start with "Use when the user asks".
3. Your words. The exact phrases you or your team type, copied from your own history, not synonyms you invented.

Put the key use case first. The description and `when_to_use` are cut together at 1,536 characters in the skill listing.

## Worked example (the real one from our run)

Before, a summary of the skill:

```
Generates changelog documents from repository history.
```

After, what + when + the words we type:

```
Writes this week's release notes from git log, grouped by scope (router, css, jsx). Use when the user asks what we shipped, what changed this week, for release notes, a changelog, or a summary of recent commits.
```

What changed:

- What: "Writes this week's release notes from git log, grouped by scope".
- When: "Use when the user asks ...".
- Your words: "what we shipped", "what changed this week", "release notes", "a changelog", "a summary of recent commits".

## Fill-in sheet

- Skill name:
- The prompt I keep retyping (copy it from your history):
- The output I want from it:
- Verb + output (what it does):
- Four phrases I really type (when):
- Phrases that sound close but are a different job (for your near misses):

## Check before you save

- Third person, no "I" and no "you".
- At least three phrases you really type.
- Under 1,536 characters in total.
- Not so wide that it matches a neighbour job. Add the near misses to your trigger test.

description-template.md on GitHub

The fresh-session checklist.

trigger-checklist.md

# Fresh-session trigger checklist

A skill that runs by name can still be invisible to a plain request. Test it the way a teammate will use it.

## Test in a fresh session

1. Open a new session. A skill already used stays in the conversation, so a test in the same session proves nothing.
2. Ask for the task in your own words, without the skill's name. Example: "what did we ship this week?"
3. Watch for the Skill tool call before Claude touches anything else. If Claude runs the commands itself and writes its own summary, the skill was skipped.
4. Repeat in several fresh sessions. A handful of runs is a demo, not a statistic.

## If it was skipped, read why before you edit

1. `/skills`: is it listed, is it switched on, is there a `user-only` badge? A `user-only` badge means Claude will not trigger it on its own.
2. Ask "What skills are available?" and check your skill is in the answer.
3. `claude --debug`, then search the log for the skills line (how many loaded) and for a "Skill listing over budget" warning. Over budget means some descriptions are cut, the least invoked first.
4. `claude plugin validate .claude/skills`: a malformed header still lets `/name` work but leaves Claude with empty metadata to match against.
5. Check the file is `.claude/skills/<name>/SKILL.md`, not a loose `<name>.md`, and that the header starts on the first line.

## Fix, then prove it

1. Rewrite the description with `description-template.md`.
2. Fresh session, same plain request, several runs: does the Skill call come first now?
3. Baseline: set the skill to `"off"` in `skillOverrides` (saved to `.claude/settings.local.json` from `/skills`), ask again, then switch it back on.

## Hold it

1. Write twenty requests: ten that should load the skill, ten near misses (`evals.json` is our set).
2. Run each in its own fresh session and note whether the skill loaded (`trigger_test.py`, `tally.py`).
3. Re-run the list every time you change the description.
4. For a skill that must never miss: call it by name, or add a UserPromptSubmit hook (`hook.json`).

trigger-checklist.md on GitHub

The twenty-request test set: ten requests that should load the skill, ten near misses that should not. Edit the queries to match your own repo.

evals.json

[
  {"query": "what did we ship this week?", "should_trigger": true},
  {"query": "can you put together the release notes for this week", "should_trigger": true},
  {"query": "what changed in the repo since monday", "should_trigger": true},
  {"query": "write the changelog for the commits we merged this week", "should_trigger": true},
  {"query": "I need a summary of this week's commits for the team update on friday", "should_trigger": true},
  {"query": "draft the weekly notes of what got fixed, grouped by area", "should_trigger": true},
  {"query": "list what landed this week, router and css changes apart", "should_trigger": true},
  {"query": "give me something to post in the team channel about this week's fixes", "should_trigger": true},
  {"query": "recap the work merged since last monday", "should_trigger": true},
  {"query": "what's new in hono this week? short version for the release post", "should_trigger": true},
  {"query": "explain what commit 08807ce1 changes in the trie router", "should_trigger": false},
  {"query": "the jsx tests are failing on cloneElement, can you fix them", "should_trigger": false},
  {"query": "write a commit message for my staged changes", "should_trigger": false},
  {"query": "who wrote most of the serve-static code?", "should_trigger": false},
  {"query": "add a CHANGELOG entry format section to CONTRIBUTING.md", "should_trigger": false},
  {"query": "show me the diff between main and my branch", "should_trigger": false},
  {"query": "bump the version in package.json to the next patch", "should_trigger": false},
  {"query": "why does the css helper minify url() like that?", "should_trigger": false},
  {"query": "squash my last three commits into one", "should_trigger": false},
  {"query": "review this PR for breaking changes before I merge it", "should_trigger": false}
]

evals.json on GitHub

The script that runs each request in its own fresh claude -p session and counts whether the Skill tool was called.

trigger_test.py

"""Twenty-prompt trigger test for one skill, the method skill-creator describes, run on the real skill folder.
For each query: a fresh `claude -p` in the repo; the run counts as loaded if the Skill tool is called with the skill's name
before the run ends (stopped early as soon as it is). Usage:
  python3 trigger_test.py <repo> <skill-name> <SKILL.md to install> <evals.json> <out.json> [workers]"""
import json, os, shutil, subprocess, sys
from concurrent.futures import ThreadPoolExecutor

repo, name, skill_md, evals_path, out = sys.argv[1:6]
workers = int(sys.argv[6]) if len(sys.argv) > 6 else 4
dest = os.path.join(repo, '.claude', 'skills', name)
os.makedirs(dest, exist_ok=True)
shutil.copy(skill_md, os.path.join(dest, 'SKILL.md'))
evals = json.load(open(evals_path))
allowed = ['Bash(rtk git log *)', 'Bash(git log *)', 'Bash(git show *)', 'Bash(date *)', 'Skill', 'Read']

def run(item):
    cmd = ['claude', '-p', item['query'], '--output-format', 'stream-json', '--verbose', '--allowedTools', *allowed]
    p = subprocess.Popen(cmd, cwd=repo, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, text=True)
    loaded = False
    try:
        for line in p.stdout:
            try:
                d = json.loads(line)
            except ValueError:
                continue
            if d.get('type') == 'assistant':
                for c in d['message'].get('content', []):
                    if c.get('type') == 'tool_use' and c['name'] == 'Skill' and name in json.dumps(c.get('input', {})):
                        loaded = True
            if loaded or d.get('type') == 'result':
                break
    finally:
        if p.poll() is None:
            p.kill()
    return {**item, 'loaded': loaded, 'pass': loaded == item['should_trigger']}

with ThreadPoolExecutor(workers) as ex:
    results = list(ex.map(run, evals))
should = [r for r in results if r['should_trigger']]
shouldnt = [r for r in results if not r['should_trigger']]
summary = {'loaded_when_it_should': sum(r['loaded'] for r in should), 'of_should': len(should),
           'loaded_when_it_should_not': sum(r['loaded'] for r in shouldnt), 'of_should_not': len(shouldnt),
           'passed': sum(r['pass'] for r in results), 'total': len(results)}
json.dump({'skill': name, 'skill_md': skill_md, 'summary': summary, 'results': results}, open(out, 'w'), indent=2)
for r in results:
    print(f"[{'PASS' if r['pass'] else 'FAIL'}] {'loaded ' if r['loaded'] else 'skipped'} expected={'load' if r['should_trigger'] else 'skip'}: {r['query']}")
print(json.dumps(summary))

trigger_test.py on GitHub

A small tally for a folder of claude -p --output-format stream-json runs.

tally.py

"""Count whether a `claude -p --output-format stream-json` run loaded a given skill. Usage: python3 tally.py <skill> run1.jsonl [run2.jsonl ...]"""
import json, sys
skill, files = sys.argv[1], sys.argv[2:]
fired = 0
for f in files:
    hit = False
    for line in open(f):
        try:
            d = json.loads(line)
        except ValueError:
            continue
        if d.get('type') == 'assistant':
            for c in d['message'].get('content', []):
                if c.get('type') == 'tool_use' and c['name'] == 'Skill' and skill in json.dumps(c.get('input', {})):
                    hit = True
    fired += hit
    print(f"{f}: {'loaded' if hit else 'skipped'}")
print(f"{skill}: loaded in {fired} of {len(files)} runs")

tally.py on GitHub

The must-fire fallback: a UserPromptSubmit hook that names the skill.

hook.json

{
  "hooks": {
    "UserPromptSubmit": [
      {
        "hooks": [
          {
            "type": "command",
            "command": "echo 'If the user asks about release notes, a changelog or what shipped this week, use the release-notes skill.'"
          }
        ]
      }
    ]
  }
}

hook.json on GitHub

Do this Monday

• Write down the prompt you retyped most last week, and the exact output you want from it.
• Create .claude/skills/<name>/SKILL.md with a name, a description and a short numbered body.
• Test it by name, then open a fresh session and ask for the task in plain words without the name.
• If it is skipped, read /skills, claude --debug and claude plugin validate .claude/skills before editing anything.
• Rewrite the description with description-template.md: what it does, when to use it, the words you type.
• Retest in several fresh sessions, then once with the skill set to "off" in skillOverrides and once back on.
• Write your twenty requests, run them with trigger_test.py, and re-run them whenever you change the description.
• For a skill that must never miss, decide: call it by name, or add the hook.

What we measured

These are small demo runs on a scratch clone of the open-source Hono repo (s20), with Claude Code 2.1.294 and the request "what did we ship this week?" on 8 non-merge conventional commits since Monday 2026-10-12. They are not a benchmark. Your model, your skills and your repo will move the numbers.

Before the rewrite, the 4 requests that loaded were the ones containing "release notes", "changelog", "notes" or "release": which is the keyword effect in plain sight. After the rewrite the wider description did not pull in a single near miss.

The warning we found, and the honest reading. With --debug our setup printed: "Skill listing over budget: 296 skills, 144833 chars > 30000 budget". Our setup carries too many skills, so descriptions get truncated. Our project skill still loaded ("Loaded 43 unique skills", one of them the project skill), and the old description survived the cut: in the twenty-request test, the request containing "changelog", a word that lives only in the description, loaded the skill. We did not prove how the budget treats a new skill in general. If you carry a lot of skills, run the check yourself. /skills on our setup listed 277 skills, and release-notes showed as a project skill, switched on, with no user-only badge.

A caveat on the tool itself: Anthropic's eval script did not see the skill at all on our version, so we ran the same requests with the short script in the kit. The method is the same, one fresh claude -p session per request.

Go further

Fix the opposite problem too. A description that matches too much makes the skill fire where it should not. This is why half the test set is near misses. Add the words you really use, not every word you can think of. s11

Let the optimizer propose the rewrite. Skill-creator's loop proposes new descriptions and keeps the best one by held-out test score, to avoid overfitting to the queries you tuned on. Use your own twenty requests as the eval set. s4

Use a substantive test prompt. Skill-creator notes Claude only consults skills for tasks it cannot easily handle on its own, so a trivial one-step request may not trigger a skill even with a perfect match. s4

Go beyond the plain hook. The third-party test above found that a forced-evaluation hook, which makes Claude list each skill as yes or no and then activate, reached 100% where a passive reminder did not, and that a native type: "prompt" hook behaved like no hook. Our hook is the simple version, and it is one demo run. s5 Reports say hook nudges can fade in a long session. s10

Keep the body short. In the audit above, the one broken skill was 1,002 lines with MANDATORY everywhere, while the healthy ones were short. s8

What we left out. We did not cover plugin skills and the plugin eval graders, which matter only to plugin authors, or how to compare a skill against the bare model over months. Some people also report that skills written for weaker models can become dead weight, so re-test them now and then. We did not measure that.

FAQ

Why does /release-notes work but plain words do not? Typing the name bypasses matching. In plain words Claude only has the name and one description line to match, so a summary that shares no word with your request is skipped. s1

Do I have to restart Claude Code after editing a skill? No. Claude Code picks up an added or edited skill under ~/.claude/skills/ or .claude/skills/ in the current session. If you create a top-level skills directory that did not exist when the session started, run /reload-skills. s1

How long can the description be? The description and when_to_use are cut together at 1,536 characters in the listing, so put the key use case first. Anthropic's platform guide separately caps a description at 1,024 characters for skills uploaded outside Claude Code. s1 s3

Is five fresh sessions enough to trust a fix? No. It shows the direction. Our result is a demo on one repo, which is why the kit ends with a twenty-request test you can re-run every time you change the description.

When should I use a hook instead of a skill? When a rule must hold every time, not just when Claude notices a match. A hook adds a line to every message, so keep it for the few skills that must fire. s9