AIDiveDeleted, measured, one rule broke
Deleting a CLAUDE.md means removing the instruction file Claude Code reads at the start of every conversation. Boris Cherny, the creator of Claude Code, said on stage to delete it every six months. Twenty-two videos repeated him in seven weeks. None of them opened a repository.
This article does what the videos did not: it takes one real app with a 177-line CLAUDE.md, three skills, four commands and a hook, runs five everyday tasks with the file and without it, and counts. Forty-four runs later, exactly one rule broke. The file cost tokens on every task, never the same amount, and one of its lines nobody could follow.
The clip, verbatim, and the two lines that are not his
The clip comes from Boris Cherny's YC Startup School talk, recorded the day after Opus 5 shipped. His words: every six months, delete your CLAUDE.md, delete your skills, delete your hooks. See what the model does, it might surprise you. For Opus 5, he says, Anthropic really does recommend trying it: the model might not need all those instructions anymore.
Thirty seconds earlier comes the caveat nobody quotes. Anthropic does not delete the entire code base. It deletes a lot, and it calls that an ablation. The written transcript carries that line in full today. Eighty percent of the Claude Code system prompt went that way: delete everything, bring it back line by line, measure each line.
Two sentences the reaction videos put in his mouth are not in the talk. "Context, goals, and a definition of done" appears nowhere; the closest he gets is task, guardrails, exit criteria. "Sixty-four agents rewriting Bun" is not his number either: it is Jarred Sumner's, in the Bun post. Cherny says eleven days and, when asked for a count, guesses thousands.
Twenty-two videos in seven weeks quoted the clip. None ran the test. So this one does what he actually described: delete, bring it back one piece at a time, measure.
The instrument: ablation, not deletion
An ablation removes one piece of a configuration at a time and measures the effect, instead of deleting everything and guessing. The CLAUDE.md is read at the start of every conversation and again on every turn; skills load only when invoked. That difference is what got measured.
Anthropic ships the delete switch: a bare flag, the same variable Cherny names on stage. The repository is a real app of mine: 177 lines of instructions, three skills, four commands, and a hook that fires on every search.
| Dimension | Value |
|---|---|
| Everyday tasks | 5 (new component, edit, refactor, store change, architecture question) |
| Configurations | 5 (full, no file, no skills, no hook, nothing) |
| Model | one, pinned |
| Environment | empty config directory, fresh clone before every run |
| Runs | 44 |
| Cost | $39 |
Every run was scored the same way, on the same clone, by a script rather than by hand. What counts as broken: the repository's own rules (imports, types, design tokens, translations) plus typecheck and lint.
The one tool built for this kind of ablation, Caliper, ablates skills and MCP servers but never touches the file; the CLAUDE.md swap was done by hand. The limits, stated once: one repository, one model, two runs per cell, no transcript. The first result set the tone: the refactor task came back identical, sixty-four cents with the file and sixty-three without.
What broke: one rule, on new files
The rule that broke is an internationalisation rule, in the file's own words: always add both English and French, never hardcode a string the user sees. On the new-component task, with the file, all four runs wrote the translation file. Without it, three runs of four hardcoded the heading. Same task, same repository, same model, same prompt.
| New component | Wrote the translation file |
|---|---|
| With CLAUDE.md | 4 of 4 |
| Without CLAUDE.md | 1 of 4 |
The edit task tells the other half. Every configuration got it right, even with nothing, because the neighbours teach it: every component beside the edited one already has a translation file. Everything else held in all forty-four runs: the import alias, type over interface, design tokens, the store pattern. The code shows those; the file repeats them.
A paper from ETH Zurich measured the same thing on 138 real issues. Its finding: context files do not improve success and cost about twenty percent more, and the model does follow the instructions. One caveat: one run without the file wrote the translation file anyway. The rule is not impossible without the file, it is unreliable. One rule broke, the one the code could not teach. And the run that skipped that work was also the cheapest.
What the file cost, task by task
The token cost of a CLAUDE.md is the difference in tokens read between a run with the file and the same run without it.
| Task | Fewer tokens read without the file |
|---|---|
| New component | 61% |
| Edit | 15% |
| Refactor | 5% |
| Store change | 4% |
| Architecture question | 14% |
| Over the ten runs | 32% tokens, 22% money |
Thirty-two percent is the number every video would headline, and it is the wrong one. The biggest saving is the run that did not write the translation file: cheaper because it did less. Where the output was identical, the file cost four to fourteen percent. That is its real price, and the paper's twenty percent sits right between those two numbers.
The mechanism is about 1,800 tokens, read again on every turn. Skills, the hook and the knowledge graph produced nothing measurable, present or absent. Noise is bigger than most of it: the same task with the same file cost $2.11 on one run and $1.33 on the other. Two runs hit the turn cap, one with the file and one without. So the file costs a little everywhere and earns its place once, unless it also lies.
The lines that lied
A lying line is an instruction the model cannot follow or that changes nothing. The line teased at the top: import the theme from the design-tokens alias. The alias does not exist: the TypeScript config maps one prefix, and the tokens folder has no theme file. Every run, with the file or without, did what the neighbours do, the same import line forty-four times.
The file has eighty-two lines of architecture overview. Same question, six answers: all six found the same nine files, backend to screen, in the same order, with the overview and without. A block points at a knowledge graph the clone does not have; with the graph present, the question cost the same.
| Measure | Value |
|---|---|
| Anthropic's size rule | under 200 lines |
| This file | 177 lines |
| Median file in the reporails 30k-repo dataset | 50 items, 12 directives |
| Directive lines in this file | 24 of 177 |
Position decides which of two conflicting rules wins, and the model never says so, about ninety points in one vendor's test. The overview might help a task not run here: one probe, one answer. Three kinds of lines, then: the one that earned its place, the ones the code teaches, the ones that lie.
Keep, move, delete: the short list
Three piles, and the measurement behind each.
| Pile | What goes there | Measurement |
|---|---|---|
| Keep | The rule the code cannot show | 6 lines that saved 4 runs |
| Move | The architecture overview and the commands | 107 lines with no measured gain |
| Delete | The lines that lie, and the lines the tree already shows | 44 identical runs |
Keep what the model got wrong twice: that is Anthropic's own criterion for the file. Move the overview and the commands into a tree of files loaded when needed; Anthropic's July post calls the central repository a myth. Hooks stay: a gate does not expire when the model gets better. Cut the prose it outgrew, keep the gate.
The file after: about seventy lines, the six that earned it on top. That is Cherny's method, read whole: delete, bring it back line by line, measure. One repository, one model, two runs per cell; your piles will differ, the method will not. Deleting cost one rule and saved little. Finding the lines that lie was the win.
