# I Deleted My CLAUDE.md And Measured What Broke

Video: https://www.youtube.com/watch?v=1oJSvzz9-kE
Article: https://aidive.dev/videos/delete-your-claude-md-measured/
Published: 2026-09-23

## Chapters

- [0:00](https://www.youtube.com/watch?v=1oJSvzz9-kE&t=0s) Deleted, measured, one rule broke
- [0:35](https://www.youtube.com/watch?v=1oJSvzz9-kE&t=35s) The clip, verbatim, and the two lines that are not his
- [1:34](https://www.youtube.com/watch?v=1oJSvzz9-kE&t=94s) The instrument: ablation, not deletion
- [2:57](https://www.youtube.com/watch?v=1oJSvzz9-kE&t=177s) What broke: one rule, on new files
- [4:00](https://www.youtube.com/watch?v=1oJSvzz9-kE&t=240s) What the file cost, task by task
- [4:54](https://www.youtube.com/watch?v=1oJSvzz9-kE&t=294s) The lines that lied
- [5:56](https://www.youtube.com/watch?v=1oJSvzz9-kE&t=356s) Keep, move, delete: the short list

## TL;DR

- Deleting a 177-line CLAUDE.md from a real app broke exactly one rule in 44 runs: the i18n rule, on new files, 3 runs out of 4. Everything else held because the surrounding code already teaches it.
- The file cost 4% to 61% of the tokens read depending on the task, 32% over ten runs; where the output was identical, its real price was 4% to 14%.
- Boris Cherny's clip says delete it every six months, but the same talk describes an ablation: delete, bring it back line by line, measure each line.
- Two lines in the file lied: an import alias that does not exist in the config, and an 82-line architecture overview that changed nothing in six answers.
- The verdict is three piles: keep the 6 lines the code cannot show, move 107 lines of overview and commands into files loaded on demand, delete the lines that lie. Hooks stay.

## Deleted, measured, one rule broke

Deleting a CLAUDE.md means removing the instruction file Claude Code reads at the start of every conversation. Boris Cherny, the creator of Claude Code, said on stage to delete it every six months. Twenty-two videos repeated him in seven weeks. None of them opened a repository.

This article does what the videos did not: it takes one real app with a 177-line CLAUDE.md, three skills, four commands and a hook, runs five everyday tasks with the file and without it, and counts. Forty-four runs later, exactly one rule broke. The file cost tokens on every task, never the same amount, and one of its lines nobody could follow.

## The clip, verbatim, and the two lines that are not his

The clip comes from Boris Cherny's YC Startup School talk, recorded the day after Opus 5 shipped. His words: every six months, delete your CLAUDE.md, delete your skills, delete your hooks. See what the model does, it might surprise you. For Opus 5, he says, Anthropic really does recommend trying it: the model might not need all those instructions anymore.

Thirty seconds earlier comes the caveat nobody quotes. Anthropic does not delete the entire code base. It deletes a lot, and it calls that an ablation. The written transcript carries that line in full today. Eighty percent of the Claude Code system prompt went that way: delete everything, bring it back line by line, measure each line.

Two sentences the reaction videos put in his mouth are not in the talk. "Context, goals, and a definition of done" appears nowhere; the closest he gets is task, guardrails, exit criteria. "Sixty-four agents rewriting Bun" is not his number either: it is Jarred Sumner's, in the Bun post. Cherny says eleven days and, when asked for a count, guesses thousands.

Twenty-two videos in seven weeks quoted the clip. None ran the test. So this one does what he actually described: delete, bring it back one piece at a time, measure.

## The instrument: ablation, not deletion

An ablation removes one piece of a configuration at a time and measures the effect, instead of deleting everything and guessing. The CLAUDE.md is read at the start of every conversation and again on every turn; skills load only when invoked. That difference is what got measured.

Anthropic ships the delete switch: a bare flag, the same variable Cherny names on stage. The repository is a real app of mine: 177 lines of instructions, three skills, four commands, and a hook that fires on every search.

| Dimension | Value |
|---|---|
| Everyday tasks | 5 (new component, edit, refactor, store change, architecture question) |
| Configurations | 5 (full, no file, no skills, no hook, nothing) |
| Model | one, pinned |
| Environment | empty config directory, fresh clone before every run |
| Runs | 44 |
| Cost | $39 |

Every run was scored the same way, on the same clone, by a script rather than by hand. What counts as broken: the repository's own rules (imports, types, design tokens, translations) plus typecheck and lint.

The one tool built for this kind of ablation, Caliper, ablates skills and MCP servers but never touches the file; the CLAUDE.md swap was done by hand. The limits, stated once: one repository, one model, two runs per cell, no transcript. The first result set the tone: the refactor task came back identical, sixty-four cents with the file and sixty-three without.

## What broke: one rule, on new files

The rule that broke is an internationalisation rule, in the file's own words: always add both English and French, never hardcode a string the user sees. On the new-component task, with the file, all four runs wrote the translation file. Without it, three runs of four hardcoded the heading. Same task, same repository, same model, same prompt.

| New component | Wrote the translation file |
|---|---|
| With CLAUDE.md | 4 of 4 |
| Without CLAUDE.md | 1 of 4 |

The edit task tells the other half. Every configuration got it right, even with nothing, because the neighbours teach it: every component beside the edited one already has a translation file. Everything else held in all forty-four runs: the import alias, type over interface, design tokens, the store pattern. The code shows those; the file repeats them.

A paper from ETH Zurich measured the same thing on 138 real issues. Its finding: context files do not improve success and cost about twenty percent more, and the model does follow the instructions. One caveat: one run without the file wrote the translation file anyway. The rule is not impossible without the file, it is unreliable. One rule broke, the one the code could not teach. And the run that skipped that work was also the cheapest.

## What the file cost, task by task

The token cost of a CLAUDE.md is the difference in tokens read between a run with the file and the same run without it.

| Task | Fewer tokens read without the file |
|---|---|
| New component | 61% |
| Edit | 15% |
| Refactor | 5% |
| Store change | 4% |
| Architecture question | 14% |
| Over the ten runs | 32% tokens, 22% money |

Thirty-two percent is the number every video would headline, and it is the wrong one. The biggest saving is the run that did not write the translation file: cheaper because it did less. Where the output was identical, the file cost four to fourteen percent. That is its real price, and the paper's twenty percent sits right between those two numbers.

The mechanism is about 1,800 tokens, read again on every turn. Skills, the hook and the knowledge graph produced nothing measurable, present or absent. Noise is bigger than most of it: the same task with the same file cost $2.11 on one run and $1.33 on the other. Two runs hit the turn cap, one with the file and one without. So the file costs a little everywhere and earns its place once, unless it also lies.

## The lines that lied

A lying line is an instruction the model cannot follow or that changes nothing. The line teased at the top: import the theme from the design-tokens alias. The alias does not exist: the TypeScript config maps one prefix, and the tokens folder has no theme file. Every run, with the file or without, did what the neighbours do, the same import line forty-four times.

The file has eighty-two lines of architecture overview. Same question, six answers: all six found the same nine files, backend to screen, in the same order, with the overview and without. A block points at a knowledge graph the clone does not have; with the graph present, the question cost the same.

| Measure | Value |
|---|---|
| Anthropic's size rule | under 200 lines |
| This file | 177 lines |
| Median file in the reporails 30k-repo dataset | 50 items, 12 directives |
| Directive lines in this file | 24 of 177 |

Position decides which of two conflicting rules wins, and the model never says so, about ninety points in one vendor's test. The overview might help a task not run here: one probe, one answer. Three kinds of lines, then: the one that earned its place, the ones the code teaches, the ones that lie.

## Keep, move, delete: the short list

Three piles, and the measurement behind each.

| Pile | What goes there | Measurement |
|---|---|---|
| Keep | The rule the code cannot show | 6 lines that saved 4 runs |
| Move | The architecture overview and the commands | 107 lines with no measured gain |
| Delete | The lines that lie, and the lines the tree already shows | 44 identical runs |

Keep what the model got wrong twice: that is Anthropic's own criterion for the file. Move the overview and the commands into a tree of files loaded when needed; Anthropic's July post calls the central repository a myth. Hooks stay: a gate does not expire when the model gets better. Cut the prose it outgrew, keep the gate.

The file after: about seventy lines, the six that earned it on top. That is Cherny's method, read whole: delete, bring it back line by line, measure. One repository, one model, two runs per cell; your piles will differ, the method will not. Deleting cost one rule and saved little. Finding the lines that lie was the win.

## FAQ

### Should you delete your CLAUDE.md like Boris Cherny said?

Not blindly. Cherny's talk describes an ablation: delete the file, bring it back line by line and measure each line. On a real 177-line file, deleting it broke one rule in 44 runs and saved little; the win was finding the lines that lie.

### What breaks when you delete a CLAUDE.md?

On this repository, one rule: always add both English and French translations. Without the file, three runs out of four hardcoded a heading on the new-component task. Every other rule held because the neighbouring code already shows it.

### How many tokens does a CLAUDE.md cost?

About 1,800 tokens read again on every turn. Task by task that was 4% to 61% fewer tokens read without the file; where the output was identical the file cost 4% to 14%, which matches the ETH Zurich paper's 20% average on 138 real issues.

### What should stay in a CLAUDE.md?

The rules the code cannot show, which is also Anthropic's own criterion: keep what the model got wrong twice. Move overviews and command lists into files loaded when needed, delete instructions that point at things that do not exist, and keep hooks, since a gate does not expire when the model improves.

### Do context files like CLAUDE.md or AGENTS.md improve coding agents?

The ETH Zurich paper on 138 real issues found that context files do not improve task success and raise inference cost by over 20% on average, even though the model does follow the instructions. This measurement on one repository landed in the same range.

### What is an ablation in Claude Code terms?

Removing one piece of the configuration at a time, with the model pinned and a fresh clone per run, and measuring the effect. Anthropic used it to cut 80% of the Claude Code system prompt; here it ran across five configurations: full, no file, no skills, no hook, nothing.

## Sources

- [Boris Cherny: We Cut 80% of Claude Code's Prompt (YC Startup School 2026)](https://www.youtube.com/watch?v=qyPCVqFUyDo) — Y Combinator
- [Boris Cherny on building Claude Code, transcript](https://www.ycombinator.com/library/UN-boris-cherny-building-claude-code) — Y Combinator
- [How Claude remembers your project (CLAUDE.md docs)](https://code.claude.com/docs/en/memory) — Anthropic
- [Claude Code CLI reference (--bare flag)](https://code.claude.com/docs/en/cli-reference) — Anthropic
- [The new rules of context engineering for Claude 5 generation models](https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models) — Anthropic
- [Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?](https://arxiv.org/abs/2602.11988) — arXiv
- [Caliper: ablation tooling for Claude Code](https://github.com/edonadei/caliper) — GitHub
- [The state of AI instruction quality: a 30k-repo analysis](https://reporails.com/articles/state-ai-instruction-quality-30k-repo-analysis) — reporails