Roteie os subagents do Claude Code para o Haiku 5.5: a tabela, os dois arquivos, o harness

TL;DR

• Os subagents do Claude Code não rodam no Haiku 5.5 a menos que um arquivo diga isso. O agente Explore embutido e todo subagent personalizado herdam o modelo principal, e a variável do modelo de subagent sozinha deixa o Explore intacto.
• Dois arquivos resolvem: um arquivo de agente Explore.md no projeto (Haiku 5.5, effort low) só para o explorador, e um bloco env em settings.json com CLAUDE_CODE_SUBAGENT_MODEL_FORCE para empurrar todos os subagents para o Haiku.
• Nossa matriz, 182 execuções em um repositório real: explorar, testes, lint, commit, refatoração, revisão e resumo, tudo aceito por um teste oculto ou uma regra fixa no Haiku 5.5, em effort baixo e médio (70 de 70 em cada).
• Roteie as sete para o Haiku 5.5 em effort low. Mantenha o Sonnet 5.5 ou o Opus 5.5 para coding agêntico complexo, que é o conselho da própria Anthropic.
• O medidor real do plano quase não se mexeu: janela de cinco horas de 21% para 21% em 70 execuções do Haiku low, janela semanal em 67% nas 182 execuções. O detalhamento do /usage ordena os modelos com pesos locais, não com cobrança.
• Fixe o id completo do modelo. O alias haiku é o Haiku 4.5 no Bedrock, no Vertex e no Foundry.

O que dizem as fontes

O Haiku 5.5 custa $0.10 por milhão de tokens de entrada para prompts de até 100k tokens e $0.50 acima disso, com saída a $0.50 e $2.50. O Haiku 4.5 custa $1.00 e $5.00, o Sonnet 5.5 custa $2.00 e $10.00 s1. A página de preços lista o Opus 5.5 a $4 e $20 por milhão de tokens e observa que o Haiku 5.5, ao contrário dos outros modelos de contexto de 1M, é precificado pelo tamanho do prompt s20. Os 90% de destaque são a parcela das requisições do Haiku anterior com prompts abaixo de 100k tokens, não um corte de preço s1.

A Anthropic posiciona o Haiku 5.5 como subagent ao lado do Opus 5.5 e do Sonnet 5.5, e diz que o Sonnet 5.5 e o Opus 5.5 continuam sendo a melhor escolha para coding agêntico complexo, com o Haiku 5.5 para trabalho estritamente delimitado s1. No Terminal-Bench 4.0, o post de lançamento dá 39,2% para o Haiku 5.5 contra 70,6% para o Sonnet 5.5 s1.

Nada vai para o Haiku por padrão. Desde a v2.1.198, o agente Explore embutido deixa de usar o Haiku como padrão e herda o modelo principal s24. A ordem de resolução documentada para o modelo de um subagent é: o parâmetro por chamada, o frontmatter da definição, CLAUDE_CODE_SUBAGENT_MODEL, e então o modelo da conversa principal s2. CLAUDE_CODE_SUBAGENT_MODEL sozinha não muda os subagents embutidos Explore e Plan s2. Um subagent de usuário ou de projeto chamado Explore substitui o embutido, e a documentação sugere definir um com o modelo Haiku s2. CLAUDE_CODE_SUBAGENT_MODEL_FORCE=1 aplica um único modelo a todos os subagents, a partir da v2.1.257 s2 s24.

A documentação de custos recomenda model: haiku para tarefas simples de subagent e afirma que as requisições de subagent continuam consumindo o mesmo uso s3. Os medidores semanais são um para todos os modelos mais um para o Fable onde incluído s22, e os outros modelos consomem dos mesmos limites s23.

No Claude Code, o thinking não pode ser desligado para o Haiku 5.5, o Opus 5.5, o Sonnet 5.5 ou o Fable: alwaysThinkingEnabled em false e MAX_THINKING_TOKENS=0 não têm efeito. O effort vai de low a max e o Haiku 5.5 usa medium como padrão s21. O alias haiku resolve para o Haiku 5.5 na API da Anthropic e para o Haiku 4.5 na Platform on AWS, no Bedrock, no Vertex e no Foundry s21.

A configuração de um praticante em um plano Pro (Opus 5.5 principal em medium, um implementador Sonnet 5.5, um scoper Haiku 5.5 somente leitura, um shipper Haiku 5.5 em low) relata 13 PRs em dez sessões, com cerca de 6% da janela de cinco horas cada, e cerca de 1% do contador semanal para uma sessão de um único PR s4. A ressalva dele: a economia vem de o contexto principal se manter pequeno, não do modelo mais barato, já que precificar os tokens dos subagents a preço de Opus acrescenta só 5% s4.

Os dois arquivos

Dois arquivos, use um ou os dois. O arquivo de agente do projeto .claude/agents/Explore.md se chama Explore, então substitui o explorador embutido. Id completo do modelo, effort low, apenas ferramentas de leitura e busca.

Explore.md

---
name: Explore
description: Read-only codebase explorer. Use it to find where something is defined or called and to summarize what it found, with file paths and function names.
model: claude-haiku-5-5
effort: low
tools: Read, Glob, Grep, Bash
---

You explore a codebase and report back. Search with Glob and Grep, read only the files you need, and never edit anything. Answer with file paths and function names, then stop.

Explore.md on GitHub

O arquivo de configurações .claude/settings.json contém o bloco env: o modelo de subagent, mais a flag force que o faz valer para todos os subagents.

settings.json

{
  "env": {
    "CLAUDE_CODE_SUBAGENT_MODEL": "claude-haiku-5-5",
    "CLAUDE_CODE_SUBAGENT_MODEL_FORCE": "1"
  }
}

settings.json on GitHub

Qual usar, para quem:

• Use o bloco env quando todos os subagents devem ser baratos. FORCE arrasta todos os agentes para o Haiku, inclusive os que você fixou de propósito no Opus.
• Use o arquivo Explore quando só o explorador deve mudar. Os outros subagents mantêm seus próprios modelos.
• Definir CLAUDE_CODE_SUBAGENT_MODEL sem FORCE não faz nada pelo Explore. Duas execuções headless nossas, sessão principal no Fable, "use o agente Explore para encontrar onde o cache de consultas é limpo": o resultado da ferramenta informou Opus 5.5 sem nenhuma configuração, e Opus 5.5 de novo com a variável definida como Haiku 5.5 e nada mais. Com o arquivo Explore, ou com as duas variáveis de ambiente, informou Haiku 5.5.

A verificação: /tasks mostra o modelo e o effort em que cada subagent rodou (v2.1.243 e posteriores s24), e o resultado da ferramenta Agent numa execução headless traz resolvedModel. Confie nessa linha, nunca no arquivo.

A armadilha do alias: escreva claude-haiku-5-5, não haiku. O alias é Haiku 5.5 apenas na API da Anthropic; no Bedrock, no Vertex e no Foundry ele ainda significa Haiku 4.5 s21.

Um relato aberto mais antigo diz que o modelo do frontmatter de um subagent era ignorado silenciosamente s25, e um posterior conta 1.149 de 1.371 subagents rodando no modelo Opus do processo pai nas versões 2.1.277 a 2.1.290 s10. No Claude Code 2.1.293, todas as nossas 182 execuções usaram o modelo configurado (model_ok True na tabela de resultados). Confira mesmo assim.

A tabela de roteamento

Sete tarefas do dia a dia no tinydb em 18d73a1, 226 testes, claude -p headless, uma verificação determinística de aceitar ou rejeitar por tarefa. Cada célula traz execuções aceitas, preço médio de tabela por execução, segundos médios de tempo total. As células do Haiku são 10 execuções por tarefa, as do Sonnet e do Opus são 3 execuções por tarefa.

A tarefa "refactor" é a pequena funcionalidade: adicionar as operações de atualização push e pull a tinydb.operations, avaliada por um arquivo de teste oculto. Custou $0.0052 no Haiku low e $0.1675 no Opus, em 17.1 s contra 30.3 s.

O que fica no Sonnet ou no Opus: coding agêntico complexo, nas palavras da própria Anthropic s1. Estas sete são tarefas delimitadas de menos de um minuto cada, em um repositório pequeno.

O harness

O benchmark que produziu a tabela. Rode-o em um terminal simples, sem mais nada usando a conta, porque o medidor do plano é compartilhado. O checkout do tinydb, os testes ocultos e os patches de referência não estão incluídos: aponte-o para o seu próprio repositório e escreva as suas próprias verificações.

Runner e medidor:

• README.md: o que é o harness e o que ficou de fora.

  README.md

# routing bench

The benchmark harness behind the routing table: run.sh runs one cell headless and reads the real meters from the rate_limit_event, matrix.sh runs cells sequentially, check.py decides accept or reject per task, analyze.py writes the results table. tasks/ holds the seven prompts, inject/ the planted bugs. The tinydb checkout (commit 18d73a1), the hidden tests and the reference patches are not included: bring your own repo and checks.

README.md on GitHub

run.sh

#!/usr/bin/env bash
# One run: fresh tinydb work copy + task injection, headless claude -p, extract tokens/meter, check.
# Usage: run.sh <config> <task> <rep>
#   config: a key of configs.json (haiku-med | haiku-low | sonnet | opus)
#   task:   a key of tasks.json   (explore | tests | lint | commit | refactor | review | summary)
set -uo pipefail
CFG="$1"; TASK="$2"; REP="${3:-1}"
HERE="$(cd "$(dirname "$0")" && pwd)"
. "$HERE/env.sh"
PY="$HERE/.venv/bin/python"
MAX_TURNS="${BENCH_MAX_TURNS:-40}"
NAME="${TASK}-${CFG}-r${REP}"
OUT="$HERE/runs/$NAME"; W="$HERE/work/$NAME"
rm -rf "$OUT"; mkdir -p "$OUT"
"$HERE/prep.sh" "$TASK" "$W" > "$OUT/prep.txt" 2>&1 || { echo "$NAME: prep failed"; cat "$OUT/prep.txt"; exit 2; }
( cd "$W" && git diff HEAD > "$OUT/pre.diff" )

# Config args (model pinned by full id, optional --effort)
CARGS=()
while IFS= read -r a; do CARGS+=("$a"); done < <("$PY" -c "
import json; [print(a) for a in json.load(open('$HERE/configs.json'))['$CFG']['args']]")
[ ${#CARGS[@]} -gt 0 ] || { echo "bad config $CFG" >&2; exit 2; }

# Fixed allow-list for every cell, including Bash(ruff:*) for the lint task.
TOOLS="Read,Edit,Write,MultiEdit,Glob,Grep,Skill,TodoWrite,Bash(uv:*),Bash(python:*),Bash(python3:*),Bash(pytest:*),Bash(ruff:*),Bash(git status:*),Bash(git diff:*),Bash(git log:*),Bash(ls:*),Bash(cat:*),Bash(mkdir:*),Bash(mktemp:*),Bash(head:*),Bash(tail:*),Bash(grep:*),Bash(wc:*)"

cd "$W"
START=$(date +%s)
claude -p "$(cat "$HERE/tasks/$TASK.md")" "${CARGS[@]}" \
  --output-format stream-json --verbose --max-turns "$MAX_TURNS" \
  --permission-mode acceptEdits --allowedTools "$TOOLS" \
  --setting-sources project --strict-mcp-config \
  > "$OUT/stream.jsonl" 2> "$OUT/stderr.txt"
echo "$?" > "$OUT/exit_code.txt"
END=$(date +%s)
echo "$((END-START))" > "$OUT/wall_s.txt"

# After-meter: one cheap Haiku probe (its own stream is kept for audit).
"$HERE/probe.sh" "$OUT/probe.jsonl" > "$OUT/after.json" 2>/dev/null

# Diff of the agent's work (tracked + untracked) vs the post-injection state
# (index left untouched: the commit task's staged changes stay staged for the check)
git diff HEAD -- . ':(exclude).coverage' > "$OUT/post.diff"
git diff HEAD --numstat -- . ':(exclude).coverage' > "$OUT/numstat.txt"
git ls-files --others --exclude-standard | while IFS= read -r f; do
  git diff --no-index -- /dev/null "$f" >> "$OUT/post.diff"
  echo "$(wc -l < "$f" | tr -d ' ')	0	$f" >> "$OUT/numstat.txt"
done

"$PY" "$HERE/extract.py" "$OUT" "$CFG" "$TASK" "$REP" > "$OUT/result.json"
"$PY" "$HERE/check.py" "$TASK" "$W" "$OUT/result.json" > "$OUT/check.txt"; echo "$?" >> "$OUT/check.txt"
"$PY" "$HERE/extract.py" "$OUT" "$CFG" "$TASK" "$REP" --score

run.sh on GitHub

matrix.sh

#!/usr/bin/env bash
# Run an ordered list of "<config> <task> <rep>" lines SEQUENTIALLY (the plan meter is shared:
# never run two cells at once). Blank lines and #comments are skipped. Appends to matrix.log.
# Usage: matrix.sh [matrix-file]   (default: matrix.txt)
#   SKIP_DONE=1 matrix.sh ...      skip cells whose runs/<task>-<config>-r<rep>/score.json exists (resume)
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"; cd "$HERE"
LIST="${1:-matrix.txt}"
# one matrix at a time: the meter is shared
mkdir "$HERE/.matrix.lock" 2>/dev/null || { echo "another matrix.sh is running (remove .matrix.lock if stale)" >&2; exit 1; }
trap 'rmdir "$HERE/.matrix.lock"' EXIT
echo "=== $(date '+%F %T') start $LIST" >> matrix.log
grep -vE '^\s*(#|$)' "$LIST" | while read -r CFG TASK REP; do
  if [ "${SKIP_DONE:-0}" = 1 ] && [ -f "runs/${TASK}-${CFG}-r${REP}/score.json" ]; then
    echo "$(date '+%F %T') skip ${TASK}-${CFG}-r${REP}" >> matrix.log; continue
  fi
  line=$(./run.sh "$CFG" "$TASK" "$REP" < /dev/null 2>&1 | tail -1)
  echo "$(date '+%F %T') $line" | tee -a matrix.log
done
echo "=== $(date '+%F %T') done $LIST" >> matrix.log

matrix.sh on GitHub

gen_matrix.py

#!/usr/bin/env python3
"""Write the ordered matrix file.

gen_matrix.py [--reps 3] [--passes 2] [--configs haiku-low,haiku-med,sonnet,opus] [--out matrix.txt]

Order: configs run as contiguous BLOCKS (all tasks of one rep set), cheap configs first; tasks are
rotated inside each block so no task always runs first. With --passes 2 the second pass runs the
configs in reverse order (ABBA), which interleaves configs over time without breaking the
contiguous segments that analyze.py needs to beat the 1% meter resolution.
"""
import argparse, json, os

HERE = os.path.dirname(os.path.abspath(__file__))
ap = argparse.ArgumentParser()
ap.add_argument("--reps", type=int, default=3, help="reps per config x task per pass")
ap.add_argument("--passes", type=int, default=2)
ap.add_argument("--configs", default="haiku-low,haiku-med,sonnet,opus")
ap.add_argument("--start-rep", type=int, default=1, help="first rep number (use 2 to keep the r1 smoke runs)")
ap.add_argument("--out", default=os.path.join(HERE, "matrix.txt"))
a = ap.parse_args()
tasks = list(json.load(open(os.path.join(HERE, "tasks.json"))))
cfgs = a.configs.split(",")
lines, rep = [], a.start_rep
for p in range(a.passes):
    order = cfgs if p % 2 == 0 else list(reversed(cfgs))
    lines.append(f"# pass {p + 1}: {' -> '.join(order)}")
    for c in order:
        lines.append(f"# block {c}")
        for r in range(a.reps):
            k = rep + r
            rot = (k - 1) % len(tasks)
            for t in tasks[rot:] + tasks[:rot]:
                lines.append(f"{c} {t} {k}")
    rep += a.reps
open(a.out, "w").write("\n".join(lines) + "\n")
n = sum(1 for l in lines if not l.startswith("#"))
print(f"{a.out}: {n} runs ({len(cfgs)} configs x {len(tasks)} tasks x {a.reps} reps x {a.passes} passes)")

gen_matrix.py on GitHub

prep.sh

#!/usr/bin/env bash
# Build a fresh work copy for one task: upstream tinydb, baseline commit, task injection.
# Usage: prep.sh <task> <workdir>
set -euo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
TASK="$1"; W="$2"
read -r INJ MODE < <("$HERE/.venv/bin/python" -c "
import json,sys; t=json.load(open('$HERE/tasks.json'))['$TASK']; print(t['inject'] or '-', t['mode'] or '-')")
rm -rf "$W"; mkdir -p "$(dirname "$W")"
cp -R "$HERE/upstream" "$W"
cd "$W"
/usr/bin/find . -name __pycache__ -type d -prune -exec rm -rf {} +
rm -f .coverage
git update-index -q --refresh || true
G() { git -c user.email=dev@example.com -c user.name=dev "$@"; }
# Injections that belong to the baseline are folded into the top upstream commit
# (amend, same message/author/date), so `git log` shows no extra "bug" commit.
if [ "$MODE" = "commit" ]; then
  git apply "$HERE/$INJ"
  git add -A
  G commit -q --amend --no-edit --no-verify
fi
case "$MODE" in
  stage)    git apply --index "$HERE/$INJ" ;;
  worktree) git apply "$HERE/$INJ" ;;
esac

prep.sh on GitHub

env.sh

# Sourced by run.sh and probe.sh.
# Auth: an isolated CLAUDE_CONFIG_DIR is not logged in
# (`claude auth status` -> loggedIn false), so runs use the user's default config dir:
# CLAUDE_CONFIG_DIR is deliberately left unset.
unset CLAUDE_CONFIG_DIR
# Fresh work copies at the same path must not inherit memory written by an earlier rep.
export CLAUDE_CODE_DISABLE_AUTO_MEMORY=1
# The bench venv (pytest, ruff, mypy) comes first, so `python -m pytest` and `ruff` just work.
export PATH="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/.venv/bin:$PATH"

env.sh on GitHub

probe.sh

#!/usr/bin/env bash
# Cheapest possible call: one Haiku turn answering "ok". Prints the current plan
# meter read from its rate_limit_event as one JSON line. The probe itself costs a
# few hundred tokens of Haiku (well under the 1% meter resolution).
# Usage: probe.sh [raw-stream-out-file]
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
[ -f "$HERE/env.sh" ] && . "$HERE/env.sh"
RAW="${1:-$(mktemp -t hrb-probe)}"; case "$RAW" in /*) ;; *) RAW="$PWD/$RAW";; esac
cd "$(mktemp -d -t hrb-probe-cwd)"
claude --model claude-haiku-5-5 -p "ok" --max-turns 1 --output-format stream-json --verbose \
  --setting-sources project --strict-mcp-config > "$RAW" 2>/dev/null
"$HERE/.venv/bin/python" "$HERE/meter.py" "$RAW"

probe.sh on GitHub

meter.py

#!/usr/bin/env python3
"""Read a stream-json file and print the plan meter carried by its rate_limit_event lines.

Output (one JSON line): {"ts", "five_hour", "five_hour_resets_at", "seven_day",
"seven_day_resets_at", "n_events", "raw_first", "raw_last"}: utilizations are the
values of the FIRST event (the meter as the run began). `--last` uses the last event.
"""
import json, sys, time

def events(path):
    out = []
    for line in open(path, errors="replace"):
        line = line.strip()
        if not line.startswith("{"):
            continue
        try:
            o = json.loads(line)
        except Exception:
            continue
        if o.get("type") == "rate_limit_event":
            out.append(o)
    return out

def windows(ev):
    info = ev.get("rate_limit_info") or {}
    uw = info.get("unifiedWindows") or {}
    def w(name):
        x = uw.get(name) or {}
        return x.get("utilization"), x.get("resetsAt")
    fh, fhr = w("five_hour")
    sd, sdr = w("seven_day")
    return {"five_hour": fh, "five_hour_resets_at": fhr, "seven_day": sd, "seven_day_resets_at": sdr}

def meter(path, which="first"):
    evs = events(path)
    if not evs:
        return {"ts": int(time.time()), "n_events": 0, "five_hour": None, "seven_day": None}
    ev = evs[0] if which == "first" else evs[-1]
    d = {"ts": int(time.time()), "n_events": len(evs)}
    d.update(windows(ev))
    d["raw_first"] = evs[0].get("rate_limit_info")
    d["raw_last"] = evs[-1].get("rate_limit_info")
    return d

if __name__ == "__main__":
    which = "last" if "--last" in sys.argv else "first"
    path = [a for a in sys.argv[1:] if not a.startswith("--")][0]
    print(json.dumps(meter(path, which)))

meter.py on GitHub

extract.py

#!/usr/bin/env python3
"""Extract one run's numbers from runs/<name>/stream.jsonl.

extract.py <outdir> <config> <task> <rep>          -> prints the stream's `result` line (result.json)
extract.py <outdir> <config> <task> <rep> --score  -> writes score.json, prints a one-line summary
"""
import json, os, sys

HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
from meter import meter  # noqa: E402

out, cfg, task, rep = sys.argv[1:5]
score_mode = "--score" in sys.argv
stream = os.path.join(out, "stream.jsonl")

lines = []
for raw in open(stream, errors="replace") if os.path.exists(stream) else []:
    raw = raw.strip()
    if raw.startswith("{"):
        try:
            lines.append(json.loads(raw))
        except Exception:
            pass
result = next((o for o in reversed(lines) if o.get("type") == "result"), None)
init = next((o for o in lines if o.get("type") == "system" and o.get("subtype") == "init"), {})

if not score_mode:
    print(json.dumps(result or {"type": "result", "result": "", "missing": True}))
    sys.exit(0)

def rd(name, default=None):
    p = os.path.join(out, name)
    try:
        return open(p).read().strip()
    except Exception:
        return default

def jl(name):
    try:
        return json.loads(rd(name) or "null")
    except Exception:
        return None

cfgs = json.load(open(os.path.join(HERE, "configs.json")))
before = meter(stream, "first") if os.path.exists(stream) else None
after = jl("after.json")
r = result or {}
usage = r.get("usage") or {}
tok = {k: usage.get(k) or 0 for k in ("input_tokens", "cache_read_input_tokens", "cache_creation_input_tokens", "output_tokens")}
mu = r.get("modelUsage") or {}
assistant_models = sorted({(o.get("message") or {}).get("model") for o in lines
                           if o.get("type") == "assistant" and (o.get("message") or {}).get("model") not in (None, "<synthetic>")})
check_txt = (rd("check.txt", "") or "").split("\n")
verdict_line = check_txt[0] if check_txt else ""
numstat = [l.split("\t") for l in (rd("numstat.txt", "") or "").split("\n") if l.strip()]
added = sum(int(a) for a, d, f in numstat if a.isdigit())
deleted = sum(int(d) for a, d, f in numstat if d.isdigit())
changed_by_agent = (rd("pre.diff", "") or "") != (rd("post.diff", "") or "")

def strip_raw(m):
    if not m:
        return m
    return {k: v for k, v in m.items() if not k.startswith("raw_")}

s = {
    "run": os.path.basename(out), "config": cfg, "task": task, "rep": int(rep),
    "expected_model": cfgs[cfg]["model"], "config_args": cfgs[cfg]["args"],
    "init_model": init.get("model"), "assistant_models": assistant_models,
    "model_usage_keys": sorted(mu.keys()),
    "model_ok": init.get("model") == cfgs[cfg]["model"] and set(mu.keys()) <= {cfgs[cfg]["model"]} and bool(mu),
    "claude_code_version": init.get("claude_code_version"),
    "meter_before": strip_raw(before), "meter_after": strip_raw(after),
    "n_rate_limit_events": (before or {}).get("n_events", 0),
    "has_result_line": result is not None,
    "subtype": r.get("subtype"), "is_error": r.get("is_error"), "stop_reason": r.get("stop_reason"),
    "num_turns": r.get("num_turns"), "duration_ms": r.get("duration_ms"), "duration_api_ms": r.get("duration_api_ms"),
    "total_cost_usd": r.get("total_cost_usd"), "usage": usage, "tokens": tok, "modelUsage": mu,
    "wall_s": int(rd("wall_s.txt", "0") or 0), "exit_code": rd("exit_code.txt"),
    "final_text": (r.get("result") or "")[-2000:],
    "diff": {"files": len(numstat), "added": added, "deleted": deleted, "changed_by_agent": changed_by_agent},
    "check": {"verdict": verdict_line.split(" ")[0] if verdict_line else None, "reason": verdict_line[5:] if verdict_line else None},
}
s["accepted"] = s["check"]["verdict"] == "PASS"
json.dump(s, open(os.path.join(out, "score.json"), "w"), indent=2)
mb, ma = s["meter_before"] or {}, s["meter_after"] or {}
print(f"{s['run']}: {s['check']['verdict']} turns={s['num_turns']} cost=${(s['total_cost_usd'] or 0):.3f} "
      f"in={tok['input_tokens']} cr={tok['cache_read_input_tokens']} cc={tok['cache_creation_input_tokens']} out={tok['output_tokens']} "
      f"wall={s['wall_s']}s 5h {mb.get('five_hour')}->{ma.get('five_hour')} 7d {mb.get('seven_day')}->{ma.get('seven_day')} "
      f"model_ok={s['model_ok']} | {s['check']['reason']}")

extract.py on GitHub

check.py

#!/usr/bin/env python3
"""Deterministic accept/reject check for one run.

Usage: check.py <task> <workdir> <result.json>
  result.json = the `result` line of the run's stream (only its "result" text is used).
Prints `PASS <reason>` or `FAIL <reason>`; exit 0 on PASS, 1 on FAIL.
"""
import json, os, re, subprocess, sys

HERE = os.path.dirname(os.path.abspath(__file__))
PY = os.path.join(HERE, ".venv", "bin", "python")
RUFF = os.path.join(HERE, ".venv", "bin", "ruff")


def final_text(path):
    try:
        r = json.load(open(path))
    except Exception:
        return ""
    return r.get("result") or "" if isinstance(r, dict) else ""


def sh(args, cwd, env=None):
    p = subprocess.run(args, cwd=cwd, capture_output=True, text=True, env=env)
    return p.returncode, p.stdout + p.stderr


def suite(wd):
    rc, out = sh([PY, "-m", "pytest", "-q", "-p", "no:cacheprovider", "-o", "addopts=", "tests"], wd)
    m = re.search(r"(\d+) passed", out)
    f = re.search(r"(\d+) failed", out)
    e = re.search(r"(\d+) error", out)
    return rc == 0, f"suite {m.group(1) if m else 0} passed, {f.group(1) if f else 0} failed, {e.group(1) if e else 0} errors"


def git_diff(wd, *paths):
    # tracked changes vs the baseline commit + untracked files under paths
    _, d = sh(["git", "diff", "HEAD", "--", *paths], wd)
    _, u = sh(["git", "ls-files", "--others", "--exclude-standard", "--", *paths], wd)
    return d.strip(), u.strip()


def words(s):
    return re.findall(r"[A-Za-z0-9_']+", s)


def strip_fences(s):
    lines = s.strip().split("\n")
    if lines and lines[0].strip().startswith("```"):
        lines = lines[1:]
    if lines and lines[-1].strip().startswith("```"):
        lines = lines[:-1]
    return "\n".join(lines).strip()


def c_explore(wd, txt):
    t = txt.lower()
    miss = []
    if "table.py" not in t:
        miss.append("table.py")
    if "clear_cache" not in t and "_query_cache" not in t:
        miss.append("clear_cache/_query_cache")
    names = ["insert_multiple", "update_multiple", "insert", "update", "upsert", "remove", "truncate"]
    found = {n for n in names if re.search(r"(?<![a-z_])" + n + r"(?![a-z_])", t)}
    if len(found) < 3:
        miss.append(f"only {len(found)} mutating methods {sorted(found)}")
    return (not miss), ("missing " + ", ".join(miss)) if miss else f"table.py + cache method + {len(found)} methods {sorted(found)}"


def c_tests(wd, txt):
    ok, s = suite(wd)
    d, u = git_diff(wd, "tests")
    if d or u:
        return False, f"tests/ modified; {s}"
    return ok, s + "; tests/ untouched"


def c_lint(wd, txt):
    rc, out = sh([RUFF, "check", "tinydb", "--output-format", "concise", "--no-cache"], wd)
    cfg_d, _ = git_diff(wd, "ruff.toml", "pyproject.toml")
    d, _ = git_diff(wd, "tinydb")
    noqa = [l for l in d.split("\n") if l.startswith("+") and "noqa" in l.lower()]
    ok, s = suite(wd)
    why = []
    if rc != 0:
        found = [l for l in out.split("\n") if l.startswith("Found ")]
        why.append("ruff: " + (found[0] if found else out.strip().split("\n")[-1]))
    if cfg_d:
        why.append("lint config changed")
    if noqa:
        why.append(f"{len(noqa)} noqa added")
    if not ok:
        why.append(s)
    return (not why), ("; ".join(why) if why else f"ruff clean; {s}")


DIFF_LINE = re.compile(r"^(diff --git |index [0-9a-f]{6,}\.\.|@@ |\+\+\+ |--- a/|--- /dev/null|[+-] {3,}\S|[+-](def|class|import|from|return) )")


def c_commit(wd, txt):
    msg = strip_fences(txt)
    if not msg:
        return False, "empty answer"
    first = msg.split("\n")[0].strip()
    why = []
    if len(first) > 72:
        why.append(f"first line {len(first)} chars")
    if "push" not in first.lower():
        why.append("no 'push' in first line")
    if "pull" not in first.lower():
        why.append("no 'pull' in first line")
    dl = [l for l in msg.split("\n") if DIFF_LINE.match(l)]
    if dl:
        why.append(f"{len(dl)} diff lines")
    return (not why), ("; ".join(why) if why else f"first line {len(first)} chars: {first!r}")


def c_refactor(wd, txt):
    env = dict(os.environ, TINYDB_TREE=wd)
    rc, out = sh([PY, "-m", "pytest", "-q", "-c", os.path.join(HERE, "hidden", "pytest.ini"),
                  "--rootdir", os.path.join(HERE, "hidden"), os.path.join(HERE, "hidden", "test_T1.py")], wd, env)
    m = re.search(r"(\d+) passed", out)
    hp = int(m.group(1)) if m else 0
    ok, s = suite(wd)
    return (hp == 5 and ok), f"hidden {hp}/5; {s}"


REVIEW_PATTERNS = [
    r"limit\s*=?=\s*0\b",
    r"limit\s+(of\s+)?(0|zero)\b",
    r"\b(0|zero)\s+limit",
    r"zero[- ]limit",
    r"limit.{0,150}\b(falsy|falsey|truthy|truthiness)",
    r"\b(falsy|falsey|truthy|truthiness)\b.{0,150}limit",
    r"if limit:.{0,150}\b(0|zero)\b",
    r"\b(0|zero)\b.{0,150}if limit:",
]


def c_review(wd, txt):
    t = txt.lower().replace("`", "")
    for p in REVIEW_PATTERNS:
        m = re.search(p, t, re.S)
        if m:
            return True, f"matched {p!r}: {m.group(0)[:80]!r}"
    return False, "planted limit=0 defect not mentioned"


def c_summary(wd, txt):
    n = len(words(txt))
    t = txt.lower()
    mods = [m for m in ["database", "table", "queries", "storages", "middlewares", "operations"] if m in t]
    why = []
    if n > 170:
        why.append(f"{n} words > 170")
    if len(mods) < 4:
        why.append(f"only {len(mods)} modules {mods}")
    return (not why), ("; ".join(why) if why else f"{n} words, modules {mods}")


CHECKS = {"explore": c_explore, "tests": c_tests, "lint": c_lint, "commit": c_commit,
          "refactor": c_refactor, "review": c_review, "summary": c_summary}

if __name__ == "__main__":
    task, wd, res = sys.argv[1], os.path.abspath(sys.argv[2]), sys.argv[3]
    ok, why = CHECKS[task](wd, final_text(res))
    print(("PASS " if ok else "FAIL ") + why)
    sys.exit(0 if ok else 1)

check.py on GitHub

analyze.py

#!/usr/bin/env python3
"""Aggregate runs/*/score.json into results.json and results.md.

Per config x task: accept rate, mean tokens, mean cost, mean turns, mean wall.
Per config: measured plan-meter deltas (five_hour, seven_day, in percentage points), computed two ways:
  - per_run_sum: sum over the config's runs of (after - before). Quantized at 1% per run, so
    runs far below 1% mostly read 0: a lower-bound-ish, very noisy number.
  - segments:    runs sorted by start time are cut into maximal contiguous segments of the same
    config; each segment contributes (after of its last run - before of its first run). This
    telescopes the quantization: one +-1 pt error per segment instead of per run. PREFERRED.
  Segments/runs whose window resetsAt changed between before and after are excluded for that window.
Plus the /usage-style local heuristic: weighted = cached*1 + uncached*10 + cache_create*12.5 + output*50,
times the model tier (haiku 1 / sonnet 3 / opus 5). The probe's own Haiku tokens after each run are
counted as overhead in the heuristic (they also move the meter).
Usage: analyze.py [--runs-dir runs] [--out-prefix results]
"""
import glob, json, os, statistics, sys

HERE = os.path.dirname(os.path.abspath(__file__))
RUNS = os.path.join(HERE, sys.argv[sys.argv.index("--runs-dir") + 1]) if "--runs-dir" in sys.argv else os.path.join(HERE, "runs")
OUTP = sys.argv[sys.argv.index("--out-prefix") + 1] if "--out-prefix" in sys.argv else "results"
CFGS = json.load(open(os.path.join(HERE, "configs.json")))
TASKS = list(json.load(open(os.path.join(HERE, "tasks.json"))).keys())
W = {"cache_read_input_tokens": 1, "input_tokens": 10, "cache_creation_input_tokens": 12.5, "output_tokens": 50}
TIER = {c: v["tier"] for c, v in CFGS.items()}


def weighted(tok, tier):
    raw = sum((tok.get(k) or 0) * w for k, w in W.items())
    return raw, raw * tier


MU = {"cacheReadInputTokens": "cache_read_input_tokens", "inputTokens": "input_tokens",
      "cacheCreationInputTokens": "cache_creation_input_tokens", "outputTokens": "output_tokens"}


def model_tier(name, fallback):
    n = (name or "").lower()
    return 5 if "opus" in n else 3 if "sonnet" in n else 1 if "haiku" in n else fallback


def heuristic_run(s):
    """Tiered heuristic per model actually used (modelUsage), so helper calls on another
    model are weighted with their own tier; falls back to `usage` x config tier."""
    mu = s.get("modelUsage") or {}
    if not mu:
        return weighted(s["tokens"], TIER[s["config"]])[1]
    return sum(weighted({MU[k]: v.get(k) or 0 for k in MU}, model_tier(m, TIER[s["config"]]))[1] for m, v in mu.items())


def probe_tokens(run_dir):
    p = os.path.join(run_dir, "probe.jsonl")
    if not os.path.exists(p):
        return {}
    for line in reversed(open(p, errors="replace").read().strip().split("\n")):
        try:
            o = json.loads(line)
        except Exception:
            continue
        if o.get("type") == "result":
            u = o.get("usage") or {}
            return {k: u.get(k) or 0 for k in W}
    return {}


def mean(xs):
    xs = [x for x in xs if x is not None]
    return statistics.mean(xs) if xs else None


def window_delta(b, a, w):
    if not b or not a or b.get(w) is None or a.get(w) is None:
        return None, "missing"
    if b.get(w + "_resets_at") != a.get(w + "_resets_at"):
        return None, "reset"
    return round((a[w] - b[w]) * 100, 2), "ok"


scores = []
for p in sorted(glob.glob(os.path.join(RUNS, "*", "score.json"))):
    s = json.load(open(p))
    if s.get("config") not in CFGS:
        continue
    s["_dir"] = os.path.dirname(p)
    s["_start"] = (s.get("meter_before") or {}).get("ts") or os.path.getmtime(os.path.join(s["_dir"], "stream.jsonl"))
    s["_probe_tok"] = probe_tokens(s["_dir"])
    scores.append(s)
# meter_before.ts is written at extraction time; order by stream file creation instead
for s in scores:
    st = os.stat(os.path.join(s["_dir"], "stream.jsonl"))
    s["_start"] = getattr(st, "st_birthtime", st.st_mtime)
scores.sort(key=lambda s: s["_start"])

cells, per_cfg = {}, {}
for s in scores:
    cells.setdefault((s["config"], s["task"]), []).append(s)
    per_cfg.setdefault(s["config"], []).append(s)

cell_rows = []
for (c, t), ss in sorted(cells.items(), key=lambda kv: (list(CFGS).index(kv[0][0]), TASKS.index(kv[0][1]) if kv[0][1] in TASKS else 99)):
    row = {"config": c, "task": t, "n": len(ss),
           "accept_rate": sum(1 for s in ss if s.get("accepted")) / len(ss),
           "mean_turns": mean([s.get("num_turns") for s in ss]),
           "mean_cost_usd": mean([s.get("total_cost_usd") for s in ss]),
           "mean_wall_s": mean([s.get("wall_s") for s in ss]),
           "mean_tokens": {k: mean([s["tokens"].get(k) for s in ss]) for k in W},
           "mean_weighted_heuristic": mean([heuristic_run(s) for s in ss]),
           "model_ok_all": all(s.get("model_ok") for s in ss)}
    cell_rows.append(row)

# contiguous segments in global start order
segments, cur = [], None
for s in scores:
    if cur and cur["config"] == s["config"]:
        cur["runs"].append(s)
    else:
        cur = {"config": s["config"], "runs": [s]}
        segments.append(cur)

cfg_rows = []
for c, ss in per_cfg.items():
    tok_sum = {k: sum(s["tokens"].get(k) or 0 for s in ss) for k in W}
    probe_sum = {k: sum(s["_probe_tok"].get(k, 0) for s in ss) for k in W}
    raw_w, _ = weighted(tok_sum, TIER[c])
    heur = sum(heuristic_run(s) for s in ss)
    probe_raw, probe_heur = weighted(probe_sum, 1)
    row = {"config": c, "runs": len(ss), "accept_rate": sum(1 for s in ss if s.get("accepted")) / len(ss),
           "tokens_sum": tok_sum, "probe_tokens_sum": probe_sum,
           "cost_sum_usd": round(sum(s.get("total_cost_usd") or 0 for s in ss), 4),
           "weighted_raw_M": raw_w / 1e6, "heuristic_M": (heur + probe_heur) / 1e6}
    for w in ("five_hour", "seven_day"):
        per_run, excl = 0.0, 0
        for s in ss:
            d, why = window_delta(s.get("meter_before"), s.get("meter_after"), w)
            if d is None:
                excl += 1
            else:
                per_run += d
        seg_sum, seg_n, seg_excl = 0.0, 0, 0
        for g in segments:
            if g["config"] != c:
                continue
            d, why = window_delta(g["runs"][0].get("meter_before"), g["runs"][-1].get("meter_after"), w)
            if d is None:
                seg_excl += 1
            else:
                seg_sum += d; seg_n += 1
        row[w] = {"per_run_sum_pts": round(per_run, 2), "per_run_excluded": excl,
                  "segments_sum_pts": round(seg_sum, 2), "segments": seg_n, "segments_excluded": seg_excl,
                  "quantization_error_pts": f"+-{seg_n}",
                  "pts_per_M_heuristic": round(seg_sum / row["heuristic_M"], 4) if row["heuristic_M"] else None,
                  "pts_per_M_weighted_raw": round(seg_sum / row["weighted_raw_M"], 4) if row["weighted_raw_M"] else None}
    cfg_rows.append(row)
cfg_rows.sort(key=lambda r: list(CFGS).index(r["config"]))

out = {"generated_from": len(scores), "cells": cell_rows, "configs": cfg_rows,
       "segments": [{"config": g["config"], "runs": [s["run"] for s in g["runs"]],
                     "before": g["runs"][0].get("meter_before"), "after": g["runs"][-1].get("meter_after")} for g in segments],
       "heuristic_weights": {"tokens": W, "tier": TIER}}
json.dump(out, open(os.path.join(HERE, OUTP + ".json"), "w"), indent=2, default=str)

f = lambda x, d=0: "-" if x is None else (f"{x:,.{d}f}")
md = ["# Results", "", f"{len(scores)} runs. Meter deltas in percentage points of the plan window (1-pt resolution).", "",
      "## Per config x task", "",
      "| config | task | n | accept | turns | input | cache read | cache create | output | cost $ | wall s | heuristic (M) | model ok |",
      "|---|---|---|---|---|---|---|---|---|---|---|---|---|"]
for r in cell_rows:
    t = r["mean_tokens"]
    md.append(f"| {r['config']} | {r['task']} | {r['n']} | {r['accept_rate']:.0%} | {f(r['mean_turns'],1)} | {f(t['input_tokens'])} | "
              f"{f(t['cache_read_input_tokens'])} | {f(t['cache_creation_input_tokens'])} | {f(t['output_tokens'])} | "
              f"{f(r['mean_cost_usd'],4)} | {f(r['mean_wall_s'],1)} | {f(r['mean_weighted_heuristic']/1e6,2)} | {r['model_ok_all']} |")
md += ["", "## Per config: measured meter vs local heuristic", "",
       "| config | runs | accept | cost $ | heuristic (M units) | 5h pts (segments) | 5h pts (per-run sum) | 7d pts (segments) | segments | 5h pts / M heuristic | 7d pts / M heuristic |",
       "|---|---|---|---|---|---|---|---|---|---|---|"]
for r in cfg_rows:
    fh, sd = r["five_hour"], r["seven_day"]
    md.append(f"| {r['config']} | {r['runs']} | {r['accept_rate']:.0%} | {r['cost_sum_usd']} | {r['heuristic_M']:.2f} | "
              f"{fh['segments_sum_pts']} (excl {fh['segments_excluded']}) | {fh['per_run_sum_pts']} | {sd['segments_sum_pts']} (excl {sd['segments_excluded']}) | "
              f"{fh['segments']} | {fh['pts_per_M_heuristic']} | {sd['pts_per_M_heuristic']} |")
md += ["", "If the /usage heuristic matched the real meter, `pts / M heuristic` would be the same for every config.",
       "Each segment carries up to +-1 pt of quantization error; trust a config's number only when its segment sum is several points.", ""]
open(os.path.join(HERE, OUTP + ".md"), "w").write("\n".join(md))
print("\n".join(md))

analyze.py on GitHub

Configuração:

• configs.json: os quatro workers, com ids completos de modelo e --effort low para o bloco low.

  configs.json

{
  "haiku-med": {"args": ["--model", "claude-haiku-5-5"], "model": "claude-haiku-5-5", "tier": 1, "note": "default effort"},
  "haiku-low": {"args": ["--model", "claude-haiku-5-5", "--effort", "low"], "model": "claude-haiku-5-5", "tier": 1},
  "sonnet":    {"args": ["--model", "claude-sonnet-5-5"], "model": "claude-sonnet-5-5", "tier": 3, "note": "default effort"},
  "opus":      {"args": ["--model", "claude-opus-5-5"], "model": "claude-opus-5-5", "tier": 5, "note": "default effort"}
}

configs.json on GitHub

tasks.json

{
  "explore":  {"inject": null,                     "mode": null,       "kind": "answer"},
  "tests":    {"inject": "inject/tests.patch",     "mode": "commit",   "kind": "code"},
  "lint":     {"inject": "inject/lint.patch",      "mode": "commit",   "kind": "code"},
  "commit":   {"inject": "reference/T1.patch",     "mode": "stage",    "kind": "answer"},
  "refactor": {"inject": null,                     "mode": null,       "kind": "code"},
  "review":   {"inject": "inject/review.patch",    "mode": "worktree", "kind": "answer"},
  "summary":  {"inject": null,                     "mode": null,       "kind": "answer"}
}

tasks.json on GitHub

ruff.toml

# Pinned lint config for the `lint` task: a small explicit rule set.
# Upstream tinydb @ 18d73a1 is clean under it (the two per-file ignores cover
# pre-existing F401s: the __version__ re-export and middlewares' Optional).
[lint]
select = ["F401", "F541", "F841", "E722"]

[lint.per-file-ignores]
"tinydb/__init__.py" = ["F401"]
"tinydb/middlewares.py" = ["F401"]

ruff.toml on GitHub

Alterações plantadas:

• inject/tests.patch: inverte uma comparação em Query.__ge__ para que um teste falhe.

  inject/tests.patch

diff --git a/tinydb/queries.py b/tinydb/queries.py
index cdab9fb..db00a2a 100644
--- a/tinydb/queries.py
+++ b/tinydb/queries.py
@@ -322,7 +322,7 @@ class Query(QueryInstance):
         :param rhs: The value to compare against
         """
         return self._generate_test(
-            lambda value: value >= rhs,
+            lambda value: value > rhs,
             ('>=', self._path, rhs)
         )
 

inject/tests.patch on GitHub

inject/lint.patch

diff --git a/ruff.toml b/ruff.toml
new file mode 100644
--- /dev/null
+++ b/ruff.toml
@@ -0,0 +1,6 @@
+[lint]
+select = ["F401", "F541", "F841", "E722"]
+
+[lint.per-file-ignores]
+"tinydb/__init__.py" = ["F401"]
+"tinydb/middlewares.py" = ["F401"]
diff --git a/tinydb/operations.py b/tinydb/operations.py
index 7101a23..f8541fe 100644
--- a/tinydb/operations.py
+++ b/tinydb/operations.py
@@ -10,6 +10,7 @@ This would delete the ``foo`` field from all documents where ``foo`` equals 2.
 
 
 from collections.abc import Callable, MutableMapping
+import sys
 from typing import Any, Union
 
 
@@ -18,7 +19,7 @@ def delete(field: str) -> Callable[[MutableMapping], None]:
     Delete a given field from the document.
     """
     def transform(doc: MutableMapping):
-        del doc[field]
+        removed = doc.pop(field)
 
     return transform
 
diff --git a/tinydb/queries.py b/tinydb/queries.py
index cdab9fb..489a2be 100644
--- a/tinydb/queries.py
+++ b/tinydb/queries.py
@@ -227,7 +227,7 @@ class Query(QueryInstance):
         :return: A :class:`~tinydb.queries.QueryInstance` object
         """
         if not self._path and not allow_empty_path:
-            raise ValueError('Query has no path')
+            raise ValueError(f'Query has no path')
 
         def runner(value):
             try:
diff --git a/tinydb/table.py b/tinydb/table.py
index f4027bf..eb8058c 100644
--- a/tinydb/table.py
+++ b/tinydb/table.py
@@ -818,7 +818,7 @@ class Table:
 
         try:
             raw_table = tables[self.name]
-        except KeyError:
+        except:
             # The table does not exist yet, so it is empty
             raw_table = {}
 
diff --git a/tinydb/utils.py b/tinydb/utils.py
index 43cad4e..07114d5 100644
--- a/tinydb/utils.py
+++ b/tinydb/utils.py
@@ -2,6 +2,7 @@
 Utility functions.
 """
 
+import os
 from collections import OrderedDict, abc
 from collections.abc import Iterator
 from typing import TypeVar, Generic, Union, Optional, TYPE_CHECKING, cast

inject/lint.patch on GitHub

inject/review.patch

diff --git a/tinydb/table.py b/tinydb/table.py
index f4027bf..4e06ba1 100644
--- a/tinydb/table.py
+++ b/tinydb/table.py
@@ -244,10 +244,21 @@ class Table:
 
         return list(iter(self))
 
-    def search(self, cond: QueryLike) -> list[Document]:
+    def search(
+        self,
+        cond: QueryLike,
+        *,
+        sort_by: Optional[str] = None,
+        reverse: bool = False,
+        limit: Optional[int] = None,
+    ) -> list[Document]:
         """
         Search for all documents matching a 'where' cond.
 
+        Results can optionally be sorted by a field (``sort_by``, descending
+        with ``reverse=True``; documents without the field always come last)
+        and truncated to the first ``limit`` results after sorting.
+
         Note: a repeated search with the same ``cond`` is served from an
         internal cache. The returned list is a fresh copy, but its
         documents are the same objects held by the cache -- treat them as
@@ -255,8 +266,34 @@ class Table:
         what a later identical search returns.
 
         :param cond: the condition to check against
+        :param sort_by: optional field name to sort the results by
+        :param reverse: sort in descending order
+        :param limit: optional maximum number of results
         :returns: list of matching documents
         """
+        if limit is not None and limit < 0:
+            raise ValueError('limit must not be negative')
+
+        # The cache always holds the plain, unsorted, untruncated result;
+        # _search returns a fresh list we are free to reorder/slice
+        docs = self._search(cond)
+
+        if sort_by is not None:
+            field = sort_by
+            present = [doc for doc in docs if field in doc]
+            missing = [doc for doc in docs if field not in doc]
+            present.sort(key=lambda doc: doc[field], reverse=reverse)
+            docs = present + missing
+
+        if limit:
+            docs = docs[:limit]
+
+        return docs
+
+    def _search(self, cond: QueryLike) -> list[Document]:
+        """
+        Plain search backed by the query cache. Returns a fresh list.
+        """
 
         # First, we check the query cache to see if it has results for this
         # query

inject/review.patch on GitHub

Prompts e verdade de referência:

• tasks/explore.md: onde o cache de consultas é invalidado e quais métodos públicos o disparam.

  tasks/explore.md

Where does TinyDB invalidate its query cache, and which public methods trigger it? Give file paths and function names.

tasks/explore.md on GitHub

tasks/tests.md

pytest fails on this repo. Find the bug in the library code and fix it. Do not change the tests.

tasks/tests.md on GitHub

tasks/lint.md

ruff reports issues in tinydb/ (run `ruff check tinydb`). Fix them all without changing behaviour.

tasks/lint.md on GitHub

tasks/commit.md

Write the commit message for the staged changes. Output only the message.

tasks/commit.md on GitHub

tasks/refactor.md

I keep writing little lambdas to append things to list fields, so can you add two update operations to `tinydb.operations`, next to `increment`, `delete` and friends? `push(field, value)` should append `value` to the list stored in `field`, keeping the existing order, and `pull(field, value)` should remove every occurrence of `value` from that list, not just the first one. If a document doesn't have the field yet, `push` should create it as a one-element list, while `pull` should just leave that document alone without raising. If the field exists but isn't a list, `push` has to raise a `TypeError` instead of doing something weird with it. They should be used exactly like the existing operations, e.g. `db.update(push('tags', 'python'), User.name == 'john')`.

tasks/refactor.md on GitHub

tasks/review.md

Review the uncommitted changes for bugs and edge cases. List findings only.

tasks/review.md on GitHub

tasks/summary.md

Summarize how TinyDB is organised for a new contributor in at most 150 words: the main modules and what each owns.

tasks/summary.md on GitHub

tasks/explore.truth.md

# explore: ground truth (tinydb @ 18d73a1)

- The query cache is `Table._query_cache`, an `LRUCache` (tinydb/utils.py) created in
  `Table.__init__` (tinydb/table.py, `query_cache_class = LRUCache`, capacity
  `default_query_cache_capacity = 10`). It is filled in `Table.search`.
- Invalidation happens in one place: `Table._update_table()` (tinydb/table.py) ends with
  `self.clear_cache()`, and `Table.clear_cache()` does `self._query_cache.clear()`.
- Every public write goes through `_update_table`, so these trigger it:
  `insert`, `insert_multiple`, `update` (all three branches: callable/fields, by doc_ids, by cond),
  `update_multiple`, `upsert` (via `update` or `insert`), `remove` (by cond or doc_ids), `truncate`.
  `Table.__init__(persist_empty=True)` also calls it. `clear_cache()` itself is public.
  `TinyDB` (tinydb/database.py) forwards these via `__getattr__` to the default table;
  `drop_table`/`drop_tables` do not clear a cache, they forget the `Table` instances (`self._tables`).

## Check rule (check.py explore)
PASS iff the final answer
1. contains `table.py`, AND
2. contains `clear_cache` or `_query_cache`, AND
3. names at least 3 distinct methods among
   insert, insert_multiple, update, update_multiple, upsert, remove, truncate (word match, case-insensitive).

tasks/explore.truth.md on GitHub

tasks/review.truth.md

# review: planted defect and matching rule

The uncommitted change is the reference T4 patch (`Table.search(cond, *, sort_by, reverse, limit)`)
with one defect: the truncation guard is `if limit:` instead of `if limit is not None:`, so
`search(cond, limit=0)` returns ALL matching documents instead of `[]` (0 is falsy).
Everything else in the change is correct (missing-field docs last in both directions,
negative limit -> ValueError, cache keeps the plain result).

## Check rule (check.py review)
PASS iff the final answer (case-insensitive) matches at least one of:
- `limit\s*=?=\s*0` (e.g. "limit=0", "limit == 0")
- `limit\s+(of\s+)?(0|zero)\b` (e.g. "limit of 0", "limit zero")
- `\b(0|zero)\s+limit` / `zero[- ]limit`
- `limit` within 150 characters of `falsy|falsey|truthy|truthiness` (either order)
- `if limit:` together with `0` or `zero` within 150 characters
Decoy answers (generic review, or only the sort order) must FAIL.

tasks/review.truth.md on GitHub

Faça isto na segunda-feira

• Rode o Claude Code 2.1.257 ou posterior, para que a flag force exista, e 2.1.243 ou posterior, para que o /tasks mostre o modelo de cada subagent.
• Decida qual arquivo você quer: o arquivo Explore só para o explorador, o bloco env para todos os subagents.
• Salve Explore.md como .claude/agents/Explore.md, ou mescle o bloco env em .claude/settings.json.
• Use o id completo claude-haiku-5-5, não o alias haiku, principalmente no Bedrock, no Vertex ou no Foundry.
• Inicie uma sessão, peça uma busca no código pelo agente Explore, depois abra o /tasks e leia o modelo e o effort em que ele rodou.
• Mantenha o Sonnet 5.5 ou o Opus 5.5 em qualquer subagent que faça coding agêntico complexo, e não defina FORCE se você fixou um agente assim de propósito.
• Rode de novo as suas próprias sete tarefas pelo harness antes de confiar na nossa tabela no seu repositório.
• Leia o /usage como "qual skill ou agente consome o meu histórico local", não como uma previsão de cota.

Medições

O medidor do plano, lido do rate_limit_event de cada execução headless, com um ponto de resolução. Cada bloco é contíguo, sem mais nada tocando a conta. A janela semanal marcou 67% em todas as 182 execuções.

Resumo da matriz: 182 execuções, a matriz inteira por cerca de $4,09 em preço de tabela, dois pontos de uma janela de cinco horas e zero ponto da semana. Cada bloco carrega até um ponto de arredondamento, então o ponto do Sonnet e o ponto do Opus não podem ser distinguidos. A tarefa delimitada é invisível nesse medidor em qualquer um dos modelos. A página de suporte sobre o custo relativo por turno dos três modelos não dá nenhum multiplicador s9.

Contra o detalhamento local, por configuração:

O detalhamento do /usage é calculado a partir do histórico local na sua máquina s3. A fórmula, lida do binário do cliente, é uma camada de modelo vezes um peso de registro. Camada do modelo: Fable 10, Opus 5, Haiku 1, qualquer outro 3. Peso do registro: tokens em cache 1, entrada sem cache 10, criação de cache 12,5, saída 50. Esses são pesos locais de atribuição do cliente, não cobrança, e não provam como o servidor conta a sua cota. A diferença aparece: o bloco Haiku medium marca 29,48 e o bloco Opus 27,30, então o painel coloca o bloco do Haiku acima do bloco do Opus, enquanto o medidor real marcou zero contra um ponto.

Onde um mês de logs de uma conta colocou os tokens: 2.631 execuções de subagent, 42 no Haiku 4.5, subagents em 48,1% de todos os tokens, Haiku 4.5 em 0,27% do custo ponderado, e um prompt de lançamento de subagent com mediana de 47.117 tokens, seja qual for o modelo.

O effort muda o apetite na mesma funcionalidade aceita (refactor): o Haiku medium leu 232.985 tokens de leitura de cache em 13,7 turnos, o Haiku low 139.801 em 7,6 turnos, o Opus 87.612 em 8,3 turnos.

Para ir além

• O penhasco de preço dos 100k: acima de 100.000 tokens de prompt, a faixa de preço mais alta vale para a requisição inteira, não só para o excedente s8. Se os tokens de cache contam para os 100k é a leitura desse autor, não uma regra citada. Importa para a economia de quem usa chave de API, não para o medidor de uma assinatura. Página de preços: s20.
• Tokenizer: o mesmo texto conta cerca de 30% mais tokens no Haiku 5.5 que no Haiku 4.5, e o adaptive thinking vem ligado por padrão na API, com o effort como alavanca s12.
• Effort e tokens por tarefa: uma execução independente mediu cerca de 162k tokens de saída por tarefa do Intelligence Index no effort máximo, e cerca de 55k no effort alto, para uma pontuação de 38 s7.
• Uma configuração completa de praticante, com seus números e a própria ressalva de que a economia vem de um contexto principal pequeno: s4.
• Issue aberta #100082, modelo do frontmatter do subagent ignorado em alguns builds: s10.
• Issue aberta #100154, transcrições de subagent subcontam output_tokens, 461 contra 15.575 para o mesmo briefing, cobrança no servidor não afetada s11. Como o /usage multiplica a saída por 50 a partir do histórico local, essa subcontagem chega ao painel.
• Não testado aqui: Haiku como modelo principal (as fontes enquadram o Haiku 5.5 como subagent), combinações de modelos entre provedores por um gateway (não são configurações do Claude Code), prompts de contexto de 1M e os créditos de API incluídos nos planos Max (Claude Platform, não a cota do Claude Code).

FAQ

Definir CLAUDE_CODE_SUBAGENT_MODEL coloca o Explore no Haiku?

Não. Sozinha, ela não muda os subagents embutidos Explore e Plan s2. Adicione o arquivo do agente Explore, ou a flag force.

O FORCE vai quebrar meus subagents no Opus?

Ele aplica um único modelo a todos os subagents, então sim: um agente que você fixou no Opus de propósito também roda no Haiku. Use o arquivo Explore se só o explorador deve ser barato.

O Haiku 5.5 é bom o bastante para trabalho de subagent?

Nessas sete tarefas delimitadas em um repositório pequeno, ele foi aceito 70 de 70 vezes em effort baixo e 70 de 70 em médio. A própria Anthropic mantém o Sonnet 5.5 e o Opus 5.5 para coding agêntico complexo s1, e a diferença no Terminal-Bench 4.0 é de 39,2% contra 70,6%.

Quanta cota do plano a troca devolve?

Para uma tarefa delimitada, menos de um ponto do medidor em qualquer modelo: a matriz inteira de 182 execuções moveu a janela de cinco horas em dois pontos e a janela semanal em zero. Para onde vai uma semana real: o contexto principal e o prompt de lançamento que cada subagent carrega.

Posso desligar o thinking para economizar mais?

Não nesses modelos no Claude Code s21. O effort é o botão que sobra.