Kieruj subagenty Claude Code na Haiku 5.5: tabela, dwa pliki, harness

TL;DR

• Subagenty Claude Code nie działają na Haiku 5.5, dopóki plik tego nie ustawi. Wbudowany agent Explore i każdy własny subagent dziedziczą model główny, a sama zmienna modelu subagenta zostawia Explore bez zmian.
• Rozwiązują to dwa pliki: projektowy plik agenta Explore.md (Haiku 5.5, effort low) tylko dla eksploratora oraz blok env w settings.json z CLAUDE_CODE_SUBAGENT_MODEL_FORCE, który przestawia wszystkich subagentów na Haiku.
• Nasza macierz, 182 uruchomienia na jednym prawdziwym repo: eksploracja, testy, lint, commit, refactor, review i podsumowanie, wszystko przyjęte przez ukryty test lub stałą regułę na Haiku 5.5, na low i na medium effort (po 70 z 70).
• Wszystkie siedem zadań kieruj na Haiku 5.5 z effort low. Sonnet 5.5 lub Opus 5.5 zachowaj do złożonego kodowania agentowego, czyli według rady samej Anthropic.
• Prawdziwy licznik planu ledwo drgnął: okno pięciogodzinne 21% do 21% przez 70 uruchomień Haiku low, okno tygodniowe 67% przez wszystkie 182 uruchomienia. Podział w /usage szereguje modele lokalnymi wagami, a nie rozliczeniem.
• Przypnij pełny identyfikator modelu. Alias haiku to Haiku 4.5 na Bedrock, Vertex i Foundry.

Co mówią źródła

Haiku 5.5 kosztuje 0,10 $ za milion tokenów wejścia dla promptów do 100k tokenów i 0,50 $ powyżej, a wyjście 0,50 $ i 2,50 $. Haiku 4.5 to 1,00 $ i 5,00 $, Sonnet 5.5 to 2,00 $ i 10,00 $ s1. Strona z cennikiem podaje Opusa 5.5 po 4 $ i 20 $ za milion tokenów i zaznacza, że Haiku 5.5, inaczej niż pozostałe modele z kontekstem 1M, jest wyceniany według długości promptu s20. Nagłówkowe 90% to odsetek próśb do poprzedniego Haiku z promptami poniżej 100k tokenów, a nie obniżka ceny s1.

Anthropic pozycjonuje Haiku 5.5 jako subagenta obok Opusa 5.5 i Sonneta 5.5 i mówi, że Sonnet 5.5 i Opus 5.5 pozostają lepszym wyborem do złożonego kodowania agentowego, a Haiku 5.5 do wąsko zakreślonej pracy s1. W Terminal-Bench 4.0 wpis premierowy podaje 39,2% dla Haiku 5.5 wobec 70,6% dla Sonneta 5.5 s1.

Nic nie idzie na Haiku domyślnie. Od wersji v2.1.198 wbudowany agent Explore przestaje domyślnie używać Haiku i dziedziczy model główny s24. Udokumentowana kolejność rozstrzygania modelu subagenta to: parametr z wywołania, frontmatter definicji, CLAUDE_CODE_SUBAGENT_MODEL, a potem model głównej rozmowy s2. Sama zmienna CLAUDE_CODE_SUBAGENT_MODEL nie zmienia wbudowanych subagentów Explore i Plan s2. Subagent użytkownika lub projektu o nazwie Explore nadpisuje wbudowanego, a dokumentacja sugeruje zdefiniowanie takiego z modelem Haiku s2. CLAUDE_CODE_SUBAGENT_MODEL_FORCE=1 stosuje jeden model do każdego subagenta, od wersji v2.1.257 s2 s24.

Dokumentacja kosztów zaleca model: haiku dla prostych zadań subagentów i stwierdza, że prośby subagentów nadal zużywają ten sam limit s3. Liczniki tygodniowe to jeden dla wszystkich modeli oraz jeden dla Fable tam, gdzie jest dostępny s22, a pozostałe modele czerpią z tych samych limitów s23.

W Claude Code nie da się wyłączyć thinking dla Haiku 5.5, Opusa 5.5, Sonneta 5.5 ani Fable: alwaysThinkingEnabled ustawione na false i MAX_THINKING_TOKENS=0 nie mają żadnego skutku. Effort sięga od low do max, a Haiku 5.5 domyślnie używa medium s21. Alias haiku wskazuje na Haiku 5.5 w API Anthropic, a na Haiku 4.5 w Platform on AWS, Bedrock, Vertex i Foundry s21.

Konfiguracja praktyka na planie Pro (główny Opus 5.5 na medium, implementer na Sonnecie 5.5, scoper Haiku 5.5 tylko do odczytu, shipper Haiku 5.5 na low) raportuje 13 PR w dziesięciu sesjach, po około 6% okna pięciogodzinnego na PR i około 1% licznika tygodniowego na sesję z jednym PR s4. Jego własne zastrzeżenie: oszczędność bierze się z tego, że główny kontekst pozostaje mały, a nie z tańszego modelu, bo wycena tokenów subagentów po stawkach Opusa dodaje tylko 5% s4.

Dwa pliki

Dwa pliki, weź jeden albo oba. Projektowy plik agenta .claude/agents/Explore.md nazywa się Explore, więc zastępuje wbudowanego eksploratora. Pełny identyfikator modelu, effort low, tylko narzędzia do odczytu i wyszukiwania.

Explore.md

---
name: Explore
description: Read-only codebase explorer. Use it to find where something is defined or called and to summarize what it found, with file paths and function names.
model: claude-haiku-5-5
effort: low
tools: Read, Glob, Grep, Bash
---

You explore a codebase and report back. Search with Glob and Grep, read only the files you need, and never edit anything. Answer with file paths and function names, then stop.

Explore.md on GitHub

Plik ustawień .claude/settings.json zawiera blok env: model subagenta oraz flagę force, która sprawia, że obowiązuje on każdego subagenta.

settings.json

{
  "env": {
    "CLAUDE_CODE_SUBAGENT_MODEL": "claude-haiku-5-5",
    "CLAUDE_CODE_SUBAGENT_MODEL_FORCE": "1"
  }
}

settings.json on GitHub

Który dla kogo:

• Użyj bloku env, gdy każdy subagent ma być tani. FORCE ściąga na Haiku każdego agenta, także tych, które celowo przypiąłeś do Opusa.
• Użyj pliku Explore, gdy ma się zmienić tylko eksplorator. Pozostali subagenci zachowują własne modele.
• Ustawienie CLAUDE_CODE_SUBAGENT_MODEL bez FORCE nic nie zmienia dla Explore. Dwa nasze uruchomienia headless, sesja główna na Fable, „use the Explore agent to find where the query cache is cleared”: wynik narzędzia wskazał Opusa 5.5 bez żadnego ustawienia i znowu Opusa 5.5 przy zmiennej ustawionej na Haiku 5.5 i niczym więcej. Z plikiem Explore albo z dwiema zmiennymi env pokazał Haiku 5.5.

Sprawdzian: /tasks pokazuje model i effort, na których działał każdy subagent (od v2.1.243 s24), a wynik narzędzia Agent w uruchomieniu headless zawiera resolvedModel. Ufaj tej linii, nigdy plikowi.

Pułapka aliasu: wpisz claude-haiku-5-5, nie haiku. Alias to Haiku 5.5 tylko w API Anthropic; na Bedrock, Vertex i Foundry nadal oznacza Haiku 4.5 s21.

Starsze otwarte zgłoszenie mówi, że model z frontmattera subagenta był po cichu ignorowany s25, a późniejsze liczy 1149 z 1371 subagentów działających na modelu Opus rodzica w wersjach od 2.1.277 do 2.1.290 s10. Na Claude Code 2.1.293 każde z naszych 182 uruchomień miało skonfigurowany model (model_ok True w tabeli wyników). Mimo to sprawdzaj.

Tabela routingu

Siedem codziennych zadań na tinydb w 18d73a1, 226 testów, headless claude -p, jedno deterministyczne sprawdzenie przyjęcia lub odrzucenia na zadanie. Każda komórka to przyjęte uruchomienia, średnia cena katalogowa za uruchomienie, średni czas w sekundach. Komórki Haiku to 10 uruchomień na zadanie, komórki Sonneta i Opusa to 3 uruchomienia na zadanie.

Zadanie „refactor” to mała funkcja: dodaj operacje aktualizacji push i pull do tinydb.operations, oceniane ukrytym plikiem testowym. Kosztowało 0,0052 $ na Haiku low i 0,1675 $ na Opusie, w 17,1 s wobec 30,3 s.

Co zostaje na Sonnecie lub Opusie: złożone kodowanie agentowe, według słów samej Anthropic s1. Te siedem zadań ma wąski zakres, każde trwa poniżej minuty, na jednym małym repo.

Harness

Benchmark, który dał tę tabelę. Uruchom go ze zwykłego terminala, gdy nic innego nie używa konta, bo licznik planu jest współdzielony. Checkout tinydb, ukryte testy i referencyjne patche nie są dołączone: wskaż własne repo i napisz własne sprawdzenia.

Runner i licznik:

• README.md: czym jest harness i co zostało pominięte.

  README.md

# routing bench

The benchmark harness behind the routing table: run.sh runs one cell headless and reads the real meters from the rate_limit_event, matrix.sh runs cells sequentially, check.py decides accept or reject per task, analyze.py writes the results table. tasks/ holds the seven prompts, inject/ the planted bugs. The tinydb checkout (commit 18d73a1), the hidden tests and the reference patches are not included: bring your own repo and checks.

README.md on GitHub

run.sh

#!/usr/bin/env bash
# One run: fresh tinydb work copy + task injection, headless claude -p, extract tokens/meter, check.
# Usage: run.sh <config> <task> <rep>
#   config: a key of configs.json (haiku-med | haiku-low | sonnet | opus)
#   task:   a key of tasks.json   (explore | tests | lint | commit | refactor | review | summary)
set -uo pipefail
CFG="$1"; TASK="$2"; REP="${3:-1}"
HERE="$(cd "$(dirname "$0")" && pwd)"
. "$HERE/env.sh"
PY="$HERE/.venv/bin/python"
MAX_TURNS="${BENCH_MAX_TURNS:-40}"
NAME="${TASK}-${CFG}-r${REP}"
OUT="$HERE/runs/$NAME"; W="$HERE/work/$NAME"
rm -rf "$OUT"; mkdir -p "$OUT"
"$HERE/prep.sh" "$TASK" "$W" > "$OUT/prep.txt" 2>&1 || { echo "$NAME: prep failed"; cat "$OUT/prep.txt"; exit 2; }
( cd "$W" && git diff HEAD > "$OUT/pre.diff" )

# Config args (model pinned by full id, optional --effort)
CARGS=()
while IFS= read -r a; do CARGS+=("$a"); done < <("$PY" -c "
import json; [print(a) for a in json.load(open('$HERE/configs.json'))['$CFG']['args']]")
[ ${#CARGS[@]} -gt 0 ] || { echo "bad config $CFG" >&2; exit 2; }

# Fixed allow-list for every cell, including Bash(ruff:*) for the lint task.
TOOLS="Read,Edit,Write,MultiEdit,Glob,Grep,Skill,TodoWrite,Bash(uv:*),Bash(python:*),Bash(python3:*),Bash(pytest:*),Bash(ruff:*),Bash(git status:*),Bash(git diff:*),Bash(git log:*),Bash(ls:*),Bash(cat:*),Bash(mkdir:*),Bash(mktemp:*),Bash(head:*),Bash(tail:*),Bash(grep:*),Bash(wc:*)"

cd "$W"
START=$(date +%s)
claude -p "$(cat "$HERE/tasks/$TASK.md")" "${CARGS[@]}" \
  --output-format stream-json --verbose --max-turns "$MAX_TURNS" \
  --permission-mode acceptEdits --allowedTools "$TOOLS" \
  --setting-sources project --strict-mcp-config \
  > "$OUT/stream.jsonl" 2> "$OUT/stderr.txt"
echo "$?" > "$OUT/exit_code.txt"
END=$(date +%s)
echo "$((END-START))" > "$OUT/wall_s.txt"

# After-meter: one cheap Haiku probe (its own stream is kept for audit).
"$HERE/probe.sh" "$OUT/probe.jsonl" > "$OUT/after.json" 2>/dev/null

# Diff of the agent's work (tracked + untracked) vs the post-injection state
# (index left untouched: the commit task's staged changes stay staged for the check)
git diff HEAD -- . ':(exclude).coverage' > "$OUT/post.diff"
git diff HEAD --numstat -- . ':(exclude).coverage' > "$OUT/numstat.txt"
git ls-files --others --exclude-standard | while IFS= read -r f; do
  git diff --no-index -- /dev/null "$f" >> "$OUT/post.diff"
  echo "$(wc -l < "$f" | tr -d ' ')	0	$f" >> "$OUT/numstat.txt"
done

"$PY" "$HERE/extract.py" "$OUT" "$CFG" "$TASK" "$REP" > "$OUT/result.json"
"$PY" "$HERE/check.py" "$TASK" "$W" "$OUT/result.json" > "$OUT/check.txt"; echo "$?" >> "$OUT/check.txt"
"$PY" "$HERE/extract.py" "$OUT" "$CFG" "$TASK" "$REP" --score

run.sh on GitHub

matrix.sh

#!/usr/bin/env bash
# Run an ordered list of "<config> <task> <rep>" lines SEQUENTIALLY (the plan meter is shared:
# never run two cells at once). Blank lines and #comments are skipped. Appends to matrix.log.
# Usage: matrix.sh [matrix-file]   (default: matrix.txt)
#   SKIP_DONE=1 matrix.sh ...      skip cells whose runs/<task>-<config>-r<rep>/score.json exists (resume)
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"; cd "$HERE"
LIST="${1:-matrix.txt}"
# one matrix at a time: the meter is shared
mkdir "$HERE/.matrix.lock" 2>/dev/null || { echo "another matrix.sh is running (remove .matrix.lock if stale)" >&2; exit 1; }
trap 'rmdir "$HERE/.matrix.lock"' EXIT
echo "=== $(date '+%F %T') start $LIST" >> matrix.log
grep -vE '^\s*(#|$)' "$LIST" | while read -r CFG TASK REP; do
  if [ "${SKIP_DONE:-0}" = 1 ] && [ -f "runs/${TASK}-${CFG}-r${REP}/score.json" ]; then
    echo "$(date '+%F %T') skip ${TASK}-${CFG}-r${REP}" >> matrix.log; continue
  fi
  line=$(./run.sh "$CFG" "$TASK" "$REP" < /dev/null 2>&1 | tail -1)
  echo "$(date '+%F %T') $line" | tee -a matrix.log
done
echo "=== $(date '+%F %T') done $LIST" >> matrix.log

matrix.sh on GitHub

gen_matrix.py

#!/usr/bin/env python3
"""Write the ordered matrix file.

gen_matrix.py [--reps 3] [--passes 2] [--configs haiku-low,haiku-med,sonnet,opus] [--out matrix.txt]

Order: configs run as contiguous BLOCKS (all tasks of one rep set), cheap configs first; tasks are
rotated inside each block so no task always runs first. With --passes 2 the second pass runs the
configs in reverse order (ABBA), which interleaves configs over time without breaking the
contiguous segments that analyze.py needs to beat the 1% meter resolution.
"""
import argparse, json, os

HERE = os.path.dirname(os.path.abspath(__file__))
ap = argparse.ArgumentParser()
ap.add_argument("--reps", type=int, default=3, help="reps per config x task per pass")
ap.add_argument("--passes", type=int, default=2)
ap.add_argument("--configs", default="haiku-low,haiku-med,sonnet,opus")
ap.add_argument("--start-rep", type=int, default=1, help="first rep number (use 2 to keep the r1 smoke runs)")
ap.add_argument("--out", default=os.path.join(HERE, "matrix.txt"))
a = ap.parse_args()
tasks = list(json.load(open(os.path.join(HERE, "tasks.json"))))
cfgs = a.configs.split(",")
lines, rep = [], a.start_rep
for p in range(a.passes):
    order = cfgs if p % 2 == 0 else list(reversed(cfgs))
    lines.append(f"# pass {p + 1}: {' -> '.join(order)}")
    for c in order:
        lines.append(f"# block {c}")
        for r in range(a.reps):
            k = rep + r
            rot = (k - 1) % len(tasks)
            for t in tasks[rot:] + tasks[:rot]:
                lines.append(f"{c} {t} {k}")
    rep += a.reps
open(a.out, "w").write("\n".join(lines) + "\n")
n = sum(1 for l in lines if not l.startswith("#"))
print(f"{a.out}: {n} runs ({len(cfgs)} configs x {len(tasks)} tasks x {a.reps} reps x {a.passes} passes)")

gen_matrix.py on GitHub

prep.sh

#!/usr/bin/env bash
# Build a fresh work copy for one task: upstream tinydb, baseline commit, task injection.
# Usage: prep.sh <task> <workdir>
set -euo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
TASK="$1"; W="$2"
read -r INJ MODE < <("$HERE/.venv/bin/python" -c "
import json,sys; t=json.load(open('$HERE/tasks.json'))['$TASK']; print(t['inject'] or '-', t['mode'] or '-')")
rm -rf "$W"; mkdir -p "$(dirname "$W")"
cp -R "$HERE/upstream" "$W"
cd "$W"
/usr/bin/find . -name __pycache__ -type d -prune -exec rm -rf {} +
rm -f .coverage
git update-index -q --refresh || true
G() { git -c user.email=dev@example.com -c user.name=dev "$@"; }
# Injections that belong to the baseline are folded into the top upstream commit
# (amend, same message/author/date), so `git log` shows no extra "bug" commit.
if [ "$MODE" = "commit" ]; then
  git apply "$HERE/$INJ"
  git add -A
  G commit -q --amend --no-edit --no-verify
fi
case "$MODE" in
  stage)    git apply --index "$HERE/$INJ" ;;
  worktree) git apply "$HERE/$INJ" ;;
esac

prep.sh on GitHub

env.sh

# Sourced by run.sh and probe.sh.
# Auth: an isolated CLAUDE_CONFIG_DIR is not logged in
# (`claude auth status` -> loggedIn false), so runs use the user's default config dir:
# CLAUDE_CONFIG_DIR is deliberately left unset.
unset CLAUDE_CONFIG_DIR
# Fresh work copies at the same path must not inherit memory written by an earlier rep.
export CLAUDE_CODE_DISABLE_AUTO_MEMORY=1
# The bench venv (pytest, ruff, mypy) comes first, so `python -m pytest` and `ruff` just work.
export PATH="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/.venv/bin:$PATH"

env.sh on GitHub

probe.sh

#!/usr/bin/env bash
# Cheapest possible call: one Haiku turn answering "ok". Prints the current plan
# meter read from its rate_limit_event as one JSON line. The probe itself costs a
# few hundred tokens of Haiku (well under the 1% meter resolution).
# Usage: probe.sh [raw-stream-out-file]
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
[ -f "$HERE/env.sh" ] && . "$HERE/env.sh"
RAW="${1:-$(mktemp -t hrb-probe)}"; case "$RAW" in /*) ;; *) RAW="$PWD/$RAW";; esac
cd "$(mktemp -d -t hrb-probe-cwd)"
claude --model claude-haiku-5-5 -p "ok" --max-turns 1 --output-format stream-json --verbose \
  --setting-sources project --strict-mcp-config > "$RAW" 2>/dev/null
"$HERE/.venv/bin/python" "$HERE/meter.py" "$RAW"

probe.sh on GitHub

meter.py

#!/usr/bin/env python3
"""Read a stream-json file and print the plan meter carried by its rate_limit_event lines.

Output (one JSON line): {"ts", "five_hour", "five_hour_resets_at", "seven_day",
"seven_day_resets_at", "n_events", "raw_first", "raw_last"}: utilizations are the
values of the FIRST event (the meter as the run began). `--last` uses the last event.
"""
import json, sys, time

def events(path):
    out = []
    for line in open(path, errors="replace"):
        line = line.strip()
        if not line.startswith("{"):
            continue
        try:
            o = json.loads(line)
        except Exception:
            continue
        if o.get("type") == "rate_limit_event":
            out.append(o)
    return out

def windows(ev):
    info = ev.get("rate_limit_info") or {}
    uw = info.get("unifiedWindows") or {}
    def w(name):
        x = uw.get(name) or {}
        return x.get("utilization"), x.get("resetsAt")
    fh, fhr = w("five_hour")
    sd, sdr = w("seven_day")
    return {"five_hour": fh, "five_hour_resets_at": fhr, "seven_day": sd, "seven_day_resets_at": sdr}

def meter(path, which="first"):
    evs = events(path)
    if not evs:
        return {"ts": int(time.time()), "n_events": 0, "five_hour": None, "seven_day": None}
    ev = evs[0] if which == "first" else evs[-1]
    d = {"ts": int(time.time()), "n_events": len(evs)}
    d.update(windows(ev))
    d["raw_first"] = evs[0].get("rate_limit_info")
    d["raw_last"] = evs[-1].get("rate_limit_info")
    return d

if __name__ == "__main__":
    which = "last" if "--last" in sys.argv else "first"
    path = [a for a in sys.argv[1:] if not a.startswith("--")][0]
    print(json.dumps(meter(path, which)))

meter.py on GitHub

extract.py

#!/usr/bin/env python3
"""Extract one run's numbers from runs/<name>/stream.jsonl.

extract.py <outdir> <config> <task> <rep>          -> prints the stream's `result` line (result.json)
extract.py <outdir> <config> <task> <rep> --score  -> writes score.json, prints a one-line summary
"""
import json, os, sys

HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, HERE)
from meter import meter  # noqa: E402

out, cfg, task, rep = sys.argv[1:5]
score_mode = "--score" in sys.argv
stream = os.path.join(out, "stream.jsonl")

lines = []
for raw in open(stream, errors="replace") if os.path.exists(stream) else []:
    raw = raw.strip()
    if raw.startswith("{"):
        try:
            lines.append(json.loads(raw))
        except Exception:
            pass
result = next((o for o in reversed(lines) if o.get("type") == "result"), None)
init = next((o for o in lines if o.get("type") == "system" and o.get("subtype") == "init"), {})

if not score_mode:
    print(json.dumps(result or {"type": "result", "result": "", "missing": True}))
    sys.exit(0)

def rd(name, default=None):
    p = os.path.join(out, name)
    try:
        return open(p).read().strip()
    except Exception:
        return default

def jl(name):
    try:
        return json.loads(rd(name) or "null")
    except Exception:
        return None

cfgs = json.load(open(os.path.join(HERE, "configs.json")))
before = meter(stream, "first") if os.path.exists(stream) else None
after = jl("after.json")
r = result or {}
usage = r.get("usage") or {}
tok = {k: usage.get(k) or 0 for k in ("input_tokens", "cache_read_input_tokens", "cache_creation_input_tokens", "output_tokens")}
mu = r.get("modelUsage") or {}
assistant_models = sorted({(o.get("message") or {}).get("model") for o in lines
                           if o.get("type") == "assistant" and (o.get("message") or {}).get("model") not in (None, "<synthetic>")})
check_txt = (rd("check.txt", "") or "").split("\n")
verdict_line = check_txt[0] if check_txt else ""
numstat = [l.split("\t") for l in (rd("numstat.txt", "") or "").split("\n") if l.strip()]
added = sum(int(a) for a, d, f in numstat if a.isdigit())
deleted = sum(int(d) for a, d, f in numstat if d.isdigit())
changed_by_agent = (rd("pre.diff", "") or "") != (rd("post.diff", "") or "")

def strip_raw(m):
    if not m:
        return m
    return {k: v for k, v in m.items() if not k.startswith("raw_")}

s = {
    "run": os.path.basename(out), "config": cfg, "task": task, "rep": int(rep),
    "expected_model": cfgs[cfg]["model"], "config_args": cfgs[cfg]["args"],
    "init_model": init.get("model"), "assistant_models": assistant_models,
    "model_usage_keys": sorted(mu.keys()),
    "model_ok": init.get("model") == cfgs[cfg]["model"] and set(mu.keys()) <= {cfgs[cfg]["model"]} and bool(mu),
    "claude_code_version": init.get("claude_code_version"),
    "meter_before": strip_raw(before), "meter_after": strip_raw(after),
    "n_rate_limit_events": (before or {}).get("n_events", 0),
    "has_result_line": result is not None,
    "subtype": r.get("subtype"), "is_error": r.get("is_error"), "stop_reason": r.get("stop_reason"),
    "num_turns": r.get("num_turns"), "duration_ms": r.get("duration_ms"), "duration_api_ms": r.get("duration_api_ms"),
    "total_cost_usd": r.get("total_cost_usd"), "usage": usage, "tokens": tok, "modelUsage": mu,
    "wall_s": int(rd("wall_s.txt", "0") or 0), "exit_code": rd("exit_code.txt"),
    "final_text": (r.get("result") or "")[-2000:],
    "diff": {"files": len(numstat), "added": added, "deleted": deleted, "changed_by_agent": changed_by_agent},
    "check": {"verdict": verdict_line.split(" ")[0] if verdict_line else None, "reason": verdict_line[5:] if verdict_line else None},
}
s["accepted"] = s["check"]["verdict"] == "PASS"
json.dump(s, open(os.path.join(out, "score.json"), "w"), indent=2)
mb, ma = s["meter_before"] or {}, s["meter_after"] or {}
print(f"{s['run']}: {s['check']['verdict']} turns={s['num_turns']} cost=${(s['total_cost_usd'] or 0):.3f} "
      f"in={tok['input_tokens']} cr={tok['cache_read_input_tokens']} cc={tok['cache_creation_input_tokens']} out={tok['output_tokens']} "
      f"wall={s['wall_s']}s 5h {mb.get('five_hour')}->{ma.get('five_hour')} 7d {mb.get('seven_day')}->{ma.get('seven_day')} "
      f"model_ok={s['model_ok']} | {s['check']['reason']}")

extract.py on GitHub

check.py

#!/usr/bin/env python3
"""Deterministic accept/reject check for one run.

Usage: check.py <task> <workdir> <result.json>
  result.json = the `result` line of the run's stream (only its "result" text is used).
Prints `PASS <reason>` or `FAIL <reason>`; exit 0 on PASS, 1 on FAIL.
"""
import json, os, re, subprocess, sys

HERE = os.path.dirname(os.path.abspath(__file__))
PY = os.path.join(HERE, ".venv", "bin", "python")
RUFF = os.path.join(HERE, ".venv", "bin", "ruff")


def final_text(path):
    try:
        r = json.load(open(path))
    except Exception:
        return ""
    return r.get("result") or "" if isinstance(r, dict) else ""


def sh(args, cwd, env=None):
    p = subprocess.run(args, cwd=cwd, capture_output=True, text=True, env=env)
    return p.returncode, p.stdout + p.stderr


def suite(wd):
    rc, out = sh([PY, "-m", "pytest", "-q", "-p", "no:cacheprovider", "-o", "addopts=", "tests"], wd)
    m = re.search(r"(\d+) passed", out)
    f = re.search(r"(\d+) failed", out)
    e = re.search(r"(\d+) error", out)
    return rc == 0, f"suite {m.group(1) if m else 0} passed, {f.group(1) if f else 0} failed, {e.group(1) if e else 0} errors"


def git_diff(wd, *paths):
    # tracked changes vs the baseline commit + untracked files under paths
    _, d = sh(["git", "diff", "HEAD", "--", *paths], wd)
    _, u = sh(["git", "ls-files", "--others", "--exclude-standard", "--", *paths], wd)
    return d.strip(), u.strip()


def words(s):
    return re.findall(r"[A-Za-z0-9_']+", s)


def strip_fences(s):
    lines = s.strip().split("\n")
    if lines and lines[0].strip().startswith("```"):
        lines = lines[1:]
    if lines and lines[-1].strip().startswith("```"):
        lines = lines[:-1]
    return "\n".join(lines).strip()


def c_explore(wd, txt):
    t = txt.lower()
    miss = []
    if "table.py" not in t:
        miss.append("table.py")
    if "clear_cache" not in t and "_query_cache" not in t:
        miss.append("clear_cache/_query_cache")
    names = ["insert_multiple", "update_multiple", "insert", "update", "upsert", "remove", "truncate"]
    found = {n for n in names if re.search(r"(?<![a-z_])" + n + r"(?![a-z_])", t)}
    if len(found) < 3:
        miss.append(f"only {len(found)} mutating methods {sorted(found)}")
    return (not miss), ("missing " + ", ".join(miss)) if miss else f"table.py + cache method + {len(found)} methods {sorted(found)}"


def c_tests(wd, txt):
    ok, s = suite(wd)
    d, u = git_diff(wd, "tests")
    if d or u:
        return False, f"tests/ modified; {s}"
    return ok, s + "; tests/ untouched"


def c_lint(wd, txt):
    rc, out = sh([RUFF, "check", "tinydb", "--output-format", "concise", "--no-cache"], wd)
    cfg_d, _ = git_diff(wd, "ruff.toml", "pyproject.toml")
    d, _ = git_diff(wd, "tinydb")
    noqa = [l for l in d.split("\n") if l.startswith("+") and "noqa" in l.lower()]
    ok, s = suite(wd)
    why = []
    if rc != 0:
        found = [l for l in out.split("\n") if l.startswith("Found ")]
        why.append("ruff: " + (found[0] if found else out.strip().split("\n")[-1]))
    if cfg_d:
        why.append("lint config changed")
    if noqa:
        why.append(f"{len(noqa)} noqa added")
    if not ok:
        why.append(s)
    return (not why), ("; ".join(why) if why else f"ruff clean; {s}")


DIFF_LINE = re.compile(r"^(diff --git |index [0-9a-f]{6,}\.\.|@@ |\+\+\+ |--- a/|--- /dev/null|[+-] {3,}\S|[+-](def|class|import|from|return) )")


def c_commit(wd, txt):
    msg = strip_fences(txt)
    if not msg:
        return False, "empty answer"
    first = msg.split("\n")[0].strip()
    why = []
    if len(first) > 72:
        why.append(f"first line {len(first)} chars")
    if "push" not in first.lower():
        why.append("no 'push' in first line")
    if "pull" not in first.lower():
        why.append("no 'pull' in first line")
    dl = [l for l in msg.split("\n") if DIFF_LINE.match(l)]
    if dl:
        why.append(f"{len(dl)} diff lines")
    return (not why), ("; ".join(why) if why else f"first line {len(first)} chars: {first!r}")


def c_refactor(wd, txt):
    env = dict(os.environ, TINYDB_TREE=wd)
    rc, out = sh([PY, "-m", "pytest", "-q", "-c", os.path.join(HERE, "hidden", "pytest.ini"),
                  "--rootdir", os.path.join(HERE, "hidden"), os.path.join(HERE, "hidden", "test_T1.py")], wd, env)
    m = re.search(r"(\d+) passed", out)
    hp = int(m.group(1)) if m else 0
    ok, s = suite(wd)
    return (hp == 5 and ok), f"hidden {hp}/5; {s}"


REVIEW_PATTERNS = [
    r"limit\s*=?=\s*0\b",
    r"limit\s+(of\s+)?(0|zero)\b",
    r"\b(0|zero)\s+limit",
    r"zero[- ]limit",
    r"limit.{0,150}\b(falsy|falsey|truthy|truthiness)",
    r"\b(falsy|falsey|truthy|truthiness)\b.{0,150}limit",
    r"if limit:.{0,150}\b(0|zero)\b",
    r"\b(0|zero)\b.{0,150}if limit:",
]


def c_review(wd, txt):
    t = txt.lower().replace("`", "")
    for p in REVIEW_PATTERNS:
        m = re.search(p, t, re.S)
        if m:
            return True, f"matched {p!r}: {m.group(0)[:80]!r}"
    return False, "planted limit=0 defect not mentioned"


def c_summary(wd, txt):
    n = len(words(txt))
    t = txt.lower()
    mods = [m for m in ["database", "table", "queries", "storages", "middlewares", "operations"] if m in t]
    why = []
    if n > 170:
        why.append(f"{n} words > 170")
    if len(mods) < 4:
        why.append(f"only {len(mods)} modules {mods}")
    return (not why), ("; ".join(why) if why else f"{n} words, modules {mods}")


CHECKS = {"explore": c_explore, "tests": c_tests, "lint": c_lint, "commit": c_commit,
          "refactor": c_refactor, "review": c_review, "summary": c_summary}

if __name__ == "__main__":
    task, wd, res = sys.argv[1], os.path.abspath(sys.argv[2]), sys.argv[3]
    ok, why = CHECKS[task](wd, final_text(res))
    print(("PASS " if ok else "FAIL ") + why)
    sys.exit(0 if ok else 1)

check.py on GitHub

analyze.py

#!/usr/bin/env python3
"""Aggregate runs/*/score.json into results.json and results.md.

Per config x task: accept rate, mean tokens, mean cost, mean turns, mean wall.
Per config: measured plan-meter deltas (five_hour, seven_day, in percentage points), computed two ways:
  - per_run_sum: sum over the config's runs of (after - before). Quantized at 1% per run, so
    runs far below 1% mostly read 0: a lower-bound-ish, very noisy number.
  - segments:    runs sorted by start time are cut into maximal contiguous segments of the same
    config; each segment contributes (after of its last run - before of its first run). This
    telescopes the quantization: one +-1 pt error per segment instead of per run. PREFERRED.
  Segments/runs whose window resetsAt changed between before and after are excluded for that window.
Plus the /usage-style local heuristic: weighted = cached*1 + uncached*10 + cache_create*12.5 + output*50,
times the model tier (haiku 1 / sonnet 3 / opus 5). The probe's own Haiku tokens after each run are
counted as overhead in the heuristic (they also move the meter).
Usage: analyze.py [--runs-dir runs] [--out-prefix results]
"""
import glob, json, os, statistics, sys

HERE = os.path.dirname(os.path.abspath(__file__))
RUNS = os.path.join(HERE, sys.argv[sys.argv.index("--runs-dir") + 1]) if "--runs-dir" in sys.argv else os.path.join(HERE, "runs")
OUTP = sys.argv[sys.argv.index("--out-prefix") + 1] if "--out-prefix" in sys.argv else "results"
CFGS = json.load(open(os.path.join(HERE, "configs.json")))
TASKS = list(json.load(open(os.path.join(HERE, "tasks.json"))).keys())
W = {"cache_read_input_tokens": 1, "input_tokens": 10, "cache_creation_input_tokens": 12.5, "output_tokens": 50}
TIER = {c: v["tier"] for c, v in CFGS.items()}


def weighted(tok, tier):
    raw = sum((tok.get(k) or 0) * w for k, w in W.items())
    return raw, raw * tier


MU = {"cacheReadInputTokens": "cache_read_input_tokens", "inputTokens": "input_tokens",
      "cacheCreationInputTokens": "cache_creation_input_tokens", "outputTokens": "output_tokens"}


def model_tier(name, fallback):
    n = (name or "").lower()
    return 5 if "opus" in n else 3 if "sonnet" in n else 1 if "haiku" in n else fallback


def heuristic_run(s):
    """Tiered heuristic per model actually used (modelUsage), so helper calls on another
    model are weighted with their own tier; falls back to `usage` x config tier."""
    mu = s.get("modelUsage") or {}
    if not mu:
        return weighted(s["tokens"], TIER[s["config"]])[1]
    return sum(weighted({MU[k]: v.get(k) or 0 for k in MU}, model_tier(m, TIER[s["config"]]))[1] for m, v in mu.items())


def probe_tokens(run_dir):
    p = os.path.join(run_dir, "probe.jsonl")
    if not os.path.exists(p):
        return {}
    for line in reversed(open(p, errors="replace").read().strip().split("\n")):
        try:
            o = json.loads(line)
        except Exception:
            continue
        if o.get("type") == "result":
            u = o.get("usage") or {}
            return {k: u.get(k) or 0 for k in W}
    return {}


def mean(xs):
    xs = [x for x in xs if x is not None]
    return statistics.mean(xs) if xs else None


def window_delta(b, a, w):
    if not b or not a or b.get(w) is None or a.get(w) is None:
        return None, "missing"
    if b.get(w + "_resets_at") != a.get(w + "_resets_at"):
        return None, "reset"
    return round((a[w] - b[w]) * 100, 2), "ok"


scores = []
for p in sorted(glob.glob(os.path.join(RUNS, "*", "score.json"))):
    s = json.load(open(p))
    if s.get("config") not in CFGS:
        continue
    s["_dir"] = os.path.dirname(p)
    s["_start"] = (s.get("meter_before") or {}).get("ts") or os.path.getmtime(os.path.join(s["_dir"], "stream.jsonl"))
    s["_probe_tok"] = probe_tokens(s["_dir"])
    scores.append(s)
# meter_before.ts is written at extraction time; order by stream file creation instead
for s in scores:
    st = os.stat(os.path.join(s["_dir"], "stream.jsonl"))
    s["_start"] = getattr(st, "st_birthtime", st.st_mtime)
scores.sort(key=lambda s: s["_start"])

cells, per_cfg = {}, {}
for s in scores:
    cells.setdefault((s["config"], s["task"]), []).append(s)
    per_cfg.setdefault(s["config"], []).append(s)

cell_rows = []
for (c, t), ss in sorted(cells.items(), key=lambda kv: (list(CFGS).index(kv[0][0]), TASKS.index(kv[0][1]) if kv[0][1] in TASKS else 99)):
    row = {"config": c, "task": t, "n": len(ss),
           "accept_rate": sum(1 for s in ss if s.get("accepted")) / len(ss),
           "mean_turns": mean([s.get("num_turns") for s in ss]),
           "mean_cost_usd": mean([s.get("total_cost_usd") for s in ss]),
           "mean_wall_s": mean([s.get("wall_s") for s in ss]),
           "mean_tokens": {k: mean([s["tokens"].get(k) for s in ss]) for k in W},
           "mean_weighted_heuristic": mean([heuristic_run(s) for s in ss]),
           "model_ok_all": all(s.get("model_ok") for s in ss)}
    cell_rows.append(row)

# contiguous segments in global start order
segments, cur = [], None
for s in scores:
    if cur and cur["config"] == s["config"]:
        cur["runs"].append(s)
    else:
        cur = {"config": s["config"], "runs": [s]}
        segments.append(cur)

cfg_rows = []
for c, ss in per_cfg.items():
    tok_sum = {k: sum(s["tokens"].get(k) or 0 for s in ss) for k in W}
    probe_sum = {k: sum(s["_probe_tok"].get(k, 0) for s in ss) for k in W}
    raw_w, _ = weighted(tok_sum, TIER[c])
    heur = sum(heuristic_run(s) for s in ss)
    probe_raw, probe_heur = weighted(probe_sum, 1)
    row = {"config": c, "runs": len(ss), "accept_rate": sum(1 for s in ss if s.get("accepted")) / len(ss),
           "tokens_sum": tok_sum, "probe_tokens_sum": probe_sum,
           "cost_sum_usd": round(sum(s.get("total_cost_usd") or 0 for s in ss), 4),
           "weighted_raw_M": raw_w / 1e6, "heuristic_M": (heur + probe_heur) / 1e6}
    for w in ("five_hour", "seven_day"):
        per_run, excl = 0.0, 0
        for s in ss:
            d, why = window_delta(s.get("meter_before"), s.get("meter_after"), w)
            if d is None:
                excl += 1
            else:
                per_run += d
        seg_sum, seg_n, seg_excl = 0.0, 0, 0
        for g in segments:
            if g["config"] != c:
                continue
            d, why = window_delta(g["runs"][0].get("meter_before"), g["runs"][-1].get("meter_after"), w)
            if d is None:
                seg_excl += 1
            else:
                seg_sum += d; seg_n += 1
        row[w] = {"per_run_sum_pts": round(per_run, 2), "per_run_excluded": excl,
                  "segments_sum_pts": round(seg_sum, 2), "segments": seg_n, "segments_excluded": seg_excl,
                  "quantization_error_pts": f"+-{seg_n}",
                  "pts_per_M_heuristic": round(seg_sum / row["heuristic_M"], 4) if row["heuristic_M"] else None,
                  "pts_per_M_weighted_raw": round(seg_sum / row["weighted_raw_M"], 4) if row["weighted_raw_M"] else None}
    cfg_rows.append(row)
cfg_rows.sort(key=lambda r: list(CFGS).index(r["config"]))

out = {"generated_from": len(scores), "cells": cell_rows, "configs": cfg_rows,
       "segments": [{"config": g["config"], "runs": [s["run"] for s in g["runs"]],
                     "before": g["runs"][0].get("meter_before"), "after": g["runs"][-1].get("meter_after")} for g in segments],
       "heuristic_weights": {"tokens": W, "tier": TIER}}
json.dump(out, open(os.path.join(HERE, OUTP + ".json"), "w"), indent=2, default=str)

f = lambda x, d=0: "-" if x is None else (f"{x:,.{d}f}")
md = ["# Results", "", f"{len(scores)} runs. Meter deltas in percentage points of the plan window (1-pt resolution).", "",
      "## Per config x task", "",
      "| config | task | n | accept | turns | input | cache read | cache create | output | cost $ | wall s | heuristic (M) | model ok |",
      "|---|---|---|---|---|---|---|---|---|---|---|---|---|"]
for r in cell_rows:
    t = r["mean_tokens"]
    md.append(f"| {r['config']} | {r['task']} | {r['n']} | {r['accept_rate']:.0%} | {f(r['mean_turns'],1)} | {f(t['input_tokens'])} | "
              f"{f(t['cache_read_input_tokens'])} | {f(t['cache_creation_input_tokens'])} | {f(t['output_tokens'])} | "
              f"{f(r['mean_cost_usd'],4)} | {f(r['mean_wall_s'],1)} | {f(r['mean_weighted_heuristic']/1e6,2)} | {r['model_ok_all']} |")
md += ["", "## Per config: measured meter vs local heuristic", "",
       "| config | runs | accept | cost $ | heuristic (M units) | 5h pts (segments) | 5h pts (per-run sum) | 7d pts (segments) | segments | 5h pts / M heuristic | 7d pts / M heuristic |",
       "|---|---|---|---|---|---|---|---|---|---|---|"]
for r in cfg_rows:
    fh, sd = r["five_hour"], r["seven_day"]
    md.append(f"| {r['config']} | {r['runs']} | {r['accept_rate']:.0%} | {r['cost_sum_usd']} | {r['heuristic_M']:.2f} | "
              f"{fh['segments_sum_pts']} (excl {fh['segments_excluded']}) | {fh['per_run_sum_pts']} | {sd['segments_sum_pts']} (excl {sd['segments_excluded']}) | "
              f"{fh['segments']} | {fh['pts_per_M_heuristic']} | {sd['pts_per_M_heuristic']} |")
md += ["", "If the /usage heuristic matched the real meter, `pts / M heuristic` would be the same for every config.",
       "Each segment carries up to +-1 pt of quantization error; trust a config's number only when its segment sum is several points.", ""]
open(os.path.join(HERE, OUTP + ".md"), "w").write("\n".join(md))
print("\n".join(md))

analyze.py on GitHub

Konfiguracja:

• configs.json: czterech wykonawców, z pełnymi identyfikatorami modeli i --effort low dla bloku low.

  configs.json

{
  "haiku-med": {"args": ["--model", "claude-haiku-5-5"], "model": "claude-haiku-5-5", "tier": 1, "note": "default effort"},
  "haiku-low": {"args": ["--model", "claude-haiku-5-5", "--effort", "low"], "model": "claude-haiku-5-5", "tier": 1},
  "sonnet":    {"args": ["--model", "claude-sonnet-5-5"], "model": "claude-sonnet-5-5", "tier": 3, "note": "default effort"},
  "opus":      {"args": ["--model", "claude-opus-5-5"], "model": "claude-opus-5-5", "tier": 5, "note": "default effort"}
}

configs.json on GitHub

tasks.json

{
  "explore":  {"inject": null,                     "mode": null,       "kind": "answer"},
  "tests":    {"inject": "inject/tests.patch",     "mode": "commit",   "kind": "code"},
  "lint":     {"inject": "inject/lint.patch",      "mode": "commit",   "kind": "code"},
  "commit":   {"inject": "reference/T1.patch",     "mode": "stage",    "kind": "answer"},
  "refactor": {"inject": null,                     "mode": null,       "kind": "code"},
  "review":   {"inject": "inject/review.patch",    "mode": "worktree", "kind": "answer"},
  "summary":  {"inject": null,                     "mode": null,       "kind": "answer"}
}

tasks.json on GitHub

ruff.toml

# Pinned lint config for the `lint` task: a small explicit rule set.
# Upstream tinydb @ 18d73a1 is clean under it (the two per-file ignores cover
# pre-existing F401s: the __version__ re-export and middlewares' Optional).
[lint]
select = ["F401", "F541", "F841", "E722"]

[lint.per-file-ignores]
"tinydb/__init__.py" = ["F401"]
"tinydb/middlewares.py" = ["F401"]

ruff.toml on GitHub

Podrzucone zmiany:

• inject/tests.patch: odwraca porównanie w Query.__ge__, żeby jeden test padał.

  inject/tests.patch

diff --git a/tinydb/queries.py b/tinydb/queries.py
index cdab9fb..db00a2a 100644
--- a/tinydb/queries.py
+++ b/tinydb/queries.py
@@ -322,7 +322,7 @@ class Query(QueryInstance):
         :param rhs: The value to compare against
         """
         return self._generate_test(
-            lambda value: value >= rhs,
+            lambda value: value > rhs,
             ('>=', self._path, rhs)
         )
 

inject/tests.patch on GitHub

inject/lint.patch

diff --git a/ruff.toml b/ruff.toml
new file mode 100644
--- /dev/null
+++ b/ruff.toml
@@ -0,0 +1,6 @@
+[lint]
+select = ["F401", "F541", "F841", "E722"]
+
+[lint.per-file-ignores]
+"tinydb/__init__.py" = ["F401"]
+"tinydb/middlewares.py" = ["F401"]
diff --git a/tinydb/operations.py b/tinydb/operations.py
index 7101a23..f8541fe 100644
--- a/tinydb/operations.py
+++ b/tinydb/operations.py
@@ -10,6 +10,7 @@ This would delete the ``foo`` field from all documents where ``foo`` equals 2.
 
 
 from collections.abc import Callable, MutableMapping
+import sys
 from typing import Any, Union
 
 
@@ -18,7 +19,7 @@ def delete(field: str) -> Callable[[MutableMapping], None]:
     Delete a given field from the document.
     """
     def transform(doc: MutableMapping):
-        del doc[field]
+        removed = doc.pop(field)
 
     return transform
 
diff --git a/tinydb/queries.py b/tinydb/queries.py
index cdab9fb..489a2be 100644
--- a/tinydb/queries.py
+++ b/tinydb/queries.py
@@ -227,7 +227,7 @@ class Query(QueryInstance):
         :return: A :class:`~tinydb.queries.QueryInstance` object
         """
         if not self._path and not allow_empty_path:
-            raise ValueError('Query has no path')
+            raise ValueError(f'Query has no path')
 
         def runner(value):
             try:
diff --git a/tinydb/table.py b/tinydb/table.py
index f4027bf..eb8058c 100644
--- a/tinydb/table.py
+++ b/tinydb/table.py
@@ -818,7 +818,7 @@ class Table:
 
         try:
             raw_table = tables[self.name]
-        except KeyError:
+        except:
             # The table does not exist yet, so it is empty
             raw_table = {}
 
diff --git a/tinydb/utils.py b/tinydb/utils.py
index 43cad4e..07114d5 100644
--- a/tinydb/utils.py
+++ b/tinydb/utils.py
@@ -2,6 +2,7 @@
 Utility functions.
 """
 
+import os
 from collections import OrderedDict, abc
 from collections.abc import Iterator
 from typing import TypeVar, Generic, Union, Optional, TYPE_CHECKING, cast

inject/lint.patch on GitHub

inject/review.patch

diff --git a/tinydb/table.py b/tinydb/table.py
index f4027bf..4e06ba1 100644
--- a/tinydb/table.py
+++ b/tinydb/table.py
@@ -244,10 +244,21 @@ class Table:
 
         return list(iter(self))
 
-    def search(self, cond: QueryLike) -> list[Document]:
+    def search(
+        self,
+        cond: QueryLike,
+        *,
+        sort_by: Optional[str] = None,
+        reverse: bool = False,
+        limit: Optional[int] = None,
+    ) -> list[Document]:
         """
         Search for all documents matching a 'where' cond.
 
+        Results can optionally be sorted by a field (``sort_by``, descending
+        with ``reverse=True``; documents without the field always come last)
+        and truncated to the first ``limit`` results after sorting.
+
         Note: a repeated search with the same ``cond`` is served from an
         internal cache. The returned list is a fresh copy, but its
         documents are the same objects held by the cache -- treat them as
@@ -255,8 +266,34 @@ class Table:
         what a later identical search returns.
 
         :param cond: the condition to check against
+        :param sort_by: optional field name to sort the results by
+        :param reverse: sort in descending order
+        :param limit: optional maximum number of results
         :returns: list of matching documents
         """
+        if limit is not None and limit < 0:
+            raise ValueError('limit must not be negative')
+
+        # The cache always holds the plain, unsorted, untruncated result;
+        # _search returns a fresh list we are free to reorder/slice
+        docs = self._search(cond)
+
+        if sort_by is not None:
+            field = sort_by
+            present = [doc for doc in docs if field in doc]
+            missing = [doc for doc in docs if field not in doc]
+            present.sort(key=lambda doc: doc[field], reverse=reverse)
+            docs = present + missing
+
+        if limit:
+            docs = docs[:limit]
+
+        return docs
+
+    def _search(self, cond: QueryLike) -> list[Document]:
+        """
+        Plain search backed by the query cache. Returns a fresh list.
+        """
 
         # First, we check the query cache to see if it has results for this
         # query

inject/review.patch on GitHub

Prompty i wzorzec prawdy:

• tasks/explore.md: gdzie unieważniany jest cache zapytań i które publiczne metody go wywołują.

  tasks/explore.md

Where does TinyDB invalidate its query cache, and which public methods trigger it? Give file paths and function names.

tasks/explore.md on GitHub

tasks/tests.md

pytest fails on this repo. Find the bug in the library code and fix it. Do not change the tests.

tasks/tests.md on GitHub

tasks/lint.md

ruff reports issues in tinydb/ (run `ruff check tinydb`). Fix them all without changing behaviour.

tasks/lint.md on GitHub

tasks/commit.md

Write the commit message for the staged changes. Output only the message.

tasks/commit.md on GitHub

tasks/refactor.md

I keep writing little lambdas to append things to list fields, so can you add two update operations to `tinydb.operations`, next to `increment`, `delete` and friends? `push(field, value)` should append `value` to the list stored in `field`, keeping the existing order, and `pull(field, value)` should remove every occurrence of `value` from that list, not just the first one. If a document doesn't have the field yet, `push` should create it as a one-element list, while `pull` should just leave that document alone without raising. If the field exists but isn't a list, `push` has to raise a `TypeError` instead of doing something weird with it. They should be used exactly like the existing operations, e.g. `db.update(push('tags', 'python'), User.name == 'john')`.

tasks/refactor.md on GitHub

tasks/review.md

Review the uncommitted changes for bugs and edge cases. List findings only.

tasks/review.md on GitHub

tasks/summary.md

Summarize how TinyDB is organised for a new contributor in at most 150 words: the main modules and what each owns.

tasks/summary.md on GitHub

tasks/explore.truth.md

# explore: ground truth (tinydb @ 18d73a1)

- The query cache is `Table._query_cache`, an `LRUCache` (tinydb/utils.py) created in
  `Table.__init__` (tinydb/table.py, `query_cache_class = LRUCache`, capacity
  `default_query_cache_capacity = 10`). It is filled in `Table.search`.
- Invalidation happens in one place: `Table._update_table()` (tinydb/table.py) ends with
  `self.clear_cache()`, and `Table.clear_cache()` does `self._query_cache.clear()`.
- Every public write goes through `_update_table`, so these trigger it:
  `insert`, `insert_multiple`, `update` (all three branches: callable/fields, by doc_ids, by cond),
  `update_multiple`, `upsert` (via `update` or `insert`), `remove` (by cond or doc_ids), `truncate`.
  `Table.__init__(persist_empty=True)` also calls it. `clear_cache()` itself is public.
  `TinyDB` (tinydb/database.py) forwards these via `__getattr__` to the default table;
  `drop_table`/`drop_tables` do not clear a cache, they forget the `Table` instances (`self._tables`).

## Check rule (check.py explore)
PASS iff the final answer
1. contains `table.py`, AND
2. contains `clear_cache` or `_query_cache`, AND
3. names at least 3 distinct methods among
   insert, insert_multiple, update, update_multiple, upsert, remove, truncate (word match, case-insensitive).

tasks/explore.truth.md on GitHub

tasks/review.truth.md

# review: planted defect and matching rule

The uncommitted change is the reference T4 patch (`Table.search(cond, *, sort_by, reverse, limit)`)
with one defect: the truncation guard is `if limit:` instead of `if limit is not None:`, so
`search(cond, limit=0)` returns ALL matching documents instead of `[]` (0 is falsy).
Everything else in the change is correct (missing-field docs last in both directions,
negative limit -> ValueError, cache keeps the plain result).

## Check rule (check.py review)
PASS iff the final answer (case-insensitive) matches at least one of:
- `limit\s*=?=\s*0` (e.g. "limit=0", "limit == 0")
- `limit\s+(of\s+)?(0|zero)\b` (e.g. "limit of 0", "limit zero")
- `\b(0|zero)\s+limit` / `zero[- ]limit`
- `limit` within 150 characters of `falsy|falsey|truthy|truthiness` (either order)
- `if limit:` together with `0` or `zero` within 150 characters
Decoy answers (generic review, or only the sort order) must FAIL.

tasks/review.truth.md on GitHub

Zrób to w poniedziałek

• Uruchom Claude Code 2.1.257 lub nowszy, żeby flaga force istniała, oraz 2.1.243 lub nowszy, żeby /tasks pokazywał model każdego subagenta.
• Zdecyduj, który plik chcesz: plik Explore dla samego eksploratora, blok env dla wszystkich subagentów.
• Zapisz Explore.md jako .claude/agents/Explore.md albo dołącz blok env do .claude/settings.json.
• Użyj pełnego identyfikatora claude-haiku-5-5, a nie aliasu haiku, zwłaszcza na Bedrock, Vertex lub Foundry.
• Rozpocznij sesję, poproś o przeszukanie bazy kodu przez agenta Explore, potem otwórz /tasks i odczytaj model oraz effort, na którym działał.
• Zachowaj Sonnet 5.5 lub Opus 5.5 dla każdego subagenta, który wykonuje złożone kodowanie agentowe, i nie ustawiaj FORCE, jeśli taki agent został przypięty celowo.
• Uruchom własne siedem zadań przez harness, zanim zaufasz naszej tabeli na swoim repo.
• Czytaj /usage jako „który skill lub agent zjada moją lokalną historię”, a nie jako prognozę limitu.

Pomiary

Licznik planu, odczytany ze zdarzenia rate_limit_event każdego uruchomienia headless, z rozdzielczością jednego punktu. Każdy blok jest ciągły, a nic innego nie dotykało konta. Okno tygodniowe pokazywało 67% w każdym ze 182 uruchomień.

Podsumowanie macierzy: 182 uruchomienia, cała macierz około 4,09 $ w cenie katalogowej, dwa punkty okna pięciogodzinnego i zero punktów tygodnia. Każdy blok niesie do punktu zaokrąglenia, więc jeden punkt Sonneta i jeden punkt Opusa są nie do odróżnienia. Zadanie o wąskim zakresie jest na tym liczniku niewidoczne na dowolnym z modeli. Strona wsparcia o względnym koszcie tury trzech modeli nie podaje żadnego mnożnika s9.

Na tle lokalnego podziału, na konfigurację:

Podział w /usage jest liczony z lokalnej historii na twojej maszynie s3. Jego wzór, odczytany z binarki klienta, to poziom modelu razy waga rekordu. Poziom modelu: Fable 10, Opus 5, Haiku 1, cokolwiek innego 3. Waga rekordu: tokeny z cache 1, wejście bez cache 10, tworzenie cache 12,5, wyjście 50. To lokalne wagi atrybucji klienta, a nie rozliczenie i nie dowód na to, jak serwer liczy twój limit. Widać rozbieżność: blok Haiku medium zdobywa 29,48, a blok Opusa 27,30, więc panel szereguje blok Haiku nad blokiem Opusa, podczas gdy prawdziwy licznik pokazał zero wobec jednego punktu.

Gdzie miesiąc logów jednego konta umieścił tokeny: 2631 uruchomień subagentów, 42 na Haiku 4.5, subagenci przy 48,1% wszystkich tokenów, Haiku 4.5 przy 0,27% ważonego kosztu oraz mediana launch promptu subagenta 47 117 tokenów, niezależnie od modelu.

Effort zmienia apetyt przy tej samej przyjętej funkcji (refactor): Haiku medium przeczytał 232 985 tokenów z cache w 13,7 tury, Haiku low 139 801 w 7,6 tury, Opus 87 612 w 8,3 tury.

Idź dalej

• Próg ceny 100k: powyżej 100 000 tokenów promptu wyższa stawka obejmuje całą prośbę, a nie nadwyżkę s8. To, czy tokeny cache wliczają się do 100k, jest odczytaniem tego autora, a nie przytoczoną regułą. Ma to znaczenie dla ekonomii kluczy API, nie dla licznika subskrypcji. Strona z cennikiem: s20.
• Tokenizator: ten sam tekst liczy się jako około 30% więcej tokenów w Haiku 5.5 niż w Haiku 4.5, a adaptive thinking jest domyślnie włączone w API, z effort jako dźwignią s12.
• Effort i tokeny na zadanie: niezależny test zmierzył około 162k tokenów wyjścia na zadanie Intelligence Index przy effort max i około 55k przy high effort dla wyniku 38 s7.
• Pełna konfiguracja praktyka, z jego liczbami i jego własnym zastrzeżeniem, że oszczędność bierze się z małego kontekstu głównego: s4.
• Otwarte zgłoszenie #100082, model z frontmattera subagenta ignorowany w niektórych buildach: s10.
• Otwarte zgłoszenie #100154, transkrypty subagentów zaniżają output_tokens, 461 wobec 15 575 dla tego samego zadania, rozliczenie po stronie serwera bez zmian s11. Ponieważ /usage mnoży wyjście przez 50 z lokalnej historii, to zaniżenie trafia do panelu.
• Nie testowaliśmy: Haiku jako modelu głównego (źródła przedstawiają Haiku 5.5 jako subagenta), mieszania modeli różnych dostawców przez bramkę (to nie ustawienia Claude Code), promptów z kontekstem 1M ani kredytów API dołączonych do planów Max (Claude Platform, a nie limit Claude Code).

FAQ

Czy ustawienie CLAUDE_CODE_SUBAGENT_MODEL przenosi Explore na Haiku?

Nie. Samo w sobie nie zmienia wbudowanych subagentów Explore i Plan s2. Dodaj plik agenta Explore albo flagę force.

Czy FORCE zepsuje moich subagentów na Opusie?

Stosuje jeden model do każdego subagenta, więc tak: agent, którego celowo przypiąłeś do Opusa, też zadziała na Haiku. Użyj pliku Explore, jeśli tani ma być tylko eksplorator.

Czy Haiku 5.5 wystarcza do pracy subagentów?

Dla tych siedmiu zadań o wąskim zakresie na jednym małym repo został przyjęty 70 razy na 70 na low effort i 70 razy na 70 na medium. Sama Anthropic zachowuje Sonneta 5.5 i Opusa 5.5 do złożonego kodowania agentowego s1, a różnica w Terminal-Bench 4.0 to 39,2% wobec 70,6%.

Ile limitu planu oddaje to przełączenie?

Dla zadania o wąskim zakresie mniej niż punkt licznika na dowolnym modelu: cała macierz 182 uruchomień ruszyła okno pięciogodzinne o dwa punkty, a okno tygodniowe o zero. Prawdziwy tydzień ucieka w głównym kontekście i w launch prompcie, który niesie każdy subagent.

Czy mogę wyłączyć thinking, żeby zaoszczędzić więcej?

Nie na tych modelach w Claude Code s21. Pozostaje effort jako pokrętło.