F4: receipts are append-only in every harness, not just the ablation

↗ view on GitHub · Eli Ziff · 2026-07-31 · 9d4eafe6

Both grounding harnesses did `writeFileSync(output, "", "utf8")` on any
non-resume run, and legalbench-rag-grounding's default --output is a FIXED
path (stage14-lbrag.jsonl). That is the mechanism that destroyed
`stage18-retrieval-arms.jsonl`, whose sha256 is pinned in the experiment
log.

The guard added to legalbench-retrieval-ablate in d6813dd7 now lives in
src/lib/experimentReceipts and all three harnesses call it: --output is
honoured in every mode, and an existing file throws unless --force (or
--resume, which appends and skips done cells and is the one
non-destructive way to reopen a receipt). Resolved AFTER the --dry-run
return so a dry run is never refused, and before any model call.

Two holes closed while porting: --output was ignored entirely by ablate's
--rerank-arms mode (it built its path inline and truncated it), and
`--output` with no following path silently fell back to the default
instead of erroring.

Smoke-checked live: the grounding harness now exits with "refusing to
overwrite an existing receipt: ...stage14-lbrag.jsonl" before any model
call, and that receipt's bytes and mtime are untouched.

Tests: src/lib/__tests__/experimentReceipts.test.ts (default, --output,
clobber refusal on both paths, --force, --resume, malformed --output).
No scoring or retrieval semantics touched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
Repository eliziff/Beaver
Author Eli Ziff <eliasziff@gmail.com>
Authored
Parents 0fdc1e81
Stats 5 files changed , +137 , -35
Part of Evaluation harness: Beaver-CAN and LegalBench-RAG adapters

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-9d4eafe6.md from inside the repo you want the change in.

⬇ Download capture-commit-9d4eafe6.md