F3: resume keys carry every dimension that changes what a cell means
legalbench-rag-grounding keyed cells on coords|model|effort|k|retriever|test_id. `arm` was written to the row but LEFT OUT of the key, and --coverage, --spec, --plain and --exclude-gold each change the arm - the prompt, and for --exclude-gold the evidence - while changing nothing the key could see. Those are exactly the flags the C1a/C1b/F2/F3 experiments ran under, so a resume could have declared one arm's cells done for another. Measured occurrences: zero (every existing receipt is raw-CRLF, and `coords` already separates those), but live. The sibling legal-grounding-experiment had the mirror-image hole: `arm` was in its key, `effort` was not, which already forced a Stage 13 ladder lane onto its own output file to stay separable. Both keys now live in src/lib/experimentReceipts beside the receipt guard: legalbenchRagCellKey coords|model|effort|arm|k|retriever|per_doc_cap|test_id legalGroundingCellKey model|effort|arm|checker|case_id|rank_policy Backwards compatible, deliberately: a missing field falls back to what the row's own labels prove it ran under, so pre-fix receipts stay resumable instead of being re-run wholesale. `arm` absent = required_slot; `per_doc_cap` absent is derived from the retriever label (product = n/a, passage:pool(...) = uncapped, +rerank = 24, otherwise 2 - the exact pre-fix semantics); `coords` absent = the raw-CRLF instrument, which still can never satisfy an LF cell. Verified on the real receipt store (read-only, nothing written): stage14 keys as ...|product|n/a|..., stage17/18 as ...|24|..., the fused-pool file as ...|uncapped|..., the plain-arm file as ...|plain|..., and no file has fewer distinct keys than non-error rows. Tests: 11 in src/lib/__tests__/experimentReceipts.test.ts, including all six arm labels distinct, every other dimension distinct, identical cells identical, and the legacy backfills. Full backend suite: 1427 passed (2 pre-existing docx/compare timeouts under parallel load, both pass in isolation, neither touches this code). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | a8cdd08c |
| Stats | 4 files changed , +246 , -18 |
| Part of | Legal agent-surface and context-compaction experiments |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-1d220fce.md
from inside the repo you want the change in.