Stage 18 instrument fix: runners load the LF-normalized corpus; D3/D4/D8
Corpus text is normalized (CRLF -> LF) at file read in legalbench-rag-run.ts, which is now the one builder of the source db every LegalBench script reads (`--build-only`). The normalized build is a SEPARATE file (a2aj-mini-lf.sqlite): passage sidecars are keyed by source-db path + chunk params and NOT by source content, so sharing one path would have silently reused CRLF-built indexes over LF text. The corpus files on disk are untouched. Every runner now points at it; the ablate sweep keeps a `--raw-coords` escape onto the historical db so before/after is one code path over two corpora, and tags arm labels/receipt filenames/rows with the coordinate space. Grounding rows carry `coords: "lf"` and the resume key includes it, so a pre-fix (raw) receipt file can never be resumed into an LF run. D3: rerankPassages' `fallback` was dropped on the floor - now recorded per cell (`rerank_fallback`) and printed as a rate; a fallback cell ran the lexical order, not the reranker. D4: the flat "overall" mean hides source-mix artifacts on partial runs - the summary now prints the per-source cell mix (with an UNBALANCED warning) and a source-balanced macro mean beside the flat one. D8: doc-hit rates were printed on inconsistent denominators (grounded over answered, baseline over all ok rows) - both denominators are now explicit and the baseline doc rate is also printed on the answered subset. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | fab8e767 |
| Stats | 5 files changed , +184 , -44 |
| Part of | Legal grounding and retrieval research |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-328eb32b.md
from inside the repo you want the change in.