F1: one explicit per-doc cap for every arm, recorded on the row

↗ view on GitHub · Eli Ziff · 2026-07-31 · a40f46ee

`legalbench-rag-grounding.ts` set `perDocCap: 24` ONLY when a rerank model
was configured. Without one the lexical path fell through to
searchPassages' default of 2, and an injected pool (--pool-jsonl) took
`slice(0, k)` with no cap at all. The registered Tier C comparison runs
plain lexical / ctx / ctx+rerank / injected fused pool, so three of the
four arms would have run at three different caps, none of them declared.

Now one flag, `--per-doc-cap`, applied to the lexical path and the
injected-pool path alike (`capHitsPerDoc` in passageRetrieval applies the
same rule searchPassages applies internally - unit-tested for equivalence),
printed at startup and written to every row as `per_doc_cap` (null on the
product path, which has no cap).

SHIPPED DEFAULT 24, deliberately:
- it is the crowned config's value everywhere else - the ablation sweeps,
  legalbench-rerank-eval, and legalbench-dense-dump, which BUILT the
  injected pool sidecars, all use 24; a2ajPassageSearch uses it too;
- any cap >= k is equivalent to uncapped, so at the composed k (4 or 6) it
  is INERT: it changes nothing for the injected-pool arm, which was
  uncapped, and reproduces the uncapped behaviour for the lexical arms;
- at the k=48 rerank pool, where a cap does bind, 24 is exactly the value
  the crowned rerank config was measured on.
So the cap is now identical across arms while both registered behaviours
are preserved in the place each was registered.

Measured, deterministic, no receipt written (96 cells, t1600/o120/w16, LF
instrument, lexical char recall):

  k=6   cap 2  R=0.2123   cap 24  R=0.4524   cap 48  R=0.4524
  k=4   cap 2  R=0.2019   cap 24  R=0.3330   cap 48  R=0.3330
  k=48  cap 2  R=0.2332   cap 24  R=0.8332   cap 48  R=0.8937

maud alone moves 0.0210 -> 0.1672 at k=6 (8x): exactly the "gold
concentrates inside one 300 KB agreement" mechanism, and ~16x the +-0.015
composer band. This is the one fix in P0 that MOVES a number, and it moves
it in the arms that were silently mis-capped. Honest disclosure: an
injected pool drawn at k=48 for reranking is now capped at 24/doc where it
was uncapped, so a fused+rerank arm is not byte-identical to Stage 18's;
at k<=24 the injected path is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
Repository eliziff/Beaver
Author Eli Ziff <eliasziff@gmail.com>
Authored
Parents 9d4eafe6
Stats 3 files changed , +115 , -4
Part of Evaluation harness: Beaver-CAN and LegalBench-RAG adapters

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-a40f46ee.md from inside the repo you want the change in.

⬇ Download capture-commit-a40f46ee.md