Stage 19 prerequisite: derive and pin the hold-out split; gold oracle passes 905/905
deriveSplitTests continues the mini walk past every document the mini derivation touched (including one truncated at the cap), so the split is document-blocked by construction; setup asserts zero overlap against the pinned mini corpus rather than trusting it. Hold-out: 3 sources, 55 docs, 582 tests - privacy_qa mini IS the complete upstream benchmark (194/194) and yields no hold-out, recorded in derivation.sources_without_split. Gates, run before any model call: gold oracle 905/905 on the normalized load (contractnli 276, cuad 311, maud 318) and 587/905 with --raw (maud 0/318), so normalization is load-bearing here too; db-level re-check on a2aj-holdout-lf.sqlite also 905/905. CR census reproduces the upstream signature on unseen documents: 18/18 maud files carry \r, the other two sources clean. Mini re-derives byte-identical to its pinned manifest (73 files). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | 15b15d2d |
| Stats | 6 files changed , +592 , -39 |
| Part of | Legal grounding and retrieval research |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-2d0e8bbb.md
from inside the repo you want the change in.