grounding: Stage 5 results - the harness carries the decision, not the model

↗ view on GitHub · Eli Ziff · 2026-07-30 · de59524e

144 cells (24 items x 2 composers x {control, tiered-same, tiered-cross}),
3 errors. H5 substantially supported; no falsification clause tripped.

The model-trust control answers cleanly: quote-anchored Claude
compositions were judged correctly on the audited sufficiency pair by
BOTH checker families (2/2 same-model, 2/2 codex-checked), Claude
checking rescued Codex compositions to 2/2, and the run''s only false
accept required Codex in BOTH roles (0/2 all-codex lane, which also
falsely rejected the sufficient row). Crossing the checker never
introduced a false accept; 11 of 12 same-vs-cross disagreements were
fail-closed and the twelfth fixed a false rejection. Codex control
target-F1 (0.40) beats Claude (0.29) - the crossing isolates
grounding-decision reliability from raw legal capability.

Deterministic tier at scale: 17 clears, all 17 re-audited genuine
verbatim under the lib''s real logic (the analysis script''s 2 crude-audit
alarms did not survive); the zero-checker-call path fired on three real
cells for the first time, all Codex compositions. Claude tiered''s four
non-audited rejections are all fail-closed partially_supported, two
repeating the audited weak-reference overreach shape on inspection -
recorded as fail-closed conservatism pending human audit (the one
conditional element of gate (a)).

Recommended experimental lane after five stages: quote-anchored tiered
composition with a Claude-family checker, composer either family; flag
stays off by default. Receipts archived with hash (stage5-h5.jsonl);
pre-registration 29bc7e6f.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
Repository eliziff/Beaver
Author Eli Ziff <eliasziff@gmail.com>
Authored
Parents 29bc7e6f
Stats 1 file changed , +64 , -2
Part of Legal grounding and retrieval research

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-de59524e.md from inside the repo you want the change in.

⬇ Download capture-commit-de59524e.md