Stage 20 verdict: premise #1 falsified - checker family carries ~36% of grounding decisions
The program's founding premise ("the harness carries grounding decisions,
not the model family") rested on 8 cells over one item pair. Replicated on
289 banked compositions, 1,156 checker calls, no composer call:
within-family replicate 14/222 = 6.3% [3.8, 10.3]
archived vs fresh same 48/444 = 10.8% [8.3, 14.0]
cross-family 159/444 = 35.8% [31.5, 40.4]
paired difference +29.5 pp [23.2, 36.0]
Both same-family figures land inside the registered H17 0-13% floor, so
the instrument is behaving; the cross-family effect is ~5x that floor.
Direction is symmetric (73 fail-closed / 86 fail-open), not Stage 5's
11/12 fail-closed. On housing:0 -- the audited-insufficient reference --
the claude checker false-accepts 6/16 and codex catches it, so "the only
false accept required Codex in both roles" does not hold.
Aggregate accept rates over identical compositions match within 3 pp
(claude 61.9%, codex 59.0%). That is exactly how an 8-cell table reads as
family-invariant while 36% of individual decisions flip.
The deterministic tier is invariant by construction: 0 mismatches over
4,951 banked claims. But shadow-crossing its 67 all-clear rows gives
codex 83.6% vs claude 52.2% acceptance, and 8 claude calls on 4 rows
returned `unsupported` on answers whose every claim is a verified
verbatim quote. Logged as an open contradiction, not explained away.
Headline counts recomputed independently from the receipt bytes before
logging. Nothing retired on this run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | bdbab160 |
| Stats | 2 files changed , +88 , -5 |
| Part of | Legal grounding and retrieval research |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-f0f675ee.md
from inside the repo you want the change in.