Structure probes: the evidence that killed two naive detectors

↗ view on GitHub · Eli Ziff · 2026-07-29 · c6cc4583

Cases: ladder_broken was 99.85% detector artifact (1,988 of 1,994 -
A2AJ text is left-flushed, quoted paragraphs are byte-identical to
host paragraphs, adjacent-break counting cannot work in principle);
the real hidden failure is endnote/authority lists scoring as perfect
ladders (10% of a 1,127-doc cross-court sample, near-total for
pre-1970 SCC). ALR paragraph_index vs naive v1 on 60 hand-verified
docs across 16 courts: presence 95% vs 43%, flag precision 93% vs 31%.
gt_truth.json is that hand-verified gold. Laws: the weak sets are four
rendering conventions, not dialects; ALR SECTION_MARK_RE + three
catalog-traced lookahead extensions + range expansion + named-heading
recovery takes them from .54-.65 to .965-1.000 - and found ALR's own
SK hyphen-gate under-fire (total refusal where .85 recall exists).
Journals harvester feeds the once-vacuous grammar entries real
extraction text. Four A2AJ corpus defects recorded (Sept-Oct 2003 BCCA
line-collapsed rows). Verdict, both fronts: port ALR, delete v1,
record precision not just recall.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
Repository eliziff/Beaver
Author Eli Ziff <eliasziff@gmail.com>
Authored
Parents dc1bde91
Stats 12 files changed , +2374
Part of Legal agent-surface and context-compaction experiments

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-c6cc4583.md from inside the repo you want the change in.

⬇ Download capture-commit-c6cc4583.md