Structure probes: the evidence that killed two naive detectors
Cases: ladder_broken was 99.85% detector artifact (1,988 of 1,994 - A2AJ text is left-flushed, quoted paragraphs are byte-identical to host paragraphs, adjacent-break counting cannot work in principle); the real hidden failure is endnote/authority lists scoring as perfect ladders (10% of a 1,127-doc cross-court sample, near-total for pre-1970 SCC). ALR paragraph_index vs naive v1 on 60 hand-verified docs across 16 courts: presence 95% vs 43%, flag precision 93% vs 31%. gt_truth.json is that hand-verified gold. Laws: the weak sets are four rendering conventions, not dialects; ALR SECTION_MARK_RE + three catalog-traced lookahead extensions + range expansion + named-heading recovery takes them from .54-.65 to .965-1.000 - and found ALR's own SK hyphen-gate under-fire (total refusal where .85 recall exists). Journals harvester feeds the once-vacuous grammar entries real extraction text. Four A2AJ corpus defects recorded (Sept-Oct 2003 BCCA line-collapsed rows). Verdict, both fronts: port ALR, delete v1, record precision not just recall. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | dc1bde91 |
| Stats | 12 files changed , +2374 |
| Part of | Legal agent-surface and context-compaction experiments |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-c6cc4583.md
from inside the repo you want the change in.