A contents page is an outline, not a span index: keep the inventory the gate refuses
MIN_HEAD_SPAN is right that a contents-only reading is not this document's
structure -- following one lands a reader on a page number. It is wrong that
the reading is worthless. Measured on LegalBench-RAG mini and the Stage 19
hold-out, the documents the gate protects compile 13-28 titled section heads
while their contents pages name 96-126 provisions, each with the page the
document prints for it.
So the contents reading is kept as a SEPARATE product. `AgreementSkeleton`
gains `outline` (a ContentsOutline, or null) and `outlineRefusal` (a typed
reason). `nodes` is untouched, by construction and by measurement: the reader
runs on the original text, outside the segmentation competition, and its
output is never merged in.
An entry carries no span. It has `label` (joinable to a node label), display,
heading, depth, parentLabel, the cited `page`, and `contentsLineStart` -- the
offset of the CONTENTS LINE, named so nobody can mistake it for the
provision's. A consumer that wants text goes through readSection or fails.
Detection reuses this module's own head vocabularies, so the outline cannot
drift from the span compiler, and cuts entries at the heads rather than at
line starts because the four extraction dialects in the corpus disagree about
where an entry ends but agree about where the next one begins. Refusal is
typed: no marker, no entries, too few, or no pages cited.
Identity, two ways over two corpora (receipts, not committed):
A2AJ English statutes 23,531 documents 0 differing node inventories
LegalBench-RAG corpus 714 documents 0 differing node inventories
digest sha256 A2AJ, HEAD module and this change alike:
6a5be60087df916d326c53ee111ca157231fe88d21a0f58222b2ac8de50afea6
digest sha256 LegalBench-RAG, both:
b379facb783ac0f16d49871f084c0d62db54440b00ed6473f54c63ad6ce9ee7d
The handed-over baseline digest-head.txt (b31fdc04...) does not reproduce from
HEAD -- 45 statutes differ between it and HEAD's own module, 6 of them
explained by 0fdc1e81. It is stale; the baseline above was re-derived.
Outline recovered: 17 of 69 mini documents (1,728 entries, 1,638 with a cited
page) and 22 of 55 hold-out (2,058 entries, 1,932 with a page). Refused: 84
documents with no contents marker, 1 whose contents entries are un-terminated
integers, and Altabancorp, which has an unmarked contents page an unanchored
scan would find -- not shipped, because the marker is the document saying so.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | 0fcc4aeb |
| Stats | 2 files changed , +436 , -1 |
| Part of | SourceDoc / DOCX engine: redlines, tracked changes, editing benchmark |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-271fbb1b.md
from inside the repo you want the change in.