statuteSpine: the label-alone extension may extend a spine, never constitute one

↗ view on GitHub · Eli Ziff · 2026-07-31 · 0fdc1e81

Diagnosis first. The reported "zero subsections" in compileAgreementSkeleton
over LegalBench-RAG-mini is a measurement artifact: toBlock() maps every
SkeletonNode kind onto SourceDocBlock.kind = "section", so a count over
doc.blocks reports 0 subsections by construction. Counted over skeleton.nodes,
pushSubsection/openEnum fire 2,334 times on the same 69 documents (contractnli
30, cuad 479, maud 1,825, privacy_qa 0), and the section/fallback numbers in
the report reproduce exactly as blocks-of-all-kinds (422/1189/3138/17, 9 docs
below 3 starts). structuralChunkText therefore already consumes subsections.

The real defect the artifact hid is one level up. BARE_MARK_RE widened ALR's
SECTION_MARK_RE with a label-alone-on-line alternative (the measured _EXT that
lifted LEGISLATION-NS 0.795 -> 0.963). Agreement text spends that same shape on
centred page numbers, and a run of page numbers is monotone, same-arity,
document-spanning and starts early, so it clears every guard and wins the scope
competition outright. Measured: 10 of the 69 agreements drew a spine that was
mostly or entirely contentless, and because a winning spine owns section
detection in legalTextSkeleton, the spurious spine also suppressed the real
headings. The ALAMOGORDO agency agreement compiled to 39 sections with empty
headings and its (a)..(n) ladder scattered across four of them; it now compiles
to the 18 real "Section N." headings with the ladder under sec4.

Fix: when the winning spine has no substantive member at all, rerun the
competition over marks that carry same-line content. The extension keeps
working wherever label-alone provisions sit among ordinary ones.

Numbers. A2AJ laws, 960 documents / 16 sets / 39,371 provider section labels
(unofficial_sections_en gold): recall 0.8714, precision 0.9926, 49 zero-label
docs, 571 full-recall docs -- byte-identical before and after. LegalBench-RAG
mini human gold spans, share of gold spans contained in a single clause chunk
(t1600/o120): 88.33% -> 88.62% overall; cuad 96.56 -> 97.25, maud 77.84 ->
78.44, contractnli and privacy_qa unchanged. Skeleton nodes 69 docs:
articles 146 -> 146, sections 2,192 -> 2,101, subsections 2,334 -> 2,347,
documents below 3 chunk starts 9 -> 9 (unchanged). The section drop is the
removal of spurious page-number sections.

Consumers: statuteSpine.test, legalTextSkeleton.test, skeletonOracleDiff.test
(49), then legalAmendOps, legalConflictScan, localAssistantTools,
passageRetrieval, sourceDocA2AJ + wiring/fixtures/jsonl/native/sameline,
a2ajPassageSearch, legalbenchRag (195), then slaWorkflow, localToolWiring,
documentOps, legalTemporalScan, legalTermDrift (45) -- all pass. tsc clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
Repository eliziff/Beaver
Author Eli Ziff <eliasziff@gmail.com>
Authored
Parents 1ca6a238
Stats 2 files changed , +80 , -3
Part of SourceDoc / DOCX engine: redlines, tracked changes, editing benchmark

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-0fdc1e81.md from inside the repo you want the change in.

⬇ Download capture-commit-0fdc1e81.md