legalTextSkeleton: contract's two unbracketed enumerator dialects, gated by the ladder itself
Adds the forms the last commit measured and left open: the closing-paren tail
("a) ...", what a PDF extractor leaves when the opening bracket is lost) and
the dotted alpha/roman ("a. ...", "iv. ...", also Ontario's sub-paragraph
ladder). No new detector and no token blacklist -- the dialects are two
surface forms fed into the existing ladder, and the existing competition
rejects the collisions.
Gating, in order of strength.
1. Module boundary. The grammar lives in legalTextSkeleton.ts, which
sourceDocA2AJ.ts does not import (it imports sourceDoc and statuteSpine
only). compileA2AJSourceDoc -- the path the A2AJ laws corpus, the 225k-case
bulk corpus and the skeleton oracle gate travel -- structurally cannot
reach these forms. Pinned by test as well as by import graph.
2. Disjointness. Both dialects carry lowercase alpha/roman only, and every
section grammar in this file needs a digit or a container/schedule word, so
no dialect line can also be a section head. The dialects are consulted at
the LAST branch of the compile loop, after container, schedule, spine and
section matchers have all declined. Sections cannot move; only subsections
can appear. Tested directly as a projection equality.
3. The ladder is the filter. "Inc.", "No.", "v. Smith" and "s. 231" have the
same surface as a dotted enumerator; what they cannot do is run. A dialect
is admitted per document only when its markers contain a strictly
increasing run of >= 3 readings in one family, opening at value 1, with a
gap of at most 2 -- min-run gating with gap tolerance, the statuteSpine /
a2aj_structure device one level down, over readings from interpretationsOf
so the (h)->(i) alphabet/roman disambiguation is inherited unchanged.
Corpus decisions. Digits and capitals are excluded from the tail dialect: over
6,123 documents all 105 "N)" lines are already claimed by SECTION_INTEGER_RE,
and "A)" occurs once corpus-wide. The paragraph-side substance guard (median
>= 12 or mean >= 20 words) was NOT transferred: the newly-fired bodies run
median 17 / mean 15.8 words and 219 of them are genuine two-word list items
("iv. Essex.", "i. Substructure.", "b. damages;"), so that guard would reject
real ladders here. It is a decision-paragraph guard, not a clause guard.
Numbers, 6,122 documents (69 LegalBench-RAG agreements + 6,053 A2AJ laws over
16 sets):
- non-subsection projection (container/section/schedule label, depth, start,
end, heading) identical: 6,122 / 6,122. Delta sections: 0 in every set.
- compileA2AJSourceDoc blocks identical: 6,122 / 6,122.
- defined-term attribution identical: 6,120 / 6,122; the two changes move a
term from its section to the subsection that actually defines it
(sec4.1 -> sec4.1(a)), a refinement.
- new enumerator nodes: +8,249 over 229 documents (agreements +77,
LEGISLATION-ON +2,093, REGULATIONS-ON +4,580, BC +751, FED +654).
- agreements: subsections 2,347 -> 2,424 (contractnli 30 -> 76, cuad
475 -> 506); sections 2,101 unchanged; fallback docs 9 unchanged.
- gold-span containment on the 69 agreements: 88.62% -> 88.62% (unchanged).
The capability adds citable units; it does not move this bed's retrieval
alignment.
Precision. All 8,249 newly-fired lines were run through the SHARED detectors
-- citationKey.hasCitationInText, legalTextAnchors.extractAnchors (statute and
cite classes), docxStructuralLint.isExternalReference: zero flagged. Token
distribution is a ladder distribution (i 1961, ii 1948, iii 1011, iv 539,
v 300, vi 195 ...; a 249, b 247, c 116, d 57 ...). Zero "s." and zero "ss."
fired despite 111 and 33 candidate lines in the corpus -- the run gate rejected
every one, with no blacklist. A 40-row random hand vet found 40/40 genuine.
Measured false positives: 0.
Consumers: 302 tests across 20 suites pass (statuteSpine, legalTextSkeleton,
skeletonOracleDiff, legalAmendOps, legalConflictScan, localAssistantTools,
passageRetrieval, sourceDocA2AJ + 5 wiring suites, a2ajPassageSearch,
legalbenchRag, legalTemporalScan, legalTermDrift, slaWorkflow, localToolWiring,
documentOps). tsc clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | 74831c08 |
| Stats | 2 files changed , +246 , -1 |
| Part of | SourceDoc / DOCX engine: redlines, tracked changes, editing benchmark |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-bdcd53e6.md
from inside the repo you want the change in.