95a11d72 | feat: add safe Word content controls | Eli Ziff | 2026-07-27 | ↗ GitHub |
2f0a3e3e | feat: make semantic Markdown the sole DOCX pipeline | Eli Ziff | 2026-07-27 | ↗ GitHub |
f054e6cc | feat: deterministic DOCX structural lint as a read-only library action | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body Implement the contract structural lint proposed in
docs/legal-skills-ecosystem-comparison.md: broken internal
cross-references, references to missing schedules/exhibits, literal
numbering gaps and duplicates, and duplicate or unused defined terms,
computed from word/document.xml instead of recalled by the model.
Every check abstains with an explicit note when the document does not
exhibit the convention it depends on (Word field numbering, detached
schedules, no quoted definitions), and the report is a receipt of what
was checked, resolved, skipped as external, and abstained from.
Exposed as library_lint_docx_structure beside the existing bounded
DOCX actions; read-only, so it is not a mutation tool.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
|
bd7235ab | fix: recover citations from truncated CITATIONS blocks | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body Live calls showed models sometimes drop the final "]" or the closing
</CITATIONS> tag while every citation object inside is well formed. The
final parse previously demanded a strict JSON array and discarded all
citations in that case, though parsePartialCitationObjects could already
recover them object-by-object for streaming previews.
parseCitationsWithDiagnostics now falls back to that recovery when the
strict parse fails or the close tag is missing; each recovered object
still passes normalizeCitation, so a truncated tail can never invent a
citation. Live harness (36 calls, gpt-5.5/5.4-mini/5.4-nano, old vs new
prompt arms): trimmed prompt 23/24, pre-trim 11/12, and the recovery
eliminated every truncation failure on both arms.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
|
4c9ba66a | docs: adopt the SourceDoc consolidation as P1.1a and index the eval plan | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body P1.1 will be executed as one consolidation: a single content-hashed
SourceDoc per fetched source, providers as compilers, consumers as
queries, each cutover gated on byte-identical parity over the
live-probed fixture corpus. Grounded in the three 2026-07-27 audits.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
|
c366efad | fix: real footnote numbering and labels in the DOCX viewer | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body Three rendering defects, all in how docx-preview's per-page model was
surfaced: fabricated 'Page N' badges from DOM index (duplicating the
document's own rendered footer), footnote references restarting at 1 on
every page, and w:footnoteRef markers dropped entirely so note bodies
had no labels. Plus a fourth found during verification: notes opened by
w:customMarkFollows symbols (*, **) were force-numbered, shifting every
real number by one.
New docxNotes.ts tags references and note bodies in the parsed model,
then numbers/links them continuously in the DOM with bidirectional
anchors, honouring custom marks and de-duplicating notes repeated
across pages. DocxView drops decorateDocxPages, trims render options
to non-defaults, and caches parseAsync per ArrayBuffer so remounts do
not re-inflate the zip. Verified against two real documents (239-note
and 13-note) matching their XML exactly; 117 frontend tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
|
de6a95ec | perf: shared lazy zip loader for the DOCX libs | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body Six libs value-imported jszip only to call loadAsync, putting its 34
modules in the boot graph. One lazy door (lib/zip.ts) now serves all
of them, matching the dynamic-import idiom documentOps already used.
Eager graph: 377 -> 327 modules (674 before this series).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
|
5fe26171 | refactor: one <CITATIONS> stream splitter for both chat transports | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body The marker-detection/tail-buffer algorithm that separates visible
prose from the hidden citations payload existed twice - once in
runLLMStream (cloud path), once in streamAnonymousChat (local path) -
the exact class of silent-drift duplication that has bitten before.
Both now consume createVisibleStreamSplitter, covered by boundary
tests (marker straddling chunks, <CIT...RUS> false-positive
reconstruction, per-iteration reset). Bridge backoff test gets a
bounded 30s wait + 45s test timeout; its detached write can starve
longer than 10s when suite workers saturate the CPU.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
|
2f78a7e4 | perf: query one SourceDoc instead of retokenizing per quote | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body Stage 3 of the SourceDoc consolidation. Every pinpoint-link consumer now
queries the compiled artifact: phrase search walks word postings built once
per document, a first query streams the text and stops early, and a resolve
shares one artifact across its handles.
Measured on the 2.26 MB Criminal Code shape: buildDirective 2737 ms -> 232 ms
per quote, appendLegalSourcePinpointLinks 693 ms -> 15 ms per source, a
64-handle evidence resolve 122 ms -> 4 ms per handle. backend/scripts/
bench-pinpoint.ts runs on either side of the change and prints the ratio.
Also fixes quotes carrying editorial alterations ("[T]he ...") losing their
link when the passage repeats: disambiguation now compares the resolved form
as well as the raw one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
|
c8873be6 | fix: malformed numeric entities no longer crash entity decoding | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body String.fromCodePoint throws on code points above 0x10FFFF, so provider
markup carrying garbage like � crashed structure parsing and
XML text decoding. Guard both sites the way legalSourcePresentation
already does: out-of-range entities pass through as raw text. Verified
live: 'a � b � c é d' now decodes with the
garbage intact and the valid entity converted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
|
5a105b9d | chore: unexport internal-only SourceDoc and presentation surfaces | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body knip-flagged: sourceDocRevision, sourceDocBlocksOfKind,
sourceDocQuoteSpans, SourceDocOrigin, SourceDocLocatorRange,
A2AJStructureSummary re-export, A2AJStructureView, UpstreamPdfLink.
All used only inside their modules; scripts import none of them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
|
53bd0dab | feat: styled DOCX rendering with page-number footer; drafting source ignores field-only chrome | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body docxMarkdown now emits named Title/Legal Table Text styles, bold
heading runs, and a page-number-only footer. documentOps and the ToA
bridge follow. extractDocxDraftingSource stops flagging review for
headers/footers whose XML carries no literal text - presence of a
page-number field is not content loss - so Beaver-generated DOCX
round-trips clean while text-bearing precedent chrome still warns.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
|
062a0144 | revert: restore upstream Mike editing rules; prompt defers to schemas | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body The fork's action-request taxonomy (and a carve-out patching around it)
replaced Mike's simple read-once-then-edit_document rules and broke
deterministic lint routing (caught live by the LIVE_E2E suite).
Restored verbatim from origin/main. ask_inputs prose that repeated the
tool's own schema is gone; the schema owns it. Verified live: lint
suite 2/2 and a full redline loop with a real model.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
|
f6990b99 | feat: native-markup SourceDoc engine replaces the legacy structure pipeline (P1.1a stage 4) | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body sourceDocNativeMarkup.ts compiles TNA Akoma Ntoso, CAP/CourtListener
casebody HTML, GOV.UK ET, GovInfo, and journal renditions into the one
SourceDoc artifact; legalSourceStructure.ts is deleted. Legacy behavior
is frozen in fixtures/nativemarkup/legacy-structure.json - a machine
capture over real provider payloads (343-entry lookup battery, receipt
payload_sha256 shapes) asserted byte-identical. Native anchors query
compiled postings instead of throwaway per-quote docs. Benchmarks
(bench-structure.ts, both trees): compile 1.3-1.6x, lookup batteries
2.0-2.6x, zero stage-3 regressions. Both browser gates pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
|
99eda63d | feat: deterministic attachment context, docxodus redline engine, tracked-edit resolution | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body Attached documents' extracted text is inlined into the turn
programmatically (20k chars/doc, 60k total; oversized docs keep the
pointer plus true size) so the model reads attachments directly instead
of round-tripping a tool call - the prompt sentences compensating for
the bare pointer are deleted. docxodus (WASM WmlComparer, MIT, no
install scripts) joins the tree as the transform-then-compare redline
engine, spike-verified in Node (native Deleted/Inserted revisions in
1.4s). Also lands the concurrent session's tracked-edit resolution
through the local document store and codex stream polish, grouped with
the tree fully green (tsc clean, 717 tests).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
|
c0edb265 | feat: deterministic text-ops tool for mechanical DOCX transforms | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body library_apply_text_ops: built-in registry of generic pure text
operations (case changes incl. title case, replace, sentence spacing,
quote/dash/ellipsis normalization, flag-only spell check) applied over
a once-resolved scope through the existing tracked-edit engine.
Spellcheck never mutates: en-CA dictionary by default, en-US only via
explicit BEAVER_SPELLING_DICTIONARY override. Drops the unused
docxodus spike dependency.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016yXxuitxX5h4wm1v88hnga
|
27f89160 | feat: weak-model DOCX robustness - recover instead of reject | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body Tolerant markdown parsing with deduplicated warnings replaces most
hard throws in the DOCX renderer; generate_docx field validation
reports every bad field in one recoverable error; prompt rules the
generator already enforces deterministically are deleted.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016yXxuitxX5h4wm1v88hnga
|
0db47d69 | feat: minimal-cluster tracked changes | Eli Ziff | 2026-07-27 | ↗ GitHub |
commit body applyTrackedEdits now splits each matched edit into minimal change
clusters via fast-diff semantic cleanup instead of collapsing to one
delete-and-reinsert span. Fixing 'paras 332-334' tracks a single deleted
'3'; a two-spot edit becomes two tiny independent w:del/w:ins changes
with the untouched middle left alone; word swaps stay whole words.
Content-control protection, atomic per-edit rejection, and the
change-consumer shape (one card per change) are preserved.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016yXxuitxX5h4wm1v88hnga
|
f5b74d72 | Skeleton-first context: compile legal structure into the SourceDoc plane | Eli Ziff | 2026-07-28 | ↗ GitHub |
commit body legalTextSkeleton.ts parses the native numbering of agreements and
statutes - ARTICLE/PART/DIVISION containers, Section N.NN and statute
"8. (1)" forms, (a)/(i)/(A)/(1) enumeration ladders, Schedules and
Exhibits - into SourceDoc blocks, so local Library documents share the
lookup/slice/quote plane that legislation and case law already use.
The ladder resolver is a TS port of Text-Fidelity's counter-stack
parse (parse_heading_ladder): strict increments across all readings
before level opens, so (h)->(i) disambiguation emerges from pass
order; forward jumps and mid-counter opens are tolerated as gaps;
restarts get @n occurrence suffixes; backward values are recorded
violations. Defined terms reuse docxStructuralLint's collector (now
exported, with romanToInt and isExternalReference).
Tool surface: library_outline returns the complete structural map
with lookup handles (real 141k-char credit agreement: 310 nodes,
~7.2k-token outline, 82 defined terms, Section 8.01 enumerated
(a)-(n) with nested (e)(ii)(A)-(D)); library_read gains a section
locator for span-scoped reads ("8.01", "Article VIII", "s. 8(2)",
"Schedule 7.01"). Compile cost: ~25ms. 15 unit tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bhP89cYSdEEjHNPnxacns
|
95e53f17 | Annotated tracked changes: reasons become anchored Word comments | Eli Ziff | 2026-07-28 | ↗ GitHub |
commit body applyTrackedEdits gains annotate mode: each edit's reason is rendered
as a real Word comment (commentRangeStart/End + commentReference)
spanning that edit's revision wrappers, with the comments part
created or extended (self-closed empty parts handled), content-type
and relationship registered, and ids continued from existing
comments. Genre contract: annotate requires a non-empty reason on
every edit - rationale-free markup is rejected, because a markup
without visible explanations is a clean draft (the failure that
zeroed LAB pilot task 02 for the reference harness).
library_revise_docx passes annotate through, pre-validates reasons,
reports comment_count, and auto-runs the structural lint on the new
version so markup deliverables get deterministic same-turn QA.
Verified end-to-end: evalValidators.checkDocxStructure reports 4
insertions / 3 deletions / 3 comments on a three-issue arbitration
markup; python-docx opens the package; range ids match the comments
part; accepted view reads correctly. 5 unit tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bhP89cYSdEEjHNPnxacns
|
69ab6b5f | Annotate mode measures rationale coverage instead of mandating it | Eli Ziff | 2026-07-28 | ↗ GitHub |
commit body Per review: requiring a reason on every edit was untested policy.
Genre safety already comes from the path (revise output is tracked
changes by construction); rationale-per-edit is now an A/B variable -
unreasoned edits apply without a comment and the receipt reports
edits_without_reason. Plan doc updated with the same framing plus the
mid-build addendum (reuse map, TFP resolver port, generalization
assets, measurement battery).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bhP89cYSdEEjHNPnxacns
|
ab09255c | Word comments are emphatically opt-in | Eli Ziff | 2026-07-28 | ↗ GitHub |
commit body Comments degrade Word performance on long documents and edit reasons
already reach the user via the tracked-edit receipt card, so the
annotate schema now actively discourages the flag: off by default,
set only on an explicit user request for in-document comments.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bhP89cYSdEEjHNPnxacns
|
4c919793 | shrink: trim redundant tracked-change comments | Eli Ziff | 2026-07-28 | ↗ GitHub |
ea39e713 | docxCompareVersions: tracked-changes redline between DOCX versions | Eli Ziff | 2026-07-28 | ↗ GitHub |
commit body compareDocxVersions(oldBytes, newBytes) marks the NEW document with
w:ins/w:del so Word's accept/reject round-trips both versions: block
alignment by normalized text (prefix/suffix trim + LCS, capped with a
typed whole-comparison abstention), gap pairing by edit-distance with a
similarity floor, then word-level clusters whose tokenizer mirrors the
reference implementation's quote_edits.py (word tokens with internal
apostrophes, punctuation separate, zero-gap merge, quote/dash/whitespace
equivalence - case folding deliberately dropped: case changes are real
redline content). Emission grammar, zip access, and normalization reuse
the existing docxTrackedChanges/zip/text machinery.
Everything the diff cannot honestly represent abstains with a stable
typed code (tables, content controls, headers/footers/notes, fields,
hyperlinks, pre-existing revisions, numbering changes, unrepresentable
deletions) and leaves the new version's content unmarked. 6 fixture
tests with raw-XML assertions; suite green; tsc clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bhP89cYSdEEjHNPnxacns
|
0521fca6 | Probe: manual-ink redlines are invisible or dangerous in every extractor | Eli Ziff | 2026-07-29 | ↗ GitHub |
commit body Struck-through deletions read back as operative text in raw extraction;
red-ink insertions are indistinguishable from normal text in mammoth
raw, mammoth HTML, and pandoc alike. Strikethrough survives only as
uninterpreted tags.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
13f3fd29 | Declare the docx dependency generate_docx already relies on | Eli Ziff | 2026-07-29 | ↗ GitHub |
docx@9.7.1 lived only in node_modules; a clean npm ci would break the
DOCX writer and most docx test fixtures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
21f7b3ea | docx/core: one OOXML kernel under the engines | Eli Ziff | 2026-07-29 | ↗ GitHub |
commit body The compare engine's header admitted it mirrored the tracked-changes
helpers because they were not exported; both engines, the supra fixer,
and the drafting source now share one parser pair, one tree-helper set,
and one backslash-tolerant zip access. Extraction only - the sole
genuine divergence (body node vs body children) is kept as two named
exports. Drafting-source test updated to the [^id] note contract its
extraction has emitted since the notes change.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
de7101ca | Pathology sniffer: know what a DOCX contains before reading it | Eli Ziff | 2026-07-29 | ↗ GitHub |
commit body Bounded raw-OOXML pass reporting auto-numbering, tracked changes,
comments, content controls, hyperlinks, text boxes, manual red/strike
ink, merged and nested tables, fields, embeddings, and literal
header/footer text - typed notes, never a throw. Generated fixture
matrix with expected-detection tests is the validation gate for the
extraction phases; real markup documents scan consistently with the
corpus census.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
0cca8efb | Numbering resolver: rendered labels from numbering.xml | Eli Ziff | 2026-07-29 | ↗ GitHub |
commit body Auto-numbered DOCX render their "2.1"/"(a)" labels only at display
time; no converter synthesizes them. This walks numPr (direct and
style-carried, basedOn chains, overrides, numStyleLink, isLgl) with
counters per level and formats decimal, letter, roman, and decimalZero;
bullets advance silently and unsupported formats leave a typed note.
Indexes mirror extractDocxBodyText's paragraph walk, pinned by tests.
400-document sweep: every numPr document labels cleanly, zero notes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
dea65332 | Stories layer: every text container addressable, accepted view byte-stable | Eli Ziff | 2026-07-29 | ↗ GitHub |
commit body extractDocxStories walks body, footnotes, endnotes, headers, footers and
text boxes as separate stories, descending w:hyperlink and w:smartTag so
their text stops being invisible to the main path. Paragraph text is the
accepted view; deleted runs stay on .runs so no reader has to reparse.
storiesBodyText is byte-identical to extractDocxBodyText on 9 fixtures
and 400 corpus documents (equal=400, mismatches=0); the hyperlink and
smart-tag fixes are the only deliberate divergences, fixture-proven.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
ffe9cb5a | Redline projection: struck text stops reading as operative | Eli Ziff | 2026-07-29 | ↗ GitHub |
commit body projectDocxRedline emits {++ins++}/{--del--}/{>>author: comment<<} for
native markup and the same markers with [ink] attribution for manual
red-strike redlines (sniffer's color threshold, byte-identical copy).
Precedence: tracked wins over ink, del over ins, struck over red.
Counts mirror scanDocxPathology exactly on both real markup docs
(78/85 and 68/0); the probe fixture's deleted rent figure and struck
indemnity clause - which mammoth returns as operative text - project
as {--...--}[ink]. 14 tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
f2490229 | Unicode traps join the pathology report | Eli Ziff | 2026-07-29 | ↗ GitHub |
commit body scanDocxPathology now counts what text can hide: bidi controls that
reorder the rendered line, zero-width characters that split words
invisibly, Latin words carrying Cyrillic or Greek homoglyphs,
private-use codepoints, C0 controls beyond tab/newline/CR, and
invisible math operators. Each class keeps up to five 36-char excerpts
with the trap named in place, so a reviewer sees where, not just how
many. Fixture generator grows matching trap fixtures; 28 tests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
c0d23447 | Bounds constants move into the docx kernel | Eli Ziff | 2026-07-29 | ↗ GitHub |
commit body docx/{pathology,stories,numbering,redline} imported
MAX_DRAFTING_DOCX_BYTES / MAX_DRAFTING_XML_ENTRY_BYTES from
docxDraftingSource while docxDraftingSource imports getZipEntry from
docx/core - a cycle that blocks docxDraftingSource from ever consuming
stories. The constants now live in docx/core.ts; docxDraftingSource
re-exports them so no other caller changes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
bcd07547 | Opt-in redline view; default reads point at it | Eli Ziff | 2026-07-29 | ↗ GitHub |
commit body library_read and read_document accept mode="redline" for DOCX: the
projectDocxRedline projection - {++ins++}/{--del--}/{>>author:
comment<<}, [ink] for strike/colour formatting standing in for tracked
markup - returned as a labeled JSON envelope with a marker legend,
counts, and the projection's own notes. Plumbing mirrors mode="drafting"
exactly (schema enum, dispatcher routing, per-mode turn-read dedup,
JSON ok-flag success check); edit paths untouched.
Default plain-text reads whose sniffer report shows tracked changes or
a likely manual redline now append one advisory to the 3i-1 notes:
"Struck or inserted text is invisible in this plain-text view; request
the redline view to see it." Clean documents carry neither markers nor
the advisory; the default text stays byte-identical throughout.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
3b0473fe | Reference, signal, and footnote-label families join the tables | Eli Ziff | 2026-07-29 | ↗ GitHub |
commit body references.json: 20 entries - supra/ibid token (one pattern, four
modules), both pure-reference dialects with the ref_pin composition as
defs, inline references, note references, signals (prefix x2, source,
aggressive, citation), history, editorial brackets, link attachment,
both short-form dialects (toa's [^]] respelled [^\]] - JS parses the
original as an empty class; compiled-identical in Python), sentence
and conjunction boundaries. footnote-labels.json: the engine's seven
core.py label grammars, with adapters.py's inline copy named in
provenance. 124 vectors green in both runtimes; discriminating pairs
asserted per dialect sibling - including the corrected pure-ref pair
(DOLA subsection splitter-only vs Oakes-ibid toa-only; the r 11.10
string matches neither lineage and is kept as a shared negative).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
dbb7b355 | structure: capture short complete ladders; end journal page circularity | Eli Ziff | 2026-07-30 | ↗ GitHub |
commit body Full-sweep vet drove three changes, each verified against the vet docs
plus an old-vs-new differential over the 1,862-doc reservoir (seed 47):
1. paragraph_index extensions (structure_ref.py): complete short [1]..[N]
bracket ladders (contiguous from 1, doc <= 6KB, header-sized prefix by
absolute <=1200 chars OR ratio <=0.5, at least one >=30-word prose
paragraph) are real structure - 17/29 of the sampled none-queue were
exactly this shape (short orders / oral reasons / costs rulings).
Substance guard gains mean>=20 and max>=30 arms so "I agree."
concurrence tails cannot sink a ladder. Short hypotheses never enter
the primary scope competition (a tail [1]..[4] list in [1969] SCR 277
shadowed the real ladder until ordered last). Cascade span now credits
the final paragraph body bounded by 2x median length.
Verification: 23/23 targeted docs (17 misses -> paragraphs; truly-none
letters/stubs stay none), 0/1862 regressions, 54 none->paragraphs
reservoir conversions (~2.9% of sample).
2. paragraph_scope_narrow flag (sweep.py): short docs (<4KB) flag only
below 0.30 - 6/12 sampled flags were header+signature share on correct
short judgments; all four true mis-selections (quoted-list hijack,
order-enumeration hijack, gap-splice chimera, collapsed-fragment lock)
still flag under the new rule, verified per doc.
3. Journal branch (sweep.py): page "recovery" scoring deleted. The
[page N] markers are rendered BY US from article_pages/page_map_json
in the journals database; detecting them was testing our own output
against our own input. Journals now record structure kind
provider_metadata and are scanned for grammar-entry rates only.
Intentional divergence from the ALR reference port is documented in the
structure_ref docstring; --parity now reports exactly the newly-captured
docs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
6ae6d330 | laws sections: dot-form labels, fr surface names, pseudo-label convention | Eli Ziff | 2026-07-30 | ↗ GitHub |
commit body Laws vet decomposed the 17.5pp full-corpus recovery gap; the real
detector gaps are now closed, measured before/after per set (fixed sets
scanned in FULL, controls reservoir-sampled seed 47):
1. SECTION_MARK_RE_EXT accepts the NT/PE dot-form drafting convention
("1. There is established...", "2.(1) In this section") and NB's
markdown-heading decimal rules ("### 61.01 Enforcement of Orders"):
LEGISLATION-NT rec 0.029 -> 0.985 prec 0.754 -> 0.989
LEGISLATION-PE rec 0.037 -> 0.986 prec 0.834 -> 0.987
REGULATIONS-NT rec 0.102 -> 0.996 prec 0.272 -> 0.734
REGULATIONS-PE rec 0.109 -> 0.991 prec 0.355 -> 0.770
REGULATIONS-NB rec 0.837 -> 0.994 prec 0.898 -> 0.943
2. named_heading_labels keeps the heading's surface language (fr oracles
key 'Annexe I'; the old fr->en translation never matched anything) and
_NAMED_HEAD_RE learns Formulaire:
REGULATIONS-YT rec 0.915 -> 0.993 (prec 0.639 -> 0.611, oracle-
omission set; dual-emission variant measured worse at 0.582 and was
rejected)
3. sweep law branch: single-pseudo-label oracles (Order / Ordonnance /
Proclamation - 1,218 REG-FED + 582 REG-NL unsectioned instruments)
are a provider naming convention, not a recovery target; recorded as
single_instrument, excluded from recovery scoring.
Controls unchanged or better: LEGISLATION-FED rec .887->.902 prec
.987->.994, ON .947->.964/.983, BC identical, REGULATIONS-FED
.897->.942/.910->.934, REGULATIONS-NL rec identical prec -0.004.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
c824f2b4 | journals: page map is the structure authority, not marker re-detection | Eli Ziff | 2026-07-30 | ↗ GitHub |
commit body The [page N] markers in article text are rendered by the journals
database export from its own page map (article_pages, in page_order).
journalSourceDoc now walks that map and locates each known label's
marker line, instead of regex-discovering numeric-looking markers and
joining anchors through a label-keyed Map.
What the ordered walk carries that discovery could not, verified by an
old-vs-new differential over all 18,595 articles in the real database
(18,574 byte-identical; 21 differ, every difference an addition or
anchor correction, zero blocks lost):
- 40 non-numeric page rows across 17 articles ("PDF 1", "-5".."-1"
front-matter) now produce page blocks; the \d{1,5} regex dropped
them.
- Repeated labels (4 articles) get their own per-row anchors; the Map
collapsed both page38 blocks of article 9202 onto pdf page 2.
lookupLegalSourceDoc gains a final alias pass-through: a locator no
grammar recognizes resolves through block aliases ("PDF 1" -> found,
anchor page=1), while numeric locators keep their normalized
requestedLabel bytes (receipt hashes unchanged; the frozen
legacy-structure.json recording still passes byte-identical).
Duplicate labels answer ambiguous, not a guess.
Suites: journalArticles, sourceDocNativeMarkup, sourceDocA2AJ,
localAssistantTools, legalSourceLinks, providerPdfConsumers,
localDocumentsEvidence all green.
Integration plan item 1 (task #28).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
3501d7ab | product spine: short-ladder capture + dot-form section fallback (TFP-vet ports) | Eli Ziff | 2026-07-30 | ↗ GitHub |
commit body Ports the full-sweep detector fixes (structure_ref.py dbb7b355 /
6ae6d330) into sourceDocA2AJ.ts, closing the ~3%-of-cases no-paragraph-
anchors gap and the NT/PE zero-recovery gap in the product.
Paragraph spine (mirrors dbb7b355 exactly):
- complete short [1]..[N] bracket ladders (contiguous from 1) become
last-resort hypotheses, ordered after every full scope so a tail
[1]..[4] list can never shadow a real ladder; accepted only when the
ladder IS the document (<=6KB, header-bounded start absolute <=1200 OR
ratio <=0.5, at least one >=30-word paragraph);
- substance guard becomes median>=12 OR mean>=20 OR max>=30 so "I
agree." concurrence tails cannot sink a ladder.
The cascade span credit stays sweep-side only: it feeds flag metrics,
and the product's last paragraph legitimately runs to EOF.
Laws text-derived fallback spine (section map stays primary):
- FLAT_SECTION_EXT_RE (dot-form "1. There is established...", "2.(1) In
this section", markdown-heading "### 61.01") runs ONLY when the plain
grammar yields no spine. A merged regex measurably corrupts spines:
Ontario enumerates paragraphs inside sections in the same shape and
the first cut lost 6/64 sections on a2aj-regs-on-oreg267-03; the typed
fallback provably cannot touch any doc that had a spine.
- the leading-subsection splice accepts the label's trailing dot so
"2.(1)" keeps its first subsection.
The 6ae6d330 named-heading fr-surface fix has no product surface (the
flat spine has no named-heading blocks); it stays benchmark-side until
the Phase 4 detect/ suite decides whether named units join the spine.
Differential over the seed-47 reservoir (1,862 cases) + 866 laws docs
(fixed sets full-sampled, controls included), scratchpad ts_diff:
- cases: 98.55% exact TS==Python parity on (number, offset) spines; 54
old->new none->paragraph conversions - the same 54 the Python
reservoir differential reported; 0 regressions; all 27 residual
mismatches classified pre-existing tie-break drift (the known
monotone-scope divergence queued for Phase 4 reconciliation), none
port-introduced.
- laws recovery (fallback spine vs corpus oracle): LEGISLATION-NT
0.0023 -> 0.8203, LEGISLATION-PE 0.0015 -> 0.8775, REGULATIONS-NT
0.0015 -> 0.6246, REGULATIONS-PE 0.0185 -> 0.8287, REGULATIONS-NB
0.8242 -> 0.9042; controls BC/FED/ON byte-identical (the fallback
never engaged).
Suites: sourceDocA2AJ (+4 pinning tests), sourceDocFixtures (oreg267-03
byte-identical recording), sourceDocSameLine, sourceDocNativeMarkup -
68 passed; tsc clean.
Integration plan item 3 (task #28).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
f96adec0 | native markup: compile the provider structure we discarded (survey findings 2-4) | Eli Ziff | 2026-07-30 | ↗ GitHub |
commit body The structural-richness survey (c35d45e3) found three providers handing
us structure the compiler threw away. All three now compile, under one
receipt-schema bump.
- CAP casebody (xml_harvard): star pagination
(<a id="p336" data-label="336" class="page-label">) becomes native
page blocks - INLINE, no text break, since star pages land
mid-sentence and the rendered text is frozen; footnote asides
(<aside data-label="N" class="footnote">) become native fn-N blocks.
The 372us335 fixture goes from 5 heuristic paragraphs to +17 pages
+20 footnotes, text byte-identical. compactOpinion needed no flip:
it already preferred xml_harvard for structure - the defect was that
the compiler could not read it, and now it can.
- TNA: lvl_N levels compile to native section blocks (cleanSectionId
learns the level/lvl prefix). The eat-2025-1 fixture gains 24
sections on a doc that indexed section:0. New nativeMarkupCitedRefs
reads the <ref uk:canonical uk:type> cited-authority markup -
deduped, first-appearance order - and TNA fetch/rehydrate carry it as
PublicLegalDocument.citedAuthorities; the fetch tool payload surfaces
cited_authorities (capped 60).
- GOV.UK ET: hidden_indexable_content HTML now reaches the compiler as
markup (title/description ride as escaped paragraphs) instead of
being tag-stripped first, so <p>[N] boundaries inform the heuristic
spine.
Receipt schema v1 -> v2 (mike.provider_legal_evidence.v2, handle and
store path v2): payload hashes cover same-kind context, so v1 SECTION
hashes on lvl-bearing docs can no longer re-verify; v1 handles are
refused by version with a typed recapture error (surfaced verbatim to
the model - it names the recovery), never a misleading integrity error.
Proven in the new recording gate: legacy-structure.json (the deleted
engine's frozen output, untouched) now asserts the enrichment
invariants - rendered text byte-identical, every legacy block survives
verbatim, and every legacy PARAGRAPH lookup replays with an identical
payload hash - while native-structure-v2.json freezes the enriched
compiler byte-exactly (283 lookups across 4 fixtures).
Suites: sourceDocNativeMarkup (11), publicLegalSources,
publicLegalSourceIntegration (+ typed v1-refusal test),
publicLegalSourcePdfFallback, docxEvidenceCitations, sourceDocFixtures,
providerPdfConsumers, legalSourceLinks, localAssistantTools - all
green; tsc clean.
Integration plan item 4 (task #28).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
0da8b4b1 | Wire canonical SourceDoc parser across legal sources | Eli Ziff | 2026-07-30 | ↗ GitHub |
ebe0be43 | citations: one shared in-text detector in citationKey.ts | Eli Ziff | 2026-07-30 | ↗ GitHub |
commit body Retires the recorded debt. `CITATION_IN_QUERY` in a2ajPassageSearch was a
bespoke in-consumer regex for spotting citation-shaped substrings of a
free-text query. The inventory found a real detector already in the tree:
citatorExcerpts' CITE_TOKEN, calibrated on 3,000 random citator edges and
a strict superset of CITATION_IN_QUERY (same neutral / [year]-reporter /
CanLII alternatives, plus volume-reporter-page and "(1985), 48 C.R. (3d)
226" first-instance reporters).
So CITE_TOKEN is promoted, not the query regex: citationKey.ts - the pure,
db-free corpus-identity module - now also exports `citationsInText` (matched
substrings with offsets) and `hasCitationInText`. citatorExcerpts and
a2ajPassageSearch both consume it; CITE_TOKEN and CITATION_IN_QUERY are
gone. Detection is now superset-or-equal for the passage lane: more
citation shapes reach the citation_lookup short-circuit, and residualTokens
blanks them out of the bm25 side by span.
Deliberately not folded in, and now recorded as such: the neutral-citation
parsers in canliiUrls.buildCanliiCaseUrl and legalSourceLinks
.answerCaseCitations, which need year/court/number capture groups and the
wider court-slug charset the CanLII route table gates - a different
contract, and merging them would drop the 9-character and hyphenated
datasets that table carries.
citationKey.test.ts pins the scanner: neutral cites, the French twin
"2015 CSC 5", [year] reporter cites with periods, CanLII ids, verbatim
offsets, alternation order, plain text, and the two reporter shapes the
excerpt classifier depends on (regression pin). No model calls, no network,
no real databases.
npx vitest run over citationKey, citatorExcerpts, a2ajPassageSearch,
a2ajPassageWiring, retrievalGate, caselawCitator, legalSourceLinks:
7 files, 79 tests passed, existing tests unmodified. tsc --noEmit clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
|
533b5dbd | Adopt one canonical legal structure engine | Eli Ziff | 2026-07-30 | ↗ GitHub |
0fdc1e81 | statuteSpine: the label-alone extension may extend a spine, never constitute one | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body Diagnosis first. The reported "zero subsections" in compileAgreementSkeleton
over LegalBench-RAG-mini is a measurement artifact: toBlock() maps every
SkeletonNode kind onto SourceDocBlock.kind = "section", so a count over
doc.blocks reports 0 subsections by construction. Counted over skeleton.nodes,
pushSubsection/openEnum fire 2,334 times on the same 69 documents (contractnli
30, cuad 479, maud 1,825, privacy_qa 0), and the section/fallback numbers in
the report reproduce exactly as blocks-of-all-kinds (422/1189/3138/17, 9 docs
below 3 starts). structuralChunkText therefore already consumes subsections.
The real defect the artifact hid is one level up. BARE_MARK_RE widened ALR's
SECTION_MARK_RE with a label-alone-on-line alternative (the measured _EXT that
lifted LEGISLATION-NS 0.795 -> 0.963). Agreement text spends that same shape on
centred page numbers, and a run of page numbers is monotone, same-arity,
document-spanning and starts early, so it clears every guard and wins the scope
competition outright. Measured: 10 of the 69 agreements drew a spine that was
mostly or entirely contentless, and because a winning spine owns section
detection in legalTextSkeleton, the spurious spine also suppressed the real
headings. The ALAMOGORDO agency agreement compiled to 39 sections with empty
headings and its (a)..(n) ladder scattered across four of them; it now compiles
to the 18 real "Section N." headings with the ladder under sec4.
Fix: when the winning spine has no substantive member at all, rerun the
competition over marks that carry same-line content. The extension keeps
working wherever label-alone provisions sit among ordinary ones.
Numbers. A2AJ laws, 960 documents / 16 sets / 39,371 provider section labels
(unofficial_sections_en gold): recall 0.8714, precision 0.9926, 49 zero-label
docs, 571 full-recall docs -- byte-identical before and after. LegalBench-RAG
mini human gold spans, share of gold spans contained in a single clause chunk
(t1600/o120): 88.33% -> 88.62% overall; cuad 96.56 -> 97.25, maud 77.84 ->
78.44, contractnli and privacy_qa unchanged. Skeleton nodes 69 docs:
articles 146 -> 146, sections 2,192 -> 2,101, subsections 2,334 -> 2,347,
documents below 3 chunk starts 9 -> 9 (unchanged). The section drop is the
removal of spurious page-number sections.
Consumers: statuteSpine.test, legalTextSkeleton.test, skeletonOracleDiff.test
(49), then legalAmendOps, legalConflictScan, localAssistantTools,
passageRetrieval, sourceDocA2AJ + wiring/fixtures/jsonl/native/sameline,
a2ajPassageSearch, legalbenchRag (195), then slaWorkflow, localToolWiring,
documentOps, legalTemporalScan, legalTermDrift (45) -- all pass. tsc clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
a8cdd08c | Reference grammar kernel: one owner for what a provision reference looks like | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body legalAmendOps and docxStructuralLint had each grown their own copy of the
reference shape; legalTextSkeleton counts references through the lint's
copy. legalReferenceGrammar now owns PROVISION_REF/FR_PROVISION_REF,
compactLabel(Fr), joinLocator and isExternalReference verbatim, and both
consumers import from it (joinLocator and isExternalReference are
re-exported from their old homes so no importer's surface moves).
Extraction is verbatim; behaviour is DEMONSTRATED unchanged, not assumed -
digest differential HEAD vs this commit, all identical:
- 16 EN+FR amending acts (legal-generalization-corpus): full AmendOp
object digests + target/newLabel/afterChild labels
- 69 LegalBench-RAG mini documents: skeleton node labels+spans,
crossReferences summary, defined terms
- 400 real DOCX: full lint findings+notes digests (18,384 references,
5,090 resolved, 1,732 external, 317 docs with findings)
59 existing tests pass, tsc --noEmit clean.
New on top of the move: findProvisionReferences, a FREE-TEXT detector the
anchored dialect cannot be (measured on the mini corpus: PROVISION_REF
fires on 9,989 maud spans, 2,035 of them a bare provision word with no
label; 69% bare on contractnli). It requires a non-empty label and adds
roman container numbering - "Article VIII" occurs 612x in maud, 7.7% of
that source's literal references, against 5 "Section <roman>" spans
corpus-wide. Known deliberate misses, each with its number, are in the
module header.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
bdcd53e6 | legalTextSkeleton: contract's two unbracketed enumerator dialects, gated by the ladder itself | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body Adds the forms the last commit measured and left open: the closing-paren tail
("a) ...", what a PDF extractor leaves when the opening bracket is lost) and
the dotted alpha/roman ("a. ...", "iv. ...", also Ontario's sub-paragraph
ladder). No new detector and no token blacklist -- the dialects are two
surface forms fed into the existing ladder, and the existing competition
rejects the collisions.
Gating, in order of strength.
1. Module boundary. The grammar lives in legalTextSkeleton.ts, which
sourceDocA2AJ.ts does not import (it imports sourceDoc and statuteSpine
only). compileA2AJSourceDoc -- the path the A2AJ laws corpus, the 225k-case
bulk corpus and the skeleton oracle gate travel -- structurally cannot
reach these forms. Pinned by test as well as by import graph.
2. Disjointness. Both dialects carry lowercase alpha/roman only, and every
section grammar in this file needs a digit or a container/schedule word, so
no dialect line can also be a section head. The dialects are consulted at
the LAST branch of the compile loop, after container, schedule, spine and
section matchers have all declined. Sections cannot move; only subsections
can appear. Tested directly as a projection equality.
3. The ladder is the filter. "Inc.", "No.", "v. Smith" and "s. 231" have the
same surface as a dotted enumerator; what they cannot do is run. A dialect
is admitted per document only when its markers contain a strictly
increasing run of >= 3 readings in one family, opening at value 1, with a
gap of at most 2 -- min-run gating with gap tolerance, the statuteSpine /
a2aj_structure device one level down, over readings from interpretationsOf
so the (h)->(i) alphabet/roman disambiguation is inherited unchanged.
Corpus decisions. Digits and capitals are excluded from the tail dialect: over
6,123 documents all 105 "N)" lines are already claimed by SECTION_INTEGER_RE,
and "A)" occurs once corpus-wide. The paragraph-side substance guard (median
>= 12 or mean >= 20 words) was NOT transferred: the newly-fired bodies run
median 17 / mean 15.8 words and 219 of them are genuine two-word list items
("iv. Essex.", "i. Substructure.", "b. damages;"), so that guard would reject
real ladders here. It is a decision-paragraph guard, not a clause guard.
Numbers, 6,122 documents (69 LegalBench-RAG agreements + 6,053 A2AJ laws over
16 sets):
- non-subsection projection (container/section/schedule label, depth, start,
end, heading) identical: 6,122 / 6,122. Delta sections: 0 in every set.
- compileA2AJSourceDoc blocks identical: 6,122 / 6,122.
- defined-term attribution identical: 6,120 / 6,122; the two changes move a
term from its section to the subsection that actually defines it
(sec4.1 -> sec4.1(a)), a refinement.
- new enumerator nodes: +8,249 over 229 documents (agreements +77,
LEGISLATION-ON +2,093, REGULATIONS-ON +4,580, BC +751, FED +654).
- agreements: subsections 2,347 -> 2,424 (contractnli 30 -> 76, cuad
475 -> 506); sections 2,101 unchanged; fallback docs 9 unchanged.
- gold-span containment on the 69 agreements: 88.62% -> 88.62% (unchanged).
The capability adds citable units; it does not move this bed's retrieval
alignment.
Precision. All 8,249 newly-fired lines were run through the SHARED detectors
-- citationKey.hasCitationInText, legalTextAnchors.extractAnchors (statute and
cite classes), docxStructuralLint.isExternalReference: zero flagged. Token
distribution is a ladder distribution (i 1961, ii 1948, iii 1011, iv 539,
v 300, vi 195 ...; a 249, b 247, c 116, d 57 ...). Zero "s." and zero "ss."
fired despite 111 and 33 candidate lines in the corpus -- the run gate rejected
every one, with no blacklist. A 40-row random hand vet found 40/40 genuine.
Measured false positives: 0.
Consumers: 302 tests across 20 suites pass (statuteSpine, legalTextSkeleton,
skeletonOracleDiff, legalAmendOps, legalConflictScan, localAssistantTools,
passageRetrieval, sourceDocA2AJ + 5 wiring suites, a2ajPassageSearch,
legalbenchRag, legalTemporalScan, legalTermDrift, slaWorkflow, localToolWiring,
documentOps). tsc clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
bdbab160 | Structure detection: segmentations compete, and a contents page is not one | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body The cross-reference resolver's ceiling was never its grammar; it was the
skeleton's section inventory. Every structural grammar in legalTextSkeleton
keys on a LINE START, and seven of the seventeen maud merger agreements
have no line starts to key on: their extraction joined each page into one
line, mean line length 226-310 characters against 98-183 for their
well-lineated siblings. Those seven detect 13-33 sections where the others
detect 78-111, and all seven are refused.
compileAgreementSkeleton now compiles the document under more than one
segmentation and lets the document choose. Recovery is offset-exact by
construction - the first space of an internal run becomes a newline and the
rest of the run stays, so the text is the same length character for
character and differs only in whitespace; the SourceDoc, the defined terms
and every span are still built from the ORIGINAL text whichever reading
wins. Nothing about any grammar or guard changes: discoverNodes applies
headingLike, the integer ceiling, spine ownership and the ladder to a
recovered line exactly as to a real one.
Two guards, because the first cut of this was wrong in two ways and the
corpus said so:
A reference may not endorse a provision minted out of ITSELF. Offer a line
start before "... See also Section 9.99 for notices." and that prose
becomes a head, the reference resolves onto it, and the reading that
invented it scores higher - a selector rewarding a detector for
hallucinating exactly the provision the document lacks. Targets beginning
inside the reference span do not count.
A hypothesis must produce heads that span the document. Space-run recovery
reveals a table of contents preferentially (a contents line is padded with
runs, a body heading is not), and 80-99 contents entries resolve to each
other beautifully. Head span over the 69 documents: 0.0077, 0.0079,
0.0083, 0.0122 for contents-only inventories, then 0.1237, then 0.30-1.00.
Gate at 0.05, in the 10x gap.
MEASURED AND REJECTED, recorded in the module so it is not retried blind:
offering a line start wherever a head grammar could begin recovers the
single-space extraction dialect and four more documents, and is unsound. A
merger agreement's definitions index is a list of "'Balance Sheet Date' has
the meaning set forth in Section 6.16(a)" - each entry becomes a head that
real references elsewhere resolve onto. CAI International compiles 272 heads
against roughly 100 real ones and its endorsement score RISES. Endorsement
cannot police a hypothesis that mints the provisions being sought.
legalCrossReference gains the reach gate the same measurement exposed, and
it catches a defect that predates any of this: Acacia Communications
compiles 85 contents heads with NO recovery at all, resolves 89 references
onto them, and reports integrity 1.00. Every target sits at 0.01-0.02 of the
document; every document whose targets are real provisions reaches 0.16 or
beyond. A graph with no reach has resolved into a contents page and now says
so. integrityThreshold 0 still yields the raw census.
CENSUS, 69 LegalBench-RAG mini documents, zero model calls, question-blind
(before -> after): documents refused 24 -> 19, references resolved 2,428 ->
3,891 (+60%), accepted references that miss 11.2% -> 10.6%. Detected (9,234)
and external (1,720) are unchanged, which is the control: no reference
grammar moved. Per source, refused: maud 8 -> 5, cuad 7 -> 5, contractnli
3 -> 3, privacy_qa 6 -> 6; maud resolved 2,085 -> 3,494, cuad 320 -> 374.
Of the five maud documents still refused, three are refused BECAUSE of this
work rather than in spite of it: Acacia, Boingo and Anworth have no visible
body structure at all, and the reading that made them look healthy was
their contents page.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
191c512f | The single-space extraction dialect, recovered by its antecedent | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body The rejected hypothesis from the previous commit, retried with the guard it
was missing. Offering a line start before any head-shaped token minted a
section out of every entry in a definitions index - "'Balance Sheet Date'
has the meaning set forth in Section 6.16(a)" - and CAI International went
to 272 heads against roughly 100 real ones.
What separates a lost line break from an ordinary space is what precedes it,
not what follows. A heading begins after the END of the previous line, so in
a single-space join the character before the lost newline is a sentence
terminator. "set forth in Section 6.16" is preceded by "in", and that is the
whole difference. The guard collapses the pathology exactly: CAI holds at 26
heads, not 272.
Corpus numbers, mini (before -> after this commit): documents refused 19 ->
16, references resolved 3,891 -> 4,414, accepted references that miss 10.6%
-> 9.1%. At Home Group 33 -> 96 heads and integrity 0.39 -> 0.90;
BioTelemetry, already accepted, 0.75 -> 0.93 and 296 -> 434 resolved;
ABILITYINC 0.33 -> 0.92; ASIANDRAGON 0.00 -> 0.75. No document went from
accepted to refused, and detected (9,234) and external (1,720) are still
unchanged from the 2d903c82 baseline - no reference grammar has moved
through any of this.
Whole round against the recorded baseline: refused 24 -> 16, resolved 2,428
-> 4,414 (+82%), miss rate 11.2% -> 9.1%. Per source refused: maud 8 -> 4,
cuad 7 -> 3, contractnli 3 -> 3, privacy_qa 6 -> 6.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
54be6001 | Head vocabulary gets one owner, and the holdout says the recovery generalizes | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body The recovery carried its own copy of the head words. That is the drift
legalReferenceGrammar was extracted to end, one module along: the first time
a jurisdiction, era or house style adds a container word to CONTAINER_RE,
a duplicated vocabulary in the recovery silently stops following it and the
detector quietly loses exactly the documents the new word was added for.
CONTAINER_WORDS / SCHEDULE_WORDS / SECTION_WORDS / DECIMAL_LABEL are now
named once and consumed by both the per-line matchers and SENTENCE_JOIN_RE.
The terminator class is widened to closing brackets and guillemets for the
same reason - the guard is about the ANTECEDENT, not a punctuation dialect.
Behaviour-identical on the mini corpus (16 refused / 4,414 resolved / 9.1%).
GENERALIZATION, the number that matters more than the mini corpus. The
LegalBench-RAG holdout split is 55 documents (17 contractnli, 20 cuad, 18
maud) derived by skipping every document the mini derivation touched, so it
is disjoint by construction and nothing in this round was tuned on it. Run
once, question-blind, baseline 285b86f7 -> HEAD:
documents refused 14 -> 9 (maud 9 -> 5, cuad 3 -> 2, contractnli 2)
references resolved 3,879 -> 5,150 (+33%; maud 3,368 -> 4,520)
accepted refs miss 17.2% -> 10.5% (maud 17.8% -> 10.3%)
detected 12,025 and external 2,008 UNCHANGED
Same direction, same shape, similar magnitude as the corpus the thresholds
were placed on, and the miss rate falls while 27% more references are
accepted. The mechanisms are extraction facts rather than drafting
conventions - a run of spaces is a lost line break, a heading follows the
end of a line - which is why they carry.
Census script takes XREF_SPLIT / XREF_CORPUS so the two splits and a
baseline worktree run the identical instrument.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
559b3841 | Navigation surface: a page map with both numbers, and edges a model can follow | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body The deterministic structure work got substantially stronger today and stayed
almost entirely unreachable. A model could read a whole document, Ctrl+F it,
read one section by handle, and see a flat rendered outline. It could not ask
for a page, walk to a sibling, or follow a single cross-reference -- so 4,414
resolved references were invisible to the only consumer that could use them,
and a contents page citing "Indemnification ... 47" was a dead end.
THE PAGE MAP. The engine already detects printed page labels
(universal-legal-pdf-engine `_assign_printed_page_labels`, header/footer
region, ambiguity left unresolved as a diagnostic), and
compileLegalPdfSourceDoc already lays both numbers onto the text plane --
`anchor: page=<physical>` with the printed label in `aliases`. Nothing needed
porting. The loss was downstream: the renderer prints `printed || physical`
as one `[page N]` marker, and extractLocalDocument kept only `.text`, so
every consumer saw one number and could not tell which sense it was.
extractLocalDocument now builds the map where the artifact is still in hand.
Recovered-from-markers is the fallback for text whose artifact is not (journal
bodies, A2AJ reports) and is labelled `source: "markers"`, with a numeric
marker carried under BOTH readings because the renderer genuinely collapsed
them -- claiming one would invent provenance.
Page requests resolve against both schemes. Front matter printed i-viii makes
"1" name two different sheets; a table of contents means the second and a
viewer means the first, so that request is REFUSED as ambiguous and the caller
qualifies it (`printed:1`, `pdf:1`). Ranges resolve endpoint by endpoint and
take the span by position, because a range across a numbering change is not
arithmetic on either label.
library_find gains `pages`: "47", "12-18", "3,5,9", "printed:iv - printed:2".
The filter runs on offsets AFTER the match so every `at` stays a document
offset that library_read offset= accepts -- scoping narrows where you look,
never renumbers what you find -- and the internal cap is raised first, since
applying max_results before the filter returns nothing whenever the leading
hits sit outside the scope.
library_links is new: stand at a provision and get its ancestors, siblings,
children, the references it makes and the references made to it, each as a
handle library_read accepts. Without a section it returns the reference census
and the hubs. A whole-document abstention is surfaced as `abstained` with its
note, because zero edges and "this skeleton is too thin to address" are
different answers and only one of them means the document has no references.
Two defects the tests found, both fixed here:
- "overlaps the page" is not "printed on the page". A node's span runs to the
next heading, so the last section of page 1 laps onto page 2 by the marker
line alone. pageSections splits `starts` from `continuedFrom`.
- a rendition whose provider compiler has not landed carries no blocks; that
is "no pages", not a crash inside a read tool.
extractLocalDocument's return gains `pages` (additive; the existing shape
assertion is updated). No existing behaviour changes: section reads, windowed
reads, unscoped find and outline are untouched.
22 unit tests plus one end-to-end tool test over a mocked artifact whose front
matter is printed "i". tsc clean; localAssistantTools, localPdfLookup,
legalCrossReference and legalTextSkeleton suites pass (69 tests).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
e4ede555 | Register the navigation surface in the structure round | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body Build order A/B assumed a navigator that did not exist. Records what landed,
the two defects the tests caught, and what is still open.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
271fbb1b | A contents page is an outline, not a span index: keep the inventory the gate refuses | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body MIN_HEAD_SPAN is right that a contents-only reading is not this document's
structure -- following one lands a reader on a page number. It is wrong that
the reading is worthless. Measured on LegalBench-RAG mini and the Stage 19
hold-out, the documents the gate protects compile 13-28 titled section heads
while their contents pages name 96-126 provisions, each with the page the
document prints for it.
So the contents reading is kept as a SEPARATE product. `AgreementSkeleton`
gains `outline` (a ContentsOutline, or null) and `outlineRefusal` (a typed
reason). `nodes` is untouched, by construction and by measurement: the reader
runs on the original text, outside the segmentation competition, and its
output is never merged in.
An entry carries no span. It has `label` (joinable to a node label), display,
heading, depth, parentLabel, the cited `page`, and `contentsLineStart` -- the
offset of the CONTENTS LINE, named so nobody can mistake it for the
provision's. A consumer that wants text goes through readSection or fails.
Detection reuses this module's own head vocabularies, so the outline cannot
drift from the span compiler, and cuts entries at the heads rather than at
line starts because the four extraction dialects in the corpus disagree about
where an entry ends but agree about where the next one begins. Refusal is
typed: no marker, no entries, too few, or no pages cited.
Identity, two ways over two corpora (receipts, not committed):
A2AJ English statutes 23,531 documents 0 differing node inventories
LegalBench-RAG corpus 714 documents 0 differing node inventories
digest sha256 A2AJ, HEAD module and this change alike:
6a5be60087df916d326c53ee111ca157231fe88d21a0f58222b2ac8de50afea6
digest sha256 LegalBench-RAG, both:
b379facb783ac0f16d49871f084c0d62db54440b00ed6473f54c63ad6ce9ee7d
The handed-over baseline digest-head.txt (b31fdc04...) does not reproduce from
HEAD -- 45 statutes differ between it and HEAD's own module, 6 of them
explained by 0fdc1e81. It is stale; the baseline above was re-derived.
Outline recovered: 17 of 69 mini documents (1,728 entries, 1,638 with a cited
page) and 22 of 55 hold-out (2,058 entries, 1,932 with a page). Refused: 84
documents with no contents marker, 1 whose contents entries are un-terminated
integers, and Altabancorp, which has an unmarked contents page an unanchored
scan would find -- not shipped, because the marker is the document saying so.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
9deeee42 | The two page schemes are two questions, not two readings of one | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body Eli, on the ambiguity refusal I had built: "no. let model decide which one to
use. these are separate things to call."
He is right, and the refusal was me inventing a problem. resolvePage now
resolves in ONE scheme chosen by the caller. Unqualified, digits are the PDF
page -- the scheme every PDF has -- and anything else ("iv", "A-3") can only
be a printed label; `pdf:` and `printed:` say so explicitly. The `ambiguous`
status is gone, and a miss now names which scheme it searched.
Also corrected, Eli again: "there can be printed page numbers even without a
table of contents. a ToC is just one way to navigate using printed page
numbers." The model-facing description had been written as though the printed
label existed to serve a contents page. It is the number ON THE SHEET, and it
is how the record gets cited -- a pinpoint in a brief, an index, an exhibit
stamp, a transcript. A contents page is one consumer among several, and
teaching the model otherwise would have narrowed when it reaches for it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
256b21dc | One address grammar across read/find/links, plus head and tail | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body The navigation tools each grew their own scope parameters, so the same place
had three names and every new way to narrow cost another parameter on every
tool that should support it. They now share one grammar:
8.01 · Article VIII · Schedule 7.01 structural (bare = the default)
pdf:52 · printed:47 · p.47 · page 3 the two page schemes, named
off:12000 a raw window
`at` carries it on library_read, library_find and library_links. The old
`section` / `page` / `pages` / `offset` parameters still work and are marked
deprecated in their descriptions rather than removed, so nothing calling the
old shape breaks while the A/B runs.
HEAD AND TAIL. `from: "start" | "end"` on library_read, over ANY addressed
span -- document, page or provision. Execution pages, schedules and the
closing words of a clause live at the end, and reaching them previously meant
probing the length and doing arithmetic the caller could not verify. This is
the coding-agent convention Eli asked for, applied to spans rather than lines.
ORIENTATION. library_outline now carries the page map: how many pages, which
schemes address them (pdf / printed / both), the first and last, and how many
sheets have a printed label that differs from the PDF page, with an example.
A page number is unusable until the caller knows which schemes exist, and the
outline is where a model already looks first.
Two bugs the tests caught. `parseAddress` matched "p" before "printed", so
"printed:1" parsed as page "rinted:1" -- longest alternative first, with a
word boundary so "part 2" stays structural. And the `at` fallbacks used `??`
against a helper that returns "" rather than undefined, so the address was
silently ignored on every tool; `||` throughout.
53 tests green across localAssistantTools, legalDocumentNavigator and
localPdfLookup; tsc clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
ecf89fde | Legislation scoping: the flag is not inert, so decide it per feed and thread it where a caller knows | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body Re-measured the differential the CompileSkeletonOptions docstring rested on:
compile every A2AJ English statute with the segmentation recovery on and off
and compare node inventories. It is NOT byte-identical. 45 of 23,531 statutes
read differently (2,923,700 nodes on, 2,924,267 off, 837 nodes of absolute
difference; the Criminal Code alone moves 10,861 -> 10,979). The docstring's
node total is exactly the OFF reading, so what it recorded as a differential
between constructions looks like two runs of the same one. Corrected in place:
scoping legislation out is load-bearing, not ceremony.
That also explains the frozen digest baseline: scratchpad/digest-head.txt IS
the recoverExtraction:false digest -- reproduced from this tree byte for byte
-- while the skel-digest.ts beside it compiles with the default. Same 45.
The flag does not mean "is this legislation". It means "may this text have
lost its line breaks to an extractor", so the question per consumer is which
FEED reaches it. An Act uploaded as a PDF is extraction output like any other;
an A2AJ consolidation is not.
legalAmendOps.applyAmendOps (:740, :957) THREADED. It serves the statute
dialects and the contract dialect, and only the caller knows the source's
provenance. ApplyAmendOptions.recoverExtraction now forwards to both
compiles -- the applied text is the source with splices, so one flag
governs both -- and through consolidateAmendment. Proven caller:
scripts/ollama-amend-loop.ts, whose cases are A2AJ consolidation text,
passes false. Contracts do not move: the default is prior behaviour, and
two new tests pin both readings.
chat/slaWorkflow.ts:84 LEFT ON. Its input is extractLocalDocument, the
PDF/DOCX lane. Scoping recovery out there would degrade an uploaded Act
exactly as it degrades an uploaded merger agreement.
legalTermDrift:197, legalConflictScan:163, legalTemporalScan:178 LEFT ON.
All take a {name,text} stack, and every production caller fills it from
the local Library. No authoritative feed reaches them, so a flag with no
caller would be speculative configuration; each call site now names the
lane instead.
legalCrossReference:190 SEAM ALREADY EXISTS. CrossReferenceOptions.skeleton
is the threading point -- pass a skeleton compiled with
recoverExtraction:false and the competition cannot run. Documented there;
no second flag.
Finding, not changed: passageRetrieval:429 scopes the A2AJ clause lane out by
construction, but a2ajPassageWiring.test.ts verifies that lane's spans with
clauseChunkText, which still compiles with recovery on. Oracle and lane are
built differently; the test passes only because the fixture has no space runs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
f7035bc1 | Delete the DOCX text fallback: a second extractor is a second character plane | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body Eli: "we should not have a docx text fallback. if the main docx call doesn't
work, we need to fix that instrument, not silently fallback."
The fallback was worse than redundant. `applyTextOpsToDocx` resolves every
edit scope against `extractDocxBodyText` and nothing else, so a document that
fell through to mammoth was READ on one character plane and EDITED on
another: every offset the reader returned pointed somewhere else in the
writer. Nothing surfaced it, because falling back looks exactly like success
-- and the address grammar now being wired into the edit layer would have
inherited the same silent divergence.
Measured before removing it (rule 2): 400 documents sampled at stride 28
across the 11,293-file corpus, zero empty and zero throwing. It was carrying
nothing. A typed refusal naming the failure replaces it, so a real extraction
defect arrives as a bug report instead of as a wrong-plane edit.
Parser version bumped to 2 -- the cache is content-addressed on parser
identity, so the old fallback's output cannot be served for the new contract.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
397b84f0 | Edits address places the same way reads do | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body The read layer names a place with `at: "8.01"`; the edit layer made the model
supply from_text/to_text -- exact document text, retyped -- to name a place it
already had a handle for. That is the highest-risk thing it does: a
fabrication or a whitespace drift silently scopes the wrong span, and the
model had no way to say "this clause" without reproducing it.
`scope: {kind: "at", at, follow?, depth?}` takes the same grammar as
library_read: a provision and everything under it, a page, or a clause plus
what it references. Nothing is retyped and the address is verifiable.
Layering: the address layer resolves, the op engine executes. Resolution
needs a skeleton and a page map; execution needs offsets, so the handler
resolves against the PINNED version's text and hands docxTextOps a
`{kind: "spans"}` scope. docxTextOps gains no structural dependency.
This only works because both layers project the DOCX with
extractDocxBodyText and there is no second extractor -- an offset resolved
for reading is valid for editing. That invariant is now load-bearing, which
is what makes deleting the mammoth fallback a prerequisite rather than
housekeeping. A resolved-empty scope throws instead of widening to the
document, and an `at` scope on an unextractable DOCX refuses.
Raw offsets are deliberately rejected as an edit scope: an edit names a
provision or a page, never a bare character position.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
146ceb3f | docx-edit-bench v1: reusable document-editing benchmark + A/B registration | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body A surface-agnostic benchmark for "can a model edit an arbitrary span of a
real document": 27 tasks over 11 fixtures, tasks as data, checkers as a
library, tool surfaces as configuration. Nothing in src/ knows about
MIKE_NAV_SHAPE, so the next surface is measurable without editing it.
Fixtures are built, not committed: six prose documents rendered through
the product's own markdown-to-DOCX path (a 25k-char agreement down to an
unnumbered letter, two of them page-marked), and five pathology documents
built with the DOCX packager reusing the shapes in the existing
docx-pathologies fixtures - auto-numbering that lives only in
numbering.xml, a signing-limit table, a real tracked deletion plus a
manual strike/colour redline that still reads as operative, comments,
parallel EN/FR with a bilingual table, OCR quote/dash/spacing damage, and
footnotes off the body plane. Identity is the sha256 of the extracted
body text; DOCX bytes are not reproducible across builds.
Validity is established before any model call, and both halves
separately. Task validity: 27/27 tasks carry a reference solution that
applies cleanly and passes the checker. Outcome validity: 27/27 reject
every wrong result and 0 site checks have never been observed failing -
wrong results come from hand-written near misses, two automatic probes
(the untouched document; a partial regeneration that satisfies the
positive half while destroying the tail), and one synthetic sensitivity
probe per guard that damages exactly what that guard protects. The
whole-document-rewrite shortcut is blocked by a zero budget for
destroying original lines the reference solution keeps.
Registration of the first consumer - the legacy/address navigation A/B -
is appended to docs/legal-grounding-experiments-2026-07-30.md and is
frozen by this commit. It records the served schema of both arms, the
edit-scope leak found in the legacy arm during setup and since fixed by a
concurrent session, and the description leaks left unpatched because they
are part of the surface under test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
e0820eac | docx-edit-bench: resumable campaign runner; resolve tsx from the backend workspace | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body run.ts spawned its child through require.resolve("tsx/cli") from the benchmark
tree, which has no node_modules; resolve it against backend/ instead.
campaign.ts runs the full matrix (tasks x surfaces x replicates), skips cells
already present in receipts.jsonl so an interrupted run continues, and takes a
lane split so two shells can share one output directory.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
fae74a2f | docx-edit-bench: report the failure census, not just the score | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body Adds three sections the deliverable turns on: a tool-error census per
surface (an address that would not resolve and a retyped string that was
not in the document are different diseases), the misquoted locating
strings themselves, and the shapes of address arguments actually passed.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
08389cf2 | docx-edit-bench: drive progressive tool disclosure from the runner | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body Arm B now ships 10 resident tools and defers 19 behind describe_tools.
The provider adapter freezes its tool list when a call starts, so a domain
opened mid-call cannot become callable inside it; the runner ends the call
at the disclosure and continues in a new one, replaying the exchange. The
replay cost is counted. A surface that defers nothing never restarts, so
one path serves both arms.
A call to a tool the surface has not served is refused by the runner rather
than executed - the handlers dispatch on name alone, so without that guard
a guessed name would make the deferral measure as free.
Two arm-agnostic task fields: resident_route_exists (true for all 27 - every
reference solution is a literal substitution the resident text-op tool can
execute, which bounds what this bed can say about deferral) and
alternative_route_domains. Task version 1 -> 2; checks unchanged, self-test
still 27/27 solvable and 27/27 rejecting every wrong result.
Registration amended (2026-07-31a) with the redefined arm, the measured
48.5% first-request schema saving under the sealed flag rather than the
60.5% of the unsealed catalogue, and the seven superseded receipts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
f906d7d1 | docx-edit-bench: --trace prints every failed run in full | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body Task, arm, which checks fired, and the whole tool sequence with arguments.
A schema recommendation that cannot be traced to a line here is a
preference, not a finding.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
e4fe1f94 | docx-edit-bench v1 result: address grammar is 19% cheaper, not more correct | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body 108 runs, 27 tasks x 2 surfaces x 2 replicates, codex:gpt-5.6-sol, neutral
prompt identical across arms.
Correctness: 81% vs 79% excluding floor tasks - one run in forty-eight, and
the within-arm replicate floor is 1/27 cells. UNDECIDED, not a win. The arms
fail exactly the same four tasks.
Cost: 56,149 vs 69,281 total tokens per run (-19.0%), while the address arm
makes MORE tool calls and takes MORE wall clock. Two sources: first-request
schema 13,282 vs 25,809 bytes, and 126 vs 276 document characters retyped
into arguments.
Four of the address arm's five new affordances went unused: library_find.at
was empty on every call, follow/depth never left their defaults across 54
runs, library_links was called 3 times, and read from=end never once. The
one that was used, scope.at, carries the whole benefit.
55 misquoted locating strings, from three causes the tools report with one
message: wrong line separator, post-edit text passed into a find scope, and
context strings spanning table cells. Six schema recommendations follow, each
traceable to runs.
Also recorded: memo-against-agreement is defective - the model found a third
misstatement the task did not anticipate and was scored for damage; headline
reported with and without it. Two instrument defects: opened_never_called was
per-tool not per-domain (fixed, 12 of 22 disclosures bought nothing), and
ask_inputs is served but unimplemented in this runner.
Adds the beaver-address-no-disclosure ablation surface.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
5eba804e | docx-edit-bench: additive task sets, and plumbing for real-world fixtures | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body v1's tasks, checks and fixtures are frozen so its published result stays
comparable. New tasks go in tasks-<set>.jsonl and the loader tags each task
with its set; selectTasks filters on it. Tasks also carry jurisdiction and
practice_area so breadth can be reported rather than asserted.
Adds a third fixture family, 'real', for documents cut from real sources.
It packs each source line as one paragraph verbatim rather than through the
markdown renderer, because real legal text has ALL-CAPS headings and '1.'
and '(a)' at line starts that markdown silently eats and renumbers. Every
real fixture must carry provenance - source URL, named licence, a
redistributable verdict, the required attribution, what was modified, and a
contamination note - and the manifest records it. A fixture whose licence
cannot be established does not ship.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
6e7078c5 | Compile structure once, and let text in | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body MEMOIZE THE COMPILE. `compileAgreementSkeleton` had 20+ call sites in
backend/src and nothing memoized it -- `library_find` compiled the same
document TWICE in one call, and the coding-shape Grep compiled one per
matching document. The extracted text beside it has been cached for ages, so
the string was free and the structure was not.
The key is (sha256(text), id, recoverExtraction), because the compile is not a
function of text alone: `id` becomes `doc.id`, and the recovery flag changes
the node inventory on 45 of 23,531 A2AJ statutes. Graphs memoize against the
SKELETON via a WeakMap -- resolution depends on the skeleton, and an entry
dies with the thing that keyed it, so no size policy is needed.
Sharing is safe: nothing in backend/src, backend/scripts or the tests mutates
a returned skeleton. The real cost is memory -- a shared SourceDoc keeps its
lazy token array (~14 bytes per source character) alive -- so the cap is 8.
IDENTITY PROOF, 47,062 compiles: the whole A2AJ statute corpus re-derived
under both constructions with the cache active reproduces both known
digests exactly -- b31fdc04... with recoverExtraction:false, 6a5be600...
with the default. A cache that changed output would be a bug, not a win.
LATENCY, at contract scale (12 hold-out documents, median 37,796 chars):
a turn's structural work goes 130.9 ms -> 8.1 ms, 94% saved. Stage 21
measured arm B +1,464 ms slower for statistically identical answers, and
arm B compiles strictly more per turn. That penalty was recomputation.
LET TEXT IN. A `.txt` or `.md` was rejected at three hard gates before any
parser saw it, all from one allowlist pair -- so the plain-text parser added
earlier was unreachable. Both allowlists now carry the plain-text types, with
content types so a `.md` is served as text/markdown rather than octet-stream.
Decoding does NOT normalise line endings. Rewriting CRLF is the same defect
that silently corrupted a quarter of a benchmark for five stages: every offset
a reader returns has to index the stored bytes.
VIEWER. Plain text previously fell through to the PDF renderer, which cannot
open it. `TextView` renders Markdown through the same react-markdown +
remark-gfm pair the assistant and tabular views already use -- one Markdown
implementation, not two -- and keeps `.txt` verbatim in a <pre>, because a
transcript is read against its own line structure.
124 tests green across seven suites; backend and frontend typecheck clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
b9c27219 | docx-edit-bench: require an interaction class on v2 tasks | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body From docs/deterministic-word-actions-catalog.md, imported by the legal-skills
survey: Class A is a direct UI action, B a bounded deterministic tool the
model invokes but never authors, C preview-then-confirm because intent is
ambiguous, D judgment that cannot be inferred from text or formatting. The
survey's rule is that only Class D deserves to stay a prompt.
The benchmark's corollary, now enforced by the loader: a task is a legitimate
model-evaluated item only at Class C or D, and a Class A or B task must be
labelled a floor task, because a deterministic tool should perform it and the
model's only job is routing to that tool. v2 tasks must also declare a
jurisdiction and practice area so breadth is reported rather than asserted.
v1 is exempt and frozen.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
f15c3895 | docx-edit-bench: pool campaigns and restrict to matched tasks | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body --out takes a comma-separated list so a later ablation can be compared with
the campaign it ablates, and --matched restricts to the tasks every surface
actually ran, printing how many receipts that excluded. Averaging over a
ragged matrix would compare surfaces on different work.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
d8d88215 | Pre-bake the landmark statutes: 14.2s of compute becomes a file read | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body Eli: "we should definitely do some pre-caching of the ITA and the criminal
code; those are important documents and they are worth pre-baking for their
graphs etc because that's heavy computation but exactly where I think having
a pre-made cache will pay off."
He is right, and the in-memory memo does not touch this case. It fixes
repetition WITHIN a process; a landmark statute compiled cold on a laptop pays
in full at every restart. These are also precisely the documents a model must
navigate rather than read, because reading them is not an option.
Measured, baking eight federal acts:
Income Tax Act 7,318,921 chars 30,812 nodes 20,763 edges
skeleton 676ms graph 12,328ms
Criminal Code 2,262,608 chars 10,979 nodes 5,737 edges
skeleton 218ms graph 518ms
+ CBCA, BIA, PIPEDA, Copyright, Trademarks, Competition
14.2s of compute, 27 MB on disk
Served: Income Tax Act cold compile 13,444ms -> baked read 658ms (20x);
Criminal Code 793ms -> 197ms (4x).
CORRECTNESS, checked rather than assumed: node inventory, edge list and graph
counts are all digest-identical between a baked read and a cold compile, and
the rehydrated SourceDoc's index is rebuilt at full size (61,624 entries for
the ITA). That last one is the trap -- `SourceDoc.index` is a Map, so a naive
JSON round trip yields `{}` and every lookup silently answers "not found".
The sidecar therefore stores the JSON-safe inventory and rebuilds the doc
through `createSourceDoc`, which is O(blocks) and does not tokenize. `tokens`
is a non-enumerable lazy accessor that structuredClone drops and cannot
re-attach, so persisting a SourceDoc in any form is off the table.
Content-addressed on the text, so a consolidation update simply misses and
re-bakes -- no staleness to manage, no invalidation to get wrong. Versioned
so a compiler change can never be served from a stale bake. A read that fails
for any reason falls through to a real compile: correctness never depends on
a hit, only speed does.
Nothing is wired to consume the sidecars yet -- that is a deliberate next
step, not an oversight, since the read path should be measured before the
product depends on it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
bf64065f | Arm B serves pre-baked structure too | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body Eli: "Put all of the features in the B arm. that's what I asked for."
Fair. I built the sidecars and then left them unconsumed, which made arm B
the whole bet minus its newest piece.
The library handlers now resolve structure through `documentStructure` /
`documentGraph`, which read a pre-baked sidecar in the address arm and compile
synchronously in legacy. A miss falls through to a real compile, so a document
nobody baked behaves exactly as before.
The blocker I cited for not doing this was that `bakedSkeleton` is async where
`compileAgreementSkeleton` is sync. It was not a real blocker: every one of
these handlers is already inside an async function that awaits
`extractLocalDocument`. Awaiting costs nothing there.
Coding-shape Read/Edit/Grep keep the synchronous path deliberately -- that is a
separate surface with its own A/B, and changing it here would confound two
experiments.
103 tests green; both arms typecheck.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
1468d284 | Register docx-edit-bench v2: build the editing half nobody has built | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body Step 1 surveyed what practitioners actually ask for, across
docs/legal-skills-ecosystem-comparison.md and Beaver's own systemWorkflows.
The headline is negative and it reshapes the design: not one of Beaver's 24
shipped workflows produces an edited document. Thirteen deliver a Markdown
table, eleven a spreadsheet; six review workflows carry the identical 'Do not
generate a downloadable Word document', and proofread says 'Do not rewrite
the whole document.' The public skill ecosystem has the same shape.
But nine assistant workflows terminate in a Recommended Change / Proposed
Change / Recommended Fix / Recommended Action column - a natural-language
edit instruction against a cited location. Beaver already emits the stimulus
half of an editing benchmark at scale and has nothing that consumes it. That
pair is v2's largest family; the survey's structural-lint list supplies a
second family in the mutation form both sources leave undefined.
Six families in all: apply-a-review-finding, structural lint as mutation,
table-bound edits (the v1 failure neither surface could do), tasks whose only
route runs through a deferred tool, multi-document, and bilingual/Quebec.
Recorded honestly: the survey contains no Canadian editing ask and no
bilingual grounding at all, so the Quebec and EN/FR family is grounded in
Bill 96 and bilingual enactment as a deliberate extension beyond the survey,
not as a finding from it. And the locally cached CUAD corpus is already
page-flattened with redactions - good evidence of bad PDF extraction, poor
material for clause-addressed editing.
v2 adopts the catalog's Class A-D taxonomy as a scoring rule: a task is a
legitimate model-evaluated item only at Class C or D, and a Class A/B task
must be labelled a floor task. Real fixtures ship only with an established
redistribution licence.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
b53c1353 | library_revise_docx takes an address in arm B | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body The edit tool required context_before and context_after -- ~40 characters
either side of the target, retyped by the model from memory of text it read
earlier. Every misquote in the edit benchmark was one of those strings: the
model joined two lines with a space where the document has a newline, or
invented a blank line where extraction has one.
With `at`, the model sends only the provision and what to change. The server
locates `find` inside that provision's span and reads the surrounding
characters off the document, so the bytes come from the document instead of
from the model's reconstruction of it.
Legacy keeps the old required shape. Arm B drops the context pair from
`required` and adds `at`.
Also hoisted NAV_TOOL_SHAPE above its first use -- the tool factory runs at
module init and read it from the temporal dead zone, which broke 13 tests.
1,195 tests pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
ddcb05cf | Ablation: the address grammar costs tokens; disclosure pays for it | Eli Ziff | 2026-07-31 | ↗ GitHub |
commit body 162 receipts, 27 tasks x 3 surfaces x 2 replicates, matched on every task.
beaver-address-no-disclosure holds the address grammar constant and removes
only progressive disclosure, which attributes the 19% the v1 headline
reported.
The finding inverts the natural reading. Served whole, the address grammar is
14.6% MORE expensive than the legacy surface it replaces (79,425 vs 69,281
tokens per run): library_links, describe_tools and the extra read/find
parameters make its catalogue 28,802 bytes against legacy's 25,809, and every
request carries the difference. Deferring 19 of 29 tools within the same
grammar takes it to 56,149 - a 29.3% saving - which nets out to the reported
19%. The lever is what ships in the first request, not the grammar.
A second-order effect is worth more than the tokens. Retyped document
characters on identical tasks: 327 with the full catalogue resident, 126 with
it deferred, and runs containing a misquote fall from 14/54 to 9/54. When
library_revise_docx is on the table the model reaches for it and retypes;
when it sits behind a domain the model uses the addressed tool instead. Every
misquote in this campaign came from a retyping-based argument, so deferral is
acting as a nudge toward the safer edit route, not merely as a saving.
Correctness is unmoved: 81/79/79 percent excluding floor tasks, one run of
forty-eight between best and worst, inside the replicate floor. No surface
made the model better at editing; they differ in cost and in route.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
|
29b5d2f9 | bench: preserve empty-review DOCX bytes | Eli Ziff | 2026-08-03 | ↗ GitHub |
61b958ba | fix(lab): documents_read_directly matches positional doc-N aliases; generate_docx counts as drafting | Eli Ziff | 2026-08-06 | ↗ GitHub |
commit body The chat surface only ever shows the model positional document IDs
(doc-0, doc-1, ...), but the counter matched upload UUID / filename /
source path - so documents_read_directly shipped 0 on every run. And the
single-conversation lab chassis never flips the harness draftingPhase
flag, so generate_docx was stamped research and drafting_tool_calls
shipped 0. Verified by recomputing over the employment v2 confirm-run
receipts: direct 0->5 (all five docs were read), drafting 0->1,
research 4->3, totals conserved.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011pfUVhNFTRvhYGXBwoKNj6
|
4ba6d0c4 | docs(lab): ledger - exec-emp-agmt native-on-flash refusal (0/1 DOCX, 18-11-08), re-run queued | Eli Ziff | 2026-08-07 | ↗ GitHub |
commit body employment-labor/analyze-counterparty-markup-of-executive-employment-agreement
authored 0/1 DOCX on mike_upstream_native_v1/deepseek-v4-flash after ~8 min
reasoning; same stochastic refusal signature as capital-closing first attempt.
No judge artifacts; re-run queued.
Co-Authored-By: Claude <noreply@anthropic.com>
|
0d8276a8 | docs(lab): ledger - exec-emp-agmt 2nd native refusal (3/1 DOCX, 18-19-45), 3rd run queued | Eli Ziff | 2026-08-07 | ↗ GitHub |
commit body Second attempt on employment-labor/analyze-counterparty-markup-of-executive-
employment-agreement authored 3/1 DOCX (over-produced; gate at lab-beaver-arm.ts:2882).
Distinct signature from the 0/1 first attempt; task-specific erratic refusal
flagged, 3rd re-run queued.
Co-Authored-By: Claude <noreply@anthropic.com>
|
8bb53b74 | docs(lab): ledger - antitrust protective-order first-attempt 0/1 DOCX refusal, re-run queued | Eli Ziff | 2026-08-07 | ↗ GitHub |
commit body antitrust-competition/analyze-counterparty-markup-of-protective-order authored
0/1 DOCX on mike_upstream_native_v1/deepseek-v4-flash (harness gate, clean).
Re-run queued per capital-closing recovery precedent.
Co-Authored-By: Claude <noreply@anthropic.com>
|
140109e3 | Ground legal citations with versioned source pills | Eli Ziff | 2026-08-08 | ↗ GitHub |
94125fde | Restore coherent citations and tracked edits | Eli Ziff | 2026-08-09 | ↗ GitHub |
204d3fba | Use fast DOCX renditions everywhere | Eli Ziff | 2026-08-09 | ↗ GitHub |
3534d3d4 | Speed up faithful DOCX browser rendering | Eli Ziff | 2026-08-09 | ↗ GitHub |
23abcfc3 | Fork DOCX renderer for native cold parsing | Eli Ziff | 2026-08-09 | ↗ GitHub |
1045ab19 | Use fast DOCX renderer on first open | Eli Ziff | 2026-08-09 | ↗ GitHub |
15ff2534 | Restore faithful DOCX pagination | Eli Ziff | 2026-08-09 | ↗ GitHub |