product spine: short-ladder capture + dot-form section fallback (TFP-vet ports)

↗ view on GitHub · Eli Ziff · 2026-07-30 · 3501d7ab

Ports the full-sweep detector fixes (structure_ref.py dbb7b355 /
6ae6d330) into sourceDocA2AJ.ts, closing the ~3%-of-cases no-paragraph-
anchors gap and the NT/PE zero-recovery gap in the product.

Paragraph spine (mirrors dbb7b355 exactly):
- complete short [1]..[N] bracket ladders (contiguous from 1) become
  last-resort hypotheses, ordered after every full scope so a tail
  [1]..[4] list can never shadow a real ladder; accepted only when the
  ladder IS the document (<=6KB, header-bounded start absolute <=1200 OR
  ratio <=0.5, at least one >=30-word paragraph);
- substance guard becomes median>=12 OR mean>=20 OR max>=30 so "I
  agree." concurrence tails cannot sink a ladder.
The cascade span credit stays sweep-side only: it feeds flag metrics,
and the product's last paragraph legitimately runs to EOF.

Laws text-derived fallback spine (section map stays primary):
- FLAT_SECTION_EXT_RE (dot-form "1. There is established...", "2.(1) In
  this section", markdown-heading "### 61.01") runs ONLY when the plain
  grammar yields no spine. A merged regex measurably corrupts spines:
  Ontario enumerates paragraphs inside sections in the same shape and
  the first cut lost 6/64 sections on a2aj-regs-on-oreg267-03; the typed
  fallback provably cannot touch any doc that had a spine.
- the leading-subsection splice accepts the label's trailing dot so
  "2.(1)" keeps its first subsection.
The 6ae6d330 named-heading fr-surface fix has no product surface (the
flat spine has no named-heading blocks); it stays benchmark-side until
the Phase 4 detect/ suite decides whether named units join the spine.

Differential over the seed-47 reservoir (1,862 cases) + 866 laws docs
(fixed sets full-sampled, controls included), scratchpad ts_diff:
- cases: 98.55% exact TS==Python parity on (number, offset) spines; 54
  old->new none->paragraph conversions - the same 54 the Python
  reservoir differential reported; 0 regressions; all 27 residual
  mismatches classified pre-existing tie-break drift (the known
  monotone-scope divergence queued for Phase 4 reconciliation), none
  port-introduced.
- laws recovery (fallback spine vs corpus oracle): LEGISLATION-NT
  0.0023 -> 0.8203, LEGISLATION-PE 0.0015 -> 0.8775, REGULATIONS-NT
  0.0015 -> 0.6246, REGULATIONS-PE 0.0185 -> 0.8287, REGULATIONS-NB
  0.8242 -> 0.9042; controls BC/FED/ON byte-identical (the fallback
  never engaged).

Suites: sourceDocA2AJ (+4 pinning tests), sourceDocFixtures (oreg267-03
byte-identical recording), sourceDocSameLine, sourceDocNativeMarkup -
68 passed; tsc clean.

Integration plan item 3 (task #28).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
Repository eliziff/Beaver
Author Eli Ziff <eliasziff@gmail.com>
Authored
Parents 73284884
Stats 2 files changed , +161 , -16
Part of SourceDoc / DOCX engine: redlines, tracked changes, editing benchmark

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-3501d7ab.md from inside the repo you want the change in.

⬇ Download capture-commit-3501d7ab.md