structure: capture short complete ladders; end journal page circularity

↗ view on GitHub · Eli Ziff · 2026-07-30 · dbb7b355

Full-sweep vet drove three changes, each verified against the vet docs
plus an old-vs-new differential over the 1,862-doc reservoir (seed 47):

1. paragraph_index extensions (structure_ref.py): complete short [1]..[N]
   bracket ladders (contiguous from 1, doc <= 6KB, header-sized prefix by
   absolute <=1200 chars OR ratio <=0.5, at least one >=30-word prose
   paragraph) are real structure - 17/29 of the sampled none-queue were
   exactly this shape (short orders / oral reasons / costs rulings).
   Substance guard gains mean>=20 and max>=30 arms so "I agree."
   concurrence tails cannot sink a ladder. Short hypotheses never enter
   the primary scope competition (a tail [1]..[4] list in [1969] SCR 277
   shadowed the real ladder until ordered last). Cascade span now credits
   the final paragraph body bounded by 2x median length.
   Verification: 23/23 targeted docs (17 misses -> paragraphs; truly-none
   letters/stubs stay none), 0/1862 regressions, 54 none->paragraphs
   reservoir conversions (~2.9% of sample).

2. paragraph_scope_narrow flag (sweep.py): short docs (<4KB) flag only
   below 0.30 - 6/12 sampled flags were header+signature share on correct
   short judgments; all four true mis-selections (quoted-list hijack,
   order-enumeration hijack, gap-splice chimera, collapsed-fragment lock)
   still flag under the new rule, verified per doc.

3. Journal branch (sweep.py): page "recovery" scoring deleted. The
   [page N] markers are rendered BY US from article_pages/page_map_json
   in the journals database; detecting them was testing our own output
   against our own input. Journals now record structure kind
   provider_metadata and are scanned for grammar-entry rates only.

Intentional divergence from the ALR reference port is documented in the
structure_ref docstring; --parity now reports exactly the newly-captured
docs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
Repository eliziff/Beaver
Author Eli Ziff <eliasziff@gmail.com>
Authored
Parents ae68a969
Stats 2 files changed , +96 , -29
Part of SourceDoc / DOCX engine: redlines, tracked changes, editing benchmark

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-dbb7b355.md from inside the repo you want the change in.

⬇ Download capture-commit-dbb7b355.md