Compile structure once, and let text in

↗ view on GitHub · Eli Ziff · 2026-07-31 · 6e7078c5

MEMOIZE THE COMPILE. `compileAgreementSkeleton` had 20+ call sites in
backend/src and nothing memoized it -- `library_find` compiled the same
document TWICE in one call, and the coding-shape Grep compiled one per
matching document. The extracted text beside it has been cached for ages, so
the string was free and the structure was not.

The key is (sha256(text), id, recoverExtraction), because the compile is not a
function of text alone: `id` becomes `doc.id`, and the recovery flag changes
the node inventory on 45 of 23,531 A2AJ statutes. Graphs memoize against the
SKELETON via a WeakMap -- resolution depends on the skeleton, and an entry
dies with the thing that keyed it, so no size policy is needed.

Sharing is safe: nothing in backend/src, backend/scripts or the tests mutates
a returned skeleton. The real cost is memory -- a shared SourceDoc keeps its
lazy token array (~14 bytes per source character) alive -- so the cap is 8.

  IDENTITY PROOF, 47,062 compiles: the whole A2AJ statute corpus re-derived
  under both constructions with the cache active reproduces both known
  digests exactly -- b31fdc04... with recoverExtraction:false, 6a5be600...
  with the default. A cache that changed output would be a bug, not a win.

  LATENCY, at contract scale (12 hold-out documents, median 37,796 chars):
  a turn's structural work goes 130.9 ms -> 8.1 ms, 94% saved. Stage 21
  measured arm B +1,464 ms slower for statistically identical answers, and
  arm B compiles strictly more per turn. That penalty was recomputation.

LET TEXT IN. A `.txt` or `.md` was rejected at three hard gates before any
parser saw it, all from one allowlist pair -- so the plain-text parser added
earlier was unreachable. Both allowlists now carry the plain-text types, with
content types so a `.md` is served as text/markdown rather than octet-stream.

Decoding does NOT normalise line endings. Rewriting CRLF is the same defect
that silently corrupted a quarter of a benchmark for five stages: every offset
a reader returns has to index the stored bytes.

VIEWER. Plain text previously fell through to the PDF renderer, which cannot
open it. `TextView` renders Markdown through the same react-markdown +
remark-gfm pair the assistant and tabular views already use -- one Markdown
implementation, not two -- and keeps `.txt` verbatim in a <pre>, because a
transcript is read against its own line structure.

124 tests green across seven suites; backend and frontend typecheck clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
Repository eliziff/Beaver
Author Eli Ziff <eliasziff@gmail.com>
Authored
Parents 5eba804e
Stats 9 files changed , +246 , -5
Part of SourceDoc / DOCX engine: redlines, tracked changes, editing benchmark

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-6e7078c5.md from inside the repo you want the change in.

⬇ Download capture-commit-6e7078c5.md