Compile structure once, and let text in
MEMOIZE THE COMPILE. `compileAgreementSkeleton` had 20+ call sites in backend/src and nothing memoized it -- `library_find` compiled the same document TWICE in one call, and the coding-shape Grep compiled one per matching document. The extracted text beside it has been cached for ages, so the string was free and the structure was not. The key is (sha256(text), id, recoverExtraction), because the compile is not a function of text alone: `id` becomes `doc.id`, and the recovery flag changes the node inventory on 45 of 23,531 A2AJ statutes. Graphs memoize against the SKELETON via a WeakMap -- resolution depends on the skeleton, and an entry dies with the thing that keyed it, so no size policy is needed. Sharing is safe: nothing in backend/src, backend/scripts or the tests mutates a returned skeleton. The real cost is memory -- a shared SourceDoc keeps its lazy token array (~14 bytes per source character) alive -- so the cap is 8. IDENTITY PROOF, 47,062 compiles: the whole A2AJ statute corpus re-derived under both constructions with the cache active reproduces both known digests exactly -- b31fdc04... with recoverExtraction:false, 6a5be600... with the default. A cache that changed output would be a bug, not a win. LATENCY, at contract scale (12 hold-out documents, median 37,796 chars): a turn's structural work goes 130.9 ms -> 8.1 ms, 94% saved. Stage 21 measured arm B +1,464 ms slower for statistically identical answers, and arm B compiles strictly more per turn. That penalty was recomputation. LET TEXT IN. A `.txt` or `.md` was rejected at three hard gates before any parser saw it, all from one allowlist pair -- so the plain-text parser added earlier was unreachable. Both allowlists now carry the plain-text types, with content types so a `.md` is served as text/markdown rather than octet-stream. Decoding does NOT normalise line endings. Rewriting CRLF is the same defect that silently corrupted a quarter of a benchmark for five stages: every offset a reader returns has to index the stored bytes. VIEWER. Plain text previously fell through to the PDF renderer, which cannot open it. `TextView` renders Markdown through the same react-markdown + remark-gfm pair the assistant and tabular views already use -- one Markdown implementation, not two -- and keeps `.txt` verbatim in a <pre>, because a transcript is read against its own line structure. 124 tests green across seven suites; backend and frontend typecheck clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | 5eba804e |
| Stats | 9 files changed , +246 , -5 |
| Part of | SourceDoc / DOCX engine: redlines, tracked changes, editing benchmark |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-6e7078c5.md
from inside the repo you want the change in.