eliziff gives PDF evidence a fixed address

This fork is making it easier to return to the same supporting passage instead of asking the system to find it again.

discoveryworkflow

eliziff has added structured PDF lookup to a longer-running effort around durable pinpoint evidence. In plain terms, the system can retain a precise reference to a passage in a PDF and retrieve that same evidence deterministically later.

That matters when a cited source needs to hold up under repeat review. Fuzzy rediscovery can surface a related passage; a fixed reference is meant to bring the reader back to the original one. The work also hardens the path from PDF parsing through to later evidence use, with compatibility and fallback behaviour still worth checking before adoption.

The GitHub thread is the place to inspect how those references are stored and what happens when a document source or provider behaves differently.

So what Litigation and legal-ops teams building evidence-backed answers should care because repeatable citations are easier to review, defend, and trust.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

21 commits from eliziff/Beaver, oldest first. Source extracted verbatim from the harvested git log.

SHA Subject Author Date
24df25b9 docs: plan legal PDF document intelligence Eli Ziff 2026-07-25 ↗ GitHub
20c3b1c3 docs: require degraded PDF ground-truth exports Eli Ziff 2026-07-26 ↗ GitHub
eb64906a refactor: let the PDF engine own section artifacts Eli Ziff 2026-07-27 ↗ GitHub
114ff47a feat: add exact structured PDF lookup Eli Ziff 2026-07-27 ↗ GitHub
ea8d6933 feat: rebuild the legal source reader Eli Ziff 2026-07-27 ↗ GitHub
968408aa feat: harden local matters and PDF evidence Eli Ziff 2026-07-27 ↗ GitHub
7d8d2c81 feat: attach verified local PDF pinpoint links Eli Ziff 2026-07-27 ↗ GitHub
01e61aef fix: make local PDF evidence durable Eli Ziff 2026-07-27 ↗ GitHub
7f31c520 fix: reject incomplete PDF cache publications Eli Ziff 2026-07-27 ↗ GitHub
e468a6eb feat: normalize exact legal PDF locators Eli Ziff 2026-07-27 ↗ GitHub
ca06c804 feat: attach exact provider PDF evidence Eli Ziff 2026-07-27 ↗ GitHub
d7eb917c feat: persist provider-native pinpoint evidence Eli Ziff 2026-07-27 ↗ GitHub
efcdd7c7 fix: sameLine parity between first-query scan and indexed search Eli Ziff 2026-07-27 ↗ GitHub
commit body
The stage-3 first-query scan counted a line break sitting at span end
(one past the last word) as inside the span, so a phrase ending exactly
at end-of-line - any quote ending at a block boundary - was dropped on
the first query but found by the indexed path on the second. Match the
indexed path: only breaks strictly inside [start, end) count.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
eb793a16 refactor: compact provider-PDF bridge plumbing Eli Ziff 2026-07-27 ↗ GitHub
commit body
prune() replaces ten conditional-spread pyramids with identical
persisted-JSON key order, attachmentFromStored() unifies the duplicate
stored-record validators, parsedState()/cachedPdf() dedupe the
resume-vs-read and downloaded-branch copies, and the lease layer
collapses SELECT+INSERT/CAS into one upsert whose semantics match the
original exactly (live lease refuses even the same owner; expired
lease taken over atomically under BEGIN IMMEDIATE). -301 lines.
mike-provider-pdf:v1 / v2 pointer / binding byte-shapes unchanged.

Verified: tsc clean; bridge suite 34/34 incl. lease-takeover and
cold-failure backoff, plus consumer suites green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QUsMXzBsEJ27DjJVPSfW5o
2959ca8c feat: content-addressed parse cache (Issue 11) Eli Ziff 2026-07-27 ↗ GitHub
commit body
Persistent parse cache keyed by file SHA-256 + parser name/version under
the app's local data home (parse-cache/), scoped per user/document so
matter content never crosses scopes. Wired into both production text
extraction paths (local extractLocalDocument and cloud
readDocumentContent) via a shared per-format parser table; content or
parser-version changes naturally miss, clearParseCache deletes a scope
or everything, hits log actually avoided work. Measured: 554KB PDF
1468ms -> 2ms, 121KB DOCX 128ms -> 1ms, byte-identical output.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016yXxuitxX5h4wm1v88hnga
a0baf300 library: denormalize the PDF parse lifecycle onto documents; surface it with retry Eli Ziff 2026-07-30 ↗ GitHub
commit body
P0.8 remainder, local lane. localDocumentResponse hardcoded status:
"ready" for every document, so the Library could not tell a parsing,
degraded, or failed structural parse from a clean one without a
per-document round trip.

Backend: new peekLocalPdfParseState reads the durable job's state file
only - no artifact validation, no diagnostics load, no writes - so a
listing of N documents costs N stat-reads. localDocumentResponse
becomes async and carries parse_state (status, error, attempts,
timestamps, engine_status, cache_hit, page_count, diagnostic_count,
structural_repair_available) for PDF versions; null means no parse lane
(non-PDF / flat text). `status` keeps its storage-readiness meaning -
the parse lifecycle is a separate field, and flat text remains
available in every parse state.

Frontend: Document.parse_state typed; DocTable renders a lifecycle chip
beside the filename (spinner while queued/parsing, amber Degraded with
diagnostic count in the title, red Parse failed with the error in the
title and a Retry action). Retry posts the existing
actions/retry-pdf-parse route via the new retryLibraryPdfParse API and
refreshes the collection; clean ready parses and non-PDF rows render no
chip. LibraryWorkspace wires retryPdfParse per library kind.

Deliberately deferred, per the plan's own ordering ("cloud lane last"):
the cloud/Supabase lane still reports storage status only - its
documents carry no parse_state and the chip stays absent there.

Suites: localDocumentStore, localLibraryPdfParse, localPdfIngestion
(33), DocTable interaction/removal + LibraryWorkspace + FileDirectory
(29, incl. 3 new chip/retry tests); tsc clean both ends. types.ts
staged hunk-selectively - the unrelated in-flight assistant-event hunk
stays uncommitted.

Integration plan item 5 (task #28), local lane.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
fab8e767 Parse Library PDFs lazily and faster Eli Ziff 2026-07-31 ↗ GitHub
15b15d2d Keep thin PDF consumers geometry free Eli Ziff 2026-07-31 ↗ GitHub
7c61867c Reparse changed PDFs and prune stale artifacts Eli Ziff 2026-07-31 ↗ GitHub
c8baf66c Clean stale PDF parse artifacts Eli Ziff 2026-07-31 ↗ GitHub
b9143a48 A PDF never reports "no pages": index the second route, and name the gap honestly Eli Ziff 2026-07-31 ↗ GitHub
commit body
Eli caught this on the test fixture: "no pages" on a PDF is a false statement
about the document.

Two things were wrong.

FALL-THROUGH. Both PDF routes end at the same engine. The ingested route
yields page BLOCKS via readLocalPdfSourceDoc; the un-ingested route runs
`legalpdf parse` and renders `[page N]` markers into the text it returns. The
map builder took the artifact when `parsed` existed and never tried the
markers, so a rendition that carried text but no page blocks was indexed as
pageless even though its own text was full of page markers. It now falls
through artifact -> markers.

VOCABULARY. "This document carries no page structure" conflated two states
that call for opposite responses. A DOCX genuinely has no fixed pagination
until something renders it. A PDF has pages whether or not we managed to read
them, so the same message there reports a gap in our pipeline as a fact about
the document -- exactly the class of error the refusal contract exists to
prevent. PageMap.source gains `unavailable`, set only where the file type is
known, and the two refusals now say different things: one tells you the
document is not paginated, the other tells you the index could not be built
and that page numbers appearing in the text are unverified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-837.md from inside the repo you want the changes in.

⬇ Download capture-thread-837.md