fix(citations): reject out-of-order, distant, and meaning-inverting ellipsis quotes
From the PR description
Summary
Fixes #391.
On main, this verifies - badge and all:
const source = "Under clause 9, the landlord is not permitted to terminate this lease during the fixed term.";
verifyQuoteAgainstSource(source, "the landlord is ... permitted to terminate");
// → { verified: true } // the quote asserts the OPPOSITE of the source
The ellipsis branch of verifyQuoteAgainstSource located each quoted fragment independently, anywhere in the source. Fragment order, omission size, and omission content were never checked, so reversed fragments, fragments quilted from distant pages, and omissions that swallow "not" all earned verified: true. In a product whose purpose is that users don't re-read the source, a badge that can endorse a meaning-inverted quote is worse than no badge.
What changed
backend/src/lib/chat/verifyCitations.ts - the ellipsis branch now enforces what an ellipsis actually claims to a reader:
- In-order matching with backtracking (
locateSegmentsInOrder): fragments must appear in document order; each search starts after the previous match. If an early match strands a later fragment or overruns the gap bound, the matcher retries from the next occurrence, under a fixed attempt budget (64) that fails closed. - Bounded omission: at most
MAX_OMITTED_SPAN_CHARS(500) may be omitted between consecutive fragments - ~two long legal sentences. Beyond that the quote is quilting distant text, not abbreviating one passage. - Meaning-inversion guard: if the omitted span contains a negation/exception word (
not,no,never,cannot,n't,without,except,unless,exclud*,prohibit*,forbid*,denie[sd]/deny,void), the badge is refused. A leading/trailing ellipsis gets the same check against the hidden same-clause context (up to the nearest.;:!?\n), closing the trivial bypass of moving the ellipsis to the quote's edge.
backend/src/lib/chat/verifyCitations.test.ts - 11 new tests. Eight attack the matcher (inversion, contraction, exception carve-out, reversed order, distant quilting, leading/trailing edge-truncation, repeated fragment); all eight verified as true under the old implementation. Three guard against over-blocking (benign in-order omission, negator safely in a previous sentence, backtracking to a later anchor).
Reproduce the bug on main
git checkout main
git checkout olp-pr/citation-quote-ordering -- backend/src/lib/chat/verifyCitations.test.ts
npm test --prefix backend -- src/lib/chat/verifyCitations.test.ts
Expected: 8 failed | 27 passed - every rejects ... test fails because main verifies the abuse quotes (run performed on 1b58c7aa; output in the demo below).
Verify the fix on this branch
git checkout olp-pr/citation-quote-ordering
npm test --prefix backend -- src/lib/chat/verifyCitations.test.ts
Expected: 35 passed - all 24 pre-existing tests unchanged and green (legitimate abbreviated quotes still verify), all 8 attacks rejected.
Demo
Live recording of the replication above - the identical test file run against main's implementation (8 attack quotes verify → 8 failures) and then against this branch (all 35 pass):

Testing performed
npm test --prefix backend -- src/lib/chat/verifyCitations.test.ts: 35/35 pass on this branch; the 8 new rejection tests fail onmain's implementation (red-before-green demonstrated by swapping only the source module).- Full
npm test --prefix backend: 68 files / 851 tests pass, 4 stack-gated files skipped as usual. npm run build --prefix backend(tsc): clean.
Tradeoffs & design decisions
- The inversion guard is a word-list heuristic, biased conservative. A false positive only withholds the badge from an abbreviation a human should eyeball (e.g. omitting a harmless "without prejudice to clause 4" now yields unverified); a false negative would endorse a misquote. Given the badge's meaning, the asymmetry favors withholding.
MAX_OMITTED_SPAN_CHARS = 500is a calibration, not a derivation - roughly two long legal sentences. Legitimate omissions of an entire long paragraph now come back unverified; reviewers may prefer a different constant, and the constant is the single knob.- Clause boundaries are approximated as
[.;:!?\n]for the edge-ellipsis checks. Qualifiers that limit rather than negate ("only", "provided that", "to the extent") are not in the word list, and a negator beyond an abbreviation like "cl. 9" is cut off by the period - residual risk accepted to keep false positives down. - Cross-page
[[PAGE_BREAK]]segments are still verified per-segment without cross-segment ordering. Deferred deliberately: the sentinel is constructed by our own viewer expansion rather than free-form model text, so the abuse surface is far smaller - but it is the same defect class, and I'll file it as a follow-up rather than widen this PR. - No char offsets for ellipsis quotes (unchanged from before): offsets remain attached only to contiguous single-segment quotes.
- Perf: backtracking re-normalizes the source suffix per attempt (O(n) each). Bounded by the 64-attempt budget; typical quotes use 2-4 attempts.
🤖 Generated with Claude Code
Our analysis
Harden ellipsis quote verification — read the full analysis →
Think the analysis missed something the PR description covers?
Capture this PR into my fork
Download a Markdown prompt that tells Claude how to port every
commit in this PR into your working tree. Run it via
claude -p < capture-pull-392.md from
inside the repo you want the changes in.