wardethan2000-eng turns document storage into matter-wide research

Groundwork for asking a question of a whole matter rather than of documents
handed over one at a time. Nothing user-facing yet.

Every document is now cut into passages when it is stored - after OCR for a
scan, since before that there is no text - and the passages are saved with the
page each came from, so a later search can cite by page. Storing happens after
the upload is answered and never fails it.

- lib/passages.ts    cuts page-labelled text into overlapping passages that
                     never span a page. Pure and unit-tested.
- lib/passageIndex.ts reads a stored version from whichever copy has the text
                     (the searchable PDF for scans, the file for a plain PDF,
                     the grid for a spreadsheet) and stores its passages,
                     replacing any held for that version so re-indexing is safe.
- migrations/20260818_01_document_passages.sql  the passages table, with
                     word-search and trigram indexes and user/project columns
                     so a search can be scoped to what the reader may see.
- scripts/backfillPassages.ts  indexes documents uploaded before this existed.

Word search over the stored passages is stage 3; the meaning fingerprint is a
later migration once the local model is chosen. Verified on the live server:
the existing scan indexed to page-numbered passages and word search found real
terms with correct pages.

New logic lives in new files; the edits to documents.ts and projects.ts only
add the call that stores passages after an upload.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PoCRDAPyEq4wwBVFFbDzck

The passage indexer read every document through its PDF rendition, falling back
to the original file when there was none - but then still parsed those original
bytes as a PDF. A Word file whose LibreOffice conversion had failed at upload
therefore failed to index with "Invalid PDF structure". Found while indexing
existing documents: three real .docx letters failed this way.

versionText now reads the PDF rendition when one exists (it carries page
numbers), and otherwise reads the original with a reader that suits its type -
mammoth for Word, the slide reader for PowerPoint, the sheet reader for a
spreadsheet, plain text for a text file - and never parses non-PDF bytes as a
PDF. A Word file with no rendition indexes without page numbers, which is
correct: a Word document reflows and has no fixed pages.

Verified on the live server: the three failed letters now index (5, 3 and 5
passages of clean text), and the full backfill finished with 52 documents and
412 passages, 0 failures.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PoCRDAPyEq4wwBVFFbDzck

The assistant can now be asked a question of a whole matter - "which files
mention the summons", "where is the indemnity clause" - and find the answer
across every document, without being handed them by name.

- migrations/20260818_02_search_passages.sql - one database function that
  searches the passages. Full-text search understands word stems and phrases;
  a word-similarity fallback tolerates the character errors a scan carries, so
  an OCRd document is still found when whole-word search misses it. Only the

Search now understands what a question means, not only the words in it.
Ask "who pays if this goes wrong" and it finds an indemnity clause that
never uses those words, alongside the exact-word and typo-tolerant
searches that were already there. The three run together and their results
are merged, so a passage found by more than one rises to the top; exact
words are weighted a little above the rest, so a party name or a section
number still lands first.

The meaning of each passage is read on our own machine, from a small model
(bge-small-en-v1.5) baked into the image, so no document text is sent
outside for this. Passages stored before now are filled in with a command
(backfillEmbeddings) that reads their text straight from the table, so no
document has to be read again.

A photograph has no text to read, so an inspection photo named "Shattered
roof skylight leaving the interior exposed to weather elements" could not
be found by a search of the matter - recognition cannot help a picture
with no words in it. Now, when a document yields no text of its own, its
filename is stored as a single searchable passage, so the photo is found
by that description, by exact words or by meaning like anything else. A
result says plainly when the match was on the file's name rather than on
text inside the file, so the assistant does not quote a picture as if it
held words.

Adds GET /projects/:projectId/search - the same word-and-meaning search the
assistant runs, reachable directly so a search box in the interface can call
it without going through a conversation. It checks the caller may open the
project, searches only that project's documents (so a matter you are not on
never surfaces), and returns the best passages with their document, page, and
how each was found (words, meaning, a fuzzy match, or the file's own name).

This is the backbone for the search box; the box itself follows.

Adds POST /projects/:projectId/search/answer - ask a whole matter a question
and get a single answer that cites the document and page for every point,
drawn only from the matter's own documents. It searches the passages (words
and meaning), hands the best ones to the model as its only source, and asks
for one grounded, cited answer; if the documents do not address the question
it says so rather than guessing. This is stage 7 of search across a matter,
reusing the same passages and page citations the assistant uses elsewhere.

Adds the search box the earlier stages were building toward. In a project's
documents view you can now search the whole matter two ways: "Find passages"
lists the best-matching passages with their document, page, and how each was
found (exact words, meaning, an approximate match, or the file's own name);
"Ask the matter" returns one consolidated answer that cites the document and
page, drawn only from the matter's documents. Both call the endpoints added
earlier (GET /search and POST /search/answer); this is the interface for them.

Type-checked against the frontend; contains no build-tool-specific code.

The consolidated-answer box was always asking the default model (Gemini),
so if that model was not set up it could never answer. It now uses whichever
model you have chosen in the app - the same one the assistant uses - so it
works with the provider you actually have configured.

The new /projects/:projectId/search and /search/answer routes matched
legacy paths like /projects/directory/search, since ":projectId" captured
"directory". A projectId that is not a UUID is never a real project, so both
routes now return 404 for one, which also restores the assertion that the old
project-directory search route is gone.

Indexing a very long document (a few hundred pages) pushed all of its
passages through the embedding model in a single call, which exhausted
memory on a small machine and could stall or kill the process. Passages are
now embedded in small chunks, so a document of any length indexes within a
steady, small amount of memory. Order is unchanged.