emileriksenkeev plugs the AI workspace into the firm's document record

Syd's library goes stale the moment an upload is forgotten. Cortex is the
group's system of record for documents - 28k+ legal documents synced
continuously from Gmail/Drive/Slack/Notion/Asana, deduplicated, OCR'd,
classified and version-aware, including ~11k contract-negotiation emails
Syd has never held. This connects the two; it does not copy the corpus.

Adds lib/cortexBridge.ts (pattern: lib/bryBridge.ts) with five read-only
tools - search_legal_corpus, list_corpus_documents, read_corpus_document,
counterparty_profile, legal_calendar - plus CORTEX_GUIDANCE, the prompt
layer that teaches the retrieval doctrine. Wired into global chat, project
chat, and the headless agent bridge. Cortex tools are safe on the agent
bridge in a way BRY_TOOLS are not: they are REST reads, not an agent hop,
so they cannot extend the Bry -> Syd chain.

Nothing changes until credentials exist: cortexEnabled() requires both
CORTEX_API_URL and CORTEX_API_KEY, and when false neither the tools nor
the prompt layer are registered at all (precedent: embeddingsEnabled).
Deploying this ahead of the key and the Cloudflare service token is a no-op.

Three things learned by running it against the real API rather than
assuming:

- /api/search takes 23-27s from off-droplet (Voyage embed + rerank + thread
  cards over the pooler), so a flat 30s timeout turned a working search into
  a timeout. Timeouts are now per-endpoint, and a timeout is never retried:
  a slow endpoint retried is just the same cost paid twice, while genuine
  transients (5xx, 429, network faults) still get two retries inside a
  100s budget.
- The signed Apotheca Partner Agreement returns 109 characters of
  "[Page 1] ... [Page 10]" - a scan with no text layer. A naive non-empty
  check reads that as a document, which is how an assistant ends up
  summarising a contract it never read. Now detected via the same
  marker-stripping rule as meaningfulPdfTextLength, including the sparse
  middle case, and reported as "you have NOT read this" with the file link.
- Filters this deployment does not support yet (function, lifecycle, q,
  legal_entity) are ignored rather than rejected, so the bridge compares the
  echoed filter and warns when a list is broader than what was asked for.
  A filter silently dropped is worse than one refused.

Entity-optional listing and /api/obligations are still shipping on the
Cortex side; both are feature-detected and degrade with an instruction
rather than an error. Every failure path returns guidance the model can
act on, because a legal assistant guessing because a service was down is
the failure mode worth engineering against.

Tested against the live read-only API with a temporary read key (since
deleted, along with its audit rows): profile of a counterparty with 5,491
documents, a 7-row contract set with 11 duplicate copies collapsed and
coverage gaps stated, full text of an 18k-character MOU, and correct
degradation for unknown entities, bad ids, missing endpoints, bad
credentials, and an unconfigured instance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Activation against live production exposed three gaps in the obligations
tool that only appear with real data behind it.

The feed is ordered by due_date ascending across the whole corpus, and the
corpus holds many years of long-overdue extracted obligations. "What
renews in the next 60 days" therefore returned 100 items dated 2005-2022 -
every one of them already past due, presented as if upcoming. The endpoint
supports due_after; the tool never exposed it. It does now, along with
limit and cursor, and both the tool description and the guidance say
plainly that a question about the future needs due_after.

Syd's system prompt carries no current date, so the model could not compute
that window even when told to. CORTEX_GUIDANCE is now cortexGuidance(),
built per request with today's date in it. Per request rather than at
module load on purpose: this backend runs for days between restarts, and a
date baked in at startup goes stale silently.

Cortex caps `count` on the obligations query and flags the cap with
`at_least`, which the tool dropped - so the model saw the cap 10000 as an
exact figure. Now surfaced as count_is_lower_bound with guidance to say
"at least N".

Also names the junk the extractor leaves behind: rows carrying a
1970-01-01 placeholder (no real date extracted) and rows already overdue
are counted in the note, so an obligation with no date never gets reported
as a deadline. If every row came back overdue and due_after was not set,
the note says so outright and tells the model to call again.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Phase 1 gave the chat loop five read tools. This is the rest of the loop -
the picker, the pass-through, and the publish button (v7 §5.S2/S3/S4).

S2 - a Cortex tab in the document picker. Search (ranked) and Browse (the
complete filtered set) are kept visibly separate because Cortex only reports
superseded/canonical on search results, and showing a version flag the corpus
never gave would be inventing reassurance. "Attach" MATERIALIZES: Syd resolves
every read through document_versions.storage_path, so a pointer row is
invisible to the assistant that is meant to read it. The backend pulls the
bytes and runs the ingestEntities pipeline as library calls - not over HTTP,
which is capped at 50 uploads/hour and would stop a counterparty set halfway.
The copy is a per-matter working copy by design; edits must not leak back.

Staleness: the plan said compare `norm_hash`, but a live /api/fetch returns no
hash, no status, and no normalize/enrich timestamps. So cortexFingerprint()
takes the best signal actually present and records which one it used -
preferring a real hash the moment Cortex returns one, and otherwise using the
blob path inside the presigned URL, which is content-addressed (putBlob keys
on sha256) and therefore changes exactly when the bytes do. A document whose
fingerprint cannot be established reports "could not check", never "current".

S3 - uploads pass through to the corpus in the background, skipping documents
that came FROM Cortex (no boomerangs). /api/ingest does not exist yet (404,
the write path is still being built), so everything degrades to "queued, not
lost": pending in metadata, retried by a sweep that only ever touches rows
this integration stamped - publishing Syd's back catalogue is Emil's decision,
not a side effect of a restart.

S4 - "File to Cortex" on the row menu. Explicit, per artifact, never
automatic (§8.4). Idempotent, so a double click comes back deduped.

Also fixes a phase-1 bug found against live Cortex: the bridge sent
fields/files as the number 1, and Cortex reads those flags as
`v === true || v === "1" || v === "true"` - so every request for extracted
contract fields and file links has been silently ignored.

Provenance lands in documents.metadata, the jsonb column reserved and unread
since schema.sql:110. No migration: document_versions.source already allows
'user_upload' for the refresh path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The automatic pass-through still skips documents that came FROM Cortex - same
bytes, same sha, a no-op. But an explicit "File to Cortex" on one is exactly
what §5.S4 is for: once a corpus document has been edited in a matter it is a
work product, and refusing to publish it would leave the best case for the
button unserved. Unedited, Cortex just dedupes it.

The filing result is recorded as cortex_filed_id rather than overwriting
cortex_id. cortex_id is the staleness anchor pointing at the ORIGINAL corpus
document; repointing it at the copy's own filing would make the copy look
permanently up to date against itself.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The FormData refactor left the fetch still calling JSON.stringify on an
already-stringified payload, so every JSON call went out as the literal
"{\"limit\":10}" - a JSON string, which Express parses to a string and from
which no handler can read an argument. Cortex answered every request with its
defaults: unfiltered, limit 100, no fields, no files.

Caught by the phase-2 smoke asking for ten contracts with file links and
getting a hundred without.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The integration only ever looked forward. 1,481 documents and 2.0 GB were
already sitting in Syd when it shipped, and a document that exists in exactly
one system is a document the company can lose.

backfillCortexSync hashes each document's bytes on the droplet, asks
/api/ingest the cheap question first (JSON sha precheck: a hit answers with the
doc_id and no byte moves, and costs no rate-limit token), and posts the bytes
only for the misses. It stamps the Syd row with the Cortex id, the sha and the
corpus fingerprint, so the retry sweep will not re-push it and the phase-2
staleness check covers the old library.

Work products are excluded by default and stamped as such rather than skipped
in silence: a generated document with an upload-sourced version in its history
sends the source material instead, and one without is left for Emil to file
deliberately (§8.4).

Supporting changes: cortexCall now surfaces Cortex's Retry-After on 429 (and
stops burning retries on a wait it cannot fit in its budget) so a bulk caller
can honour it; ingestToCortex takes a `tool` override; checkCortexStaleness
admits backfilled rows, whose source_system stays syd_os because Syd made them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Syd has pushed to the corpus since §5.S3 and never pulled. The only way a
working copy learned that Cortex had moved on was somebody opening the project
page: /cortex/staleness, one /api/fetch per row, capped at 25. A matter nobody
opened was never checked, and a reclassification or a supersede edge is not
something a fingerprint diff can see at all.

/api/changes is a cursor over everything Cortex touched. This adds the resident
consumer (backend runs for days on the droplet, so setInterval, same shape as
the 15-minute pass-through sweep it sits beside), bootstrapping at cursor='now'
because the feed is not a backfill tool and the back catalogue is already
reconciled.

What it does with an event is deliberately almost nothing: it stamps
metadata.cortex_stale / cortex_stale_reason / cortex_stale_at on local documents
matching metadata.cortex_id, plus cortex_superseded when Cortex says so. No
auto-refresh - that mints a version row and re-embeds, and doing it underneath a
lawyer mid-review is worse than being a day behind. The flag is the signal; the
existing "Refresh from Cortex" button is the remedy.

Three rules carry the correctness:
  - the cursor advances only past changes that actually finished. A failed page,
    a failed write, or the per-run page cap leaves it on the last completed item
    and the next tick resumes there. Re-reading is free (stamping is idempotent
    and a repeat event is a no-op write); skipping is unrecoverable.
  - an unrecognized reason is still news. The pinned v8 consumer rule: never
    switch on an exhaustive list, so anything Cortex invents later is flagged
    with its reason recorded verbatim and logged once as new vocabulary.
  - a doc_id with no local match is ignored. Syd tracks working copies, not the
    corpus.

The cursor lives in a new sync_state(key, value jsonb) table - generic on
purpose, so the next resident consumer costs a row rather than a migration -
with expression indexes on the documents.metadata keys the feed and the health
counts now query every ten minutes.

/cortex/status grows the other half of the picture: cursor age, last run, the
sweep's backlog and the stale-flag counts, with WARN lines in the journal past
3h of cursor age or 25 queued documents. A corpus link that has silently stopped
looks exactly like a corpus where nothing changed, which is the whole reason
this endpoint exists.

Also documents CORTEX_API_URL/KEY, the CF Access pair and both interval vars in
.env.example, which have been load-bearing and undocumented since the bridge
landed.

Build Plan v8 §W3.1, §W3.4.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Three changes to the materialize path, all of which stop guessing at something
the corpus can now answer directly.

STALENESS ASKS THE FLAGS FIRST (§W3.1(4)). checkCortexStaleness used to fetch
every row it was given. Now: a stamped cortex_stale is the answer; a copy the
feed has been watching continuously since it was taken (bootstrap older than the
copy, cursor not behind, last run ok) is answered "not stale" from the absence
of an event; and only what neither covers costs an /api/fetch - whose verdict is
then stamped, so the same document is free next time. The 25-cap now bounds
fetches rather than coverage, and the route's own ceiling goes 50 → 200 with
project access memoized, because a matter's documents share one project and
asking 200 times is how a raised ceiling becomes a slow page.

EXACT COMPARISON WHEN THERE IS ONE (§W3.2). materialize now records
sha256 of the bytes it stored in metadata.cortex_blob_sha256 - computed locally,
not copied from the response, so it describes what Syd HAS. When /api/fetch
starts returning blob_sha256 (v8 §W1.2) the comparison is two shas and nothing
else; until then the whole existing heuristic chain still decides, unchanged.

The chain gained one guard on the way past. blob_sha256 was already in its hash
bucket, which meant the day Cortex started returning it, every copy anchored on
a blob PATH would have compared a sha against a path and reported the entire
pre-W1 library stale in one page load. It now carries its own fingerprint_kind
and comparison requires matching kinds: mismatched kinds are "we could not tell",
which is the answer this code has always distinguished from "it is current".

NO SECOND OCR (§W3.3). A materialized scan was transcribed twice - once by
Cortex on ingest, once again here by Claude vision the moment embedDocument
reported no extractable text. /api/fetch already returned the transcription in
the same response we downloaded the bytes with. It is now persisted to ocr_text
in the statement that marks the document ready, before the embed runs, so the
embedder finds text and the vision tail never fires. Only when it is worth
trusting: not for docx/xlsx (tracked changes and grid fidelity need the local
parse), not when text_truncated (a cut transcription embedded as the whole
document is a search index that omits the end of every long contract), and not
for the ~12% of the corpus still queued for OCR upstream, where Syd's own pass
is the only thing that will produce text. Bytes still land in R2 unchanged.

Refresh keeps all three honest: it re-imports or clears the imported text for
the new bytes, re-pins the sha, clears the stale flags it just answered - and
keeps cortex_superseded, because pulling the same doc_id's newer bytes does not
make a document Cortex ruled superseded operative again. It also now merges into
existing metadata instead of replacing the column, which had been quietly
dropping cortex_filed_id, cortex_sync and cortex_backfill on every refresh.

The UI reads the flags straight off the row it already has: a distinct
"superseded in Cortex" badge, and "newer in Cortex" the moment the page loads
rather than after a round trip - with the reason in the tooltip, including
reasons this build has never heard of. Staleness badging stays gated on rows
with a real corpus anchor: a Syd upload that was pushed to Cortex is not
"behind" anything.

Build Plan v8 §W3.1(4), §W3.2, §W3.3.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>