emileriksenkeev makes long-running legal AI chats far less fragile

This fork is hardening the moments when complex chat work is most likely to disappear or stall.

chat-uiinfrastructure

Long, multi-step requests can now keep working through more of the real-world failure points that interrupt legal AI workflows.

  • Longer task runs can complete rather than stopping after a short sequence of actions.
  • Streaming connections send keepalive signals, reducing the chance that proxies cut off work that is still underway.
  • A dropped connection no longer silently loses the user's submitted request.
  • Large document collections are handled in smaller batches, preserving chat and wiki context for projects with hundreds of documents.
  • Repeated AI instructions can be cached, which may reduce cost and delay during tool-heavy conversations.
So what Legal teams using AI against substantial document sets should care because reliability during longer research and knowledge-building runs matters as much as the answer itself.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

7 commits from emileriksenkeev/sydOS, oldest first. Source extracted verbatim from the harvested git log.

SHA Subject Author Date
82bffb5d Syd OS: raise chat tool-iteration cap 10 -> 40 (env CHAT_MAX_TOOL_ITERATIONS) emileriksenkeev 2026-07-11 ↗ GitHub
The matter-wiki build (and other agentic workflows) need far more than 10
tool rounds per turn - the build kept stalling mid-run.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
823d76aa Syd OS: fix mid-sentence truncation in agentic runs emileriksenkeev 2026-07-11 ↗ GitHub
commit body
- Raise Claude max output tokens 16384 -> 32000 (env CLAUDE_MAX_OUTPUT_TOKENS)
- On stop_reason=max_tokens, continue the loop instead of breaking: run any
  completed tool calls, or push the partial text with a 'continue where you
  left off' nudge. Previously the turn silently ended mid-sentence.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
b60d872d Syd OS: SSE keepalive heartbeats on all four streaming routes emileriksenkeev 2026-07-11 ↗ GitHub
commit body
Long tool-JSON generation (wiki page writes, tabular cell generation) streams
no visible bytes for minutes; Cloudflare drops proxied connections after
~100s of silence, surfacing as 'Sorry, something went wrong' while the
backend kept working. Send an SSE comment every 15s; the client parser only
reads 'data:' lines so comments are ignored.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
9aaaadc2 Syd OS: enable Anthropic prompt caching in the agentic loop emileriksenkeev 2026-07-11 ↗ GitHub
commit body
Cache breakpoints on the system prompt and the last message of each request:
each tool-loop iteration now reuses the growing conversation prefix instead of
re-processing all previously read documents (~100k+ tokens per round on wiki
builds). Large latency win between steps and ~10x input-cost cut on long runs.
Request-only copies - stored messages never mutated. CLAUDE_PROMPT_CACHE=0
disables.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ce3bdad2 fix: batch large .in() lookups in documentVersions emileriksenkeev 2026-07-14 ↗ GitHub
commit body
attachActiveVersionPaths / attachLatestVersionNumbers passed every id in a
single PostgREST .in() - beyond ~300 ids the GET URL exceeds the server
limit, supabase-js surfaces the failure only in the discarded error field,
and every document silently loses its storage path. Net effect: any project
with enough documents (RX Labs, 525) got an EMPTY doc context - project
chat saw no documents and wiki builds produced nothing. Batch the lookups
100 ids at a time and log per-batch errors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fe464fe8 Long-run chat: persist user turns, recover from proxy disconnects emileriksenkeev 2026-07-27 ↗ GitHub
commit body
Three defects surfaced by a 13-minute, 31-document project-chat review
that ended in a bare "Sorry, something went wrong" while the answer sat
finished in the database.

1. No user message was ever saved. Both chat routes inserted a
   `workflow` field into chat_messages, which has no such column;
   PostgREST rejected every insert and the unchecked result hid it. Drop
   the field and check the error. Reloaded chats now show the questions,
   not just the answers.

2. Cloudflare caps a proxied request at ~10 minutes no matter how often
   the SSE heartbeat fires, so a long agentic run loses its client while
   the backend keeps going and persists the turn. The client now says so
   and polls the saved chat until the answer lands, instead of leaving a
   dead error in place.

3. The client ignored the SSE `error` event entirely, so a server-side
   failure ended the stream as a blank assistant turn. Surface it (and
   send the real message rather than "Stream error").

Also add start/disconnect/finish logging to the project chat route,
which had none - the reason this run left almost no trace.
45ff8f21 Merge fix/long-run-stream-resilience: survive proxy disconnects, persist user turns emileriksenkeev 2026-07-27 ↗ GitHub

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-911.md from inside the repo you want the changes in.

⬇ Download capture-thread-911.md