In-chat memory: persist tool calls and results so the model stops re-reading every document

nwhitehouse fixed the root cause of "the model forgets what it read last turn." Tool calls and results are now persisted to `chat_messages` and replayed to the LLM on every subsequent turn, using the OpenAI canonical `tool_calls`/`tool_call_id` pairing that Olava already speaks.

chat-uiinfrastructure

The original system prompt explicitly admitted the limitation: "You do NOT retain document content between conversation turns. You MUST call read_document at the start of every response." Every turn re-read every document regardless of whether anything had changed. The fix is a schema extension and a replay path, not a prompt workaround.

Migration 003_memory_persistence.sql adds four nullable columns to chat_messages: tool_call_id and tool_name (for role='tool' rows), assistant_text (raw model output including <tool_call> markup, stored separately from the events[] array used by the frontend), and assistant_tool_calls jsonb (the structured tool_calls array aggregated per turn). The migration also allows role='tool' rows alongside the existing user and assistant rows. All changes are additive; existing rows stay valid.

During tool execution, runToolCalls now inserts a role='tool' row for each result. The route handlers aggregate tool_calls from the turn and write assistant_text to the assistant row. On the next turn, buildMessages reads the full history - user rows, assistant rows with their tool_calls, and tool-result rows - and feeds the canonical sequence to the LLM. The system prompt warning is replaced with a positive instruction: use tool results already in the history rather than re-reading documents you've already inspected.

A follow-up commit fixed a display bug introduced in the first pass: mikeApi.getChat() was coercing every non-user row to role='assistant', so persisted tool-result rows were appearing as empty assistant bubbles on chat reload. The fix preserves role='tool' through the API response; ChatView's existing filter handles hiding them.

The prior workaround, enrichWithPriorEvents, which summarized prior tool names into the last assistant message, is deleted. Full replay makes it obsolete.

So what This is the strongest candidate in nwhitehouse's recent batch. The schema choice is sound and the replay approach is clean. Two things to be aware of before porting: first, this commit is closely coupled with feat-015 (audit log via `recordEvent`) and feat-014 (LoopController), both of which depend on the `chatId` parameter this commit introduces to `runLLMStream`. Port the three together to avoid a partial-refactor state. Second, deleting `enrichWithPriorEvents` is only safe because full replay is in place - a port that takes only the schema without the replay path would break the workaround it depends on.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

3 commits from nwhitehouse/mike, oldest first. Source extracted verbatim from the harvested git log.

SHA	Subject	Author	Date
`24e7b703`	[feat-017] Add Sprint 3 plan + scope feat-017 to Tier 1 in-chat memory	Nick Whitehouse	2026-05-07	↗ GitHub
commit body Documents the seven-story Sprint 3 (harness hardening + memory) and scopes feat-017 to in-chat tool result replay only. Project-memory (Tier 2) moved to feat-018 per user direction "I just want chat to remember what we're talking about through the whole conversation at a minimum." Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`563ffde8`	[feat-017] Persist tool calls + results so chat remembers across turns	Nick Whitehouse	2026-05-07	↗ GitHub
commit body Tier 1 of the memory hierarchy: in-chat tool result replay. Eliminates the "finch loses context" failure where every new turn re-read the same documents because the system prompt at chatTools.ts:730 told the model it could not retain doc content between turns. Schema (migration 003_memory_persistence.sql): - chat_messages.tool_call_id, tool_name (for role='tool' rows) - chat_messages.assistant_text (raw model output incl. <tool_call> markup) - chat_messages.assistant_tool_calls jsonb (structured tool_calls per turn) - All columns nullable; existing rows remain valid. Persistence + replay flow: - runLLMStream now takes chatId. Inside the runTools callback it inserts one role='tool' row per tool result and aggregates the structured tool_calls onto a turn-level array returned to the route handler. - Route handlers (chat.ts, projectChat.ts) persist assistant_text=fullText and assistant_tool_calls on the assistant row at end-of-turn (alongside the existing events[] which the frontend still uses for display). - Both routes now load conversation history from chat_messages directly rather than trusting the frontend payload, so prior tool results + assistant tool_calls flow into buildMessages on every turn. - buildMessages emits role='tool' rows with tool_call_id, and role='assistant' rows with their tool_calls + assistant_text - the canonical OpenAI {assistant.tool_calls} → {tool.tool_call_id} pairing. - LlmMessage type extended to support role='tool', tool_call_id, and tool_calls. Olava adapter passes them through; claude.ts and gemini.ts drop them (those paths are dead per coerceToOlava). Removed enrichWithPriorEvents - it was a partial workaround that summarised prior tool names into the last assistant message. Full tool result replay makes it obsolete. System prompt updated at chatTools.ts:752 - the "you do NOT retain document content" warning is replaced with positive guidance pointing the model at the tool history below. Frontend (ChatView.tsx) filters role='tool' rows from the rendered chat; visible UI is unchanged. MikeMessage type updated to admit the new role. Tier 2 (per-project persistent facts across chats) is deferred to feat-018 per scope decision. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`d55d7575`	[feat-017] Hide tool rows in getChat so they don't render as empty bubbles	Nick Whitehouse	2026-05-07	↗ GitHub
commit body Bug from the first pass: ChatView's role!=='tool' filter never fired because mikeApi.getChat() coerced every non-user row to role='assistant'. Result: each persisted tool result row became an empty assistant bubble on chat reload, crowding out the real tool cards (which render from the adjacent assistant row's events[] array). Fix: getChat now preserves role='tool' on the round-trip; ChatView's existing filter handles it from there. ServerMessage.role widened to include 'tool'. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-117.md from inside the repo you want the changes in.

⬇ Download capture-thread-117.md