[feat-009] Vision perf: parallel render + tiered cache + progress UI
Four wins for the vision-mode wait time, ordered by user impact: 1) PROGRESS UI. Backend emits vision_render_start/done SSE events around the pdftoppm call. Frontend renders a "Reading <filename>..." block (matching the existing DocReadBlock pattern) instead of a dead spinner. SSE stream now opens BEFORE the render so the placeholder reaches the browser immediately. ~10s+ wait now feels intentional rather than broken. 2) PARALLEL RENDER. pdftoppm is CPU-bound; one process handles only one page at a time. Split into 4 workers via -f/-l page ranges, each writing to its own subdir to avoid filename collisions. 75 pages went from 28s → 11.7s on bench. Page count discovered via pdfinfo before splitting (also from poppler-utils). 3) IN-MEMORY LRU CACHE (visionCache.ts). 5-entry cap (composites are 1-2MB each, ~30MB per 75-page doc - keeps worst-case ≤150MB resident on the 512MB Railway box). Subsequent turns against the same doc skip render entirely; sub-millisecond hit. No SSE placeholder events on a memory hit so the UI doesn't flicker. 4) R2 PERSISTENT CACHE (visionR2Cache.ts). Sits behind memory cache. Single JSON manifest at vision-cache/<base64url(storagePath|p|d)>.json contains the array of base64 composites. Survives backend restarts and Railway redeploys. Render → memory write → fire-and-forget R2 write; subsequent processes hit R2 once, then promote to memory. Errors swallowed - cache is best-effort. Combined effect on a 75-page PDF: - First chat ever: ~12s render, ~5MB R2 write - Same chat session: sub-ms (memory) - After backend restart: ~1-2s (R2 read + parse) - New process or doc: back to first-chat numbers Files: - backend/src/lib/pdfRender.ts: parallelise pdftoppm; pdfinfo page count - backend/src/lib/visionCache.ts: new - in-memory LRU - backend/src/lib/visionR2Cache.ts: new - R2-backed manifest - backend/src/lib/visionContext.ts: tiered lookup + SSE events + write hookup - backend/src/routes/chat.ts: open SSE before render so placeholder ships - frontend/src/app/hooks/useAssistantChat.ts: handle vision_render_start/done - frontend/src/app/components/assistant/AssistantMessage.tsx: VisionRenderBlock - frontend/src/app/components/shared/types.ts: vision_render variant on AssistantEvent Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
| Repository | nwhitehouse/mike |
|---|---|
| Author | Nick Whitehouse <nick.whitehouse@mccarthyfinch.com> |
| Authored | |
| Parents | 26ef15f4 |
| Stats | 8 files changed , +472 , -45 |
| Part of | Vision mode: PDF page images → Olava (feat-007a / 008 / 009 / 010 / bug-005) |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-6bf6d52d.md
from inside the repo you want the change in.