nwhitehouse makes thousand-document reviews survive a closed laptop

The fork rebuilds its bulk document-review engine so a long run no longer dies when a tab closes, a connection times out, or the server restarts.

discoveryinfrastructure

Running an AI review across a few documents is easy. Running it across five to ten thousand - the scale this fork is actually built for - is where the old approach broke. Everything ran inside a single live web request, so a closed browser tab, an idle connection, or a server hiccup mid-run wiped hours of work with nothing to resume from.

nwhitehouse swapped that for a proper job queue: every document is logged as its own task, work is handed out to background workers, and progress is saved as it goes. If a worker crashes, another picks up where it left off within minutes. You kick off a run, walk away, and check back on its status later - no babysitting the browser. Failed items are flagged automatically rather than disappearing.

So what Anyone running AI across large document sets - bulk discovery, diligence, big contract portfolios - should look here before their first overnight run dies at 80%.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

1 commit from nwhitehouse/mike, oldest first. Source extracted verbatim from the harvested git log.

SHA Subject Author Date

a663f8df [bug-007] Tabular generate as durable job + worker pool (5K-10K-doc scale) Nick Whitehouse 2026-05-07 ↗ GitHub

SHA	Subject	Author	Date
`a663f8df`	[bug-007] Tabular generate as durable job + worker pool (5K-10K-doc scale)	Nick Whitehouse	2026-05-07	↗ GitHub
commit body Replaces the previous SSE-streaming generate handler with a durable job table + in-process worker pool + frontend polling. The previous design worked at 200-doc scale but fell over at the actual product target (5K-10K-doc tabular projects): - vLLM cannot serve N concurrent inference requests at that scale - the request handler tied up an Express worker for hours - SSE was a single point of failure (proxy idle, browser tab close, backend restart all killed the run with no recovery) - in-flight progress was lost on a restart Schema (migration 005): - tabular_jobs(id, review_id, status, total_items, started_at, completed_at, cancel_requested_at, error, ...) - tabular_job_items(id, job_id, document_id, status, attempt_count, lease_expires_at, error, ...) - claim_tabular_job_item(lease_seconds) RPC: atomic worker claim via FOR UPDATE SKIP LOCKED. Multi-instance safe. - RLS via the existing can_access_review() predicate. Backend: - lib/tabularJobs.ts: extraction + LLM helpers moved here so the worker doesn't import a route file (no circular deps); added createGenerateJob, claimNextItem, processOneJobItem, maybeFinalizeJob, TabularWorkerPool. - routes/tabular.ts: POST /generate now creates a job and returns immediately. New endpoints: GET /jobs/:id, GET /jobs/:id/cells, POST /jobs/:id/cancel, GET /reviews/:id/active-job. - index.ts: TabularWorkerPool started after app.listen(); SIGTERM/ SIGINT shutdown stops the loops gracefully (in-flight items expire their lease and the next worker reclaims them). Frontend: - mikeApi.ts: removed streamTabularGeneration; added startTabularGenerate, getTabularJob, getTabularJobCells, cancelTabularJob, getActiveTabularJob. - TabularReviewView.tsx: EventSource reader replaced with a pollJob loop that surfaces 12/200 progress live and resumes automatically on remount via getActiveTabularJob. Env knobs: TABULAR_GENERATE_CONCURRENCY (workers, default 10), TABULAR_JOB_LEASE_SECONDS (300), TABULAR_WORKER_IDLE_MS (500), NEXT_PUBLIC_TABULAR_POLL_MS (1500). Verified: - tsc --noEmit clean (backend + frontend) - all 16 backend tests pass (no regressions) - migration applied to local Supabase; 6 RLS policies + claim RPC + tables + indexes in place. Known limitation: in-flight cells (worker mid-way through one doc's columns) aren't surfaced to the frontend until the item reaches a terminal state. Matches the user's mental model that a doc's row of cells appears together when its turn finishes. If live per-cell streaming becomes a requirement, add tabular_cells.updated_at + a separate query path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

commit body

Replaces the previous SSE-streaming generate handler with a durable job
table + in-process worker pool + frontend polling. The previous design
worked at 200-doc scale but fell over at the actual product target
(5K-10K-doc tabular projects):
  - vLLM cannot serve N concurrent inference requests at that scale
  - the request handler tied up an Express worker for hours
  - SSE was a single point of failure (proxy idle, browser tab close,
    backend restart all killed the run with no recovery)
  - in-flight progress was lost on a restart

Schema (migration 005):
  - tabular_jobs(id, review_id, status, total_items, started_at,
    completed_at, cancel_requested_at, error, ...)
  - tabular_job_items(id, job_id, document_id, status, attempt_count,
    lease_expires_at, error, ...)
  - claim_tabular_job_item(lease_seconds) RPC: atomic worker claim via
    FOR UPDATE SKIP LOCKED. Multi-instance safe.
  - RLS via the existing can_access_review() predicate.

Backend:
  - lib/tabularJobs.ts: extraction + LLM helpers moved here so the
    worker doesn't import a route file (no circular deps); added
    createGenerateJob, claimNextItem, processOneJobItem,
    maybeFinalizeJob, TabularWorkerPool.
  - routes/tabular.ts: POST /generate now creates a job and returns
    immediately. New endpoints: GET /jobs/:id, GET /jobs/:id/cells,
    POST /jobs/:id/cancel, GET /reviews/:id/active-job.
  - index.ts: TabularWorkerPool started after app.listen(); SIGTERM/
    SIGINT shutdown stops the loops gracefully (in-flight items
    expire their lease and the next worker reclaims them).

Frontend:
  - mikeApi.ts: removed streamTabularGeneration; added
    startTabularGenerate, getTabularJob, getTabularJobCells,
    cancelTabularJob, getActiveTabularJob.
  - TabularReviewView.tsx: EventSource reader replaced with a
    pollJob loop that surfaces 12/200 progress live and resumes
    automatically on remount via getActiveTabularJob.

Env knobs: TABULAR_GENERATE_CONCURRENCY (workers, default 10),
TABULAR_JOB_LEASE_SECONDS (300), TABULAR_WORKER_IDLE_MS (500),
NEXT_PUBLIC_TABULAR_POLL_MS (1500).

Verified:
  - tsc --noEmit clean (backend + frontend)
  - all 16 backend tests pass (no regressions)
  - migration applied to local Supabase; 6 RLS policies + claim RPC
    + tables + indexes in place.

Known limitation: in-flight cells (worker mid-way through one doc's
columns) aren't surfaced to the frontend until the item reaches a
terminal state. Matches the user's mental model that a doc's row of
cells appears together when its turn finishes. If live per-cell
streaming becomes a requirement, add tabular_cells.updated_at + a
separate query path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-150.md from inside the repo you want the changes in.

⬇ Download capture-thread-150.md