[Fix 01] Batch the directory modal's project fetch (N+1 getProject storm)
From the PR description
Summary
Every directory picker (Add Documents modal, Use Workflow modal, the assistant project selector) loads its Projects tab through useDirectoryData, which fired one GET /projects/:id for every existing project the moment the modal opened. Each of those requests costs a GoTrue auth verification plus ~6 PostgREST queries, so an account with N projects hit the Supabase gateway with an ~8xN-request burst per modal open. This PR replaces the fan-out with a single batched request.
Changes
backend/src/routes/projects.ts:GET /projectsaccepts?include=documentsand returns each project's documents from one batcheddocumentsquery (.in("project_id", ...)), running the same attach helpers asGET /projects/:idonce across all documents. Response shape without the flag is unchanged.frontend/src/app/lib/mikeApi.ts:listProjects()takes an optional{ includeDocuments }.frontend/src/app/components/shared/useDirectoryData.ts:loadProjects()makes onelistProjects({ includeDocuments: true })call instead oflistProjects()+Promise.all(projects.map(getProject)).
Why
The burst is not just slow - it takes the gateway down. Reproduced against the Supabase CLI stack with a 43-project account: overlapping modal-open storms (6 concurrent opens, 8 rounds, a project-create POST fired mid-burst each time, i.e. exactly what the e2e suite does) drove GoTrue into Postgres connection exhaustion - failed to connect to host=supabase_db_mike user=supabase_auth_admin ... context deadline exceeded - producing 467x 500 + 641x 504 on /auth/v1/user in a single run; 38/48 project creates and 1458/2064 project GETs failed. On slower CI runners Kong surfaces the same collapse as the intermittent 502 ("An invalid response was received from the upstream server") that the e2e suite (#220) had to retry around. Real users with accumulated projects hit the same wall.
Testing
- Same storm harness after the fix (one batched request per modal open, same overlap and mid-burst POSTs): 0 failures - 48/48 GETs and 48/48 creates OK, zero 5xx anywhere in the gateway logs.
- Backend
tsc --noEmitclean; frontend production build clean. - Full Playwright e2e suite run 5x consecutively against a production build with this fix (including 3 runs after the retry scaffolding was stripped from the specs in #220): 23/23 non-LLM specs green every run, zero Playwright retries, with ~40 accumulated projects in the test account.
🤖 Generated with Claude Code
Our analysis
Batch project document loading for directory pickers — read the full analysis →
Think the analysis missed something the PR description covers?
Commits in this PR (1)
| SHA | Subject | Author | Date | |
|---|---|---|---|---|
36cddb25 | fix: batch directory-modal project fetch instead of N+1 getProject burst | QA Runner | 2026-07-20 | ↗ GitHub |
commit bodyEvery directory picker (AddDocumentsModal, UseWorkflowModal, the assistant
project selector) loads its "Projects" tab through useDirectoryData, which
fired one GET /projects/:id for EVERY existing project the moment the modal
opened. Each of those requests costs an auth verification against GoTrue
plus ~6 PostgREST queries, so an account with N projects produced an
~8xN-request burst on the Supabase gateway per modal open.
Under that burst the gateway genuinely falls over: measured locally against
the Supabase CLI stack, overlapping modal-open storms drove GoTrue into
Postgres connection exhaustion ("failed to connect ... context deadline
exceeded") and produced 467x 500 + 641x 504 on /auth/v1/user in a single
run - surfacing to users as failed project creates/loads, and to the e2e
suite as the intermittent Kong 502s its specs currently retry around.
Fix: GET /projects now accepts ?include=documents and returns each
project's documents from one batched query (same attach helpers as
GET /projects/:id, run once across all documents), and useDirectoryData
uses it. A modal open is now 1 API request and a fixed number of DB
queries regardless of project count. After the change the same storm
harness produced zero 5xx and zero auth failures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
| ||||
Capture this PR into my fork
Download a Markdown prompt that tells Claude how to port every
commit in this PR into your working tree. Run it via
claude -p < capture-pull-238.md from
inside the repo you want the changes in.