feat: add resilient direct upload sessions
From the PR description
Summary
- Add direct-to-object-storage upload sessions for the web app and Word add-in.
- Process each file independently as soon as its upload completes, while allowing multiple concurrent sessions.
- Improve document-table selection, actions, optimistic state, upload placeholders, and context-aware default sorting.
- Bound memory and fan-out for document processing and folder ZIP downloads.
What changed
Direct upload architecture
- Replaced multipart API uploads with signed R2/S3 PUT URLs and upload-session manifests.
- Added per-file completion, independent processing jobs, lease-based worker claims, retries, and partial-success reporting.
- Removed the redundant bulk session-completion path so clients use one per-file protocol.
- Added resilient status polling with backoff and transient-error retries.
- Made mutation, polling, and hourly session limits configurable with safe defaults.
- Persisted uploader email on sessions so worker-generated audit events retain attribution.
- Added browser upload support to both the web app and Word add-in.
Document tables and actions
- Unified file and folder selection, including Shift-range selection, Cmd/Ctrl additive selection, and select-all.
- Applied context-menu actions to the full current selection and added relevant View/Open actions.
- Added mixed file/folder download and delete behavior with optimistic row removal.
- Persisted upload placeholder rows across table remounts and animated the
Uploading./Uploading../Uploading...state. - Kept library files and templates sorted by last updated by default, while project-root contents default to name sorting.
- Updated workflow actions so reference files can be viewed and Edit routes to the workflow editor.
Memory and reliability hardening
- Streamed ZIP exports from object storage through the backend instead of buffering all source files and the complete archive.
- Added resolved-document and total-byte caps, bounded metadata/download work, and collision-safe archive names.
- Streamed sealed upload objects to temporary files for document processing to avoid holding whole files and duplicate copies in memory.
- Improved per-file client error collection so failures surface immediately instead of becoming delayed
upload_incompleteerrors. - Added deterministic cleanup ownership and storage error handling.
Database, CI, and deployment
- Consolidated the branch's upload-session SQL into a single pre-merge migration and synchronized
backend/schema.sql. - Added MinIO CORS configuration and verification to browser-upload CI and local-stack setup.
- Updated deployment documentation and environment examples for upload-session limits and browser-reachable storage endpoints.
- Updated the legacy upload regression test to cover both the new session validation path and the retired endpoint's
410response.
Why
Large uploads previously passed through the API and delayed document creation until an entire batch finished. This change sends bytes directly to object storage, lets completed files enter processing immediately, supports overlapping upload batches, and avoids request-sized memory spikes in processing and ZIP export paths. The table changes make mixed file/folder operations consistent with familiar desktop selection behavior.
Testing
npm test --prefix backend- 76 files passed, 4 skipped; 896 tests passed, 26 skipped.npm test --prefix frontend- 104 files passed; 667 tests passed.npm run build --prefix backendnpm run build --prefix frontendnpm run typecheck --prefix word-addin- Targeted upload-session, storage, processing, selection, and ZIP-export tests.
git diff --check
Full Playwright browser E2E was not run locally; the CI storage CORS setup is included for that suite.
Our analysis
Add direct object-storage upload sessions — read the full analysis →
Think the analysis missed something the PR description covers?
Capture this PR into my fork
Download a Markdown prompt that tells Claude how to port every
commit in this PR into your working tree. Run it via
claude -p < capture-pull-397.md from
inside the repo you want the changes in.