[Mac app] Self-contained local mode - the whole stack runs inside Mike.app (stacked on #302)

🟢 open · #351 · open-legal-products/mike ← amal66/mike · opened 21d ago by amal66 · +9,340-20 across 42 files · ↗ on GitHub

From the PR description

Self-contained Mike.app - the whole stack runs inside the desktop app

Stacked on #302 (desktop shell; hosted default). This PR adds the third and final mode from desktop/docs/self-contained-plan.md: Run locally on this Mac. The app supervises the entire Mike stack - Postgres 17, GoTrue, PostgREST, gateway, backend, frontend - as loopback-only child processes. No Docker, no server, no account anywhere; documents and data never leave the machine.

The shell-over-rewrite principle survives intact: the web app and backend run byte-identical to the docker-compose stack. desktop/src/local/supervisor.js is docker-compose.yml reimplemented as a process tree; where a step looks odd, the compose file is the reference.

Rebased onto main 54681b55 (2026-08-24). All 10 required CI checks green on 8b398d93. Three commits are newer than the original text of this description and are now folded in below: 817a5439 (native drag regions - a real "nothing is clickable" blocker), 44eb7f5b (guest mode), and 8b398d93 (four CodeQL high alerts).

The hands-on testing writeup in this comment is still worth reading, but its commit SHAs died in the rebase. Current equivalents: cca05962b608f6c, 62b8314817a5439, 5500e8444eb7f5b.

What's in here

Backend (works for any single-node deploy, not just the desktop):

  • STORAGE_DRIVER=fs - a filesystem driver inside lib/storage.ts behind the same public API as S3/R2. Documents become plain files under STORAGE_FS_ROOT; S3 semantics (string-prefix listing, delete-missing-ok) are preserved and unit-tested.
  • Blob-token signed URLs - fs mode has no S3 presigner, so getSignedUrl returns an expiring HMAC capability URL (1 h at both call sites) on a new unauthenticated route GET /download/signed/:token with exactly presigned-URL trust semantics: minted by an authenticated route after its own access check, scoped to one object, expiring. Domain-separated from the permanent /download/:token HMAC by a literal "blob:" prefix inside the MAC input, so the two token kinds can't be replayed as each other (tested).

Frontend: output: "standalone" behind NEXT_OUTPUT_STANDALONE=1 - opt-in, Docker image and dev workflow untouched. Plus a Continue as guest button that only ever renders inside the local desktop product (below).

Desktop:

  • src/local/supervisor.js - ordered boot with health-check chain (postgres :42810 → gotrue :42811 → schema → postgrest :42812 → gateway :42813 → backend :42814 → frontend :42815), all bound 127.0.0.1; per-service log files under userData/local/logs/; clean shutdown on quit (postgres fast shutdown, reverse boot order, 10 s grace then SIGKILL); stale-pidfile recovery that only clears a pid nothing owns; fail-fast if any child dies during boot.
  • First-run bootstrap: initdb into userData/local/pgdata, the Supabase role ladder (incl. service_role with BYPASSRLS - schema.sql enables RLS with no policies, so the backend's role must bypass, exactly as the supabase/postgres image sets it), GoTrue's own auth migrations, then schema.sql + a migration ledger (public.mike_schema_migrations): fresh installs baseline all shipped migrations; upgrades apply only unseen ones.
  • Per-install secrets minted on first run (JWT secret, anon/service keys, DB password, download + API-key-encryption secrets, and the guest credentials) - no secret ships in the build, and secrets.json is written 0600. The frontend bundle carries only the well-known Supabase demo anon key as a placeholder; the local gateway proxy (src/local/gateway.js, a Node port of supabase/gateway.conf) swaps it for the per-install anon JWT in flight, by exact match, on apikey and Authorization only. Anon-for-anon only - the service-role key never passes through the gateway, so nothing routed through it can escalate.
  • First-launch chooser - a pristine launch of a stack-carrying build asks the one question every new user has: Use Mike Cloud / Run everything on this Mac (two cards) with a Connect to your own server... link underneath. Local-first is now a zero-knowledge decision, not a keyboard shortcut. Either choice persists and retires the chooser; any explicit signal (--server-url=, --local, a saved mode or serverUrl) skips it, so automation and returning users never see it; builds without the stack go straight to the hosted default.
  • Guest mode - the local login page offers Continue as guest: one click in, no typing, for a database living in your own ~/Library.
  • Connect screen also offers "Run locally on this Mac" (only when the build actually carries the stack); a boot progress page streams supervisor status; --server-url= still beats local mode so e2e can always retarget.
  • Scripts: local:fetch (pinned binaries: zonky Postgres 17.10 via npm, PostgREST v14.12 official macOS build, GoTrue v2.189.0 built from source - the upstream release asset named darwin-arm64 actually contains Linux ELF binaries), local:build (backend tsc + Next standalone with local URLs baked), local:stage + dist:local (package Mike.app with the whole stack in resources).
  • e2e/local.e2e.mjs - wipes its userData and drives the true first-run every time: cold initdb → six services → signup → project → library upload → row-menu download → both guest rounds, asserting bytes land under userData/local/storage and the download URL is a blob-token signed URL.

Replicate the base case (before this PR)

The point is to see that there is no no-cloud path that isn't Docker, and that a local deploy would have had nowhere to put documents even if there were.

  1. Check out main (or #302's tip) and run the shell: cd desktop && npm install && npm start. On #302's tip it loads the hosted service; point it somewhere unreachable (npm start -- --server-url=http://localhost:9) and you get the connect screen. The only thing the app can do is aim at a server. There is no local option in the menu, on the connect screen, or behind any flag - grep -r "local" desktop/src/main.js on #302 finds nothing of the sort.
  2. docker compose up is therefore the only way to run Mike without an account somewhere. Note what that costs a non-technical user: Docker Desktop installed, a terminal, a .env, and ~6 containers.
  3. Now try the piece that would be needed anyway. On main, grep -n 'STORAGE_DRIVER' backend/src/lib/storage.ts → no match: storage is S3/R2-only. Start the backend with no R2_* config and upload a document - storage is simply disabled. There is no filesystem driver, and getSignedUrl has no non-S3 path, so even a hand-rolled local deploy could not hand a browser a download link.

Replicate this PR

Prerequisites for local:fetch (it builds GoTrue from source and hydrates Postgres dylib symlinks): go (brew install go), python3, git, curl, npm. Each of its three fetch steps is skipped if the artifact is already present.

git checkout <branch>
cd desktop && npm install
npm run local:fetch     # pinned pg 17.10 (npm) + postgrest v14.12 + gotrue v2.189.0 from source
npm run local:build     # backend tsc + Next standalone with local URLs baked (few minutes)
MIKE_E2E_DEV=1 npm run e2e:local   # automated first-run proof, dev mode

Interactively:

npm run start:local     # === electron . --local ; boots the stack, lands on the local login

Then, by hand:

  1. Watch the boot progress page stream supervisor status. First run does a cold initdb and takes about a minute; later runs are much faster.
  2. Land on the local login at http://localhost:42815/login. Click Continue as guest → straight into the product, no typing. (Or sign up normally - it autoconfirms offline.)
  3. Create a project, upload a document in Library, download it back from the row menu.
  4. Look at the disk: ~/Library/Application Support/Mike/local/ now holds pgdata/, storage/ (your document as a plain file), logs/ (one per service), and secrets.json (mode 0600).
  5. Prove the download is the new capability URL: in DevTools → Network, the download URL contains /download/signed/, not an X-Amz-Signature presigned URL.
  6. Prove it's offline: DevTools → Network with "record" on through boot → login → signup → authenticated app. Every request targets localhost:42813/42814/42815. (The only outbound calls local mode ever makes are to whatever LLM provider you configure - or localhost:11434 with Ollama - and CourtListener if you use it.)

Packaged build (what a downloader actually gets):

npm run local:stage && npm run dist:local
open dist/mac-arm64/Mike.app

Wipe ~/Library/Application Support/Mike first to see the true first launch: the chooser appears with Use Mike Cloud / Run everything on this Mac / Connect to your own server.... Pick local. Then click something with a real mouse - see the drag-region note below for why that specific instruction matters.

Run the packaged first-run e2e (same script, no MIKE_E2E_DEV):

npm run e2e:local

Two fixes from hands-on first-run testing

Testing the packaged app as a real downloader surfaced one blocker and one UX gap. Full writeup in the PR comment; summarized here because a description shouldn't hide its own blocker fix.

817a5439 - nothing in the product was clickable after local boot

Base case (on 2b608f6c): package with local:stage + dist:local, wipe ~/Library/Application Support/Mike, open the app, choose "Run everything on this Mac", wait for the login page → no physical click lands anywhere. The page is healthy (it renders and hydrates; a CDP-injected click focuses #email) - native clicks simply vanish.

Cause: local-boot.html marked its whole body -webkit-app-region: drag with no no-drag island, and Chromium only replaces a window's native drag regions when the next document reports regions of its own. The web app declares none, so the boot page's full-window drag region stayed in force over the product - and a drag region consumes every mouse event (electron/electron#1354).

Why every test stayed green: the e2e suites click over CDP, which injects below the native drag layer. This bug is invisible to the entire automated suite, on both sides of the fix.

Fix: after every committed http(s) navigation the shell inserts html, body { -webkit-app-region: no-drag; } on did-navigate, forcing the renderer to report a fresh (empty) region set; the boot page also gets the same no-drag content island as welcome/connect.

Replicate the fix: same steps on 817a5439 or later - the login page accepts clicks. The decisive check is a physical click.

44eb7f5b - "Continue as guest" in local mode

Base case: local mode demanded an email+password signup for a database living in your own ~/Library.

Now: the login page shows Continue as guest - one click in. The guest is a real GoTrue user whose random per-install password lives with the other first-run secrets in userData/local/secrets.json; auth, sessions, and the data model stay byte-identical to a server deploy. The shell hands the credentials over one gated IPC call (mike:guest-credentials): local mode and a sender frame on the local frontend's origin (http://localhost:42815). A hosted page gets null; in a plain browser window.mikeDesktop doesn't exist, so the button never renders and the page behaves identically everywhere else.

Note this is the one privileged bridge call the product web app can make. #302's invariant was "the web app gets zero privileged APIs"; this PR qualifies it to "one origin-gated, local-mode-only read of a local credential", and the preload comment says so. That qualification is the price of the one-click local experience - flagged here rather than buried.


Verification

Automated gates - all 10 required CI checks green on 8b398d93 (rebased onto 54681b55):

Gate Result
Backend build and tests 744 passed / 25 skipped, 62 files
Frontend build and tests 590 passed, 95 files
Supabase stack integration tests pass
Fresh install vs upgraded deployment pass
playwright 27 passed
CodeQL / Analyze (javascript-typescript) pass (four high alerts fixed by 8b398d93)
Eval harness · gitleaks (full history) · CLA pass

Corrected: earlier text here claimed "621 backend tests". That number predates several commits; CI on the current tip reports 744 passed / 25 skipped.

New backend tests in this PR - storageFs.test.ts, 11 it() blocks / 14 runtime cases:

  • filesystem storage driver (6) - enabled by STORAGE_DRIVER=fs with no R2 config; upload → download → delete round-trip; deleteFile tolerates a missing key (S3 semantics); listFiles matches S3 string-prefix semantics, not directories; getSignedUrl returns an expiring blob-token URL on the backend; rejects keys that escape the storage root.
  • storage-root containment (3 blocks, 6 cases, added by 8b398d93) - parent traversal, deep traversal, a traversal that lands back inside, an absolute key, and the sibling-prefix off-by-one (/data/store-evil vs /data/store) all throw across every fs entry point; a normal key still lands inside the root. Both mutations these tests exist to catch - deleting the guard, and comparing against bare root - fail the suite.
  • blob tokens (2) - expired tokens verify as null; blob and permanent download tokens are not interchangeable.

Local-mode e2e (MIKE_E2E_DEV=1 npm run e2e:local) - PASSED on a wiped userData (true first-run) every time. All seven assertions:

✓ local stack booted from scratch; app routed to /login
✓ signed up + auto-signed-in as local-e2e-...@example.com (no network)
✓ project "Local E2E ..." created (data in local postgres)
✓ upload landed as plain files under userData/local/storage (1 object[s])
✓ downloaded back via blob-token URL: local-doc-....pdf
✓ guest mode, first click (creates the guest account): /login → signed-in product
✓ guest mode, second click (signs into it): /login → signed-in product
LOCAL E2E PASSED

Packaged build proven - local:stage + dist:local produce a Mike.app carrying the whole stack (548 MB in Contents/Resources/local-stack, pg dylib symlinks preserved), and the same first-run e2e run against the packaged app (no MIKE_E2E_DEV) passes end to end. First-launch chooser verified on the packaged app, pristine userData each time: chooser renders → "Run everything on this Mac" boots the stack and lands in the local product; "Use Mike Cloud" loads the hosted app and a relaunch goes straight there (chooser retired).

No-web proof - with the renderer's network recorded through boot → login → full signup → authenticated app load, every request targeted loopback (localhost:42813/42814/42815) - zero external hosts.

Remote-mode e2e re-run against the compose stack with this exact shell build: flows.e2e.mjs 11/11 and app.e2e.mjs PASSED (the S3 presigned-URL path is untouched by this PR). Both also re-passed against the packaged build.

✅ Stale e2e signup steps - FIXED (source-verified; execution pending the next packaged run)

All desktop e2e results above were recorded before the 2026-08-24 rebase, which brought in main's rewritten signup page (Email / Password (Min. 10 Characters) / Confirm Password, then a hand-off to the two-step /onboarding wizard). All three suites still filled the old Your name / Your organisation / Create a password (min. 6 characters) form, so the signup step would have hung on a missing placeholder and thrown - killing each run before a single shell behavior was asserted.

Fixed in test(desktop): teach the e2e suites the real signup flow and the host's arch:

  • Signup now fills the real Email / Password / Confirm Password fields by label (the inputs carry no placeholders any more) and submits by accessible name, so it can never resolve to the Google button in the same form.
  • A new completeOnboardingIfRequired() walks the wizard - profile step (Name / Organisation → Continue) then practice step (Skip) → /assistant. It is a port of the web suite's e2e/onboarding.ts, including the trick that makes it correct: because OnboardingGate lets a returning account touch /onboarding/profile for a moment before bouncing it to /assistant, the URL alone can't distinguish the two cases - so it waits for either the step-1 Continue button or the assistant composer.
  • Local mode's two guest rounds both run it, and that asymmetry is now part of the proof: round 1 (new guest account) walks the wizard, round 2 (same account returning, onboarding complete) is bounced straight to /assistant and the helper no-ops.
  • The existing "URL is no longer /login|/signup" assertions stay and stay correct (/onboarding/profile satisfies them); the sidebar Assistant wait moved to after the wizard, where onboarding's sidebar-less shell makes it a real assertion again.
  • Same commit fixes a second latent break: the hardcoded dist/mac-arm64 app path is now derived from process.arch (arm64 → mac-arm64, x64 → mac) at one construction point - on an Intel Mac the suites spawned a nonexistent path and died with ENOENT.
  • All four shared pieces (arch path, signup, onboarding, first-run-overlay dismissal) now live in one desktop/e2e/helpers.mjs instead of three divergent copies.

Verification status, stated plainly: the suites could not be executed for this fix - they need a packaged Mike.app plus a running stack. Every touched .mjs passes node --check, the helper module loads and resolves its exports, and every selector was source-verified against this branch's own components (signup/page.tsx, onboarding/profile, onboarding/practice, OnboardingGate.tsx, login/page.tsx's guest handler, ChatInput's role=combobox named "How can I help?", passwordPolicy.ts's min. 10). A green packaged run is still owed and the numbers above should be re-recorded on the next one.

The stack, storage, guest, and packaging code was never affected - only the test's selectors. No CI job runs desktop/, which is precisely why this rotted unnoticed. Everything in "Replicate this PR" that you drive by hand still holds.


Tradeoffs / design decisions (explicit)

  1. Bundle real GoTrue + PostgREST + Postgres binaries instead of reimplementing their APIs: 436 PostgREST call sites and the full GoTrue MFA surface make shims the expensive, drift-prone path. Cost: ~80 MB of binaries and a from-source GoTrue build step (upstream's macOS release assets are mislabeled Linux binaries), so local:fetch needs a Go toolchain and python3 on the builder's machine.
  2. pg node client instead of psql for bootstrap/migrations - the zonky Postgres build ships no psql. Multi-statement files run over the simple protocol in one implicit transaction; verified nothing in schema.sql/migrations is non-transactional. All-or-nothing per migration file is a feature for a ledgered runner.
  3. Fixed ports (42810-42815) because Next bakes its API origins at build time. A port conflict fails loudly with a clear message instead of auto-shifting (which would strand the baked URLs). Caveat worth knowing: only the in-process gateway (:42813) raises a dedicated bind error; a collision on any of the other five surfaces as a health-check timeout or a child-exit message. Legibility of that path is on the follow-up list.
  4. Placeholder-anon-key swap at the gateway rather than baking a real key: per-install secrets never exist in the bundle, and the well-known demo JWT secret is never trusted by the local services. The swap is exact-match and anon-for-anon only; the service-role key is never routed through the gateway.
  5. No SMTP in v1 - signup autoconfirms offline; email password recovery is unavailable in local mode and the connect screen says so. (An SMTP-sink surfacing mails in-app is a possible follow-up.) Guest mode reduces how often this matters.
  6. LibreOffice detect-don't-bundle - supervisor sets SOFFICE_BINARY_PATH when /Applications/LibreOffice.app exists; otherwise the product's existing docx-preview fallback applies. Bundling would add ~700 MB.
  7. Blob-token route is unauthenticated by design - it replaces S3 presigned URLs, which carry no session either; the token is the expiring, object-scoped capability. The permanent /download/:token route (auth + DB access check) is untouched, and the two HMAC domains are separated by a literal "blob:" prefix in the MAC input.
  8. LLM calls still need a key or Ollama - unchanged product behavior; keyless demo mode remains a follow-up (#260/#259 lineage).
  9. Guest credentials live in userData/local/secrets.json (0600). Readable by anything on the machine that can read that directory - but so is pgdata sitting beside it, so this is the same trust boundary, not a new exposure. Wiping secrets.json while keeping pgdata orphans the guest account; the button surfaces the auth error rather than silently recovering, because the shell should never delete account data on its own. Supported reset stays "delete userData/local". Existing installs get the guest fields topped up in place on next read, with no version bump.
  10. Guest mode qualifies #302's zero-privileged-APIs invariant to exactly one origin-gated, local-mode-only IPC read. Rejected alternative: a fixed credential baked into the build - that would put a real secret in the bundle, which the whole per-install-secrets design exists to avoid.
  11. Drag-region reset on did-navigate, not a boot-page-only fix. Fixing only local-boot.html would leave the same trap for the next full-window drag region anyone adds. Resetting after every committed http(s) navigation makes the product's clickability independent of what the previous document declared. Cost: the CSS insert runs on every navigation, and did-navigate-in-page (SPA route changes) is deliberately not hooked
    • the initial commit into the product already resets the regions, and the product declares no drag regions of its own.
  12. CodeQL fixes (8b398d93) changed the shape of two checks, not their strength. The path guard was already safe, but if (resolved !== root && !resolved.startsWith(root + sep)) only establishes a disjunction on fall-through, which CodeQL's barrier-guard analysis cannot discharge - hence three js/path-injection alerts on mkdir/writeFile/unlink. Reduced to a single if (!resolved.startsWith(rootPrefix)) throw, falling through proves exactly one thing: containment. Behavioural delta is one case - a key resolving exactly to the root is now rejected instead of returned; no call site can produce one, and handing back the root directory as a file key was never meaningful. The fourth alert (js/incomplete-url-substring-sanitization) is the same array-vs-string .includes heuristic described in #302. Where a safe check is illegible to a machine, make it legible rather than suppress the alert.

Signing (wired + documented; needs the org certificate to run)

Notarization requires every nested Mach-O to carry its own hardened-runtime signature - scanning local-stack/ found 115 (postgres + dylibs, gotrue, postgrest, plus natives nobody would hand-list: skia in the backend's @napi-rs/canvas, sharp/libvips in the frontend standalone). So the signed config is generated at build time (scripts/make-signed-local-config.mjs scans for Mach-O magic bytes and emits electron-builder.release.local.json with mac.binaries filled), and npm run dist:local:signed builds a notarized DMG under the same APPLE_* env contract as the shell's dist:signed. The operator runbook - including nested-binary verification and the two constraints that bite (a signed bundle is sealed, so pg symlinks must be intact at packaging time; no speculative entitlement exceptions) - is in desktop/README.md → "Signing the self-contained build". Like #302's signed path, it can only be run with the org's certificate, so it is wired and documented but not claimed as verified here.

For a human reviewer to check

  1. A physical click after local boot on a freshly packaged build. The single most important manual check on this branch: CDP clicks pass on both sides of the drag-region bug, so no automated suite can prove the fix. Use a real trackpad.
  2. npm run local:fetch end to end. Network- and toolchain-dependent (GitHub release for PostgREST, a git clone of supabase/auth at v2.189.0, npm for the zonky package; go and python3 locally). The claim that upstream's darwin-arm64 GoTrue asset is actually Linux ELF is also only verifiable by running it.
  3. The packaged-bundle numbers - 548 MB local-stack, 115 nested Mach-O binaries, pg dylib symlinks surviving packaging - all require running local:stage + dist:local and inspecting the bundle.
  4. The upgrade path over an existing pgdata. The ledger's non-fresh branch (apply only unseen migrations) is never exercised by the e2e, which wipes userData every run. Worth a manual pass: boot once, add a migration, boot again.
  5. service_role on a pre-existing install. bootstrapRoles sets BYPASSRLS only on the CREATE path and never ALTERs an existing role. Unreachable for anyone who first installs from this build, but not proven safe for an install whose roles predate it.
  6. Port-conflict legibility - see tradeoff 3. Confirm the error page is actually readable when something else already owns :42815.
  7. Cold-boot timing on a clean Mac with no Postgres installed. welcome.html promises "about a minute"; the e2e allows 180 s.
  8. Executing the signed build (org Apple Developer certificate - human-only).

Not in this PR (follow-ups)

  • A CI job that actually runs desktop/e2e - nothing does today, which is how the signup steps rotted unnoticed (the steps themselves are now fixed; see the note above).
  • Executing the signed build (org Apple Developer certificate - human-only).
  • Backup/export menu item; Postgres major-version upgrade path (pinned to 17).
  • SMTP sink; demo mode.

🤖 Generated with Claude Code

Our analysis

Run the full Mike stack inside the Mac app — read the full analysis →

Think the analysis missed something the PR description covers?

Capture this PR into my fork

Download a Markdown prompt that tells Claude how to port every commit in this PR into your working tree. Run it via claude -p < capture-pull-351.md from inside the repo you want the changes in.

⬇ Download capture-pull-351.md