amal66 makes Mike prove its core journeys before shipping

This fork adds a real browser check that can catch broken legal-workflow basics before a pull request lands.

workflowinfrastructure

amal66 has set up a release-quality test run that opens Mike in a browser and walks through the work people actually do. It uses a production-style environment, so it is checking the product users receive rather than a more forgiving development setup.

  • Sign-in and account settings, so access changes get checked end to end.
  • Projects and documents, including the path from upload to asking a question.
  • Chat, with AI-answer checks enabled when an Anthropic key is deliberately supplied.
  • Tabular reviews and workflows, covering two structured ways teams organise legal work.

The work also fixed a request surge that could cause temporary gateway failures when users opened project-selection screens with many projects. Failed test runs retain diagnostic records, making regressions easier to investigate.

So what Legal teams evaluating Mike should care because its most visible work paths are now far less likely to break unnoticed between releases.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

17 commits from amal66/mike, oldest first. Source extracted verbatim from the harvested git log.

SHA Subject Author Date
94987e6b test: Playwright e2e suite (auth, chat, projects, tabular reviews, workflows) Amal 2026-07-17 ↗ GitHub
commit body
Port of the amal66/mike fork's Playwright end-to-end suite onto the
upstream backend/ + frontend/ layout:

- e2e/: auth.setup (bootstraps a confirmed e2e@mike.local user via the
  Supabase admin API and saves storageState), auth-flows, critical-path
  (create project -> upload PDF -> ask a question -> streamed response),
  chat-management, project-management, tabular-reviews,
  workflows-account; fixtures/test.pdf
- playwright.config.ts: single-worker (shared test user), setup project
  + chromium project with saved auth state; webServer adapted from the
  fork's monorepo command (npm run dev --workspace apps/web) to
  upstream's layouts: backend `npm run dev` (health-checked on :3001)
  and frontend `npm run dev` (:3000)
- root package.json (upstream has none): @playwright/test, typescript
  dev-deps and test:e2e scripts, trimmed from the fork's root manifest
- root tsconfig.json scoped to e2e/ + playwright.config.ts so
  `npx tsc --noEmit` typechecks the suite
- .gitignore: playwright artifacts + e2e/.auth (session tokens)

Specs select by ARIA role/name and placeholder text only - no
data-testid attributes, so no app-code changes are required.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CEguyEgXa9JjCciXCcVemC
ffad860e fix(e2e): realign selectors with the olp UI and upstream route shape Amal 2026-07-17 ↗ GitHub
commit body
upstream-main (olp's tip) ships the #215/#216 liquid-surfaces UI, so the same
selector drifts fixed on the fork's suite (commit 3d46814 on main) also apply
here. Ports those realigned hunks, plus one upstream-specific route fix.

  * Chat input placeholder is "How can I help?", not
    "Ask a question about your documents..." (ChatInput/TRChatInput). Updated in
    chat-management (rename/delete/project-assistant) and critical-path.
  * The project-assistant empty state replaced the "+ Create New" text link with
    a PillButton reading "Create" (ProjectAssistantTable) - critical-path and
    chat-management now use getByRole("button", { name: "Create" }).
  * The sidebar chat row's active marker is APP_SURFACE_ACTIVE_CLASS
    ("bg-app-surface-active"), not "bg-gray-200/60"; the row wrapper is now h-8,
    not h-9 (SidebarChatItem). The rename/delete tests locate the row accordingly.
  * The documents-toolbar folder button is "Folder" (TabPillButton wired to the
    root createFolderAction in ProjectDocumentsView), not "Add Subfolder"; it
    still opens the autofocused "Folder name" root input.
  * NewTRModal's footer submit and the tabular page CTA both read "Create", so
    the create-review helper scopes to the modal submit
    (button[name="modalAction"][value="create-review"]).
  * The built-in workflow detail test navigated to the flat /workflows/[id],
    which upstream does not route - only the typed /workflows/assistant/[id] and
    /workflows/tabular-review/[id] exist. builtin-cp-checklist is assistant-type,
    so it now navigates to /workflows/assistant/builtin-cp-checklist (the path
    the app itself links to via workflowDetailPath; works on the fork too).

Verified: full 27-test suite green against upstream-main + test-harness +
demo-mode(+provider-registry), backend + local Supabase (upstream schema.sql +
backend/migrations) + MinIO, demo model, e2e@mike.local.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CEguyEgXa9JjCciXCcVemC
d8d38174 ci(e2e): run Playwright suite on PRs as a merge gate QA Runner 2026-07-17 ↗ GitHub
commit body
Adds .github/workflows/e2e.yml adapted to this repo's backend/ + frontend/
layout: on every pull_request into main (or upstream-main), boot MinIO +
local Supabase (loading backend/schema.sql), start the API and web dev
servers, and run the Playwright suite, uploading the HTML report/traces on
pass or fail. playwright.config.ts already disables its local webServer when
CI=true, so the job owns the stack.

docs/e2e-ci.md documents the one required secret (ANTHROPIC_API_KEY) and the
branch-protection steps that make the 'e2e / playwright' check required, so a
red run blocks the merge button.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
d427cc26 ci(e2e): upload the Playwright report even on timeout/cancel QA Runner 2026-07-17 ↗ GitHub
commit body
Switch the artifact step from !cancelled() to always() so a run that hits the
30-minute job timeout (the failure mode when many specs retry) still uploads
the HTML report and traces instead of skipping the upload - that partial report
is exactly what's needed to diagnose the failing specs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
9daee405 ci(e2e): grant service_role table access after loading schema.sql QA Runner 2026-07-17 ↗ GitHub
commit body
The backend queries exclusively as service_role (lib/supabase.ts, service key,
'bypasses RLS'). schema.sql revokes client grants (anon/authenticated) but
assumes a hosted Supabase where service_role already has full table access - on
a fresh CLI stack loaded via psql it gets none, so the first backend write 500s
with 'permission denied for table user_profiles' and every project/chat spec
fails. Granting service_role after the schema load reproduces the production
grant posture.

Verified in CI: POST /projects 500 -> 201, GET /chat 500 -> 200.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
42346d69 ci(e2e): apply migrations, serve prod build, skip LLM specs without a key QA Runner 2026-07-17 ↗ GitHub
commit body
Three fixes that take the suite from ~everything-failing to green (verified end
to end against a local stack):

1. schema.sql lags the migrations (missing e.g. workflow_open_source_submissions,
   which GET /workflows/:id queries → 500). Apply every dated migration on top of
   schema.sql (idempotent; 0 errors on a fresh DB), grant service_role AFTER so
   new tables are covered, then reload PostgREST. Fixes the workflow specs.

2. Serve a production build (next build + next start) instead of next dev. The dev
   server's on-demand compilation (slow first hit → waitForResponse timeouts) and
   hydration-error overlay (injects nextjs__container_errors DOM that pollutes text
   locators, e.g. getByText('All') matching 'Call Stack') made the suite flaky.
   Fixes tabular-reviews list render + the timing flakes.

3. LLM-dependent specs (chat rename/delete/submit, critical-path) require a model
   key to send a message. Guard them with test.skip(!hasLlmKey) (e2e/llm.ts) and
   expose ANTHROPIC_API_KEY to the Playwright process, so a keyless run is green on
   the ~23 other specs and the LLM specs run + enforce only when the secret is set.

Local result (no key, with MinIO+LibreOffice as on ubuntu-latest): 23 passed,
4 skipped, 0 failed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
094fe565 docs(e2e): document keyless-green behavior, migrations, and prod build QA Runner 2026-07-17 ↗ GitHub
commit body
The suite is green with no secret (LLM specs skip). Update docs/e2e-ci.md and the
workflow header to reflect the migrations step, the production build, and the
optional (not required) ANTHROPIC_API_KEY.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
8bb44e9c test(e2e): update built-in workflow id after upstream rename QA Runner 2026-07-20 ↗ GitHub
commit body
The workflows UI refresh (olp/main fa21ac8) renamed the built-in workflow id
builtin-cp-checklist -> builtin-draft-cp-checklist (title 'Draft CP Checklist'
unchanged). The read-only-detail spec navigated to the old id URL and got a
'workflow not found' page. Point it at the new id.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
5f8e4ab4 fix: batch directory-modal project fetch instead of N+1 getProject burst QA Runner 2026-07-20 ↗ GitHub
commit body
Every directory picker (AddDocumentsModal, UseWorkflowModal, the assistant
project selector) loads its "Projects" tab through useDirectoryData, which
fired one GET /projects/:id for EVERY existing project the moment the modal
opened. Each of those requests costs an auth verification against GoTrue
plus ~6 PostgREST queries, so an account with N projects produced an
~8xN-request burst on the Supabase gateway per modal open.

Under that burst the gateway genuinely falls over: measured locally against
the Supabase CLI stack, overlapping modal-open storms drove GoTrue into
Postgres connection exhaustion ("failed to connect ... context deadline
exceeded") and produced 467x 500 + 641x 504 on /auth/v1/user in a single
run - surfacing to users as failed project creates/loads, and to the e2e
suite as the intermittent Kong 502s its specs currently retry around.

Fix: GET /projects now accepts ?include=documents and returns each
project's documents from one batched query (same attach helpers as
GET /projects/:id, run once across all documents), and useDirectoryData
uses it. A modal open is now 1 API request and a fixed number of DB
queries regardless of project count. After the change the same storm
harness produced zero 5xx and zero auth failures.

[Cherry-pick overlap note: this same commit is submitted separately as the
olp-pr/directory-fetch-storm PR, where it belongs (production fix). It is
carried on this e2e branch only so the de-retried suite passes here; on
merge of both branches git resolves the duplicate cleanly.]

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
af38a450 test(e2e): drop transient-5xx retry scaffolding now that the fetch storm is fixed QA Runner 2026-07-20 ↗ GitHub
commit body
With the directory fetch storm eliminated (previous commit: batched
listProjects?include=documents instead of a getProject burst per modal
open), the local Supabase gateway no longer collapses under modal-open
load, so the specs' 5xx-oriented scaffolding is removed:

- create-with-retry loops (project create x2, chat create x2, workflow
  create, tabular-review create x10-attempt, review add-document x6)
- reload-guards for transient "Project not found" (waitForProjectLoaded,
  gotoProjectRow, project-assistant tab loop)
- networkidle settle-waits whose purpose was to let the per-project fetch
  burst drain before submitting (NewProjectModal / NewTRModal)
- oversized per-test timeouts justified by the storm (180s -> 60-120s)

Kept, deliberately:
- the title-generation settle logic in chat rename/delete
- chat submit's model-seeding retry loop (guards a documented
  useSelectedModel localStorage hydration race, unrelated to the gateway)
- account display-name toPass block (guards a documented profile
  hydration race, unrelated to the gateway)
- generic short networkidle settles after login/logout navigation
- Playwright's own retries: 2 on CI

Verified: with the fix, 3 consecutive full-suite runs (plus 2 more before
de-retry) were green with zero Playwright retries, running against an
account seeded with ~40 projects - a larger directory fan-out than CI's
fresh database ever produces.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
d4448fa8 test(e2e): select a real Claude model in LLM-gated specs (demo model is fork-only) QA Runner 2026-07-20 ↗ GitHub
commit body
The 4 LLM-gated specs (chat-management rename/delete/project-assistant and
the critical-path project flow) selected a "Demo (no key needed)" model in
the ModelToggle. That model exists only in a fork's demo provider - upstream
ModelToggle.MODELS has no such entry - so on this repo, setting the
ANTHROPIC_API_KEY secret unskipped the specs and they then failed at the
model-selection step.

Fix:
- Replace the selectDemoModel helpers with selectClaudeModel, which picks
  "Claude Sonnet 4.6" (the cheapest Anthropic entry in ModelToggle.MODELS).
  With ANTHROPIC_API_KEY exported to the backend, userApiKeys.envApiKey()
  reports the claude provider as configured, so the model is available and
  the submit is not blocked by the ApiKeyMissingModal.
- critical-path Step 8: the canned-reply assertion getByText("Demo mode")
  becomes a presence + nonempty assertion on the assistant answer container
  (MarkdownContent's "div.prose.font-serif.text-gray-900", unique to
  assistant answer content) - deterministic against nondeterministic real
  LLM output, same 60s budget.
- Comments rewritten to describe the upstream reality: specs run only when
  ANTHROPIC_API_KEY is set (e2e/llm.ts), the backend uses the env key, and
  title generation resolves to claude-haiku-4-5 via resolveTitleModel.

Verified against a local prod-build stack (backend :3001, frontend :3002,
local Supabase + MinIO), only ANTHROPIC_API_KEY configured:
- keyless Playwright env: 23 passed / 4 skipped
- ANTHROPIC_API_KEY exported: 27 passed / 0 skipped (1.2m)
- npx tsc --noEmit clean

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
62da94be test(e2e): self-bootstrapping local Supabase stack for local runs willchen96 2026-07-25 ↗ GitHub
commit body
npm run test:e2e:local (or plain test:e2e via the webServer hook) boots
Docker-checked local Supabase, loads schema + migrations + grants, wires
backend/.env and frontend/.env.local, and raises the backend rate-limit
caps the same way CI does - a full suite run exceeds the default
300-requests/15-min general cap, after which every call 429s and the
profile/list waits time out.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cb2306c5 fix(frontend): unify full-screen loaders to remove hydration mismatch willchen96 2026-07-25 ↗ GitHub
commit body
The provider Suspense fallback, the MFA gate loader, and the (pages)
layout auth loader were three near-identical spinners with different
Tailwind classes. Server and client can resolve those gates differently
on first paint (auth settles from localStorage before the boundary
hydrates), so the differing attributes surfaced as a hydration mismatch
on nearly every dev page load. One shared FullScreenLoader makes every
branch render byte-identical DOM.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
4728fd19 chore(frontend): pin turbopack.root to silence multi-lockfile warning willchen96 2026-07-25 ↗ GitHub
The repo has lockfiles at the root (Playwright harness) and in
frontend/, so Next.js guesses the workspace root and warns on every
dev-server start. Pin it explicitly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
c196f9b2 Merge remote-tracking branch 'origin/main' into olp-pr/e2e-playwright willchen96 2026-07-25 ↗ GitHub
f3173a9d ci(e2e): narrow post-migration service_role grants to match schema.sql willchen96 2026-07-26 ↗ GitHub
commit body
schema.sql now grants service_role its own narrowed privileges
(upstream "narrow service role schema grants"), so the broad GRANT ALL
workaround is obsolete - and worse, it widened the test database beyond
what prod gets, letting e2e pass on privileges a real deployment lacks.
Keep a post-migration re-grant (schema.sql's ON ALL statements can't
cover tables the migrations create) but with the identical narrowed set.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fb3ec2d6 chore: gitignore the local Supabase scaffold willchen96 2026-07-26 ↗ GitHub
commit body
backend/supabase/ is supabase-init output used only by the local e2e
stack; CI scaffolds its own fresh copy each run, so it must not be
committed - hide it from status/source-control noise.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-1335.md from inside the repo you want the changes in.

⬇ Download capture-thread-1335.md