test(e2e): select a real Claude model in LLM-gated specs (demo model is fork-only)

↗ view on GitHub · QA Runner · 2026-07-20 · d4448fa8

The 4 LLM-gated specs (chat-management rename/delete/project-assistant and
the critical-path project flow) selected a "Demo (no key needed)" model in
the ModelToggle. That model exists only in a fork's demo provider - upstream
ModelToggle.MODELS has no such entry - so on this repo, setting the
ANTHROPIC_API_KEY secret unskipped the specs and they then failed at the
model-selection step.

Fix:
- Replace the selectDemoModel helpers with selectClaudeModel, which picks
  "Claude Sonnet 4.6" (the cheapest Anthropic entry in ModelToggle.MODELS).
  With ANTHROPIC_API_KEY exported to the backend, userApiKeys.envApiKey()
  reports the claude provider as configured, so the model is available and
  the submit is not blocked by the ApiKeyMissingModal.
- critical-path Step 8: the canned-reply assertion getByText("Demo mode")
  becomes a presence + nonempty assertion on the assistant answer container
  (MarkdownContent's "div.prose.font-serif.text-gray-900", unique to
  assistant answer content) - deterministic against nondeterministic real
  LLM output, same 60s budget.
- Comments rewritten to describe the upstream reality: specs run only when
  ANTHROPIC_API_KEY is set (e2e/llm.ts), the backend uses the env key, and
  title generation resolves to claude-haiku-4-5 via resolveTitleModel.

Verified against a local prod-build stack (backend :3001, frontend :3002,
local Supabase + MinIO), only ANTHROPIC_API_KEY configured:
- keyless Playwright env: 23 passed / 4 skipped
- ANTHROPIC_API_KEY exported: 27 passed / 0 skipped (1.2m)
- npx tsc --noEmit clean

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Repository open-legal-products/mike
Author QA Runner <qa@local>
Authored
Parents af38a450
Stats 2 files changed , +78 , -61
Part of Add Playwright coverage for critical legal-AI flows

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-d4448fa8.md from inside the repo you want the change in.

⬇ Download capture-commit-d4448fa8.md