feat(llm): replace direct provider SDKs with Vercel AI Gateway

↗ view on GitHub · z · 2026-05-13 · 07a46ceb

Batch 4 - route all LLM traffic through the Vercel AI Gateway via
AI SDK v6. Tokens are now paid by the platform, not the customer.

Backend:
- New gateway adapter at backend/src/lib/llm/gateway.ts (uses
  ai@^6.0.182, "provider/model" strings auto-route via the bundled
  gateway provider).
- 12 unit tests in backend/src/lib/llm/__tests__/gateway.test.ts.
- Deleted backend/src/lib/llm/{claude,gemini,openai,tools}.ts and
  backend/src/lib/userApiKeys.ts - the per-provider SDKs and the
  customer-key vault are gone.
- backend/src/lib/llm/index.ts now re-exports from gateway and routes
  streamChatWithTools / completeText through it.
- backend/src/lib/llm/types.ts: added StreamChatAttribution
  ({ userId?, orgId? }) for gateway cost-slicing; UserApiKeys kept
  as @deprecated alias so existing call-sites still typecheck.
- backend/src/lib/userSettings.ts: drop per-user api_keys lookup.
- backend/src/lib/chatTools.ts: runLLMStream now forwards attribution
  instead of apiKeys and surfaces aggregate usage.totalTokens.
- backend/src/routes/{chat,projectChat,tabular}.ts: attribution
  ({ userId }) replaces apiKeys; legacy queryGemini/title helpers
  refactored to take userId.
- backend/src/routes/user.ts: deleted GET/PUT /user/api-keys.
- backend/.env.example: replaced GEMINI_/ANTHROPIC_/OPENAI_API_KEY
  and USER_API_KEYS_ENCRYPTION_SECRET with AI_GATEWAY_API_KEY.

Frontend:
- Deleted frontend/src/app/(pages)/account/models/page.tsx - the
  user-facing API key UI no longer applies.
- frontend/src/app/(pages)/account/layout.tsx: removed the
  "Models & API Keys" tab (Billing tab arrives in batch 6).

Intermediate state notes:
- frontend mikeApi.ts still exports getApiKeyStatus + saveApiKey
  against now-404 routes; UserProfileContext still imports them.
  Both are cleaned up in batches 6/8.
- public.user_api_keys table is not dropped here - orphaned but
  harmless; can be removed in a follow-up migration.

Agent ID: a5511de9cafb2355a (vercel:ai-architect).
Repository hosman20/mike-2.0
Author z <z@zs-MacBook-Pro.local>
Authored
Parents 571af72d
Stats 19 files changed , +708 , -1353
Part of LLM traffic routed through Vercel AI Gateway; customer-supplied keys removed

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-07a46ceb.md from inside the repo you want the change in.

⬇ Download capture-commit-07a46ceb.md