feat(llm): replace direct provider SDKs with Vercel AI Gateway
Batch 4 - route all LLM traffic through the Vercel AI Gateway via
AI SDK v6. Tokens are now paid by the platform, not the customer.
Backend:
- New gateway adapter at backend/src/lib/llm/gateway.ts (uses
ai@^6.0.182, "provider/model" strings auto-route via the bundled
gateway provider).
- 12 unit tests in backend/src/lib/llm/__tests__/gateway.test.ts.
- Deleted backend/src/lib/llm/{claude,gemini,openai,tools}.ts and
backend/src/lib/userApiKeys.ts - the per-provider SDKs and the
customer-key vault are gone.
- backend/src/lib/llm/index.ts now re-exports from gateway and routes
streamChatWithTools / completeText through it.
- backend/src/lib/llm/types.ts: added StreamChatAttribution
({ userId?, orgId? }) for gateway cost-slicing; UserApiKeys kept
as @deprecated alias so existing call-sites still typecheck.
- backend/src/lib/userSettings.ts: drop per-user api_keys lookup.
- backend/src/lib/chatTools.ts: runLLMStream now forwards attribution
instead of apiKeys and surfaces aggregate usage.totalTokens.
- backend/src/routes/{chat,projectChat,tabular}.ts: attribution
({ userId }) replaces apiKeys; legacy queryGemini/title helpers
refactored to take userId.
- backend/src/routes/user.ts: deleted GET/PUT /user/api-keys.
- backend/.env.example: replaced GEMINI_/ANTHROPIC_/OPENAI_API_KEY
and USER_API_KEYS_ENCRYPTION_SECRET with AI_GATEWAY_API_KEY.
Frontend:
- Deleted frontend/src/app/(pages)/account/models/page.tsx - the
user-facing API key UI no longer applies.
- frontend/src/app/(pages)/account/layout.tsx: removed the
"Models & API Keys" tab (Billing tab arrives in batch 6).
Intermediate state notes:
- frontend mikeApi.ts still exports getApiKeyStatus + saveApiKey
against now-404 routes; UserProfileContext still imports them.
Both are cleaned up in batches 6/8.
- public.user_api_keys table is not dropped here - orphaned but
harmless; can be removed in a follow-up migration.
Agent ID: a5511de9cafb2355a (vercel:ai-architect).
| Repository | hosman20/mike-2.0 |
|---|---|
| Author | z <z@zs-MacBook-Pro.local> |
| Authored | |
| Parents | 571af72d |
| Stats | 19 files changed , +708 , -1353 |
| Part of | LLM traffic routed through Vercel AI Gateway; customer-supplied keys removed |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-07a46ceb.md
from inside the repo you want the change in.