hosman20 puts every AI model behind one gateway

The fork stops talking to Anthropic, Google, and OpenAI directly and routes all of them through a single switchboard instead.

infrastructureintegration

Until now this fork wired up each AI provider separately, with its own keys and its own code. hosman20 has swapped all of that for the Vercel AI Gateway - a single service that takes a request, sees which model it names, and forwards it to the right provider. One credential replaces three, and individual users never handle their own keys.

The more interesting part for anyone running this as a business: every request now carries a tag identifying the user and organisation behind it, and the gateway counts the tokens each call burns. That's the plumbing a subscription or per-seat billing model needs. hosman20 is candid that the live path was only tested against mock providers so far, and some old key-management screens are still being cleaned up.

So what Worth a look if you're weighing how a legal-AI product meters and bills usage across customers - this is the foundation for that.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

1 commit from hosman20/mike-2.0, oldest first. Source extracted verbatim from the harvested git log.

SHA Subject Author Date

07a46ceb feat(llm): replace direct provider SDKs with Vercel AI Gateway z 2026-05-13 ↗ GitHub

SHA	Subject	Author	Date
`07a46ceb`	feat(llm): replace direct provider SDKs with Vercel AI Gateway	z	2026-05-13	↗ GitHub
commit body Batch 4 - route all LLM traffic through the Vercel AI Gateway via AI SDK v6. Tokens are now paid by the platform, not the customer. Backend: - New gateway adapter at backend/src/lib/llm/gateway.ts (uses ai@^6.0.182, "provider/model" strings auto-route via the bundled gateway provider). - 12 unit tests in backend/src/lib/llm/__tests__/gateway.test.ts. - Deleted backend/src/lib/llm/{claude,gemini,openai,tools}.ts and backend/src/lib/userApiKeys.ts - the per-provider SDKs and the customer-key vault are gone. - backend/src/lib/llm/index.ts now re-exports from gateway and routes streamChatWithTools / completeText through it. - backend/src/lib/llm/types.ts: added StreamChatAttribution ({ userId?, orgId? }) for gateway cost-slicing; UserApiKeys kept as @deprecated alias so existing call-sites still typecheck. - backend/src/lib/userSettings.ts: drop per-user api_keys lookup. - backend/src/lib/chatTools.ts: runLLMStream now forwards attribution instead of apiKeys and surfaces aggregate usage.totalTokens. - backend/src/routes/{chat,projectChat,tabular}.ts: attribution ({ userId }) replaces apiKeys; legacy queryGemini/title helpers refactored to take userId. - backend/src/routes/user.ts: deleted GET/PUT /user/api-keys. - backend/.env.example: replaced GEMINI_/ANTHROPIC_/OPENAI_API_KEY and USER_API_KEYS_ENCRYPTION_SECRET with AI_GATEWAY_API_KEY. Frontend: - Deleted frontend/src/app/(pages)/account/models/page.tsx - the user-facing API key UI no longer applies. - frontend/src/app/(pages)/account/layout.tsx: removed the "Models & API Keys" tab (Billing tab arrives in batch 6). Intermediate state notes: - frontend mikeApi.ts still exports getApiKeyStatus + saveApiKey against now-404 routes; UserProfileContext still imports them. Both are cleaned up in batches 6/8. - public.user_api_keys table is not dropped here - orphaned but harmless; can be removed in a follow-up migration. Agent ID: a5511de9cafb2355a (vercel:ai-architect).

commit body

Batch 4 - route all LLM traffic through the Vercel AI Gateway via
AI SDK v6. Tokens are now paid by the platform, not the customer.

Backend:
- New gateway adapter at backend/src/lib/llm/gateway.ts (uses
  ai@^6.0.182, "provider/model" strings auto-route via the bundled
  gateway provider).
- 12 unit tests in backend/src/lib/llm/__tests__/gateway.test.ts.
- Deleted backend/src/lib/llm/{claude,gemini,openai,tools}.ts and
  backend/src/lib/userApiKeys.ts - the per-provider SDKs and the
  customer-key vault are gone.
- backend/src/lib/llm/index.ts now re-exports from gateway and routes
  streamChatWithTools / completeText through it.
- backend/src/lib/llm/types.ts: added StreamChatAttribution
  ({ userId?, orgId? }) for gateway cost-slicing; UserApiKeys kept
  as @deprecated alias so existing call-sites still typecheck.
- backend/src/lib/userSettings.ts: drop per-user api_keys lookup.
- backend/src/lib/chatTools.ts: runLLMStream now forwards attribution
  instead of apiKeys and surfaces aggregate usage.totalTokens.
- backend/src/routes/{chat,projectChat,tabular}.ts: attribution
  ({ userId }) replaces apiKeys; legacy queryGemini/title helpers
  refactored to take userId.
- backend/src/routes/user.ts: deleted GET/PUT /user/api-keys.
- backend/.env.example: replaced GEMINI_/ANTHROPIC_/OPENAI_API_KEY
  and USER_API_KEYS_ENCRYPTION_SECRET with AI_GATEWAY_API_KEY.

Frontend:
- Deleted frontend/src/app/(pages)/account/models/page.tsx - the
  user-facing API key UI no longer applies.
- frontend/src/app/(pages)/account/layout.tsx: removed the
  "Models & API Keys" tab (Billing tab arrives in batch 6).

Intermediate state notes:
- frontend mikeApi.ts still exports getApiKeyStatus + saveApiKey
  against now-404 routes; UserProfileContext still imports them.
  Both are cleaned up in batches 6/8.
- public.user_api_keys table is not dropped here - orphaned but
  harmless; can be removed in a follow-up migration.

Agent ID: a5511de9cafb2355a (vercel:ai-architect).

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-406.md from inside the repo you want the changes in.

⬇ Download capture-thread-406.md