Add self-hosted local LLM provider support
From the PR description
Summary
- Adds a server-configured
local:serverLLM provider backed by OpenAI-compatible Chat Completions streaming. - Exposes backend-derived local provider availability/capabilities without exposing local endpoint secrets.
- Gates local tool usage unless
LOCAL_LLM_SUPPORTS_TOOLS=true, and keeps local out of tabular preferences unless tool support is enabled. - Adds backend tests for provider routing, local config parsing, SSE parsing, and no-tool local streaming.
- Documents Ollama and LM Studio self-hosting examples.
Verification
npm test --prefix backendnpm run build --prefix backendnpx eslint 'src/app/(pages)/account/models/page.tsx' 'src/app/components/assistant/ModelToggle.tsx' 'src/app/lib/modelAvailability.ts' 'src/contexts/UserProfileContext.tsx' 'src/app/lib/mikeApi.ts' 'src/app/components/shared/ApiKeyMissingModal.tsx'fromfrontend/NEXT_PUBLIC_SUPABASE_URL=https://example.supabase.co NEXT_PUBLIC_SUPABASE_PUBLISHABLE_DEFAULT_KEY=placeholder npm run build --prefix frontend
Note: full npm run lint --prefix frontend still fails on pre-existing lint issues outside this change set.
Our analysis
Add self-hosted local LLM provider with OpenAI-compatible streaming — read the full analysis →
Think the analysis missed something the PR description covers?
Commits in this PR (2)
| SHA | Subject | Author | Date | |
|---|---|---|---|---|
c0361944 | feat: add self-hosted local llm provider | Manuel Conde | 2026-05-09 | ↗ GitHub |
c0361944 | feat: add self-hosted local llm provider | Manuel Conde | 2026-05-09 | ↗ GitHub |
Capture this PR into my fork
Download a Markdown prompt that tells Claude how to port every
commit in this PR into your working tree. Run it via
claude -p < capture-pull-2.md from
inside the repo you want the changes in.