Self-hosted local LLM provider added as a fourth inference path
manueljpconde added a `local:server` provider to mikeEU that routes inference to any OpenAI-compatible `/v1/chat/completions` endpoint - Ollama, LM Studio, vLLM, and similar runtimes. Configuration is entirely server-side; the browser only sees a selectable model name.
The adapter lives in backend/src/lib/llm/local.ts (206 lines) and handles SSE frame splitting, system-plus-message shaping, and optional tool-call passthrough. Six env vars control the provider: ENABLE_LOCAL_LLM, LOCAL_LLM_BASE_URL, LOCAL_LLM_MODEL, LOCAL_LLM_API_KEY, LOCAL_LLM_LABEL, LOCAL_LLM_SUPPORTS_TOOLS (defaults false), and LOCAL_LLM_TIMEOUT_MS (defaults 120 000 ms). The backend calls the endpoint directly; the browser never receives the URL, model ID, or key.
The provider is wired into llm/index.ts, the model registry, the chat-tools fanout, userApiKeys, userSettings, the /user routes, and the frontend account/models page including ModelToggle and ApiKeyMissingModal. When LOCAL_LLM_SUPPORTS_TOOLS is false the provider is hidden from the tabular-model preferences surface and the active tool list is emptied - the model won't be offered for workflows it can't run.
232 lines of unit tests cover the SSE parser (local.test.ts), config reading (localConfig.test.ts), and model registry (models.test.ts). The round-trip path from frontend toggle to backend stream was manually verified. No retry logic and no rate-limit handling. Streaming assumes standard OpenAI SSE framing; runtimes that deviate will break silently.
This is the v1 of a larger managed-provider system. PR #7 later replaced it as the preferred path for per-user configuration while keeping this server-wide env approach as a fallback.
Spotted something wrong? Or know the PR text has fresher detail than the writeup above?