Add `local` provider for any OpenAI-compatible self-hosted LLM
manueljpconde adds a fourth LLM provider to the abstraction layer, letting self-hosters point the backend at Ollama, LM Studio, or any `/v1/chat/completions`-compatible endpoint. The integration is entirely backend-configured; the browser never learns the URL, model ID, or API key.
Seven env vars control the feature: ENABLE_LOCAL_LLM, LOCAL_LLM_BASE_URL, LOCAL_LLM_MODEL, LOCAL_LLM_LABEL, LOCAL_LLM_API_KEY, LOCAL_LLM_SUPPORTS_TOOLS, and LOCAL_LLM_TIMEOUT_MS (default 120 000 ms). The browser gets a single canonical model id local:server, and the status object that reaches the frontend deliberately omits all sensitive fields - just the label, configured, and tool-support flags.
The LOCAL_LLM_SUPPORTS_TOOLS=false path is where the interesting engineering is. When tools are disabled, runLLMStream empties the active-tools array and injects a LOCAL MODEL LIMITATION clause into the system prompt explicitly warning the model not to claim it used tools. Tabular-review and document-editing workflows are hard-gated: serializeProfile, getUserModelSettings, and validateProfilePayload all refuse local:server for tabular unless supportsTools=true. Frontend dropdowns mirror this - the model only appears if the backend reports it configured, and it's excluded from the tabular dropdown unless tool support is on.
The diff also includes node:test unit tests for the SSE chunk extractor (extractChatCompletionSseJson), streamLocal, and getLocalLlmConfig, plus a new npm test --prefix backend script wired into the README Checks section.
Two things to flag before pulling. The .env.example diff switches OPENROUTER_API_KEY to OPENAI_API_KEY - unrelated to the local-LLM scope and easy to miss if your fork still routes through OpenRouter. Also, a useEffect that reset the API-key input on hasSavedKey change is removed from account/models/page.tsx without any connection to the local-LLM work; it appears to be an incidental UX fix.
Spotted something wrong? Or know the PR text has fresher detail than the writeup above?