WS3 backend: local/on-premises OpenAI-compatible model provider
New "local" LLM provider for any OpenAI-compatible chat-completions server (Ollama, LM Studio, vLLM) alongside Claude/Gemini/OpenAI, per docs/MIGRATION_SPEC.md §5 + §6 decisions #2/#5. Positioning is data sovereignty, not cost. - lib/llm/localConfig.ts (new): resolves LOCAL_LLM_BASE_URL (primary), OPENAI_BASE_URL (documented alias only when the former is unset, never affects the cloud OpenAI client), LOCAL_LLM_MODELS, LOCAL_LLM_API_KEY (defaults to bearer "ollama"). - lib/llm/localOpenAI.ts (new): streamLocal/completeLocalText, a new chat-completions streaming client (not a base-URL override of the existing Responses-API openai.ts) mirroring its callback contract and tool loop. Malformed tool-call JSON is never executed - it's fed back as a tool error so the model can retry, never crashing the stream. - types.ts/models.ts/index.ts: "local" provider registration, local: prefix dispatch, registry bypass for local: ids once configured. - routes/user.ts: local status merged additively into all apiKeyStatus responses; routes/tabular.ts: local models bypass the per-user-key gate (server-env configured, not key-gated). - .env.example + CLAUDE.md env registry updated. - vitest added (first backend test suite): 25 tests covering config resolution, provider/model dispatch, and the streaming client against canned SSE / mocked fetch (content deltas, tool-call fragment accumulation, malformed-JSON handling, second-turn message shape, default bearer token) - no live server required. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
| Repository | Sloth-ninja/JessicaOSS |
|---|---|
| Author | sloth-ninja <ezanahaddis@gmail.com> |
| Authored | |
| Committed | |
| Parents | b02b2dd3 |
| Stats | 15 files changed , +2174 , -13 |
| Part of | WS3: local/on-premises OpenAI-compatible model provider |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-510eda76.md
from inside the repo you want the change in.