jellz77/mike: Ollama as a third LLM provider for on-prem inference

jellz77 adds Ollama alongside Claude and Gemini - full streaming, tool calls, and a model-picker dropdown that queries the local Ollama server for available tags. The `ollama:<name>` model ID convention routes through the existing provider dispatch cleanly.

infrastructuresecurity

The backend work lives in backend/src/lib/llm/ollama.ts (301 lines). It implements streamOllama and completeOllamaText against Ollama's /api/chat NDJSON endpoint. Tool calls are handled via the OpenAI-style streamed delta shape, normalized into the same NormalizedToolCall contract the rest of the codebase uses; the tool loop is bounded by maxIterations ?? 10. thinking deltas pass through for models that support them. A small route (backend/src/routes/ollama.ts) proxies GET /api/tags from the Ollama server as GET /ollama/tags, gated behind the normal auth middleware.

The ollama: prefix in models.ts is the ID convention. isOllamaModelId validates it (checks for whitespace/control characters in the suffix), ollamaModelName strips the prefix, and providerForModel / resolveModel are extended to pass ollama:* IDs through without throwing. The OLLAMA_BASE_URL env var (default http://localhost:11434) and optional OLLAMA_API_KEY configure the server; per-user key overrides are supported via a new ollama slot in UserApiKeys.

On the frontend, modelAvailability.ts is promoted from a thin convenience wrapper to the actual source of truth for model IDs. It now exports isAllowedModelId, toOllamaModelId, getModelLabel, and the canonical MODELS and ALLOWED_MODEL_IDS lists. ModelToggle.tsx and the tabular model selector render an "Ollama" group that fetches tags on dropdown open, with loading and empty states. useSelectedModel switches from a static Set check to isAllowedModelId so ollama:* IDs persist correctly in localStorage.

Before pulling this: isProviderAvailable returns true unconditionally for Ollama, so the UI offers Ollama models even with no server running. There is no per-user Ollama endpoint - everyone hits the same OLLAMA_BASE_URL. The useSelectedModel change includes an unrelated setTimeout(..., 0) deferral that looks like a hydration fix; worth verifying before lifting. frontend/package.json bumps next to ^16.0.11 with corresponding lockfile churn.

So what Worth a look if you need self-hosted inference or want a reference for plumbing a third provider into mike's LLM dispatch. The `ollama.ts` implementation and the `modelAvailability.ts` consolidation are well-factored and can be imported with limited conflict. Skip if all your users need cloud providers only, or if a single shared Ollama server per deployment is a problem for your setup.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

2 commits from jellz77/mike, oldest first. Source extracted verbatim from the harvested git log.

SHA Subject Author Date
6d0431cb added ollama support jon 2026-05-06 ↗ GitHub
04d126f6 added ollama support jon 2026-05-06 ↗ GitHub

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-128.md from inside the repo you want the changes in.

⬇ Download capture-thread-128.md