Ollama wired in as a local LLM provider with routing and model discovery

stakwork/mikeoss adds Ollama as a first-class provider alongside Claude and Gemini. The change covers the backend adapter, request routing, and a new endpoint that enumerates installed models from the Ollama server.

infrastructuresecurity

Maintained by Paul Itoi · verified on MikeWatch

The adapter fits into the existing provider pattern. A new "ollama" provider type gets registered, and model ids use an ollama/ prefix so the routing layer can dispatch them without touching the existing Claude or Gemini paths. The Ollama adapter talks to Ollama's OpenAI-compatible /v1 endpoint using the standard openai npm package rather than any Ollama-specific client.

Model discovery goes through a new GET /user/ollama-models route that proxies Ollama's /api/tags endpoint. This lets the frontend enumerate whatever models the Ollama server has installed - useful because local Ollama installs vary, so hardcoding model names isn't viable.

The PR was opened and merged in seconds, consistent with Hive-driven automated authoring landing straight to main. Two commits cover the work.

One thing to know before adopting: Ollama models get tools stripped at the routing layer. The backend passes an empty tools array and emits a tools_unavailable SSE event when Ollama is selected. Doc editing and citations don't work with local models. The matching frontend PR (c629bb00) handles the UI side of that signal.

So what Worth a look if you want a self-hosted inference path with no API keys. The adapter is small and the OpenAI-compat shim is the correct approach. Check whether the tool-stripping behavior and server-wide `OLLAMA_BASE_URL` constraint work for your deployment before pulling.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?