elitan adds Ollama as a fourth LLM provider for fully local inference

elitan wired Ollama's streaming chat API into the existing provider stack, so Mike can run entirely on local models with no external API key. The change is self-contained enough to evaluate on its own merits.

infrastructuresecurity

A new 227-line backend/src/lib/llm/ollama.ts speaks directly to Ollama's streaming chat endpoint, defaulting to http://localhost:11434 with an override via OLLAMA_BASE_URL. Models prefixed with ollama: are routed through it; the prefix is stripped before the request goes on the wire. Streaming, tool calls, and text completion all go through the same StreamChatResult contract the other providers use, so the rest of the backend sees no difference.

OLLAMA_ENABLED=false in .env.example means the feature is opt-in. Frontend changes in ModelToggle.tsx and modelAvailability.ts mark Ollama models as always-available, removing the API-key-missing warning that appears for cloud providers. The account/models page is updated to match. Pre-registered models are ollama:llama3.1 and ollama:qwen3:8b.

Two things worth reviewing before relying on tool use: the streaming JSON parser in ollama.ts is hand-rolled rather than using a library, and tool-call argument coercion accepts both object and JSON-string shapes permissively. That's fine for general chat but could produce odd results with small local models that don't format tool responses reliably.

This commit sits alongside Supabase removal and docker-compose work in the same day's run - elitan is clearly driving toward a self-hostable, zero-external-dependency deployment.

So what Worth a look if your fork targets air-gapped or fully on-prem deployments. The LLM module integration is clean and isolated; dropping it into another fork is low-risk. Skip if you have no Ollama infrastructure and no local-only requirement - the hand-rolled stream parsing adds surface area you'd be maintaining.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

1 commit from elitan/mike, oldest first. Source extracted verbatim from the harvested git log.

SHA Subject Author Date
ee082703 feat(llm): add ollama support Johan Eliasson 2026-05-15 ↗ GitHub

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-412.md from inside the repo you want the changes in.

⬇ Download capture-thread-412.md