Ollama provider added: local models with document preloading

brauliogusmao wired Ollama in as a fourth LLM backend alongside Claude, OpenAI, and Gemini. The key design decision: instead of letting local models call document-fetch tools reactively, the adapter preloads document content into the system prompt before the first LLM call.

infrastructuresecurity

The implementation is a 309-line streaming adapter in backend/src/lib/llm/ollama.ts plus routing, a model-discovery endpoint, and frontend model-picker changes. Model IDs use an ollama/ prefix -- providerForModel() checks for this prefix and routes accordingly, and resolveModel() lets ollama/* IDs bypass the static model allowlist.

A GET /ollama/models route proxies the local Ollama instance's model list to the frontend. The model picker's "Local (Ollama)" group populates from this at runtime, so it only appears when Ollama is running and has at least one model installed. The OLLAMA_URL env var (defaulting to http://localhost:11434) controls which Ollama instance the backend talks to.

The document-preloading strategy is the most opinionated part. Two tools -- read_document and fetch_documents -- are in a CONTEXT_TOOLS set. When either fires, the adapter injects the result into the system prompt and removes the tool from the list so the model can't call it again. Smaller local models are unreliable at reactive tool use, so the adapter front-loads what they'll need rather than hoping they ask for it correctly.

Existing remote-provider flows are untouched. The integration adds a new branch in the dispatcher; nothing in the Claude or OpenAI paths changed.

So what Worth a look if you're running on-prem, need offline capability, or want to cut API costs on less sensitive workloads. The provider-prefix routing is clean and the `CONTEXT_TOOLS` preload pattern is reusable for any other constrained model you might add. One caveat: document-tool behavior diverges between Ollama and the remote adapters, so test carefully if you allow users to switch providers mid-conversation.

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

1 commit from brauliogusmao/mike, oldest first. Source extracted verbatim from the harvested git log.

SHA	Subject	Author	Date
`cb3e3b59`	feat: integração com Ollama para modelos locais	Braulio Gabriel Gusmao	2026-05-13	↗ GitHub
commit body Adiciona suporte a modelos rodando localmente via Ollama, com estratégia de pré-carregamento de documentos no contexto antes da chamada ao LLM - necessário porque modelos menores não lidam bem com tool calling reativo. - Backend: adapter de streaming (ollama.ts), rota GET /ollama/models, dispatcher atualizado para provider "ollama/", models.ts e types.ts atualizados - Frontend: grupo "Local (Ollama)" dinâmico no ModelToggle, busca de modelos via fetchOllamaModels(), useSelectedModel aceita IDs ollama/*, modelAvailability atualizado para provider ollama Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

SHA

Subject

Author

Date

cb3e3b59

feat: integração com Ollama para modelos locais

Braulio Gabriel Gusmao

2026-05-13

↗ GitHub

commit body

Adiciona suporte a modelos rodando localmente via Ollama, com estratégia
de pré-carregamento de documentos no contexto antes da chamada ao LLM -
necessário porque modelos menores não lidam bem com tool calling reativo.

- Backend: adapter de streaming (ollama.ts), rota GET /ollama/models,
  dispatcher atualizado para provider "ollama/", models.ts e types.ts
  atualizados
- Frontend: grupo "Local (Ollama)" dinâmico no ModelToggle, busca de
  modelos via fetchOllamaModels(), useSelectedModel aceita IDs ollama/*,
  modelAvailability atualizado para provider ollama

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-456.md from inside the repo you want the changes in.