Ollama local model support with context preloading

brauliogusmao added Ollama as a provider, routing `ollama/<model>` IDs through a new 309-line streaming adapter. The interesting part isn't the wiring -- it's that document-fetch tools are preloaded into context rather than called reactively, because small local models don't handle reactive tool use reliably.

infrastructuresecurity

Backend changes: backend/src/lib/llm/ollama.ts handles streaming, message conversion, and tool-call normalization against Ollama's /api/chat. The dispatcher in llm/index.ts checks for the ollama/ prefix and routes accordingly. models.ts bypasses the static model allowlist for ollama/* IDs. A GET /ollama/models route in routes/ollama.ts proxies the Ollama instance's model list. OLLAMA_URL defaults to http://localhost:11434.

Frontend changes: the model picker gains a "Local (Ollama)" group that populates from fetchOllamaModels(). It only renders when Ollama is reachable and returns models. useSelectedModel and modelAvailability.ts were extended to recognize the new provider.

The CONTEXT_TOOLS set (read_document, fetch_documents) defines which tools get preloaded. When one fires, its result goes into the system prompt and the tool is removed so the model won't call it again. The commit is explicit that this exists to compensate for weaker tool-calling in smaller models -- a pragmatic accommodation. The downside is that this behavior differs from the Anthropic and OpenAI adapters, which rely on the model to call these tools when it needs them.

One thing to flag: the GET /ollama/models endpoint returns whatever the configured Ollama host reports, with no filtering. Fine for localhost, worth auditing if OLLAMA_URL ever points at a shared server.

So what Worth a look if offline capability, on-prem deployment, or cost reduction on lower-stakes queries matters to your product. The integration is additive -- it doesn't touch existing provider paths. The `CONTEXT_TOOLS` preloading pattern is also directly reusable if you add other constrained models later.

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

2 commits from brauliogusmao/mike, oldest first. Source extracted verbatim from the harvested git log.

SHA	Subject	Author	Date
`cb3e3b59`	feat: integração com Ollama para modelos locais	Braulio Gabriel Gusmao	2026-05-13	↗ GitHub
commit body Adiciona suporte a modelos rodando localmente via Ollama, com estratégia de pré-carregamento de documentos no contexto antes da chamada ao LLM - necessário porque modelos menores não lidam bem com tool calling reativo. - Backend: adapter de streaming (ollama.ts), rota GET /ollama/models, dispatcher atualizado para provider "ollama/", models.ts e types.ts atualizados - Frontend: grupo "Local (Ollama)" dinâmico no ModelToggle, busca de modelos via fetchOllamaModels(), useSelectedModel aceita IDs ollama/*, modelAvailability atualizado para provider ollama Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
`09bb9af9`	Merge pull request #1 from brauliogusmao/slm-local	Bráulio Gabriel Gusmão	2026-05-13	↗ GitHub
feat: integração com Ollama para modelos locais

SHA

Subject

Author

Date

cb3e3b59

feat: integração com Ollama para modelos locais

Braulio Gabriel Gusmao

2026-05-13

↗ GitHub

commit body

Adiciona suporte a modelos rodando localmente via Ollama, com estratégia
de pré-carregamento de documentos no contexto antes da chamada ao LLM -
necessário porque modelos menores não lidam bem com tool calling reativo.

- Backend: adapter de streaming (ollama.ts), rota GET /ollama/models,
  dispatcher atualizado para provider "ollama/", models.ts e types.ts
  atualizados
- Frontend: grupo "Local (Ollama)" dinâmico no ModelToggle, busca de
  modelos via fetchOllamaModels(), useSelectedModel aceita IDs ollama/*,
  modelAvailability atualizado para provider ollama

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

09bb9af9

Merge pull request #1 from brauliogusmao/slm-local

Bráulio Gabriel Gusmão

2026-05-13

↗ GitHub

feat: integração com Ollama para modelos locais

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-391.md from inside the repo you want the changes in.