[Extensibility 3/4] feat: local LLMs via Ollama, behind ENABLE_OLLAMA (stacked on #259)

⛔ closed · #261 · open-legal-products/mike ← amal66/mike · opened 3mo ago by amal66 · closed 3mo ago · +678-95 across 13 files · ↗ on GitHub

From the PR description

Stacked on #259 - merge that first; until it merges, this diff shows its commit too. This PR's own change is the single Ollama commit.

Closes #23.

Summary

Opt-in local LLM support via Ollama's OpenAI-compatible endpoint: a self-hosted deployment can run chat inference entirely on its own hardware - documents never leave the building, and with this flag neither does inference. Everything is behind ENABLE_OLLAMA=true (default off): with the flag unset, no new models appear and nothing changes.

Changes

  • providers/ollama.ts (new): registry entry that routes local model IDs (llama3.3, phi4, qwen2.5, + OLLAMA_MODELS extras) through the existing OpenAI adapter pointed at http://localhost:11434/v1. No new dependencies, no new streaming code path.
  • baseUrl.ts (new) + openai.ts: OPENAI_BASE_URL resolution with an SSRF guard - http/localhost/private-IP endpoints rejected in production unless explicitly allowed. Defaults to https://api.openai.com/v1 (unchanged when unset).
  • privateIp.ts (new): shared private/reserved IP classifier (IPv4 + IPv6 incl. mapped/NAT64/6to4).
  • .env.example: documents the opt-in config.
  • Tests: ollama gate, baseUrl, privateIp (21 tests).

Why

The most-requested self-hosting capability (#23), done as a registry entry instead of a bespoke client - the first external provider to use the registry, which is exactly what it was built for.

Testing

On this branch: backend npm test - 300 passed, 5 skipped (includes the 21 new tests). Flag gate verified: only the exact string "true" registers the provider.

Provenance

Mechanical port of code running in amal66/mike (main); privateIp.ts and providers/ollama.ts byte-identical to the fork. Full provenance in amal66#34.

🤖 Generated with Claude Code

Our analysis

Add opt-in local Ollama inference — read the full analysis →

Think the analysis missed something the PR description covers?

Capture this PR into my fork

Download a Markdown prompt that tells Claude how to port every commit in this PR into your working tree. Run it via claude -p < capture-pull-261.md from inside the repo you want the changes in.

⬇ Download capture-pull-261.md