jellz77/mike

jellz77/mike adds Ollama for fully on-prem LLM inference

One topic: Ollama provider support across two commits (effectively one change with a README follow-up). The implementation is end-to-end and reasonably well-factored. Key constraint: the Ollama server URL is set per deployment, not per user, so all users share one Ollama host. Worth evaluating if on-prem inference is a requirement; the provider implementation and model-ID helpers are self-contained enough to lift without major surgery.

View on GitHub →

jellz77's fork wires Ollama into mike as a third LLM provider alongside Claude and Anthropic. The backend gets a new ollama.ts provider (~301 lines) implementing streaming and tool calls against Ollama's /api/chat endpoint, plus an auth-gated /ollama/tags route that exposes the local server's model list. Model IDs follow an ollama:<name> prefix convention registered in the existing provider dispatch. The frontend adds a model picker group that queries available tags on dropdown open, and modelAvailability.ts is refactored into the canonical source of truth for model-ID validation across the codebase.

What's in it

On-prem LLM via Ollama Adds Ollama as a first-class provider alongside Claude and Gemini, so you can point Mike at a locally-hosted model instead of a hosted API.
Streaming responses from local models Local inference is wired into the same streaming chat experience as the hosted providers - no second-class treatment for the on-prem path.
Self-hosted deployment story Makes a fully self-contained Mike deployment plausible for firms that can't or won't send client material to third-party LLM APIs.

Direction

infrastructureintegrationsecurity

Activity

Themed changes and pull requests touching this fork, newest first. Themed changes that haven't been turned into a public post yet still appear — they're real work even without a published writeup.

📝 jellz77 takes Mike fully on-prem 2 commits 2mo ago infrastructuresecurity draft

A new build wires Mike up to AI models running on your own hardware instead of a cloud provider.

Threads of work (detailed view)

2 threads have been distilled into posts.

jellz77 takes Mike fully on-prem

A new build wires Mike up to AI models running on your own hardware instead of a cloud provider.

infrastructuresecurity

2 commits · latest 2mo ago

jellz77/mike: Ollama as a third LLM provider for on-prem inference

jellz77 adds Ollama alongside Claude and Gemini - full streaming, tool calls, and a model-picker dropdown that queries the local Ollama server for available tags. The `ollama:<name>` model ID convention routes through the existing provider dispatch cleanly.

infrastructuresecurity

2 commits · latest 2mo ago