jpbreda pitched a self-hosted model option for Mike

A proposal to let firms run Mike against their own inference servers instead of Anthropic or Google - closed without merging.

infrastructuresecurity

jpbreda wired a third model provider into Mike alongside its existing Claude and Gemini paths: any OpenAI-compatible endpoint, the kind exposed by vLLM and similar self-hosted inference stacks. The motivation was operational rather than technical - give firms that don't want client matter data flowing to a commercial API the option of pointing Mike at hardware they control, with the endpoint, credentials, and model names all set by the operator rather than the end user.

The model picker gained a LocalLLM group with full and lightweight variants, and the system treated it as always available since it's configured at the server level. jpbreda tested it against their own endpoint and confirmed document generation worked end to end. Upstream closed the PR on 10 May without merging, so Mike's official provider surface stayed at two.

So what Worth a look for any firm weighing whether legal-AI tooling can live entirely inside their own perimeter - the plumbing exists in a fork even if it didn't land in the trunk.

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

3 commits from jpbreda/mike, oldest first. Source extracted verbatim from the harvested git log.

SHA	Subject	Author	Date
`0c84ef49`	feat: Add LocalLLM (vLLM) provider support	Joseph Breda	2026-05-02	↗ GitHub
commit body - Add OpenAI-compatible LLM provider for local vLLM endpoints - Support for configurable model names via environment variables - Add LocalLLM Main and LocalLLM Lite as default models - Update model selector to include LocalLLM options - Fix generate_docx title fallback for missing parameters - Add LibreOffice dependency note for document conversion
`cc951105`	feat: Replace unsloth model with placeholder in .env.example	Joseph Breda	2026-05-04	↗ GitHub
`fe3fd823`	docs: Add LLM configuration options to README	Joseph Breda	2026-05-04	↗ GitHub

SHA

Subject

Author

Date

0c84ef49

feat: Add LocalLLM (vLLM) provider support

Joseph Breda

2026-05-02

↗ GitHub

commit body

- Add OpenAI-compatible LLM provider for local vLLM endpoints
- Support for configurable model names via environment variables
- Add LocalLLM Main and LocalLLM Lite as default models
- Update model selector to include LocalLLM options
- Fix generate_docx title fallback for missing parameters
- Add LibreOffice dependency note for document conversion

cc951105

feat: Replace unsloth model with placeholder in .env.example

Joseph Breda

2026-05-04

↗ GitHub

fe3fd823

docs: Add LLM configuration options to README

Joseph Breda

2026-05-04

↗ GitHub

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-313.md from inside the repo you want the changes in.