Add LocalLLM/vLLM provider support
From the PR description
Overview
Adds support for OpenAI-compatible local LLM endpoints (vLLM) as a third provider alongside Claude and Gemini. This enables users to connect Mike to self-hosted LLM instances while maintaining compatibility with commercial APIs.
Key Changes
Backend:
- New
openai.tsprovider adapter for OpenAI-compatible endpoints - Configurable model names via environment variables (
VLLM_MAIN_MODEL,VLLM_LIGHT_MODEL) - Server-side configuration (no user API key management required)
- Added fallback for missing
titleparameter ingenerate_docxtool
Frontend:
- Added "LocalLLM" model group to model selector
- Default models:
LocalLLM MainandLocalLLM Lite - Updated model availability checks to treat LocalLLM as always available
- Updated account settings page to reflect LocalLLM configuration
Configuration:
- New environment variables:
VLLM_BASE_URL- Endpoint URL for OpenAI-compatible APIVLLM_API_KEY- Optional API key for authenticationVLLM_MAIN_MODEL- Model name for main chat tasksVLLM_LIGHT_MODEL- Model name for lightweight tasks (title generation)
Testing
- Tested with vLLM endpoint at
https://bredaai.com/api/v1 - Verified document generation workflow
- Confirmed model selection and routing works correctly
Notes
- LibreOffice is required for document conversion (DOCX ↔ PDF)
- LocalLLM models are server-configured and available to all users
- Maintains backward compatibility with existing Claude/Gemini providers
Our analysis
Add vLLM as a third LLM provider — read the full analysis →
Think the analysis missed something the PR description covers?
Commits in this PR (3)
| SHA | Subject | Author | Date | |
|---|---|---|---|---|
0c84ef49 | feat: Add LocalLLM (vLLM) provider support | Joseph Breda | 2026-05-02 | ↗ GitHub |
commit body- Add OpenAI-compatible LLM provider for local vLLM endpoints - Support for configurable model names via environment variables - Add LocalLLM Main and LocalLLM Lite as default models - Update model selector to include LocalLLM options - Fix generate_docx title fallback for missing parameters - Add LibreOffice dependency note for document conversion | ||||
cc951105 | feat: Replace unsloth model with placeholder in .env.example | Joseph Breda | 2026-05-04 | ↗ GitHub |
fe3fd823 | docs: Add LLM configuration options to README | Joseph Breda | 2026-05-04 | ↗ GitHub |
Capture this PR into my fork
Download a Markdown prompt that tells Claude how to port every
commit in this PR into your working tree. Run it via
claude -p < capture-pull-20.md from
inside the repo you want the changes in.