nforum wires Mike up to a self-hosted AI backend

The fork can now run on a private inference server instead of (or alongside) the big cloud model vendors.

infrastructurepersonas

nforum has added support for vLLM, an open-source engine for hosting large language models on your own hardware. With a couple of environment variables, the fork will route prompts to a private endpoint rather than Anthropic, OpenAI, or Google. Two new model options appear in the picker, and a single provider module quietly handles both cloud and local backends through the same OpenAI-compatible plumbing.

The architecture is clean - one code path, two destinations - and it slots in alongside the existing cloud providers rather than replacing them. There is one wrinkle worth flagging for anyone borrowing the work: once a local server is configured, the fork silently makes it the default for everything, including the small model that names chat threads. That is a policy choice, not a technical one, and any downstream fork should decide it deliberately.

So what Relevant for any firm or legal-tech team that can't send client data to a third-party AI vendor and needs the option to keep inference inside its own walls.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

4 commits from nforum/mike, oldest first. Source extracted verbatim from the harvested git log.

SHA	Subject	Author	Date
`0c84ef49`	feat: Add LocalLLM (vLLM) provider support	Joseph Breda	2026-05-02	↗ GitHub
commit body - Add OpenAI-compatible LLM provider for local vLLM endpoints - Support for configurable model names via environment variables - Add LocalLLM Main and LocalLLM Lite as default models - Update model selector to include LocalLLM options - Fix generate_docx title fallback for missing parameters - Add LibreOffice dependency note for document conversion
`cc951105`	feat: Replace unsloth model with placeholder in .env.example	Joseph Breda	2026-05-04	↗ GitHub
`fe3fd823`	docs: Add LLM configuration options to README	Joseph Breda	2026-05-04	↗ GitHub
`86dab800`	merge: resolve PR #20 (vLLM/LocalLLM) conflicts with PR #16 (OpenAI)	Bojan Plese	2026-05-07	↗ GitHub
commit body Unified LLM provider architecture: - openai.ts: dual client factory (OpenAI cloud + vLLM local) via baseURL - models.ts: all 4 provider groups (LocalLLM, Anthropic, Google, OpenAI) - userSettings.ts: DB openai key with VLLM_API_KEY env fallback - ModelToggle.tsx: 4-group type union and GROUP_ORDER - modelAvailability.ts: LocalLLM always available (server-configured) - All frontend apiKeys: use profile.openaiApiKey from DB

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-49.md from inside the repo you want the changes in.

⬇ Download capture-thread-49.md