feat(llm): add model registry and OpenAI-compatible provider foundation
From the PR description
Summary
Lets a deployment declare a self-hosted or third-party OpenAI-compatible model
through MIKE_MODEL_CONFIG_JSON, without a code change, and makes those models
usable even when the endpoint behind them cannot be trusted to emit structured
tool calls.
Declared models are served by the existing AI SDK provider layer via
@ai-sdk/openai-compatible, so they inherit its transport, retries and
streaming rather than introducing a second HTTP client.
What changed
Registry
lib/llm/registry.ts- parsesMIKE_MODEL_CONFIG_JSONonce per process, validating entries and dropping malformed ones. Invalid JSON fails loudly at startup.lib/llm/models.ts-providerForModelandresolveModelconsult the registry, so a declared id resolves and persists like any catalog model.lib/llm/providers.ts- anopenai-compatiblebranch builds the adapter from the declaredbaseUrl, key andapiModel.
Only openai-compatible endpoints are declarable. The hosted providers are
already covered by the static catalog and the openrouter/, vercel/ and
opencode-go/ prefixes, so a configured entry for one of them would be a
second, subtly different way to say the same thing.
Tool-call tolerance
Self-hosted Qwen, DeepSeek and GLM builds routinely describe tool calls in
prose and wrap reasoning in <think> tags. Rather than a bespoke client, this
is a language-model middleware (wrapLanguageModel):
lib/llm/toolCallParsing.ts- pure text handling:<think>routing, tool markup suppression across chunk boundaries, and recovery of tool calls from JSON in<tool_call>markers, XML-ish<function=name>blocks, DeepSeek DSML invocations and single-key maps, withjsonrepairfor malformed arguments.lib/llm/localModelMiddleware.ts- applies that to the AI SDK stream: reasoning parts instead of visible text, real tool-call parts instead of prose, and a corrected finish reason.
A tolerant model answering a tool-declaring turn is served through the
endpoint's non-streaming path, because these models interleave markup with
prose in a way a partial stream cannot be reassembled from. Tolerance defaults
on for location: "local" and is overridable per model.
Other
routes/tabular.ts- availability messages understand declared models: a local endpoint needs no key, a cloud one is checked against its own declaration.docs/configured-models.md- configuration reference and the tolerance rules.
Why
Mike could only talk to the providers it was compiled to know about. Operators running a model on their own hardware had no way in, and the endpoints they actually run are exactly the ones that need the tolerance layer.
Testing
backend:tsc --noEmitclean;vitest run- 764 passed, 25 skipped, including 39 new tests acrosstoolCallParsing,localModelMiddlewareandregistry. The middleware tests drive it with fakedoStream/doGenerateresults, so they exercise the parsing and stream rewriting without mocking the network.
13 files changed, 1818 insertions.
Our analysis
Add configurable OpenAI-compatible model registry — read the full analysis →
Think the analysis missed something the PR description covers?
Capture this PR into my fork
Download a Markdown prompt that tells Claude how to port every
commit in this PR into your working tree. Run it via
claude -p < capture-pull-339.md from
inside the repo you want the changes in.