LLM dispatch bug hunt: local/* prefix ignored, stale model ids crash chat
Seven commits tracing one user-visible symptom - `POST /chat` throwing "External AI providers are disabled" on a deployment where only Ollama was provisioned - through four layers of the dispatch stack before landing on the real root cause: `resolveModel` in `lib/llm/models.ts` didn't recognise `local/*` model ids and silently reset them to the Gemini default.
The deployment had EXTERNAL_AI_DISABLED=true and no external provider keys. Users saw the external-disabled error despite the model picker visibly showing a local model.
Layer 1: diagnostics (953bd2a3). assertModelAllowed gets two extra characters of context: the error message now includes the offending model id and resolved provider. Two lines in lib/llmPolicy.ts. Would have shortened the whole investigation; added first.
Layer 2: title and tabular policy gap (afc21cea). resolveTitleModel and a new resolveTabularModel were hardcoded to prefer Gemini/OpenAI-nano/Claude-Haiku regardless of org policy. On an externals-disabled install, every title generation and every tabular extraction threw even when the main chat worked. Both functions now consult LlmPolicy: if externals are allowed and a key is present, use the cheapest external; otherwise fall through to the first curated local/<id>; otherwise return null. Callers are updated: title-gen writes a truncated copy of the user message as the title and returns 200 (title is best-effort); tabular returns 503 with "ask an admin" instead of 500.
Layer 3: frontend race (30fa8010). Clicking Send before /me/models resolved meant useSelectedModel still held a stale localStorage value (commonly gemini-3-flash-preview). The hook now fetches available models on mount, validates the stored selection against the live list, and only then sets a ready flag. ChatInput returns early on submit if !modelReady, and the Send button is disabled until the list resolves. ModelToggle's own fallback-onChange is removed to stop it fighting the hook's validation.
Layer 4: route-level tolerance (532738c5). resolveAllowedModel(requested, policy) added to llmPolicy.ts: returns the requested id if the policy allows it, the first allowed id otherwise, or null if nothing is enabled. POST /chat and POST /projects/:id/chat now resolve before dispatching and log substitutions. No-provider-at-all returns 503 with a clear message.
Layer 5: dispatcher self-heal (dc31b293). The same tolerant resolution moves into streamChatWithTools / completeText in lib/llm/index.ts. Any future caller passing a stale id gets a substitution and a warning log; the only remaining hard-error is an empty policy.
The actual root cause (ac3ff6a9). All five layers above and the original error persisted on the live install with no [chat/stream] substituting warning. A trace log (4a5fa1f0) revealed it: runLLMStream in chatTools.ts re-resolved its incoming model via resolveModel in lib/llm/models.ts, which checked a hardcoded ALL_MODELS set containing only the external provider catalogue. local/* ids were not in the set, so even after the route correctly substituted gemini-3-flash-preview → local/llama3.2:3b, resolveModel silently reset it back to DEFAULT_MAIN_MODEL (Gemini). The fix is one character: ALL_MODELS.has(id) || id.startsWith("local/"). Trace log removed in the same commit.
Spotted something wrong? Or know the PR text has fresher detail than the writeup above?