Pre-Phase-7: model routing seam added before connector work begins
Archibald312 deferred local inference (Ollama, vLLM) to post-launch and instead landed the routing surface that connectors and the eventual local-inference adapter will plug into. The behavioral change today is zero - every chat still resolves to the user's requested model - but the decision point now exists as a tested, audited abstraction.
The schema change is minimal: nullable model_preference text columns on projects and documents. Existing rows are unaffected.
backend/src/lib/llm/routing.ts (156 lines) implements resolveModelRouting(). Precedence runs document → project → request: any document in the request with a non-null preference wins first; if none, the project preference; if none, the caller's requested model. When two documents in the same request carry different preferences, the resolver captures the conflict with a structured reason and takes the first non-null value rather than blocking. Unknown model IDs are rejected with a recorded reason. All decisions write into the routing_policy_applied jsonb column in audit_log - that column was provisioned in Phase 6 specifically for this.
streamChatWithTools now accepts a routing context, dispatches with the resolved model, and records the policy decision. The main chat path passes the chat's project ID and the document IDs already tracked in the in-memory document index.
Eight Vitest tests cover: document-level precedence over project-level, project-level over request, conflict capture, unknown-model rejection at both layers, DB error tolerance when the preference lookup fails, and the early-exit path when no document IDs are supplied. Two manual verification steps - confirming a document-level override actually routes and audits correctly end-to-end, and confirming a no-document chat records a request-sourced resolution - were left unchecked in the test plan at merge time.
Phase 7 connectors will populate documents.model_preference at ingest from their declared preference. Phase 14 wires in a local-inference adapter as another model option. Neither requires touching dispatch sites.
Spotted something wrong? Or know the PR text has fresher detail than the writeup above?