Pre-Phase-7: per-source LLM routing seam
From the PR description
Summary
- Defer original Phase 7 (local inference / Ollama / vLLM) to post-launch, and land the per-source LLM routing seam now so it's in place before Phase 7 connectors. See
decisions.md(2026-05-15) and the renumbered build plan inCLAUDE.md. - New
resolveModelRouting()resolver with document → project → request precedence. Stored prefs are validated against the canonical model set; unknown IDs are rejected and recorded, then the resolver walks down the chain. Conflicts between documents are captured but the first non-null wins (no policy engine yet). streamChatWithToolsaccepts an optionalroutingcontext, dispatches with the resolved model, and records the resolution into the existingaudit_log.routing_policy_appliedjsonb column shipped in Phase 6.- Main chat path wired to pass the chat's project + the document IDs from
docIndex. Today this resolves to the user's requested model unchanged; Phase 7 connectors will populatedocuments.model_preferenceat ingest and the same code path will route accordingly without further changes. - New
model_preference(nullable text) columns onprojectsanddocuments. No UI yet - by design.
Why now, not in Phase 7
The local-inference adapter is cheap to add later (a fourth file alongside claude.ts/gemini.ts/openai.ts). The routing policy surface is the part that would otherwise get baked into every connector and dispatch site in Phase 7+, and retrofitting it would force a much wider edit later. Landing the seam now keeps Phase 7 connectors free to declare a model_preference at ingest without inventing the surface from scratch.
Test plan
-
backend/npx tsc --noEmitclean -
backend/npx vitest run- 73/73 green (8 new routing tests covering precedence, conflicts, unknown-model rejection at doc + project layers, db-error tolerance, and missing-document-ids skip path) - Manual: with the migration applied, set a
documents.model_preferenceto a known model id on one doc; confirm a chat that includes that doc resolves to the override and writes the policy intoaudit_log.routing_policy_applied - Manual: confirm a chat that includes no documents and no project override still works and
audit_log.routing_policy_appliedrecordssource: "request"
🤖 Generated with Claude Code
Our analysis
Land the per-source model routing seam before Phase 7 connectors — read the full analysis →
Think the analysis missed something the PR description covers?
Capture this PR into my fork
Download a Markdown prompt that tells Claude how to port every
commit in this PR into your working tree. Run it via
claude -p < capture-pull-7.md from
inside the repo you want the changes in.