Pre-Phase-7: per-source LLM routing seam

✅ merged · #7 · Archibald312/GordonOSS ← Archibald312/GordonOSS · opened 11d ago by Archibald312 · merged 11d ago by Archibald312 · self · +452-16 across 9 files · ↗ on GitHub

From the PR description

Summary

  • Defer original Phase 7 (local inference / Ollama / vLLM) to post-launch, and land the per-source LLM routing seam now so it's in place before Phase 7 connectors. See decisions.md (2026-05-15) and the renumbered build plan in CLAUDE.md.
  • New resolveModelRouting() resolver with document → project → request precedence. Stored prefs are validated against the canonical model set; unknown IDs are rejected and recorded, then the resolver walks down the chain. Conflicts between documents are captured but the first non-null wins (no policy engine yet).
  • streamChatWithTools accepts an optional routing context, dispatches with the resolved model, and records the resolution into the existing audit_log.routing_policy_applied jsonb column shipped in Phase 6.
  • Main chat path wired to pass the chat's project + the document IDs from docIndex. Today this resolves to the user's requested model unchanged; Phase 7 connectors will populate documents.model_preference at ingest and the same code path will route accordingly without further changes.
  • New model_preference (nullable text) columns on projects and documents. No UI yet - by design.

Why now, not in Phase 7

The local-inference adapter is cheap to add later (a fourth file alongside claude.ts/gemini.ts/openai.ts). The routing policy surface is the part that would otherwise get baked into every connector and dispatch site in Phase 7+, and retrofitting it would force a much wider edit later. Landing the seam now keeps Phase 7 connectors free to declare a model_preference at ingest without inventing the surface from scratch.

Test plan

  • backend/npx tsc --noEmit clean
  • backend/npx vitest run - 73/73 green (8 new routing tests covering precedence, conflicts, unknown-model rejection at doc + project layers, db-error tolerance, and missing-document-ids skip path)
  • Manual: with the migration applied, set a documents.model_preference to a known model id on one doc; confirm a chat that includes that doc resolves to the override and writes the policy into audit_log.routing_policy_applied
  • Manual: confirm a chat that includes no documents and no project override still works and audit_log.routing_policy_applied records source: "request"

🤖 Generated with Claude Code

Our analysis

Land the per-source model routing seam before Phase 7 connectors — read the full analysis →

Think the analysis missed something the PR description covers?

Capture this PR into my fork

Download a Markdown prompt that tells Claude how to port every commit in this PR into your working tree. Run it via claude -p < capture-pull-7.md from inside the repo you want the changes in.

⬇ Download capture-pull-7.md