feat: user-defined LLM committees
From the PR description
Summary
Adds user-defined model committees: a committee answers one prompt with several
models at once and has a chair model synthesize their replies into the single
answer the user sees. Committees are built under Settings > Models, stored
on the profile (user_profiles.model_committees), and selected from a
Committee group in the model picker like any other model.
Stacked on #339 - the first commit here is that PR's foundation. Retarget or rebase once it merges.
What changed
Backend
lib/llm/committee.ts- members run concurrently against the ordinary provider layer, then the chair synthesizes. Guards against self-chairing, self-membership and cycles.lib/userCommittees.ts- normalization for reads (malformed rows are dropped, never thrown, so a bad row cannot lock a user out of Settings) and strict validation for writes (max 8 committees, 2-8 members).lib/llm/models.ts-missingCommitteeApiKeyModels/modelHasApiKey/resolveUsableModel: availability resolves recursively through a committee's members and chair.lib/routerModels.ts-resolveRequestedModelrecognises committee ids and gates them on availability, taking committees the caller already loaded so no request reads the profile twice.lib/userSettings.ts,lib/chat/streaming.ts,routes/chat.ts,routes/tabular.ts- committees are loaded once with the rest of the profile and threaded to dispatch.routes/user.ts-modelCommitteeson the profile (select / serialize / PATCH) plusGET /user/models. Profile reads get their own retry tier, so a database without the column keeps every other live column.migrations/20260823_02_user_model_committees.sql+schema.sql.
Frontend
components/settings/CommitteeSettingsSection.tsx- create, edit and delete committees. Only models the user can actually run are offered as members.ModelToggle- a Committee group, shown only when every member and the chair is usable.mikeApi/UserProfileContext- committee types, profile state, save.
Design notes
Availability is all-or-nothing. A committee is hidden from the picker and refused at request time unless every member and its chair is usable. Half a committee is not a usable answer.
A deleted committee is an explicit error, not a silent fallback to another model - quietly answering with a different model misrepresents which model wrote the response.
No tool-calling loop. When the caller supplies tools (the main chat path always does) they are dropped and the members are told so, rather than being left to claim document or case-law work they could not do.
Cost. Members are separate model calls, so a committee costs roughly the sum of its members plus the chair. Running them concurrently keeps the latency close to the slowest member instead of the sum.
Why
Different models disagree in useful ways on legal questions. A committee makes that disagreement visible to a synthesizing model rather than making the user run the same prompt several times by hand.
Testing
backend:tsc --noEmitclean;vitest run- 789 passed, 25 skipped (+25 over #339's foundation, covering committee dispatch, concurrency, cycle/self-reference guards, and committee validation).frontend:tsc --noEmitclean;npm run lint0 errors;vitest run- 564 passed, 14 failed. All 14 failures are pre-existing onmain(mikeApi.test.tsBlob handling,useSelectedModel,ChatInput.modelSelection) and were confirmed against the baseline before these changes.- Migration follows the dated convention with
schema.sqlupdated to match.
28 files changed in this commit (1681 insertions); 37 files and 3498 insertions including #339's foundation.
Our analysis
Add user-defined model committees — read the full analysis →
Think the analysis missed something the PR description covers?
Capture this PR into my fork
Download a Markdown prompt that tells Claude how to port every
commit in this PR into your working tree. Run it via
claude -p < capture-pull-341.md from
inside the repo you want the changes in.