feat: add Concentrate AI as an optional model router with ZDR-aware routing and live catalogs

🟢 open · #167 · Open-Legal-Products/mike ← concentrate-ai/mike · opened 2mo ago by concentrate-ai · +1,888-53 across 28 files · ↗ on GitHub

From the PR description

What this adds

This PR adds Concentrate AI (https://concentrate.ai) as an optional, additive model provider. Users who add a Concentrate API key get:

  1. Live model catalogs - the model picker no longer relies on a hard-coded list. On first open it fetches the catalog for each configured provider (Anthropic, Google, OpenAI, and Concentrate), filters to chat-capable models, and merges them into a single grouped list. A 5-minute in-memory cache makes repeat opens instant. Users can also force-refresh via the new Refresh model list button in Settings → Models.

  2. ZDR-aware routing - Concentrate's API exposes a per-model zdr (Zero Data Retention) flag. For models tagged as ZDR, if the user has both a direct provider key and a Concentrate key, the backend routes through Concentrate - the provider never sees the plaintext prompt. For non-ZDR models the backend continues to prefer the direct provider key; Concentrate is only used as a fallback if no direct key is configured. The routing is surfaced in the UI via a small ZDR pill badge in the picker.

  3. Concentrate is completely optional - no behavior changes if the Concentrate key field is left blank. All four provider fields appear in Settings → Models with plain-English descriptions of when each is used for routing.

  4. I added some pre-push hooks with a security audit + typecheck on every push. I'd love to add some ci/cd

Schema / migration

One additive change: 'concentrate' added to the provider check constraint in user_api_keys. Fresh installs run schema.sql as usual. Existing deployments apply backend/migrations/001_add_concentrate_provider.sql.

Manual test plan

  • No keys - picker shows static fallback list; no background network calls.
  • Add a Gemini key - open picker; Google models load from live catalog.
  • Add an OpenAI key - open picker; OpenAI models load.
  • Then remove a key - re-open picker; ensure provider's models disappear.
  • Add a Concentrate key - ZDR models appear with the ZDR pill badge; tooltip reads "Zero Data Retention...".
  • Send a chat message - backend log shows the selected model ID, not the default fallback.
  • Settings → Models → Refresh model list - clears cache and refetches.

Known gaps (not blockers)

  • ZDR set warms lazily - routing doesn't enforce ZDR until /concentrate/models has been hit once
  • No backend enforcement that the posted model ID is in any catalog.
  • OpenAI family regex needs manual extension when OpenAI ships new families.

If you are interested in some free tokens please email me at todd@concentrate.ai and I'll be sure to set you up with a generous grant.

Our analysis

Add Concentrate AI as an optional ZDR-aware model provider — read the full analysis →

Think the analysis missed something the PR description covers?

Capture this PR into my fork

Download a Markdown prompt that tells Claude how to port every commit in this PR into your working tree. Run it via claude -p < capture-pull-167.md from inside the repo you want the changes in.

⬇ Download capture-pull-167.md