feat(mcp): add Connectors - URL+headers and OAuth 2.1

🟢 open · #32 · willchen96/mike ← ZachLaik/mike · opened 21d ago by ZachLaik · +3,100-6 across 24 files · ↗ on GitHub

From the PR description

Summary

Adds Connectors: users can plug their own Model Context Protocol servers into Mike via Settings → Connectors, with no code change. Tools discovered from each enabled connector are merged into the chat assistant's per-request tool list and dispatched back to the right server at call time.

Two auth modes are supported:

  • API key / headers - paste an MCP URL + optional custom headers (e.g. Authorization: Bearer ...). Works for self-hosted servers and any token-issuing service.
  • OAuth 2.1 (auto-discover) - for spec-conformant servers (RFC 9728 + RFC 7591 dynamic client registration + PKCE). One-click sign-in via popup, no token to paste, auto-refresh on expiry. Verified end-to-end against https://legaldatahunter.com/mcp.

What's in

Backend (Express/TS)

  • user_mcp_servers table (RLS owner-only) with two migrations + the same DDL inlined into 000_one_shot_schema.sql.
  • lib/mcp/{client,servers,types,oauth}.ts - Streamable-HTTP client wrapper around the official @modelcontextprotocol/sdk, per-request loader, MCP inputSchema → Mike's OpenAIToolSchema converter (with 64-char tool-name guard), OAuthClientProvider impl backed by the row's oauth_* columns.
  • routes/mcpServers.ts (/user/mcp-servers) - CRUD + /test (connect + list_tools probe) + /oauth/start (returns authorize URL for popup).
  • routes/mcpOauth.ts (/mcp/oauth/callback) - public callback that verifies an HMAC-signed state token, finishes the SDK auth() flow, and postMessages the opener.
  • lib/chatTools.ts - extends runLLMStream and runToolCalls with an mcp__ dispatch branch; emits mcp_tool_result SSE events with capped args/output for in-chat observability.
  • routes/{chat,projectChat}.ts - load enabled connectors at request start, close clients in finally.

Frontend (Next.js)

  • New Settings tab /account/mcp: add / edit / delete / enable-disable connectors. Auth-mode radio (headers vs OAuth). For OAuth: Save & sign in opens a popup, the page polls until tokens land, then auto-runs tool discovery. For headers: tools are auto-discovered immediately on save.
  • Connectors button in the chat input next to Documents / Workflows - popover with per-server toggle switches (hides itself when no connectors).
  • mcp_tool_result block in assistant messages: Called <Server> · <tool> with a one-line preview and an expand for full pretty-printed JSON args + raw output.
  • Trust warning at the top of the page; secret-leak guard that redacts the Name field if it looks like a Bearer token was pasted into it.
  • mikeApi.ts typed client wrappers.

Env

  • New optional BACKEND_PUBLIC_URL (defaults to http://localhost:${PORT}) used to build the OAuth callback URL. Added to .env.example.

What's not in (deliberate, follow-up PRs)

  • Curated marketplace of trusted connectors with one-click install - straightforward layer on top once this lands.
  • Per-row encryption of header values + oauth_tokens. Currently they're stored at-rest in jsonb (RLS owner-only), which matches the existing precedent set by user_profiles.{claude,gemini}_api_key. Worth a dedicated hardening PR.

Security notes for reviewers

  • Backend uses the Supabase service-role key, so all /user/mcp-servers/* handlers explicitly filter by user_id = res.locals.userId. RLS on the table is belt-and-suspenders for any direct client access.
  • URL validation rejects non-HTTPS URLs except for localhost/127.0.0.1.
  • Headers capped at 20 entries × 4 KB.
  • The GET /user/mcp-servers response returns header keys only and a boolean oauth_authorized - header values and access tokens never round-trip to the browser, even to the row's owner (defense in depth - RLS would allow it).
  • OAuth state token: HMAC-signed with DOWNLOAD_SIGNING_SECRET, 5 min TTL, carries user_id + server_id only. CSRF-safe across the popup hop with no server-side session needed.
  • Mike registers as a public OAuth client (token_endpoint_auth_method=none, PKCE-protected) - no client secret to store confidentially.
  • mcp_tool_result SSE events truncate args + output to 4 KB before persistence to keep chat_messages.content from bloating; the model still sees the full untruncated tool output.

Test plan

  • npm run build --prefix backend clean
  • npx eslint clean on changed frontend files
  • Manual: add a public no-auth header-mode connector (e.g. https://mcp.deepwiki.com/mcp) → tools auto-discovered → chat invokes a tool → mcp_tool_result block shows args/output → reload chat, persisted blocks render the same.
  • Manual: bad URL → graceful failure, last_error populated, chat still works.
  • Manual: disabled connector → tools not exposed.
  • Manual: connector with Authorization header against a real authed MCP → end-to-end tool call succeeds.
  • Manual: OAuth-mode connector against https://legaldatahunter.com/mcp → popup opens at LDH → sign in → popup closes → row flips to OAuth · signed in → tools auto-discover → chat invokes an mcp__legal-data-hunter__search tool successfully.

Note (not introduced by this PR)

npm run build --prefix frontend fails at prerender on /account/* pages because frontend/src/lib/supabase.ts calls createClient("", "") at module load when env vars are absent. This affects main identically; dev mode is unaffected. Happy to fix in a separate PR if helpful.

🤖 Generated with Claude Code

Our analysis

Add user-configurable MCP connectors with OAuth 2.1 support — read the full analysis →

Think the analysis missed something the PR description covers?

Commits in this PR (3)

SHA Subject Author Date
277339f6 feat(mcp): add user-configurable MCP servers (URL + custom headers) Zacharie Laik 2026-05-04 ↗ GitHub
commit body
Lets users register Streamable-HTTP MCP servers from the Settings page.
Tools discovered from each enabled server are merged into the per-request
tool set under the `mcp__<slug>__<tool>` prefix and dispatched back to the
right server via runToolCalls. Headers (e.g. `Authorization: Bearer ...`)
are stored on the row.

Backend
- New `user_mcp_servers` table (RLS owner-only) with migration 001 + the
  same DDL inlined in the one-shot schema.
- `lib/mcp/{client,servers,types}.ts`: thin wrapper around
  @modelcontextprotocol/sdk's StreamableHTTPClientTransport, per-request
  loader, schema converter (MCP `inputSchema` -> Mike's OpenAIToolSchema)
  with 64-char tool-name truncation.
- `runLLMStream` and `runToolCalls` accept an optional `mcpServers` list;
  chat routes load + close clients in a try/finally.
- New `routes/mcpServers.ts` mounted at `/user/mcp-servers` with
  GET/POST/PATCH/DELETE plus `/test` for connect-and-list-tools probing.
  All handlers filter by user_id since the backend uses the service role
  key.

Frontend
- New `account/mcp` settings tab and page: add/edit/delete servers, toggle
  enabled, run test connection. Header values are masked in the form
  (type=password) and the GET endpoint returns header keys only.
- `mikeApi.ts`: typed CRUD wrappers.

Notes for review
- Header values are stored via the same RLS-only model used today for
  `user_profiles.claude_api_key`/`gemini_api_key`. Per-row encryption is
  a clean follow-up.
- OAuth-protected MCP servers are out of scope for this PR; a follow-up
  will add an OAuth 2.1 client (PKCE + dynamic client registration) so
  spec-conformant servers (e.g. https://legaldatahunter.com/mcp) work
  without manual token paste.
fad06aca feat(mcp): rename to Connectors, prettier tool calls, observability Zacharie Laik 2026-05-04 ↗ GitHub
commit body
Polish on top of the initial MCP support commit. Same scope (no auth/marketplace yet),
just smoothing the rough edges from a real test session.

UX
- Settings tab + chat-input button renamed to "Connectors". MCP is mentioned in
  the page description (with a link to modelcontextprotocol.io) so the protocol
  is still discoverable.
- New `Connectors` button next to Documents / Workflows in the chat input opens a
  popover with a per-server toggle switch. Hides itself when the user has no
  connectors configured.
- Tool calls in chat now render `Running <Server> · <tool>` (friendly) instead of
  the raw `mcp__<slug>__<tool>` prefix; the original name still routes correctly.
- After each MCP tool call, a result block shows ✓/✗ + first line of output, with
  a "Show details" toggle that expands pretty-printed JSON arguments and the full
  text output.
- New connectors auto-discover their tool list immediately on save (no extra Test
  click). Re-enabling a disabled connector also auto-tests.
- Settings card redesigned: status pill, header chips, expandable per-tool
  descriptions with More/Less. Sanitises Name field if it looks like a Bearer
  token was pasted into it (best-effort safety net).
- Amber "only add connectors you trust" notice at the top of the page and a
  compact restated form inside the Add panel.

Backend
- New SSE event type `mcp_tool_result` with `{ server, tool, ok, args, output }`.
  args/output capped at 4 KB each before persistence (the model still receives
  the untruncated tool output - only the user-visible preview is capped).
- `tool_call_start` now optionally carries `display_name`; the renderer
  prefers it.
52749e6e feat(mcp): OAuth 2.1 sign-in for connectors Zacharie Laik 2026-05-05 ↗ GitHub
commit body
Adds OAuth 2.1 (RFC 9728 discovery + RFC 7591 dynamic client registration +
PKCE) so spec-conformant MCP servers like https://legaldatahunter.com/mcp
work without the user pasting any token.

The MCP TypeScript SDK does almost all the heavy lifting via its `auth()`
helper - discovery, DCR, PKCE, code exchange, refresh. We only have to plug
in an OAuthClientProvider whose getters/setters read and write the row's
oauth_* columns, plus an HMAC-signed state token so the popup callback can
look the row up without a server-side session.

DB
- migration 002 + inline patch to the one-shot:
  alter table user_mcp_servers
    add auth_type ('headers'|'oauth' default 'headers'),
    add oauth_metadata jsonb,
    add oauth_tokens jsonb,
    add oauth_code_verifier text;

Backend
- New `lib/mcp/oauth.ts`:
  - `DbOAuthProvider` implements OAuthClientProvider, persists everything
    on the user_mcp_servers row.
  - "initiate" mode (used by /oauth/start) captures the authorize URL into
    a property so the route can return it for the popup; "use" mode (used
    by chat) throws ReauthRequiredError when the SDK wants the user back,
    so the caller can mark the row reauth_required.
  - signOAuthState/verifyOAuthState - HMAC over user_id+server_id (5 min
    TTL) reusing DOWNLOAD_SIGNING_SECRET. No DB round-trip on callback.
- `lib/mcp/client.ts`: accepts an optional authProvider passed through to
  StreamableHTTPClientTransport - the SDK auto-attaches Authorization
  headers and auto-refreshes on 401.
- `lib/mcp/servers.ts`: builds a DbOAuthProvider for OAuth rows that have
  tokens; rows without tokens are skipped (UI surfaces a "Sign in" button
  in settings instead).
- New `routes/mcpOauth.ts` mounted at /mcp/oauth: public callback that
  verifies state, finishes the SDK auth() flow, and returns a small HTML
  page that postMessage()s the opener and closes the popup.
- `routes/mcpServers.ts`:
  - POST /:id/oauth/start kicks off discovery + DCR via the SDK and
    returns { authorize_url } for the frontend popup.
  - POST creates honor `auth_type`; PATCH/test/list now project + return
    auth_type and a boolean oauth_authorized (the access_token itself
    never round-trips to the browser).
- `BACKEND_PUBLIC_URL` env var (defaults to http://localhost:${PORT}) used
  to build the OAuth redirect URI; documented in `.env.example`.

Frontend
- `account/mcp/page.tsx`:
  - Authentication mode radio in the Add form: "API key / headers" vs
    "OAuth (auto-discover)". Headers section hides itself in OAuth mode.
  - Save button label switches to "Save & sign in" for OAuth, which
    immediately opens the authorize popup. The page polls listMcpServers
    until oauth_authorized flips, then auto-runs tool discovery.
  - Per-card status pills: "OAuth · signed in" (blue) / "OAuth · sign-in
    required" (amber). Cards in the latter state show a "Sign in" button
    instead of "Test".
  - Simplified copy per user feedback: dropped the OAuth explainer block,
    redundant "By saving..." trust pill, and helper text under Name and
    URL inputs. Single load-bearing trust warning at top of page remains.
- `mikeApi.ts`: `startMcpOauth(id)` wrapper.

Security notes for reviewers
- access_token / refresh_token / oauth_metadata are stored at-rest in
  jsonb (RLS owner-only). Per-row encryption deferred to a separate
  hardening PR - matches existing precedent for user_profiles.{claude,
  gemini}_api_key.
- State token is HMAC-signed with DOWNLOAD_SIGNING_SECRET, 5 min TTL,
  carries user_id + server_id only. CSRF-safe across the popup hop with
  no server-side session needed.
- Public client (token_endpoint_auth_method=none, PKCE-protected) - no
  client secret needed for confidential storage.

Capture this PR into my fork

Download a Markdown prompt that tells Claude how to port every commit in this PR into your working tree. Run it via claude -p < capture-pull-32.md from inside the repo you want the changes in.

⬇ Download capture-pull-32.md