docs+test: surface and pin OPENAI_BASE_URL gateway routing for OpenAI models

🟢 open · #402 · open-legal-products/mike ← open-legal-products/mike · opened 10d ago by amal66 · self · +103-1 across 2 files · ↗ on GitHub

From the PR description

Summary

Mike can already serve its OpenAI models from an Azure OpenAI endpoint or a LiteLLM-style proxy with zero code changes - @ai-sdk/openai falls back to the OPENAI_BASE_URL environment variable whenever createOpenAI() is called without an explicit baseURL, which is exactly how backend/src/lib/llm/providers.ts calls it. Nothing in the repository documents this, and nothing protects it: the capability lives entirely in a dependency's fallback path. This PR makes it discoverable (.env.example) and pins it with an end-to-end regression test. No runtime code changes.

Context: a community request for "more pluggable backends so that the Azure backend could be an option on base" - the Azure fork it mentions exists to solve a problem base Mike already solves invisibly.

OPENAI_BASE_URL demo

Live demo: a local OpenAI-protocol stub stands in for a corporate gateway; Mike's real compiled provider layer sends POST /v1/responses to it and round-trips the completion.

What changed

  • backend/.env.example - documents OPENAI_BASE_URL next to OPENAI_API_KEY, including the two real constraints (gateway must support the Responses API; deployments must be named after model ids in Mike's catalog).
  • backend/src/lib/__tests__/llmProviders.test.ts - two new tests pin the routing end to end by capturing requests at aiSdkFetch (the fetch implementation every provider adapter hands to its SDK): the default resolves to https://api.openai.com/v1/responses, and with OPENAI_BASE_URL=http://gateway.test/v1 the same call lands on http://gateway.test/v1/responses and the completion parses.

Why

Behavior that is undocumented might as well not exist - the community's lagging Azure fork is the proof. And behavior that lives in a dependency's fallback (loadSetting(..., environmentVariableName: "OPENAI_BASE_URL") inside @ai-sdk/openai) is one SDK upgrade away from silently disappearing. Documentation makes it usable; the test makes it safe to rely on.

Reproduce the base behavior on main

  1. git checkout main && npm ci --prefix backend && npm run build --prefix backend
  2. grep -rn OPENAI_BASE_URL backend/src backend/.env.example docs/zero hits: nothing tells an operator this variable exists.
  3. Yet routing already honors it:
    OPENAI_BASE_URL=http://localhost:9/v1 OPENAI_API_KEY=sk-x node -e \
      "require('./backend/dist/lib/llm/providers.js').completeWithProvider({model:'gpt-5.4',user:'hi'}).catch(e=>console.log(e.message))"
    
    The failure is ECONNREFUSED 127.0.0.1:9 - the request went to the override, not to api.openai.com. Working, invisible, unpinned.

Verify on this branch

  1. Same setup on this branch.
  2. backend/.env.example now documents the variable and its constraints.
  3. npm test --prefix backend -- src/lib/__tests__/llmProviders.test.ts → 4 tests pass, including both routing pins. Delete the SDK's env fallback (or stub OPENAI_BASE_URL handling out) and the test fails - that's the regression net.
  4. Optional live check: run any OpenAI-protocol stub locally and repeat the step-3 command from the base case with the stub's URL; the completion round-trips (see GIF).

Tradeoffs & design decisions

  • Deliberately no runtime code. An earlier draft passed baseURL explicitly in providers.ts; live A/B testing showed it was a functional no-op because the SDK already reads the variable, so it was dropped. If a future @ai-sdk/openai removes the fallback, the pinned test fails and the fix is a one-line explicit baseURL in createOpenAI().
  • Responses API required. The adapter calls openai.responses(model), so chat-completions-only gateways won't work via this variable. Those are already served by the OLLAMA_BASE_URL + ollama/<tag> path (openai-compatible adapter), at the cost of "Local (Ollama)" labeling.
  • Deployment naming constraint. Model validation still runs against Mike's catalog, so Azure deployments must be named after Mike's model ids (gpt-5.4, gpt-5.6-terra, ...). Accepting arbitrary deployment names would need catalog/validation work (a provider-registry discussion, out of scope here).
  • UI provider labels are untouched - requests routed through a gateway still display as "OpenAI".

Testing performed

  • npm test --prefix backend -- src/lib/__tests__/llmProviders.test.ts (4 passed)
  • npm test --prefix backend (842 passed, 25 skipped)
  • npm run build --prefix backend (tsc clean)
  • Live A/B against a local OpenAI-protocol stub: with the variable set, Mike's compiled provider layer hit the stub and returned its completion on this branch and on unmodified main - which is precisely why this ships as docs + test rather than code (recorded in the GIF above).

🤖 Generated with Claude Code

https://claude.ai/code/session_01BpdjmF3BsrGfjVX6qEx2G2

Our analysis

Document OpenAI gateway routing — read the full analysis →

Think the analysis missed something the PR description covers?

Capture this PR into my fork

Download a Markdown prompt that tells Claude how to port every commit in this PR into your working tree. Run it via claude -p < capture-pull-402.md from inside the repo you want the changes in.

⬇ Download capture-pull-402.md