willchen96/mike#122: Add local Ollama/Qwen support through the OpenAI-compatible provider

Add local Ollama/Qwen support through the OpenAI-compatible provider

🟢 open · #122 · willchen96/mike ← punyaslokdutta/HarveyOss · opened 13d ago by punyaslokdutta · +2,170-50 across 25 files · ↗ on GitHub

From the PR description

Summary

add OpenAI-compatible local runtime support for self-hosted endpoints like Ollama
add Qwen model options in Mike's model picker and document the Docker-plus-native-Ollama setup
make the OpenAI-compatible path stream local model output incrementally so Mike surfaces answers instead of appearing blank while Ollama is still generating

What changed

made the OpenAI adapter configurable via OPENAI_COMPAT_BASE_URL, OPENAI_COMPAT_MODEL, and OPENAI_COMPAT_ENDPOINT_MODE
added support for chat-completions style OpenAI-compatible backends used by Ollama and vLLM
treated env-configured OpenAI-compatible runtimes as available even without a hosted OpenAI API key
added Qwen 3 8B and 14B options to the model picker
documented the local Docker app stack plus native Ollama workflow for testing Mike locally

Validation

npm run build --prefix backend
local Docker stack rebuilt with the Ollama-compatible env path
verified local Supabase keys were wired into the Docker env files for the local dev stack

Notes

frontend lint still reports existing repo-wide issues unrelated to this branch
this branch is intentionally scoped to local inference enablement first; broader benchmark work can build on top

Screenshot

Mike with Qwen on Ollama

Our analysis

Add OpenAI-compatible local inference for Ollama and Qwen — read the full analysis →

Think the analysis missed something the PR description covers?

Commits in this PR (2)

SHA Subject Author Date

39cdf1ca chore: local setup - swap R2 for Supabase Storage, install frontend deps Punyaslok Dutta 2026-05-09 ↗ GitHub

SHA	Subject	Author	Date
`39cdf1ca`	chore: local setup - swap R2 for Supabase Storage, install frontend deps	Punyaslok Dutta	2026-05-09	↗ GitHub
commit body - storage.ts: replaced @aws-sdk/client-s3 + R2 with Supabase Storage (upload, download, delete, signed URLs all via @supabase/supabase-js) - .env.example: removed R2 vars, added STORAGE_BUCKET=mike - frontend/package-lock.json: updated after npm install --legacy-peer-deps Local setup recap: - Backend :3001, frontend :3000 - Supabase project: gbdfkvaigunfvrgurkwk (ap-northeast-1 Tokyo) - Storage bucket: mike (private, Supabase Storage) - DB schema applied via 000_one_shot_schema.sql - AI provider: Gemini - Secrets in .env / .env.local - gitignored, not committed Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
`4e73a45d`	Merge remote-tracking branch 'origin/main'	Punyaslok Dutta	2026-05-09	↗ GitHub

commit body

- storage.ts: replaced @aws-sdk/client-s3 + R2 with Supabase Storage
  (upload, download, delete, signed URLs all via @supabase/supabase-js)
- .env.example: removed R2 vars, added STORAGE_BUCKET=mike
- frontend/package-lock.json: updated after npm install --legacy-peer-deps

Local setup recap:
- Backend :3001, frontend :3000
- Supabase project: gbdfkvaigunfvrgurkwk (ap-northeast-1 Tokyo)
- Storage bucket: mike (private, Supabase Storage)
- DB schema applied via 000_one_shot_schema.sql
- AI provider: Gemini
- Secrets in .env / .env.local - gitignored, not committed

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

4e73a45d Merge remote-tracking branch 'origin/main' Punyaslok Dutta 2026-05-09 ↗ GitHub

Capture this PR into my fork

Download a Markdown prompt that tells Claude how to port every commit in this PR into your working tree. Run it via claude -p < capture-pull-122.md from inside the repo you want the changes in.

⬇ Download capture-pull-122.md