Reference index: the amal66 fork, decomposed into 25 reviewable PRs

⛔ closed · #205 · open-legal-products/mike ← amal66/mike · opened 4mo ago by amal66 · closed 3mo ago · +77,610-34,011 across 734 files · ↗ on GitHub

From the PR description

This PR is a reference index - it is not meant to be merged. It maps everything in the amal66 fork onto small, individually reviewable PRs, so you can point at any row and take exactly that piece (or port it yourself - CLA signed, credit is enough).

Status update (2026-08-06): the index now doubles as a scoreboard - see the Status column in every table below. Where things stand: 9 rows merged into this repo's main, 1 partially merged (CORS landed as #245; its tool-confirmation half is open as #247), 4 open as live PRs here (#259 provider registry, #267 + #268 organizations, #294 durable queues, #295 service layers), and the remaining 11 rows refreshed - re-ported onto current main on top of everything that merged, gates green, ready to open as PRs here on request.

What this is

The amal66 fork has tracked this repo since May (kept continuously in sync with main; your UI kept verbatim as the source of truth - my changes are re-ported onto yours after every sync, never the reverse). This index decomposes the fork's entire delta into 25 standalone PRs, each:

  • based on a mirror of this repo's main (branch upstream-main on amal66/mike), in this repo's own layout (backend/ + frontend/) - so every diff reads exactly as it would arrive here;
  • one concern per PR, in the CONTRIBUTING format (summary / changes / why / testing);
  • carrying its tests, and a Provenance section proving each hunk is a mechanical port of code running in the fork - no logic invented during extraction;
  • one push away from becoming a PR on this repo, whenever wanted (see Recommended merge order below for the only sequencing that matters).

Recommended merge order

(Written before anything merged - kept for reference. For the rows still pending, only two constraints survive: #259 (provider registry) before amal66#28 / amal66#48, and #294 (durable queues) before amal66#41 RAG; #295 (service layers) stays last.)

Every row below is an independent diff against upstream-main and applies cleanly on its own, so any single row can still be taken in isolation. If you take several, the order only matters in the three places below - this is the sequence that keeps them conflict-free:

  1. Safety net first - amal66#24 (vitest harness). Every other row's tests ride on it. ✅ done (#228)
  2. Small security & self-contained rows - any order among themselves - amal66#26 SSRF, amal66#25 citation verification, amal66#30 download-token expiry, amal66#32 connector-secret encryption, amal66#31 CORS, amal66#29 spotlighting. Each edits routes/libs in place; no cross-conflicts.
  3. Extensibility base, then its stack - amal66#27 provider registry before amal66#28 demo mode, amal66#34 Ollama, amal66#48 Vertex (they build on #27). Independent siblings amal66#35, amal66#33, amal66#39, amal66#47, amal66#44 fit anywhere in this band.
  4. Architecture (read the ADR first) - amal66#38 organizations; amal66#40 durable queues before amal66#41 RAG (RAG builds on the queue).
  5. Clients & proof - anytime - amal66#43 e2e, amal66#45 evals, amal66#46 CI, amal66#37 Word add-in. Fully self-contained; zero route overlap.
  6. The backend reorganization LAST - amal66#42 service layers. It is the one row that relocates code: it retires 7 backend/src/routes/*.ts files into 41 backend/src/modules/* files. Taken last, it reconciles against the finished route set in a single pass; taken earlier, every route-touching row above (#40, #38, #29, #44, #33, #41) would need re-porting onto the new module layout. If you'd rather not adopt the module layout at all, skip it - nothing else depends on it.

Two hard rules regardless of the above: #27 → #28 / #34 / #48 and #40 → #41 - a stacked row needs its base merged first. The only routine friction is lockfile churn: rows that add a dependency touch backend/package.json / package-lock.json; regenerate the lockfile on merge and it's gone.

The tables below are the same 25 rows grouped by theme, with sizes and detail. Status legend: ✅ merged here · 🔓 open PR on this repo · 🔄 refreshed (re-ported onto current main, ready to open on request).

Start here (small, self-contained, zero new hosting cost)

PR What it does Size Status
amal66#24 Minimal vitest harness (this repo has no test runner; dev-deps only) - the safety net every other row's tests ride on S ✅ Merged as #228
amal66#26 SSRF guardrails: connector URLs can't make the server fetch internal/cloud-metadata addresses (incl. DNS-rebinding pinning) S ✅ Merged as #242
amal66#25 Citation verification: every quoted citation is checked against the actual document text server-side - pure string matching, no extra LLM calls S ✅ Merged as #249
amal66#28 Demo mode: first-run users (and journalist demos) get a real answer with zero API keys instead of an error (stacks on #27) M 🔄 Refreshed (earlier attempt #260 closed); awaits #259
amal66#29 Prompt-injection spotlighting: untrusted document text is nonce-fenced so it can't impersonate instructions S ✅ Merged as #248

Security pack (one topic per PR)

PR What it does Size Status
amal66#32 MCP connector secrets encrypted with per-record derived keys (no migration needed; legacy rows still decrypt) S 🔄 Refreshed (earlier attempt #244 closed)
amal66#30 Download tokens without an expiry are rejected instead of living forever S 🔄 Refreshed (earlier attempt #250 closed)
amal66#31 Production CORS allowlist + external tools default to requiring confirmation S ✅ CORS merged as #245 · 🔓 tool confirmation open as #247

Reliability & interchange

PR What it does Size Status
amal66#33 Tabular review streams auto-resume after network drops - cells keep filling instead of silently freezing (server heartbeats deferred to #170, which covers them) M 🔄 Refreshed
amal66#35 .mikeworkflow.json export/import with optional language/jurisdictions metadata - a UK pack isn't a US pack (feeds the SKILL.md workflows direction) M 🔄 Refreshed (reconciled with the YAML packs from #270)

Extensibility & local inference ("files never leave the building")

PR What it does Size Status
amal66#27 Provider registry: adding an LLM provider stops requiring edits at every call site M 🔓 Open as #259
amal66#34 Local LLMs via Ollama (OpenAI-compatible endpoints - the approach endorsed in #20), off unless ENABLE_OLLAMA is set (stacks on #27) M ✅ Landed via #282 (Ollama ships with the self-host stack)
amal66#39 Optional OpenTelemetry + Sentry - provably inert unless configured; AIRGAPPED=true is a hard kill switch M 🔄 Refreshed
amal66#44 DMS connectors: iManage and NetDocuments behind a pluggable interface - Mike reads documents where the firm already keeps them M 🔄 Refreshed
amal66#47 Pluggable storage adapter: R2 behavior preserved verbatim, GCS added, S3 region configurable M 🔄 Refreshed
amal66#48 Vertex AI provider (stacks on #27) S 🔄 Refreshed (earlier attempt #262 closed); awaits #259

Architecture (each opens with a design doc - read the ADR before the diff)

PR What it does Size Status
amal66#38 Multi-tenant organizations with owner/admin/member roles - a firm is not one user L 🔓 Open as #267 + #268
amal66#40 Durable job queues (BullMQ) for conversion/extraction - all flags default off; no Redis required for a small self-host L 🔓 Open as #294
amal66#41 RAG: pgvector retrieval + a search_documents tool, inside the firm's own database (stacks on #40) L 🔄 Refreshed; awaits #294
amal66#42 Service layers: route monoliths decomposed into tested modules - endpoint parity verified, 93 tests carried. Relocates route files - merge last (see Recommended merge order). XL 🔓 Open as #295 (stacked on #294)
amal66#36 Self-host & air-gap stack: one-step Docker, embedded Supabase, AIRGAPPED mode - fully self-contained (no dependency on any other PR), opt-in files that change nothing for non-Docker users. Where it overlaps the community's open Docker PRs (#196/#149), happy to reconcile with their authors rather than compete - but it stands alone either way L ✅ Merged as #282 (superseding #196)

Clients & end-to-end proof

PR What it does Size Status
amal66#43 Playwright e2e suite (auth, chat, projects, tabular, workflows) - 27 tests, verified green against a live stack built from this repo's own schema + migrations M ✅ Merged as #220 (a11y follow-up #243 also merged)
amal66#45 Evals harness - dependency-free prompt-safety/behavior gates (8 checks, failure gate proven) S 🔄 Refreshed (earlier attempts #231/#257 closed)
amal66#46 CI workflow - build + test jobs that no-op gracefully until the pieces they cover merge, safe in any order S ✅ Merged as #232
amal66#37 Word add-in: chat, tracked-changes rewrites, workflows and projects inside Word - 57 hermetic browser scenarios. Lives in the fork's namespace; PR exists so the work is reviewable in this repo's shape XL ✅ Merged as #221

Deliberately not offered upstream

Stripe billing (amal66#11) and the programmatic API/SDK surface (amal66#12) stay fork-only unless you ever want them - you've said there are no commercialisation plans, and they'd be new maintenance surfaces.

Ground rules

Small PRs on your terms, in any order including none; anything you'd rather port yourself, take it - credit is enough; everything I land, I maintain (each PR states a triage commitment, and if I go dark for 60 days, revert freely - the rungs are sequenced to be independently revertible); nothing lands while your licence decision is open without us talking first.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CEguyEgXa9JjCciXCcVemC

Our analysis

Input bounds and resource limits — read the full analysis →

Think the analysis missed something the PR description covers?

Commits in this PR (45)

SHA Subject Author Date
62f73a51 chore(chapter-01): organize the repo into readable workspaces Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 01 - Repository shape.

Plain-English map:
Move the project from flat `backend/` and `frontend/` folders into a
workspace layout: `apps/api`, `apps/web`, and shared `packages/*`.
This makes the codebase easier to walk because each folder has a clear job.

Why it matters:
A contributor should not need private context to know where the API, web app,
shared types, client package, or SDK live. The repo structure should teach the
project before anyone opens a README.

Principle:
Make the project legible before making it bigger. Monorepos work best when
each workspace owns one understandable responsibility.

Precedent borrowed:
The fork report showed active forks splitting concerns, adding SDK surfaces,
and creating deployment-specific structure. This commit turns that pressure
into a stable workspace map.

Upstream base: willchen96/mike@d39f580.
Original local commit: b587d63.
5ad6715c docs(chapter-02): add the community scaffolding for contributors Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 02 - Open-source collaboration.

Plain-English map:
Add a code of conduct, security policy, issue templates, and a pull request
template. These files tell people how to report bugs, propose changes, and
disclose vulnerabilities without guessing.

Why it matters:
Open-source projects are not only code. They need a shared operating manual
so first-time contributors know the rules of the room and maintainers get the
information they need.

Principle:
Healthy collaboration is documented. A project should make the safe path the
obvious path, especially for security reports.

Precedent borrowed:
GitHub community-health conventions and the security-reporting guidance
requested in upstream PR #147.

Upstream base: willchen96/mike@d39f580.
Original local commit: 045bdb7.
c72b8c74 feat(chapter-03): add Supabase migrations as database history Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 03 - Database evolution.

Plain-English map:
Introduce Supabase CLI configuration and timestamped migrations so database
changes can be reviewed, applied, and repeated over time.

Why it matters:
A single schema file shows what the database looks like today, but not how it
got there. Migrations tell future maintainers when and why the shape changed.

Principle:
Schema changes should be versioned like application code. Existing deployments
need an incremental path instead of a one-shot reset.

Precedent borrowed:
The fork report's Docker/local-development cluster, Supabase practice, and
upstream PR #113's database-integrity work.

Upstream base: willchen96/mike@d39f580.
Original local commit: bc72e34.
9f15ed1c fix(chapter-04): fail fast on config and compare secrets safely Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 04 - Runtime safety basics.

Plain-English map:
Validate required environment variables at startup, reuse Supabase and storage
clients instead of recreating them per request, and remove a timing leak from
download-token comparison.

Why it matters:
Bad configuration should be caught before users hit the app. Shared clients
avoid needless connection churn. Secret checks should not reveal clues through
response timing.

Principle:
Fail fast, reuse expensive resources, and compare secrets in constant time.

Precedent borrowed:
Upstream PRs #81, #106, and #109, which independently targeted timing-safe
comparison, env validation, and singleton client reuse.

Upstream base: willchen96/mike@d39f580.
Original local commit: c2d2a9a.
bfc55399 feat(chapter-05): add structured logs with request IDs Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 05 - Observability.

Plain-English map:
Add Pino JSON logging and attach a request ID to each request so related log
lines can be traced together in production.

Why it matters:
Plain text logs are hard to search once many users are active. Structured logs
let operators answer practical questions like "what happened during this one
failed upload?"

Principle:
Production logs should be queryable, consistent, and tied to request context.

Precedent borrowed:
Upstream PR #156 and common production logging practice in hosted services.

Upstream base: willchen96/mike@d39f580.
Original local commit: 6f59af5.
91f95485 fix(chapter-06): remove console logs and avoid PII in logs Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 06 - Privacy-aware logging.

Plain-English map:
Replace ad hoc console logging with structured logger calls and remove log
lines that exposed user/document activity identifiers.

Why it matters:
Legal-document activity is sensitive. Logs often flow to third-party systems,
so they should contain only what is useful for debugging and operations.

Principle:
Minimize sensitive data. Treat logs as production data, not scratch paper.

Precedent borrowed:
Upstream PR #80 and GDPR-style data-minimization practice.

Upstream base: willchen96/mike@d39f580.
Original local commit: 9398be6.
0a153ed6 fix(chapter-07): add security headers and readiness checks Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 07 - Deployable hardening.

Plain-English map:
Add API and frontend security headers, wire the structured request logger into
the Express app, and add a readiness endpoint that checks whether dependencies
are available.

Why it matters:
Browsers and platforms need clear signals. Security headers reduce browser-side
risk, while readiness checks tell deploy platforms when the app can actually
serve traffic.

Principle:
Defense in depth plus honest health checks.

Precedent borrowed:
Upstream PR #78 and standard container/load-balancer practice that separates
liveness from readiness.

Upstream base: willchen96/mike@d39f580.
Original local commit: 73ff97a.
be60ae90 chore(chapter-08): give the API workspace its own project config Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 08 - Workspace independence.

Plain-English map:
Add the API package metadata, TypeScript config, and Vitest config needed for
`apps/api` to build, run, and test as a clear workspace.

Why it matters:
After the monorepo move, each workspace needs its own local controls. A new
contributor should be able to enter `apps/api` and understand its commands.

Principle:
Every package should be independently understandable even inside a monorepo.

Precedent borrowed:
Monorepo conventions used by larger TypeScript open-source projects and by
forks that split Mike into clearer package boundaries.

Upstream base: willchen96/mike@d39f580.
Original local commit: 377d685.
eea82f6a chore(chapter-09): move API source into the workspace path Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 09 - Physical source move.

Plain-English map:
Move the API implementation files from the old backend location into
`apps/api` so the directory structure and package boundaries match.

Why it matters:
The repo should not say one thing in package metadata and another thing in its
file layout. Matching structure reduces mental bookkeeping for readers.

Principle:
Make ownership visible in the filesystem.

Precedent borrowed:
The same workspace and modularization pattern introduced in Chapter 01.

Upstream base: willchen96/mike@d39f580.
Original local commit: d1ec379.
bfb2466e refactor(chapter-10): split routes into modules with validation and access guards Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 10 - API boundaries.

Plain-English map:
Introduce a module layer for API routes, validate incoming request shapes with
Zod, and add authorization checks around tabular document access.

Why it matters:
Large route files make it hard to see where trust boundaries are. Validation
and access checks need to sit at the boundary before data reaches deeper code.

Principle:
Validate inputs and authorize data access close to the edge of the system.

Precedent borrowed:
Upstream PR #155 and the fork report's strongest security signal: multiple
forks independently patched CWE-639 style tabular-document access bugs.

Upstream base: willchen96/mike@d39f580.
Original local commit: f913869.
16de3244 test(chapter-11): cover token storage and API-key helpers Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 11 - First regression tests.

Plain-English map:
Add Vitest tests for download tokens, storage-path helpers, and user API-key
encryption behavior.

Why it matters:
These helpers sit near sensitive boundaries. If they regress, users can lose
access, leak secrets, or store files in the wrong place.

Principle:
Test the sharp edges first.

Precedent borrowed:
The fork report highlighted several forks with meaningful test suites. This
starts that same protection around core helper behavior.

Upstream base: willchen96/mike@d39f580.
Original local commit: 4fdc779.
7c1b463b ci(chapter-12): add automated checks for pull requests Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 12 - Continuous integration.

Plain-English map:
Add a GitHub Actions pipeline for tests, builds, migration checks, and deploy
gating.

Why it matters:
Reviewers should not have to run every check by hand. CI gives maintainers and
contributors a shared, repeatable answer about whether a change is ready.

Principle:
Make correctness checks part of the merge path.

Precedent borrowed:
Downstream AGPL disclosures and testing-focused forks that added CI around
backend and document-processing behavior.

Upstream base: willchen96/mike@d39f580.
Original local commit: e3d2728.
54dcdf77 fix(chapter-13): time-limit stalled LLM streams Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 13 - Bounded external calls.

Plain-English map:
Add a three-minute timeout around LLM server-sent-event streams so a stalled
provider call cannot hold a connection open forever.

Why it matters:
External services can hang. Without a timeout, one stuck call can consume
server resources until something outside the app kills it.

Principle:
Every external dependency call should have a clear time boundary.

Precedent borrowed:
Upstream PR #112.

Upstream base: willchen96/mike@d39f580.
Original local commit: 8992d98.
112a9a53 fix(chapter-14): give download tokens an expiration time Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 14 - Secret lifetime.

Plain-English map:
Add a 30-day expiration to signed download tokens while keeping old tokens
backward compatible.

Why it matters:
A signed link should not be useful forever. Shorter lifetimes reduce the
damage if a link is copied, leaked, or found later.

Principle:
Secrets and bearer tokens should expire.

Precedent borrowed:
Upstream PR #77.

Upstream base: willchen96/mike@d39f580.
Original local commit: e6bfeb1.
e3efabf0 fix(chapter-15): default public tables to deny-all RLS Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 15 - Database defense in depth.

Plain-English map:
Enable Row Level Security fallback policies that deny browser/client roles by
default on public tables.

Why it matters:
The backend uses a service role, but mistakes happen. If a future grant or
client path exposes a table, the database should still default to no access.

Principle:
Least privilege by default, with explicit access instead of accidental access.

Precedent borrowed:
Upstream PR #145.

Upstream base: willchen96/mike@d39f580.
Original local commit: faa098c.
08dd7159 fix(chapter-16): normalize shared-email access checks Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 16 - Consistent identity matching.

Plain-English map:
Lowercase emails before checking whether a user has shared access to a
project.

Why it matters:
People do not think of email case as meaningful. Access checks that disagree
with that expectation create confusing denials and uneven security behavior.

Principle:
Normalize identity fields before comparing them.

Precedent borrowed:
Upstream PR #79.

Upstream base: willchen96/mike@d39f580.
Original local commit: e740508.
f38ea30d fix(chapter-17): paginate chat history requests Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 17 - Resource limits.

Plain-English map:
Add `limit` and `before` cursor support to chat history loading so the API
does not return an unbounded number of chats in one request.

Why it matters:
Unbounded endpoints are easy to misuse accidentally and easy to stress on
purpose. Pagination keeps response sizes predictable.

Principle:
Public API reads should be bounded and page through large data sets.

Precedent borrowed:
Upstream PR #110.

Upstream base: willchen96/mike@d39f580.
Original local commit: b52b04f.
8664afc9 fix(chapter-18): bound JSON bodies and log process failures Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 18 - Operational guardrails.

Plain-English map:
Reduce the accepted JSON body size and add handlers that log unhandled
promise rejections and uncaught exceptions.

Why it matters:
Large request bodies can create memory pressure. Unexpected crashes should
leave useful logs so operators can understand what happened.

Principle:
Bound inputs and surface failures clearly.

Precedent borrowed:
The fork report's security-hardening cluster, where many forks added basic
abuse and reliability guardrails.

Upstream base: willchen96/mike@d39f580.
Original local commit: d70fde7.
93b1f1a0 feat(chapter-19): add API linting with security rules Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 19 - Automated review.

Plain-English map:
Add ESLint for the API workspace, including TypeScript and security-oriented
rules.

Why it matters:
Linters catch repetitive mistakes before a human reviewer spends attention on
them. That frees review time for product and architecture questions.

Principle:
Automate the boring parts of code review.

Precedent borrowed:
Open-source CI hygiene and the fork report's broad hardening pattern.

Upstream base: willchen96/mike@d39f580.
Original local commit: 276d1fa.
fd362463 ci(chapter-20): add dependency updates and ownership gates Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 20 - Maintenance flow.

Plain-English map:
Add Dependabot configuration and CODEOWNERS so dependency updates arrive
regularly and sensitive paths route to designated reviewers.

Why it matters:
Open-source maintenance gets easier when updates are routine and review
responsibility is explicit.

Principle:
Make maintenance systematic instead of heroic.

Precedent borrowed:
GitHub dependency-management and ownership conventions used across mature
open-source projects.

Upstream base: willchen96/mike@d39f580.
Original local commit: 5723a73.
bb249e0f ci(chapter-21): run audit and security lint checks in CI Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 21 - Security checks in the merge path.

Plain-English map:
Add npm audit and ESLint security scanning to the CI pipeline.

Why it matters:
Dependency risk and unsafe patterns should show up during pull request review,
not after deployment.

Principle:
Security automation belongs where code is merged.

Precedent borrowed:
The fork report's dominant security PR cluster and common CI hardening
practice.

Upstream base: willchen96/mike@d39f580.
Original local commit: 8cda4ca.
5762ee71 fix(chapter-22): derive API-key encryption keys with HKDF salts Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 22 - User API key protection.

Plain-English map:
Replace static SHA-256 key derivation with HKDF and a unique salt per stored
user API key row.

Why it matters:
Users may store real provider keys in Mike. Those keys deserve standard,
reviewable cryptography instead of one shared derived key for every row.

Principle:
Use established key-derivation functions and per-record salts for encrypted
secrets.

Precedent borrowed:
Upstream PR #76.

Upstream base: willchen96/mike@d39f580.
Original local commit: e0a64bd.
41ede550 fix(chapter-23): enforce monthly message credits before LLM calls Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 23 - Server-side quota enforcement.

Plain-English map:
Check a user's monthly message credits before making chat LLM calls.

Why it matters:
A UI counter alone does not enforce anything. The backend must stop expensive
provider calls when a user has reached the configured limit.

Principle:
Billing and quota rules must be enforced server-side.

Precedent borrowed:
Upstream PR #157.

Upstream base: willchen96/mike@d39f580.
Original local commit: fa433e1.
761f6129 fix(chapter-24): spotlight untrusted content for prompt-injection defense Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 24 - LLM threat modeling.

Plain-English map:
Fence document text, filenames, and other untrusted content with nonce-marked
spotlighting so the model can better separate data from instructions.

Why it matters:
Legal documents can contain malicious or simply confusing text. The model
should be told which text came from the user, which came from a document, and
which instructions are trusted.

Principle:
An LLM is not a security boundary. Prompts should preserve provenance and make
untrusted content explicit.

Precedent borrowed:
Upstream PR #158 and the threat model documented in `docs/SECURITY-MODEL.md`.

Upstream base: willchen96/mike@d39f580.
Original local commit: bededdd.
b358c585 fix(chapter-25): validate uploads by file signature Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 25 - Upload trust boundary.

Plain-English map:
Check uploaded files by their magic bytes, not only by their filename or
extension.

Why it matters:
A file can be renamed to look harmless. The backend should inspect the actual
file signature before accepting it as a supported document type.

Principle:
Trust content, not labels.

Precedent borrowed:
Upstream PR #78 and standard upload-validation practice.

Upstream base: willchen96/mike@d39f580.
Original local commit: c6dd2ec.
ca69fd8c refactor(chapter-26): bring chat tools into structured logging Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 26 - Sensitive-path observability.

Plain-English map:
Replace remaining console logging in `chatTools.ts` with the same structured
Pino logger used elsewhere.

Why it matters:
`chatTools.ts` assembles prompts and handles model tool calls. It should have
the same traceability and privacy discipline as the rest of the API.

Principle:
Observability conventions should be consistent, especially in sensitive code.

Precedent borrowed:
Upstream PR #156 and the structured logging baseline from Chapter 05.

Upstream base: willchen96/mike@d39f580.
Original local commit: 921ef5b.
5719dd1d feat(chapter-27): add Docker Compose for local self-hosting Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 27 - Self-hosting path.

Plain-English map:
Add Dockerfiles and Docker Compose configuration so contributors can run a
local web/API setup with less manual environment work.

Why it matters:
If local setup is fragile, forks drift because everyone solves deployment
differently. A supported path keeps more work shareable upstream.

Principle:
Self-hosting should be boring and documented.

Precedent borrowed:
Upstream PRs #44, #63, and #149 plus the fork report's large local/self-hosted
stack cluster.

Upstream base: willchen96/mike@d39f580.
Original local commit: b2ae01c.
e7a09761 test(chapter-28): add HTTP integration tests around the Express app Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 28 - Request-level testing.

Plain-English map:
Extract an app factory from process startup and add Supertest integration
tests that exercise the API through HTTP-style requests.

Why it matters:
Unit tests protect small functions. Integration tests catch mistakes in route
wiring, middleware order, and app startup assumptions.

Principle:
Test important boundaries at the level where users and clients hit them.

Precedent borrowed:
The fork report's testing leaders and downstream backend test coverage.

Upstream base: willchen96/mike@d39f580.
Original local commit: 9549258.
b7ef398b feat(chapter-29): retry transient LLM provider failures Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 29 - Provider resilience.

Plain-English map:
Wrap LLM provider calls with exponential-backoff retries for temporary errors
like rate limits, timeouts, and provider outages.

Why it matters:
LLM providers occasionally fail for reasons the user cannot fix. A short,
careful retry can turn a temporary outage into a normal response.

Principle:
Classify external failures before retrying. Retry transient problems, not bad
requests or authentication errors.

Precedent borrowed:
The fork report's alternative-provider cluster and provider-abstraction work
across active forks.

Upstream base: willchen96/mike@d39f580.
Original local commit: 1417930.
5b82eff9 fix(chapter-30): convert user IDs to UUID foreign keys Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 30 - Database identity integrity.

Plain-English map:
Convert nine `user_id` columns from text to UUID and attach them to Supabase
Auth users with foreign keys.

Why it matters:
Supabase user IDs are UUIDs. Storing them as plain text allows invalid values
and orphaned rows that the database cannot protect against.

Principle:
Let the database enforce identity shape and relationships.

Precedent borrowed:
Upstream PR #113.

Upstream base: willchen96/mike@d39f580.
Original local commit: 5c28df3.
4378b6af feat(chapter-31): make workflows exportable and importable Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 31 - Workflow portability.

Plain-English map:
Add endpoints for exporting and importing workflows as `.mikeworkflow.json`
files.

Why it matters:
Workflows capture reusable legal know-how. Users should be able to move them
between Mike instances, teams, and environments without database access.

Principle:
Useful knowledge should be portable.

Precedent borrowed:
Upstream PR #59.

Upstream base: willchen96/mike@d39f580.
Original local commit: 212fbbe.
c32e9307 docs(chapter-32): define the workflow pack format Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 32 - Shared workflow contract.

Plain-English map:
Add a JSON Schema and human documentation for workflow packs, including
examples and field explanations.

Why it matters:
Portable files need a shared contract. A schema lets editors, importers, and
forks validate workflow packs in the same way.

Principle:
Schema-first formats reduce fragmentation.

Precedent borrowed:
Upstream PR #34 and workflow-heavy jurisdiction and practice-area forks from
the fork report.

Upstream base: willchen96/mike@d39f580.
Original local commit: 557e647.
0de61709 docs(chapter-33): warn contributors how to test safely Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 33 - Safe local development.

Plain-English map:
Document how to avoid using production databases, production buckets, real
client documents, or browser-exposed service-role keys during local testing.

Why it matters:
New contributors can accidentally leak sensitive data or admin credentials
while only trying to run the app. The docs should prevent that mistake early.

Principle:
Good setup docs are a security control.

Precedent borrowed:
Upstream PR #133.

Upstream base: willchen96/mike@d39f580.
Original local commit: 5e3608b.
27694ca4 test(chapter-34): cover message-credit enforcement edges Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 34 - Quota regression tests.

Plain-English map:
Add unit tests for the monthly message-credit gate, including reset dates,
limit boundaries, null counters, and database-error behavior.

Why it matters:
A quota bug can either block legitimate users or make the limit meaningless.
The important edge cases need to be pinned down.

Principle:
Server-side gates need boundary tests.

Precedent borrowed:
Chapter 23 and the fork report's testing-focused active forks.

Upstream base: willchen96/mike@d39f580.
Original local commit: 919665e.
1f77a3fe test(chapter-35): cover spotlight nonces and document label lookup Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 35 - Prompt assembly tests.

Plain-English map:
Add tests for spotlight nonce generation and document label resolution in the
prompt/tool pipeline.

Why it matters:
Prompt fencing depends on unpredictable nonces, and tool calls depend on the
model resolving document labels correctly. Both failures are subtle.

Principle:
Security-sensitive prompt helpers deserve direct regression tests.

Precedent borrowed:
Chapter 24's prompt-injection defense work.

Upstream base: willchen96/mike@d39f580.
Original local commit: d198da9.
d2406f43 test(chapter-36): cover model routing and safe fallbacks Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 36 - Provider contract tests.

Plain-English map:
Add tests that verify known model IDs route to the right provider and unknown
model preferences fall back safely.

Why it matters:
Provider routing mistakes are hard for users to diagnose. They show up only
when a chat request hits the wrong external API or an unknown model string.

Principle:
Provider abstraction needs tests that define its contract.

Precedent borrowed:
The fork report's alternative-provider cluster.

Upstream base: willchen96/mike@d39f580.
Original local commit: a706a79.
213ee269 docs(chapter-37): make the README a guide to the whole fork Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 37 - Reader scaffolding.

Plain-English map:
Rewrite the README so it explains the fork's thesis, the open-source bets, the full change index, and the way to read this branch's commits as chapters.

Why it matters:
The branch changes a lot: repo shape, security, CI, Docker, workflows, tests, and docs. A less technical reader needs a map before they walk the history.

Principle:
Documentation should explain why the work exists, not only how to run it.

Precedent borrowed:
The fork report itself: repeated community patterns around security, self-hosting, provider choice, workflow portability, database integrity, and testing.

Upstream base: willchen96/mike@d39f580.
Original local commit: d598c70.
3031c9e6 fix(chapter-38): close action-plan gaps that can be automated Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 38 - Action-plan gap closure.

Plain-English map:
Add the missing foundation pieces that can be safely implemented now: a standard error helper with legacy detail compatibility, API-client parsing for the new error envelope, validated env usage for rate limits/CORS/storage, real storage readiness checks, MinIO-backed local object storage in Docker Compose, LLM circuit breaking, backend abort wiring for timed-out provider streams, access-helper tests, HTTP helper tests, and a manual follow-up register.

Why it matters:
The previous chapters handled the main security and infrastructure pulls, but the action plan still had loose ends. This commit tightens the operational boundary without pretending that larger product work like MCP connectors, a durable queue, or seeded browser E2E can be completed safely as drive-by changes.

Principle:
Automate what can be proven, document what needs manual credentials or product direction, and keep compatibility while improving contracts.

Precedent borrowed:
ACTION_PLAN.md, the fork report's testing/self-hosting/security clusters, and independent agent review of the remaining gaps.

Verification:
- npm test --workspace apps/api
- npm run build --workspace apps/api
- tsc -p packages/api-client/tsconfig.json --noEmit
- git diff --check

Upstream base: willchen96/mike@d39f580.
2be2fa8b feat(chapter-39): automate safe manual follow-ups Amal 2026-05-24 ↗ GitHub
commit body
Chapter: 39 - Manual follow-up reduction.

Plain-English map:
Move safe pieces out of MANUAL_FOLLOW_UP.md and into the repo: add Redis to Docker Compose as the queue dependency, add OpenAI-compatible base URL support with production HTTPS guardrails, and add a skipped-by-default live Supabase access harness for proving document ID filtering against a real local Supabase database.

Why it matters:
Some follow-up items genuinely need credentials or product decisions, but the repo can still provide the rails: local dependencies, documented commands, and tests that activate when the right environment is available.

Principle:
Turn manual work into repeatable hooks wherever possible; leave only the irreducibly manual or product-shaped decisions in the follow-up document.

Precedent borrowed:
ACTION_PLAN.md, the fork report's local/self-hosting and alternative-provider clusters, and Supabase's local migration/testing workflow.

Verification:
- npm test --workspace apps/api (118 passed, 1 skipped live Supabase harness)
- npm run build --workspace apps/api
- tsc -p packages/api-client/tsconfig.json --noEmit
- git diff --check

Upstream base: willchen96/mike@d39f580.
05be49c0 Cleanup Amal 2026-05-24 ↗ GitHub
f98c2092 refactor: provider registry, storage adapter, table-driven api-key lookup Amal 2026-05-24 ↗ GitHub
commit body
Implements three open-source extensibility primitives so adding a new LLM
provider, storage backend, or law-library integration is a single-file
operation with no edits to core dispatch logic.

**LLM provider registry** (lib/llm/registry.ts)
- LLMProviderAdapter interface: id, matchesModel, stream, complete, models
- registerProvider() / findProviderForModel() / allRegisteredModels()
- Built-in providers (Claude, Gemini, OpenAI) self-register in index.ts on
  load; Vitest mock hoisting still captures them so retryCircuit tests pass
- providerForModel() checks registry first, then falls back to prefix
  heuristics - external providers override routing with zero core edits
- resolveModel() checks live registry union then static ALL_MODELS fallback
  (derived from *_MODELS spreads, not hand-maintained)
- Dispatch in streamChatWithTools / completeText: registry lookup + explicit
  throw on missing adapter - no silent fallthrough
- registerProvider() re-exported from lib/llm for consumer convenience
- Example: lib/llm/providers/ollama.ts shows full third-party integration

**Storage adapter pattern** (lib/storage/adapter.ts + r2.ts)
- StorageAdapter interface: upload, download, delete, getSignedUrl, checkReady
- R2StorageAdapter implements the interface with singleton S3Client
- lib/storage.ts gains setStorageAdapter() - swap to Supabase Storage, local
  filesystem, AWS S3, or MinIO with zero changes to any route or middleware
- All original function exports preserved with identical signatures

**Table-driven API-key lookup** (core/apiKeyProviders.ts)
- Provider env-var names live in a Map, not in a switch statement
- registerApiKeyProvider() lets new providers declare their env vars once
- getUserApiKeyStatus() and getUserApiKeys() iterate getRegisteredProviders()
  so a fourth provider appears in both automatically - no literal-object edits
- UserApiKeys index signature extended so third-party keys flow through
  without type changes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
c0dfd7ec feat(law-libraries): plugin registry for jurisdiction-specific law integrations Amal 2026-05-24 ↗ GitHub
commit body
Adds a self-contained plugin system so teams can wire in Danish law,
EU law, Australian law - or any jurisdiction - without touching chatTools.ts
or any route file. One file + one registerLawLibrary() call activates the
plugin; its system prompt fragment and tools are picked up automatically.

- lib/lawLibraries/registry.ts: LawLibraryPlugin interface, registerLawLibrary(),
  buildLawLibrarySystemPrompt(), getAllLawLibraryTools()
- lib/lawLibraries/index.ts: clean public re-export surface
- lib/lawLibraries/examples/danishLaw.ts: full working example (Retsinformation.dk
  citation conventions, danish_law_search tool schema)
- chatTools.ts: two-line integration - system prompt and tool list now include
  all registered law library plugins automatically

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
97b37ca4 feat(python-sdk): httpx-based sync/async client for the Mike API Amal 2026-05-24 ↗ GitHub
commit body
Adds sdks/python/ - a fully typed Python SDK covering every key API
surface so community forks can build integrations without writing HTTP
boilerplate from scratch.

Structure mirrors the Anthropic and OpenAI Python SDK conventions:
- MikeClient / AsyncMikeClient - thin httpx wrappers; swap in a custom
  httpx client for proxy/testing, pass session_token for auth
- Resources: chat, documents, projects, tabular, workflows, user - each
  exposed as client.<resource>.<verb>()
- SyncStream / AsyncStream - iterate SSE events from chat and tabular
  generate endpoints as plain Python dicts
- Pydantic v2 models for every response type; open to model_validate from
  raw API JSON
- Exception hierarchy (MikeError → APIError → AuthenticationError,
  NotFoundError, RateLimitError, InternalServerError, StreamError)
- 15 pytest tests with respx mocking (sync + async paths, error handling)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
abe78b7d refactor(chatTools): split 3412-line monolith into focused modules Amal 2026-05-24 ↗ GitHub
commit body
chatTools.ts was a 3412-line file mixing static constants, types, context
builders, and tool execution. Now split into three focused files so each
concern is navigable and independently testable:

- lib/chatToolDefs.ts  - static constants + shared types (SYSTEM_PROMPT,
  TOOLS, PROJECT_EXTRA_TOOLS, TABULAR_TOOLS, WORKFLOW_TOOLS, DocStore,
  WorkflowStore, DocIndex, ToolCall, ChatMessage)
- lib/chatContext.ts   - chat context builders (buildMessages,
  enrichWithPriorEvents, buildDocContext, buildProjectDocContext,
  buildWorkflowStore, generateSpotlightNonce)
- lib/chatTools.ts     - runtime tool execution, down to 2604 lines;
  re-exports everything from both sub-modules so no callers changed

All 128 tests pass, zero type errors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
57c1c1b5 feat(google-cloud): GCS storage adapter + Vertex AI Gemini provider (#2) Amalanand Muthukumaran 2026-05-24 ↗ GitHub
commit body
* feat(google-cloud): GCS storage adapter + Vertex AI Gemini provider

Adds two drop-in Google Cloud integrations following the existing
extension points (StorageAdapter + LLMProviderAdapter) so no core files
needed editing beyond exporting the new classes.

**GCS storage adapter** (lib/storage/gcs.ts)
- Implements the StorageAdapter interface backed by @google-cloud/storage
- Auth via Application Default Credentials - covers service account key
  files (GOOGLE_APPLICATION_CREDENTIALS), Workload Identity on GKE /
  Cloud Run, and gcloud CLI for local dev (zero secrets to manage)
- Enabled when GCS_PROJECT_ID or GOOGLE_APPLICATION_CREDENTIALS is set
- Signed URLs use V4 signing; Content-Disposition header supported
- Swap R2 for GCS with one setStorageAdapter() call at startup:
    import { setStorageAdapter } from "./lib/storage";
    import { GCSStorageAdapter } from "./lib/storage/gcs";
    setStorageAdapter(new GCSStorageAdapter());

**Vertex AI Gemini provider** (lib/llm/providers/vertexAI.ts)
- Routes all Gemini model IDs through Google Cloud Vertex AI instead of
  AI Studio - same model IDs, different billing + auth + data residency
- Uses @google/genai (already a dep) with vertexai: true mode - no new
  package required
- Auth via ADC: Workload Identity on GKE/Cloud Run, or
  GOOGLE_APPLICATION_CREDENTIALS locally
- Replaces the built-in gemini provider on re-registration (same id):
    import { setupVertexAI } from "lib/llm/providers/vertexAI";
    setupVertexAI();

**Tests**: 23 new tests (16 GCS adapter, 7 Vertex AI provider); full
suite 151 tests passing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* docs(readme): rewrite as setup and contribution guide

The previous README was a project retrospective - design rationale,
open-source bets, a 40-row change index - with the actual setup steps
buried partway through. This rewrites it as a task-oriented doc:

- Quick Start: clone → install → env → database → run (5 steps)
- Configuration: all env vars grouped by category with descriptions
- Extending Mike: concrete code snippets for each extension point
  (custom LLM provider, Vertex AI, GCS, custom storage, law library
  plugins, Python SDK)
- Development: test/lint/build/typecheck commands in one place
- Troubleshooting: the four most common issues
- Contributing: short version inline, CONTRIBUTING.md for the rest

The commit history and design rationale move to a collapsed footnote
(<details> block) so they're available without dominating the page.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Capture this PR into my fork

Download a Markdown prompt that tells Claude how to port every commit in this PR into your working tree. Run it via claude -p < capture-pull-205.md from inside the repo you want the changes in.

⬇ Download capture-pull-205.md