cpatpa rips out the rented database and rebuilds the plumbing in-house

Phase 2 swaps a hosted database service for a self-run Postgres setup - and hardens how user data lives and dies along the way.

infrastructuresecurity

cpatpa spent nine commits pulling Supabase - a hosted database-and-auth service - out of the back end and replacing it with a plain Postgres setup the team controls directly. The migration was staged so the app kept running the whole way through, smallest pieces first, biggest last.

The interesting part for legal teams isn't the plumbing, it's what got tightened in passing. Every table now defaults to locked-down access that fails closed rather than open. Account deletion is now genuinely destructive: it requires re-authenticating, then cascades through and removes the user's owned data - but the audit trail of what happened is deliberately preserved. The default settings also bake in a seven-year retention posture out of the box.

So what Anyone evaluating Mike forks for handling client data should note this one now controls its own data layer and treats deletion, access, and audit logging as first-class concerns.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

9 commits from cpatpa/PIP, oldest first. Source extracted verbatim from the harvested git log.

SHA Subject Author Date
76691b24 Phase 2 foundation: pg pool, migration runner, new schema Claude 2026-05-14 ↗ GitHub
commit body
This is the additive first step of the Phase 2 cutover from Supabase
to plain Postgres + Auth.js. No existing behaviour changes; the
existing Supabase paths still work. The new tables and infrastructure
land alongside the upstream schema and become the target state that
later sub-phases migrate onto.

Added:
- backend/src/lib/db.ts: a single pg Pool per process, with query,
  transaction, ping, and closePool helpers. Reads DATABASE_URL or the
  discrete PG* vars. SSL controlled by DATABASE_SSL.
- backend/src/lib/migrate.ts: tiny SQL-file applier with a
  pip_migrations tracking table. Each file runs inside a transaction.
  No down-migrations: recovery from destructive change is restore from
  backup. CLI entry runs via `npm run migrate --prefix backend` and
  will run on container boot from Phase 7.
- backend/migrations/0001_extensions.sql: pgcrypto and citext.
- backend/migrations/0002_users.sql: public.users with onboarding
  fields (jurisdictions, practice_areas, sectors, office_location,
  timezone), role and status, custom_instructions,
  ai_disclaimer_at, and a touch_updated_at trigger.
- backend/migrations/0003_org_settings.sql: single-row organisation
  configuration. Seeded with the canonical PA system prompt
  (AU English, AGLC4, hallucination guards, AI reminder), the AU
  jurisdictions, the PA practice areas and sectors taken from
  piperalderman.com.au, a 7-year default retention, the
  piperalderman.com.au email allowlist, and the external-AI banner
  text.
- backend/migrations/0004_audit_events.sql: append-only audit trail
  referencing public.users with ON DELETE SET NULL so audit history
  survives user deletion.
- Dependencies: pg, @types/pg, bcryptjs, @types/bcryptjs.
- docs/developer/03-database-schema.md and 08-migrations.md.
- backend/.env.example documents the new DATABASE_URL,
  DOCX_*, and JSON_*_LIMIT variables.

The Auth.js frontend cutover, backend middleware swap, and the
user_id text -> uuid FK migration (H1) are the next sub-phases.
d384aee8 Phase 2 schema completion + RLS + L1 (H1, L5, L1 closed) Claude 2026-05-15 ↗ GitHub
commit body
Migrations 0005-0011 create the remaining PIP application tables.
Every user_id is now uuid REFERENCES public.users(id) ON DELETE
CASCADE, closing audit finding H1. The previous user_id text columns
that had no foreign key allowed orphaned data after user deletion.

Added migrations:

  0005_user_profiles_and_api_keys.sql  user_profiles, user_api_keys
  0006_projects.sql                    projects, project_subfolders
  0007_documents.sql                   documents, document_versions,
                                       document_edits (chat_message_id
                                       FK added later in 0008)
  0008_chats.sql                       chats, chat_messages, document
                                       _edits.chat_message_id FK
  0009_workflows.sql                   workflows, hidden_workflows,
                                       workflow_shares
  0010_tabular_reviews.sql             tabular_reviews, tabular_cells,
                                       tabular_review_chats,
                                       tabular_review_chat_messages
  0011_rls.sql                         ENABLE + FORCE ROW LEVEL SECURITY
                                       on every backend-owned table;
                                       revokes any inherited grants to
                                       anon/authenticated

L5 closed by 0011_rls.sql. No row-level policies are defined; the
application connects via a BYPASSRLS role and is itself the access-
control boundary. RLS sits in place as defence-in-depth so any future
direct grant fails closed. The role provisioning commands are
documented in the migration file and in docs/developer/08-migrations.md.

L1 closed in backend/src/lib/userApiKeys.ts. USER_API_KEYS_ENCRYPTION
_SECRET is now required (minimum 32 characters); the previous fallback
to SUPABASE_SECRET_KEY is removed. Rotating the Supabase key no longer
renders stored model provider API keys undecipherable.

backend/schema.sql is marked deprecated. PIP defines its schema only
through the incremental migration files. The Supabase-flavoured file
is retained as a historical reference only.

End-to-end verification against a fresh Postgres 16 database:
- All 11 migrations apply cleanly in order.
- 20 tables created (19 application tables + pip_migrations).
- RLS enabled on every backend-owned table; pip_migrations excluded.
- org_settings seeded with 10 jurisdictions, 17 PA services, 16 PA
  sectors, 7-year retention, piperalderman.com.au allowlist.
- /health and /ready return ok with db: true.
- POST /auth/register grants admin role to BOOTSTRAP_ADMIN_EMAIL on
  first registration.
- POST /auth/register rejects disallowed domains and short passwords
  with clear messages.
- POST /auth/verify-credentials returns the same 401 for wrong
  password and unknown email (no user enumeration).
- users_ensure_profile trigger auto-creates the user_profiles row.
- audit_events table records user.register and user.login.credentials.
a0895b22 Phase 2: migrate lib helpers off Supabase JS onto pg Claude 2026-05-15 ↗ GitHub
commit body
Begins the route-migration sub-phase by converting the four lib
helpers that every route depends on. Helper signatures keep their
db parameter as optional and ignored so callers compile unchanged;
they get cleaned up once each calling route migrates.

  src/lib/access.ts          checkProjectAccess, ensureDocAccess,
                             ensureReviewAccess,
                             filterAccessibleDocumentIds,
                             listAccessibleProjectIds. Switched to
                             pg queries. listAccessibleProjectIds
                             now uses the @> JSONB containment
                             operator on shared_with (the PostgREST
                             cs equivalent), still parameterised.

  src/lib/userApiKeys.ts     getUserApiKeyStatus, getUserApiKeys,
                             saveUserApiKey. Insert/upsert/delete
                             now via parameterised SQL with ON
                             CONFLICT (user_id, provider).

  src/lib/userSettings.ts    getUserModelSettings, getUserApiKeys.
                             tabular_model preference now read with
                             a single parameterised SELECT against
                             user_profiles.

  src/lib/documentVersions.ts loadActiveVersion,
                             attachActiveVersionPaths,
                             attachLatestVersionNumbers. The
                             attach* helpers use ANY($1::uuid[]) for
                             batched lookups.

End-to-end verified against the Phase 2 schema in Postgres 16:
  - saveUserApiKey upsert and delete behave correctly;
    AES-256-GCM round-trip is intact (sk-test-abc-123-xyz saved
    encrypted and decrypted back identically).
  - getUserApiKeyStatus surfaces 'user' / null sources correctly.
  - listAccessibleProjectIds returns empty for a user with no
    projects.
  - checkProjectAccess returns {ok:false} for a non-existent
    project id.
  - Backend still boots, /ready returns ok, /auth/verify-credentials
    signs in an existing user with the previously-reset password.

Routes still use the Supabase JS client for their own queries and
will migrate one or two at a time in the next commits.
c198e804 Phase 2: migrate downloads/user/workflows routes onto pg Claude 2026-05-15 ↗ GitHub
commit body
Three routes converted from the Supabase JS client to parameterised
pg queries.

backend/src/routes/downloads.ts
  - 70-line file, two db.from calls. Migrated.

backend/src/routes/user.ts
  - /user/profile (GET/POST/PATCH), /user/api-keys (GET/PUT),
    /user/account (DELETE).
  - ensureProfileRow uses INSERT ... ON CONFLICT DO NOTHING.
  - loadProfile parameterised, monthly-credit reset still applied
    inline.
  - PATCH builds the UPDATE column list dynamically from the
    supplied fields.
  - DELETE /user/account now removes the public.users row directly.
    Schema-level ON DELETE CASCADE on every owned table tears down
    the data; audit_events.user_id is ON DELETE SET NULL so history
    is preserved. Storage-object cleanup and re-auth confirmation
    land in the follow-up M1 commit.

backend/src/routes/workflows.ts
  - Largest of the three (430 lines, 15 db.from calls).
  - Sharer-name lookup no longer uses Supabase Auth admin; a single
    SELECT id, display_name, email FROM users WHERE id = ANY($1)
    returns names with fallback to email.
  - workflow_shares upsert built as a multi-row VALUES with ON
    CONFLICT (workflow_id, shared_with_email) DO UPDATE.

End-to-end verification against Postgres 16:
  - /user/profile GET auto-creates row, PATCH applies display name
    and organisation.
  - /user/api-keys GET reports state, PUT openai encrypts and
    stores, GET shows source=user.
  - /workflows POST creates, GET lists (count=1), PATCH renames,
    POST /share with two emails persists two share rows, GET
    /shares lists both, POST /hidden inserts, GET /hidden returns
    the id, DELETE /workflows/<id> returns 204.

Type-check clean. Backend boots and serves all migrated routes.

projects.ts, documents.ts, chat.ts, projectChat.ts, tabular.ts, and
chatTools.ts are next.
bc2391db Phase 2: migrate projects.ts onto pg Claude 2026-05-15 ↗ GitHub
commit body
Largest single route conversion so far. 881 lines, 29 db.from calls,
all now parameterised SQL via lib/db.

Highlights:
- GET / lists own + shared via shared_with @> $::jsonb (PostgREST cs
  equivalent), then per-project count queries for documents, chats,
  tabular_reviews.
- POST / inserts and returns the new row with documents:[] attached.
- GET /:id loads project + documents + folders, attaches active
  version paths via the already-migrated documentVersions helpers.
- GET /:id/people replaces db.auth.admin.listUsers({ perPage: 1000 })
  with a single SELECT id, email, display_name FROM users WHERE
  lower(email) = ANY($1). Owner is resolved by id. Members are
  resolved by email; missing accounts surface as
  display_name: null in the response.
- PATCH /:id builds the UPDATE column list dynamically from supplied
  fields, with shared_with normalised to lowercase + deduped, capped
  at 100 entries.
- DELETE /:id relies on schema-level ON DELETE CASCADE (Phase 2
  migrations 0007-0010) to tear down documents, chats, reviews,
  folders.
- /:id/folders POST + PATCH + DELETE. Cycle detection on parent
  reassignment walks up the proposed ancestor chain; folder delete
  moves direct documents to root in a transaction before removing
  the folder.
- handleDocumentUpload (shared with documents.ts) migrated. Its db
  parameter is retained for the still-Supabase documents.ts call
  site and ignored.

End-to-end verification against Postgres 16:
  - POST creates project, returns 201 with documents:[].
  - GET / returns count=1 with doc_count/chat_count/review_count=0
    and is_owner=true.
  - GET /:id returns the full project payload with documents and
    folders arrays.
  - GET /:id/people returns owner (with display_name) and members
    (display_name null for not-yet-registered emails).
  - PATCH rename applies.
  - Folder create, subfolder under parent, cycle detection on
    PATCH (400 with 'Cannot move a folder into itself or a
    descendant'), folder delete 204.
  - Project delete 204; GET / returns count=0.

Remaining: documents.ts (16), chat.ts (12), projectChat.ts (4),
tabular.ts (19), chatTools.ts (12). Then @supabase/supabase-js comes
out of the backend.
401f5c7c Phase 2: migrate documents.ts onto pg Claude 2026-05-15 ↗ GitHub
commit body
1046-line file converted from the Supabase JS client to parameterised
pg queries. 16 db.from calls migrated.

Key changes:
- handleDocumentUpload duplicate removed. Both routes (single-docs
  POST and projects POST) now import the shared implementation from
  projects.ts. This also drops ~140 lines of duplicate
  countPdfPages/extractStructureTree code.
- Edit-resolution flow (accept / reject tracked changes):
  - The pending-edits count uses `SELECT count(*)::text` instead of
    the Supabase PostgREST head-count pattern.
  - status / resolved_at updates use parameterised UPDATE.
  - All idempotent-already-resolved and not-found-in-docx branches
    still emit the same response shapes for backward compatibility.
- DELETE /single-documents/:id keeps the storage-fanout cleanup
  (download every version row's storage_path + pdf_storage_path,
  delete the underlying object) before removing the documents row.
- POST /:id/versions:
  - Computes the next version_number with `ORDER BY version_number
    DESC NULLS LAST LIMIT 1`.
  - Conditionally rewrites documents.filename to honour a
    user-supplied display_name, preserving the original or uploaded
    extension.

End-to-end verification against Postgres 16:
  - GET /single-documents lists own docs with storage_path attached.
  - GET /single-documents/:id/versions returns current_version_id
    plus the versions list.
  - PATCH version display_name applies.
  - DELETE returns 204 and removes the row + cascades.
  - download-zip rejects an empty array with 400.
  - Existing /projects and /workflows routes still pass regression.

Remaining on Supabase JS: chat.ts (12), projectChat.ts (4),
tabular.ts (19), chatTools.ts (12).
a24b8c2e Phase 2: migrate chat.ts, projectChat.ts, chatTools.ts onto pg Claude 2026-05-15 ↗ GitHub
commit body
Closes the chat and assistant-tools surface. ~20 db.from sites across
chatTools.ts (3285 lines) converted to parameterised SQL, plus full
rewrites of routes/chat.ts and routes/projectChat.ts.

Key changes:
- routes/chat.ts:
  - GET /chat replaces the .or() PostgREST filter that
    string-concatenated project ids with
    'user_id = $1 OR project_id = ANY($2::uuid[])', still
    parameter-safe.
  - chat_messages inserts now JSON.stringify content/annotations so
    they reach Postgres as proper JSONB rather than relying on the
    Supabase client's serialisation.
- routes/projectChat.ts:
  - Full rewrite. Same flow, all queries through pg.
- lib/chatTools.ts:
  - enrichWithPriorEvents, generateDocx, loadCurrentVersionBytes,
    runEditDocument (with multi-row bulk INSERT for document_edits
    using positional placeholders), the replicate_document inline
    block (bulk INSERTs for documents and document_versions with
    multi-row VALUES + per-row UPDATE for current_version_id),
    buildDocContext, buildProjectDocContext, and buildWorkflowStore
    all converted.
  - Public functions that previously required a Supabase client
    keep their db parameter as optional and ignored, so the
    still-Supabase tabular.ts keeps compiling unchanged until its
    own migration.

End-to-end verification against Postgres 16:
- /ready ok with db: true.
- GET /chat initially empty.
- POST /chat/create returns a uuid.
- GET /chat/:id returns chat + empty messages array.
- PATCH rename applies.
- GET /chat lists 1 with the new title.
- DELETE returns 204.
- GET /chat returns empty.
- Regression: /projects, /workflows, /single-documents still pass.

Remaining on Supabase JS: tabular.ts (19). After that, drop
@supabase/supabase-js from the backend entirely.
e114baf9 Phase 2: migrate tabular.ts onto pg, drop @supabase/supabase-js Claude 2026-05-15 ↗ GitHub
commit body
Closes the final backend route on Supabase JS. 1733-line tabular.ts
with 49 PostgREST query sites converted to parameterised SQL.

Highlights:
- GET / collapses three Supabase queries (own + shared-by-project +
  shared-by-email) into the same pattern used elsewhere
  (shared_with @> $::jsonb). Per-review cell count now uses a single
  SELECT COUNT(DISTINCT document_id) GROUP BY review_id instead of
  N round trips.
- POST / and PATCH /:reviewId build the bulk tabular_cells insert
  with multi-row VALUES positional placeholders, same shape as the
  edits insert in chatTools.
- /:reviewId/people drops db.auth.admin.listUsers({ perPage: 1000 })
  in favour of a single SELECT id, lower(email::text), display_name
  FROM users WHERE lower(email::text) = ANY($1::text[]). No more
  Supabase Auth admin dependency anywhere in the backend.
- Streaming /:reviewId/generate writes all status transitions and
  cell content through parameterised UPDATE. The cell-update SSE
  events are unchanged.

Dependency removal:
- backend/src/lib/supabase.ts deleted.
- Every ReturnType<typeof createServerSupabase> type alias collapsed
  to `unknown`. Affected files: lib/access.ts, lib/userApiKeys.ts,
  lib/userSettings.ts, lib/documentVersions.ts, lib/chatTools.ts,
  routes/projects.ts. The aliases were transitional only.
- @supabase/supabase-js uninstalled from backend/package.json.

End-to-end verification against Postgres 16:
- tabular: POST create, GET list, GET detail, GET people (owner
  display_name + member display_name), PATCH rename, GET chats,
  DELETE returning 204.
- Full regression sweep: /chat, /projects, /workflows,
  /single-documents, /user/profile all still return the expected
  shapes and 2xx codes.

The backend no longer depends on the Supabase JS client at all.
Auth is Auth.js (HS256 JWT verification via jose); data access is
plain pg with parameterised SQL.

Remaining for Phase 2:
- M1 (account-deletion teardown of storage objects + re-auth
  confirmation).
- Frontend Supabase dependencies (frontend already migrated, but
  may have stray imports to audit).
ed34140e Phase 2: M1 closed - re-auth on account delete + storage teardown Claude 2026-05-15 ↗ GitHub
commit body
DELETE /user/account is now properly destructive and properly gated.

Re-authentication:
- Credentials users (password_hash present) must supply `password`
  in the JSON body. Missing returns 401 with code password_required;
  wrong returns 401 with detail "Incorrect password" (timing-safe
  via bcrypt.compare).
- Entra-only users (no password_hash) must supply `confirm_text`
  matching their lowercase email. Missing or wrong returns 401 with
  code email_confirmation_required and the expected email in the
  detail so the UI can echo it back. Defeats casual misuse of a
  still-valid session via browser extension, stolen Bearer, or
  shared device.

Storage teardown:
- Before deleting the row, enumerate every storage_path and
  pdf_storage_path across the user's document_versions (joined to
  documents). Set of unique paths is best-effort deleted after the
  cascade commits. Failures are logged but do not block the user
  delete; orphaned storage objects are easier to detect later than
  missing data is to recover.

Audit:
- audit_events row written inside the same transaction as
  DELETE FROM users so the actor identity survives the cascade
  (audit_events.user_id is ON DELETE SET NULL).
- Metadata captures storage_objects count, had_password,
  had_entra so an admin scrolling the log can see the deletion's
  scope without joining to the now-missing user row.

Frontend:
- Account > Delete Account panel grew password + email-confirm
  inputs. Validates locally that at least one is non-empty before
  enabling the destructive button. On 401 the server's detail is
  surfaced inline (with the expected email for Entra-only users).
- mikeApi.deleteAccount now takes an options object
  { password?, confirmText? }.

End-to-end verified against Postgres 16:
- Credentials: missing pw 401, wrong pw 401, correct pw 204 with
  full cascade (project/document/version all removed) and audit
  row with metadata { storage_objects: 2, had_password: true,
  had_entra: false }.
- Entra-only: missing confirm 401, wrong email 401, correct email
  204. Two audit rows recorded across both deletions.

Closes audit finding M1.

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-360.md from inside the repo you want the changes in.

⬇ Download capture-thread-360.md