⚖MikeWatch Forks of the Mike legal-AI codebase, distilled for legal-tech readers
Home · cpatpa/PIP · this thread

Phase 10: two-phase document retention with grace window and admin recovery UI

A daily retention job that flag-then-hard-deletes documents, chats, and tabular reviews past their retention horizon. A 7-day grace window between flag and deletion lets admins recover items before bytes are gone, and an `/admin/retention` tab makes that recovery visible.

complianceworkflow

Backend (490f8271, migration 0017, +510/-19 across 10 files). Migration adds pending_deletion_at (timestamptz) to documents, chats, and tabular_reviews, and retention_grace_days (default 7) to org_settings. Partial indexes cover each pending_deletion_at IS NOT NULL case.

lib/retention.ts runs two phases daily. Phase 1 (flagPastRetention): rows whose age exceeds the effective retention horizon are stamped with pending_deletion_at = now(). Retention horizon resolves workspace-first (workspaces.retention_days), falling through to org_settings.retention_days; 0 means keep forever. Phase 2 (hardDeleteAged): rows that have sat flagged past the grace window are hard-deleted and their storage objects are unlinked through the storage facade. Both phases write audit events - retention.flag and retention.hard_delete - with row counts. The job is disabled in dev with RETENTION_DISABLED=true.

User-facing list endpoints - single documents, project docs, chats, project chats, tabular review list - are updated to exclude pending_deletion_at IS NOT NULL rows. Flagged items disappear from the UI immediately, before hard delete.

Admin UI (c91262ac, +420/-24 across 10 files). New admin endpoints: GET /admin/retention/pending returns every currently-flagged row across documents/chats/reviews with created-at, flagged-at, and days-since-flag. POST /admin/retention/restore clears pending_deletion_at on a specific row. POST /admin/retention/run kicks the job immediately. The frontend /admin/retention tab renders the pending list and surfaces restore controls per row.

The same commit also drops projects.shared_with and tabular_reviews.shared_with via migration 0018. These JSONB columns were kept after Phase 3 as a one-cycle rollback safety net; no application code reads or writes them any more. Their GIN indexes drop implicitly with DROP COLUMN. API responses still surface shared_with: string[] derived from the members tables.

So what A solid standalone port candidate. The flag-then-grace-then-hard-delete shape with workspace-overrides-org fallback is generic enough to lift directly, and the audit-event-per-batch pattern matches the rest of this fork's approach. The admin recovery UI is the part that makes the UX defensible for a legal-sector tool - accidental deletion with no recovery path is a bad failure mode. One dependency to check: Phase 2 calls the storage facade introduced in Phase 5 to unlink storage objects. A port without that facade would need to wire into whatever storage abstraction you use.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

2 commits from cpatpa/PIP, oldest first. Source extracted verbatim from the harvested git log.

SHA Subject Author Date
490f8271 Phase 10: document retention enforcement Claude 2026-05-15 ↗ GitHub
commit body 
Two-phase retention so accidental data loss is recoverable:

  1. Flag    rows past the effective retention horizon
             (workspace.retention_days, falling through to
             org_settings.retention_days; 0 = keep forever) are
             marked with pending_deletion_at.
  2. Hard delete rows still flagged after
             org_settings.retention_grace_days (default 7) are
             removed; document storage objects unlinked then.

Schema (0017_retention.sql) adds pending_deletion_at to documents,
chats, tabular_reviews and a retention_grace_days column on
org_settings. Both phases write audit events
(retention.flag / retention.hard_delete) with row counts.

backend/src/lib/retention.ts owns the SQL and a setInterval(24h)
boot scheduler, disable with RETENTION_DISABLED=true. The user
list/get surfaces (single-docs, project docs, chats, project chats,
tabular review list) hide pending-deletion rows so the UI reflects
the flag immediately.

Admin endpoints for visibility and recovery:
  GET  /admin/retention/pending
  POST /admin/retention/restore  { kind, id }
  POST /admin/retention/run
c91262ac Retention admin UI + drop legacy shared_with JSONB Claude 2026-05-15 ↗ GitHub
commit body 
New /admin/retention tab visualises every row currently flagged for
deletion (documents, chats, tabular reviews) with created-at,
flagged-at, and days-since-flag. Admins can restore individual rows
inside the grace window via POST /admin/retention/restore or kick
the job immediately with POST /admin/retention/run.

Migration 0018 drops projects.shared_with and
tabular_reviews.shared_with JSONB columns. No app code reads or
writes them any more; project_members and review_members have been
the source of truth since Phase 3. The companion GIN indexes drop
implicitly. API responses still surface a shared_with array
derived from the members tables, so the frontend wire shape is
unchanged.

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-371.md from inside the repo you want the changes in.

⬇ Download capture-thread-371.md