feat: read scans, photographs and plain text

↗ view on GitHub · Ethan Ward · 2026-08-18 · 35f39239

Mike accepted eight file types and could only read a PDF's existing text
layer, so a scanned document came back as page numbers and nothing else.

Accepted types now also cover jpg, jpeg, png, tif, tiff, bmp, gif, heic, heif,
webp, txt, md, csv, rtf and odt. An uploaded picture becomes a PDF and is read
by OCR; a PDF that turns out to be a scan is given a text layer. Either way the
document previews, quotes and cites by page like any other, and text that came
from recognition is flagged so it is quoted with care. A low-resolution image
warns the user at upload, while it can still be rescanned.

Reading happens after the upload is answered - roughly eight seconds a page,
minutes for a long bundle - and the document shows as processing until done.
A PDF that already has text is checked in about a second and never waits.

Notes for later:
- ocrmypdf is pinned to --jobs 1. Left to itself it runs one page per CPU and
  is killed by the OOM killer on a machine this size; a killed run reports no
  error at all, which is what that silence means.
- ImageMagick is never asked to write a PDF: Debian's policy forbids it. Images
  are normalised to PNG and wrapped with img2pdf instead.
- backfillOcr.js re-reads documents uploaded before this existed.

New code lives in new files (lib/ocr.ts, lib/documentRendition.ts) with the
smallest possible edits to upstream files, so merges stay cheap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PoCRDAPyEq4wwBVFFbDzck
Repository wardethan2000-eng/mike
Author Ethan Ward <ethanward321@hotmail.com>
Authored
Parents 3b8fac31
Stats 20 files changed , +1216 , -116
Part of OCR and extended document ingestion

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-35f39239.md from inside the repo you want the change in.

⬇ Download capture-commit-35f39239.md