feat: read scans, photographs and plain text
Mike accepted eight file types and could only read a PDF's existing text layer, so a scanned document came back as page numbers and nothing else. Accepted types now also cover jpg, jpeg, png, tif, tiff, bmp, gif, heic, heif, webp, txt, md, csv, rtf and odt. An uploaded picture becomes a PDF and is read by OCR; a PDF that turns out to be a scan is given a text layer. Either way the document previews, quotes and cites by page like any other, and text that came from recognition is flagged so it is quoted with care. A low-resolution image warns the user at upload, while it can still be rescanned. Reading happens after the upload is answered - roughly eight seconds a page, minutes for a long bundle - and the document shows as processing until done. A PDF that already has text is checked in about a second and never waits. Notes for later: - ocrmypdf is pinned to --jobs 1. Left to itself it runs one page per CPU and is killed by the OOM killer on a machine this size; a killed run reports no error at all, which is what that silence means. - ImageMagick is never asked to write a PDF: Debian's policy forbids it. Images are normalised to PNG and wrapped with img2pdf instead. - backfillOcr.js re-reads documents uploaded before this existed. New code lives in new files (lib/ocr.ts, lib/documentRendition.ts) with the smallest possible edits to upstream files, so merges stay cheap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PoCRDAPyEq4wwBVFFbDzck
| Repository | wardethan2000-eng/mike |
|---|---|
| Author | Ethan Ward <ethanward321@hotmail.com> |
| Authored | |
| Parents | 3b8fac31 |
| Stats | 20 files changed , +1216 , -116 |
| Part of | OCR and extended document ingestion |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-35f39239.md
from inside the repo you want the change in.