Fix scanned-PDF blindness: marker-poisoned text checks + grounded tabular
Root cause of wrong tabular extractions (and 278 workspace docs being invisible to search): extractPdfText emits a "[Page N]" marker for every page even when the page is image-only, so scanned PDFs produce marker-only text whose raw length passes every "does this document have a text layer?" threshold. Those docs were embedded as junk markers, never OCR'd, and fed to tabular reviews as blank - where the model invented plausible values (dates, reference numbers) from the filename and flagged them green. - chatTools: meaningfulPdfTextLength() measures marker-stripped text; read_document's OCR fallback now uses it. - embedDocument: marker-aware fallback to ocr_text, and marker-only text no longer counts as extractable. - ocrDocument: "has text layer" check is marker-aware, so scanned PDFs actually get OCR'd. - tabular: falls back to documents.ocr_text when the text layer is empty (generate + single-cell paths); when a document has no readable text at all the model is not called - cells get an explicit "Not Found / no readable text" result instead of hallucinations; the single-cell path no longer nulls content before regenerating. - tabular grounding: every [[quote]] citation is verified against the supplied document text server-side - values whose quotes don't appear are flagged red as unverified. Prompts now forbid inferring values from filenames, define flags as confidence signals, and instruct the model to distinguish operative dates from form-template/print dates. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
| Repository | emileriksenkeev/sydOS |
|---|---|
| Author | emileriksenkeev <emil@keev.co> |
| Authored | |
| Parents | 6c7f015d |
| Stats | 4 files changed , +155 , -24 |
| Part of | Text extraction and OCR hardening |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-39c86e4f.md
from inside the repo you want the change in.