emileriksenkeev gives project search a semantic backbone

This fork can search project documents by meaning, while fixing the quiet scale problems that make early search demos unreliable.

searchknowledge-management

The search uses Voyage, an AI service that turns text into meaning-based representations, and pgvector, a database extension built to compare them. A search for a concept can therefore find relevant material even when the wording differs.

Semantic document search: users can retrieve project material by the ideas it contains, not only exact keywords.
Safer embedding runs: temporary service failures are retried, and repeat runs leave documents already processed alone.
Complete large-workspace coverage: the backfill now works through documents in batches, avoiding a previous 1,000-document ceiling that could silently leave newer files out of search.

So what Legal teams building searchable matter knowledge should care because incomplete search results are hard to spot and easy to trust.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

3 commits from emileriksenkeev/sydOS, oldest first. Source extracted verbatim from the harvested git log.

SHA	Subject	Author	Date
`918eeb38`	Syd OS: Plan B1 - semantic search over project documents (pgvector + Voyage)	emileriksenkeev	2026-07-11	↗ GitHub
commit body - pgvector document_chunks table + match_document_chunks() search fn (migration applied) - voyage.ts (voyage-law-2 embeddings, configurable), chunk.ts, embedDocument.ts pipeline - search_documents tool wired into project assistant + system prompt (find by content, not filename) - auto-embed hook on upload (fire-and-forget, no-op without VOYAGE_API_KEY) - backfillEmbeddings.ts script for existing docs Foundation for the wiki-memory layer (B2). No-ops safely until VOYAGE_API_KEY is set. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`9fe117db`	Syd OS: harden embedder - retry 429/5xx with backoff; backfill skips already-embedded docs	emileriksenkeev	2026-07-11	↗ GitHub
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`0b44a4f6`	Backfill scripts: paginate past PostgREST's 1000-row cap	emileriksenkeev	2026-07-17	↗ GitHub
commit body The workspace outgrew 1000 documents, so the full-corpus embeddings backfill silently fetched only the oldest 1000 and dropped the newest ~200 (the People-HR batch among them) while reporting success. The document_chunks select hit the same cap, so the already-embedded set was also incomplete. Page both queries with .range() in both scripts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-910.md from inside the repo you want the changes in.

⬇ Download capture-thread-910.md