Davemaina1 crams a legal-search engine onto a free server

A multi-step rebuild to make AI-powered case-law search run inside a 512MB hosting budget, at zero cost.

searchinfrastructure

Davemaina1 pulled the AI search machinery out of the main app and into a small dedicated service, then spent several rounds shrinking it to fit a hard ceiling. Free hosting tiers - here, Render's - cap memory at 512MB, and the usual machine-learning toolkit blows straight past that at roughly 1.5GB. So Davemaina1 swapped the heavyweight components for leaner equivalents, compressed the underlying case-law data, and stopped loading the whole library into memory at once, reading it from disk on demand instead.

The reference corpus now lives in cloud storage (Supabase, a hosted database-and-files service) and is pulled in at startup. Final memory use lands around 320MB - comfortably under budget.

So what Anyone trying to ship a legal-AI tool without a hosting bill should study this: it's a working blueprint for running real semantic search on free infrastructure.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

5 commits from Davemaina1/iroh_, oldest first. Source extracted verbatim from the harvested git log.

SHA	Subject	Author	Date
`c0684a41`	RAG sidecar: Python service for embedding+BM25+rerank; Node becomes HTTP client	Davemaina1	2026-05-13	↗ GitHub
`bed9cc2f`	fix(rag): pin Python 3.11, relax torch/numpy version constraints	Davemaina1	2026-05-14	↗ GitHub
Render defaults to Python 3.14 which doesn't have torch 2.5.1 wheels. Pin to 3.11 via .python-version and allow any torch 2.x. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
`7a0da671`	feat(rag): replace torch with onnxruntime to fit 512MB RAM	Davemaina1	2026-05-14	↗ GitHub
commit body Eliminates torch (1.5GB) and sentence-transformers entirely. Uses onnxruntime + tokenizers + huggingface-hub to run the same all-MiniLM-L6-v2 model via its ONNX export (~90MB). Drops CrossEncoder reranker - RRF fusion alone is sufficient for the testing phase. Estimated memory: ~340MB. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
`0b5251e8`	feat(rag): eliminate ChromaDB dependency, load corpus from Supabase Storage	Davemaina1	2026-05-14	↗ GitHub
commit body Removes both torch/sentence-transformers AND ChromaDB from production. Corpus (86K chunks, embeddings, metadata) is pre-exported to .npz files hosted on Supabase Storage (public bucket). On startup, the service downloads ~50MB, dequantizes int8 embeddings, and builds a BM25 index. Semantic search is brute-force numpy dot product (~180ms/query for 86K vectors). Total runtime memory: ~350MB (fits in Render's 512MB free tier). Zero additional services required. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
`3ec32238`	feat(rag): mmap embeddings + on-demand metadata - peak 320MB RSS	Davemaina1	2026-05-14	↗ GitHub
commit body Downloads corpus files to /tmp, memory-maps the embeddings (zero RAM cost), and reads metadata only for the top-K results on each query. Drops BM25 (150MB overhead) - semantic-only search is good enough for testing phase. Removes chromadb and rank-bm25 dependencies entirely. Measured peak RSS: 320MB (well within Render's 512MB free tier). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-380.md from inside the repo you want the changes in.

⬇ Download capture-thread-380.md