umerkay adds Olostep web research endpoints behind optional dependency

Four backend POST endpoints at `/api/research` - scrape a URL to Markdown, run an AI Q&A against a page, start a multi-page crawl, or discover site URLs - all gated behind an optional `olostep` SDK install and `OLOSTEP_API_KEY` env var. Absent the key, every call returns an error string; nothing breaks on boot.

searchintegration

The implementation spans six files: backend/src/routes/research.ts (the router), backend/src/lib/olostep.ts (the SDK wrapper), and plumbing changes to package.json, package-lock.json, index.ts, and .env.example.

The olostep package lands in optionalDependencies and the lib file loads it with a one-shot import("olostep") that caches the result - null if the module isn't installed, the module if it is. Missing-key errors and missing-package errors both surface as "Error: ..." strings rather than thrown exceptions. That pattern costs almost nothing to replicate and means the feature can ship to a codebase before everyone has an Olostep account.

The four endpoints map to Olostep's four API surfaces: scrape (Markdown, images stripped), search (AI-powered answers with source URLs), crawl (async multi-page, returns only a crawl ID), and map (URL discovery with optional query). All four require auth and validate the URL scheme.

A few gaps to weigh before pulling. The /api/research mount prefix breaks the pattern of every other router in the file. The crawl endpoint returns an ID but there is no status or result endpoint yet. No rate limiting and no audit logging on outbound paid API calls. The Olostep Format enum and client.scrapes/client.answers/client.crawls/client.maps shapes are used directly, with no provider interface in between.

So what Worth a look if you want URL-to-Markdown or AI-Q&A-on-a-page in a legal-research workflow and are comfortable with Olostep vendor lock-in. The optional-dep pattern and graceful degradation are clean and portable. Skip if you need the crawl endpoint to be end-to-end functional, or if consistent route naming and audit trails matter before you ship.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

1 commit from umerkay/mike, oldest first. Source extracted verbatim from the harvested git log.

SHA Subject Author Date
007f8123 feat(research): add Olostep web research endpoints umerkay 2026-05-01 ↗ GitHub

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-140.md from inside the repo you want the changes in.

⬇ Download capture-thread-140.md