eliziff turns Claude's command-line tool into a faster AI engine

This fork makes it possible to run legal-AI conversations through Claude's command-line interface, with session reuse cutting measured response loops by more than half.

infrastructureintegration

eliziff is building an alternative route to large language models that uses a Claude subscription through its command-line interface rather than the usual API connection. The attraction is practical: retain conversational context between turns, avoid repeatedly sending the full history, and make agent-style work feel materially quicker.

  • Persistent conversations reuse a live Claude session, with a reported reduction in agent-loop time from 14.5 to 6.0 seconds.
  • Resilience measures handle rate-limit queues, stalled responses, malformed streaming output and failed sessions, reverting to a complete fresh replay when needed.
  • An Ollama option supports locally run models, while a direct connection to Meta's model service broadens provider choice.

Legal-product teams evaluating AI cost, speed and vendor-connection options should care: this is a serious attempt to make subscription-backed AI workflows dependable enough to use.

So what Worth a GitHub look for teams weighing faster, more flexible AI infrastructure behind their legal workflows.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

15 commits from eliziff/Beaver, oldest first. Source extracted verbatim from the harvested git log.

SHA Subject Author Date
b5506122 Add claude-p and ollama transports to the Beaver model layer Eli Ziff 2026-07-28 ↗ GitHub
commit body
claude-p:<model> reaches Anthropic models over headless Claude Code
(subscription flat rate; JSON-in-text tool protocol, spawns claude.exe
directly to dodge the .CMD shim); ollama:<model> reaches local models
(desktop Qwen) over ollama's native /api/chat tool calling. Both are
transport-only so the LAB experiment can hold the model constant across
the LAB and Beaver harnesses (benchmarks/lab/PROTOCOL.md). Keyless
providers skip the tabular API-key gate. tsc clean; llm suite 13/13;
live smoke: tool round-trip + usage through streamChatWithTools.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bhP89cYSdEEjHNPnxacns
525ed809 Stream claude-p transport with inactivity watchdog and effort Eli Ziff 2026-07-28 ↗ GitHub
commit body
stream-json + --include-partial-messages keeps bytes flowing during
long single-shot generations, so the watchdog kills on 240s of silence
rather than total wall time (whole-document turns legitimately run
10-30 min; hard cap 3600s). params.reasoningEffort now passes through
as --effort. tsc clean; live smoke green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bhP89cYSdEEjHNPnxacns
405341bb Record thinking/effort/output-cap facts for the claude-p transport Eli Ziff 2026-07-28 ↗ GitHub
commit body
Probes falsified the earlier "no thinking control" note: adaptive
thinking is active over claude -p, --effort passes through (runs pin
high, Harvey's own config label), and the CLI caps output at 32k
tokens/call. Streaming + inactivity watchdog replaces total-time caps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bhP89cYSdEEjHNPnxacns
5d6ad2c0 claude-p: TTFB grace for rate-limit queueing, retry backoff Eli Ziff 2026-07-28 ↗ GitHub
commit body
A queued -p call streams nothing until the server admits it; the
240s inactivity watchdog was killing queued-not-wedged calls under
concurrent lane use. 600s grace to first byte, 240s after; retries
back off 15s/30s instead of immediately re-entering the queue.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bhP89cYSdEEjHNPnxacns
3b712e31 claude-p watchdog: liveness counts model stream events only Eli Ziff 2026-07-28 ↗ GitHub
commit body
Same mechanism as the LAB adapter fix: init lines print at spawn and
were flipping the activity flag, so the 240s clock killed first
calls still in prompt/cache processing (>240s TTFT on Beaver-sized
contexts, probe-confirmed no rate-limit queueing). 900s grace to the
first model event, 240s inactivity after.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018bhP89cYSdEEjHNPnxacns
e640b740 claude-p: opt-in persistent sessions (MIKE_CLAUDE_P_PERSIST=1) - 2.4x faster agent loops Eli Ziff 2026-07-30 ↗ GitHub
commit body
One long-lived claude -p --input-format stream-json process per
conversation: first turn carries the full payload, later iterations
send only tool_results (the session holds context; probed live on CLI
2.1.220). Any failure falls back to a fresh session with full replay,
so recovery is stateless and the incumbent per-call path is untouched
by default. Live A/B (sonnet, 2-tool-call loop): 14.5s -> 6.0s with
prompt-cache reads engaged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
b7584a05 claude-p transport: tolerant parsing, jsonrepair fallback, corrective retry Eli Ziff 2026-07-30 ↗ GitHub
commit body
Stage 12 invalidated both Claude-5 lanes with content-shaped JSON
failures: unescaped quotes/newlines inside TOOL_CALLS string values
when tool inputs embed legal text. Three layers: protocol now demands
strict escaped JSON; parseReply tolerates fences/preamble/same-line
markers and falls back to jsonrepair (safe - the deterministic
verbatim gate re-verifies every quoted span, so a mutating repair
fails typed rather than passing falsely); retries feed the parse
error back as a corrective turn instead of resending an identical
payload. Usage now accumulates on failed parses too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
52b8bf8a @ feat(lab): md-swap 2x2 arms, deepseek usage + 32K budget, claude-p auth isolation Eli Ziff 2026-08-04 ↗ GitHub
commit body
- lab-beaver-arm.ts: add mike_markdown_swap_v1 / mike_markdown_e2e_v1 /
  mike_markdown_read_upstream_draft_v1 arms + surface conformance checks
  (2x2 read/write matrix: control, markdown-swap, markdown-e2e, reverse-swap)
- upstreamMikeBenchmarkSurface.ts: MARKDOWN_SWAP_GENERATE_DOCX_TOOL (markdown
  draft) + UPSTREAM_MIKE_MARKDOWN_SWAP_LAB_TOOLS
- localAssistantTools.ts: markdown_swap/e2e tool shapes + MIKE_READ_DOCX_MARKDOWN
  (serve Pandoc drafting-source markdown on docx reads for the e2e arm)
- deepseek.ts: aggregate streamed usage (include_usage) into NormalizedLlmUsage;
  MAX_TOKENS 16K->32K (reasoning stream ate the whole budget on LAB smoke run)
- claudeP.ts: delete (not blank) ANTHROPIC_* auth env for claude -p child
  isolation; surface stdout result-envelope hint when stderr is empty
- chat.ts: wire markdown_swap/e2e shape flags into run surface
- AGENTS.md: forbid OpenAI per-token spend (flat-rate surfaces only)

Co-Authored-By: Claude <noreply@anthropic.com>
@
248e864e feat(lab): metrics plumbing - claude-p context rounds, provider_round_count, deliverable_chars Eli Ziff 2026-08-05 ↗ GitHub
commit body
Audit fix wave A4:
- streamClaudeP now emits a content-free LlmContextRoundReceipt per loop
  iteration (attempts, continuation mode, payload/tool byte+sha receipts,
  per-round usage deltas, tool result bytes). The manifest previously carried
  rounds: [] for claude-p, so context_round_count was always 0 on that lane.
- metrics.json gains provider_round_count (contextRounds count, falling back
  to invocation count on the deepseek lane where each round is one
  invocation) - separates context volume from turn count, which the
  cache-adjusted input headline conflated (the HSR "index worse on input"
  reading was entirely a round-count artifact).
- deliverable receipts gain text_chars (docx body text via
  extractDocxBodyStructure, raw text for md/txt) and metrics.json gains
  deliverable_chars - deliverable length is the strongest single score
  predictor in the deepseek family and was never recorded.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011pfUVhNFTRvhYGXBwoKNj6
e14b83b8 feat(lab): unblock claude-p lane + typed terminal failures Eli Ziff 2026-08-05 ↗ GitHub
commit body
Audit fix wave A5:
- The default-tier and session-prompt-cache receipt gates now exempt the
  claude-p lane alongside deepseek (flatRateLane): the CLI envelope reports
  no service tier and no prompt-cache key, and the tier gate killed every
  claude-p arm run of the last wave ("requires the provider-reported default
  tier; received no tier receipt").
- Removed MIKE_DEEPSEEK_MAX_TOKENS=65536 from the index arm env - an
  uncontrolled second delta vs every other arm (triple audit D2).
- claudeP.ts classifies deterministic CLI failures as typed
  ClaudePFatalError: context_overflow ("Prompt is too long"/blocking_limit)
  and quota_exhausted (weekly limit). Both rethrow immediately instead of
  burning the 3-attempt retry loop.
- The arm script's failure handler records the typed status + error into
  run-state.json. A context overflow on a 200K-class model is the MEASURED
  RESULT of a whole-read arm on a no-fit task, not a crash; the results tree
  previously kept a permanent provider_call_pending stub.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011pfUVhNFTRvhYGXBwoKNj6
f1b5bc28 perf(lab): persistent claude-p session is the lane default for arm runs Eli Ziff 2026-08-05 ↗ GitHub
commit body
The stateless claude-p mode spawns a fresh CLI process per tool round and
re-uploads the ENTIRE conversation each time - a 20-round run pays process
spawn + full-context re-upload + prefix re-ingest 20 times. ClaudePSession
(already built, behind MIKE_CLAUDE_P_PERSIST) keeps one live process and
sends only the new tool results per round, with stateless full-replay
recovery on any session death. Our arms' fixed tool schemas never trigger
the session-killing schema-change condition, so the arm launcher now sets
MIKE_CLAUDE_P_PERSIST=1 whenever the model is claude-p:*. Transport-level,
not an experimental variable; contextRounds receipts record continuation
"provider" vs "none" per round so the mode stays visible in metrics.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011pfUVhNFTRvhYGXBwoKNj6
8b8c7173 feat(llm): direct Meta Model API provider for Muse Spark Eli Ziff 2026-08-05 ↗ GitHub
commit body
Meta Model API speaks the OpenAI Responses dialect verbatim, so the
adapter is a thin wrapper over the shared streamResponsesApi -- same
shape as openrouter.ts. Probed against muse-spark-1.2 before writing
it: store:false, reasoning.effort, function tools and the standard
response.* SSE event names are all accepted, and the tool loop,
streaming deltas and normalized usage (incl. reasoning + cache-read
tokens) come back correct through Beaver's own dispatch.

Transport rides on the id shape, as it already does for codex: /
claude-p: / ollama: -- bare `muse-spark-*` goes direct, the `meta/`
slug stays on OpenRouter. The existing OpenRouter route is unchanged;
its picker entry is relabelled so the two are distinguishable.

Key resolution takes META_API_KEY first, falling back to MODEL_API_KEY
(the name Meta's docs export).

The -contributor tier is wired but off the default picker: it is ~12x
cheaper because Meta trains on the prompts and completions sent to it,
which is the wrong default for client documents. It is reachable only
once an account enrols; until then it returns an attributed 404 rather
than falling back silently.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GRKSheQsXFWp1ophSt6X11
d3b1076c fix(claude-p): effort-scaled watchdog, stderr liveness, sonnet 64k output cap, retry-cause telemetry Eli Ziff 2026-08-06 ↗ GitHub
commit body
Both HSR DNFs (2026-08-06) were transport defects: a parse failure discarded
a completed 16-min generation, and the effort-blind 240s inactivity limit
killed healthy effort-max generations at thinking-flush seams. Inactivity
now scales to the 900s grace at effort max; stderr resets liveness on both
paths; sonnet lanes raise the CLI's 32k per-message output cap to 64k
(truncate-rewrite cycles discarded 83%/72% of employment/insurance drafting
output); the terminal retry error carries per-attempt failure kinds; and
unparseable-but-completed replies are preserved to tmpdir for post-mortem.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011pfUVhNFTRvhYGXBwoKNj6
0ae82913 fix(claude-p): salvage dominant-string TOOL_CALLS defects; stop_reason gates repairs Eli Ziff 2026-08-06 ↗ GitHub
commit body
One raw interior double quote in a 60KB generate_docx markdown string
discarded a completed DPA draft (2026-08-06) and bought a corrective
regeneration ~30% shorter. parseReply now recovers that class safely:

- jsonrepair keeps first authority (it preserves all fields for small
  structural slips), but for large payloads its result must preserve
  the raw key inventory (one-sided) - a repair that swallowed a field
  or call is worse than no repair.
- Structural salvage when jsonrepair fails: anchor the dominant
  string's closing quote at the envelope tail, re-derive the logical
  string from the model's own bytes (valid escapes decode, defects
  pass through verbatim, nothing invented), re-escape strictly, accept
  only candidates that re-parse strictly, satisfy the calls shape, and
  preserve the key inventory. Latest surviving opening wins; 64-candidate cap.
- Truncation is decided by the transport, not tail bytes: stop_reason
  captured from assistant/message_delta events on both spawn paths;
  stop_reason=max_tokens blocks BOTH repair paths so a partial draft
  can never pass as complete.
- Raw replies preserved to tmpdir on both salvage and parse failure.

Adversarially reviewed (opus subagent): the initial version had four
blocking findings (sibling-field/call absorption, jsonrepair
preemption, tail-byte truncation guard fakeable and over-refusing) -
all closed. Reviewer probe matrix 0/23 regressed vs pre-change
behavior; the real preserved DPA reply recovers its full 59,155-char
draft byte-exact. Residual, documented: odd dangling-backslash content
takes jsonrepair's normalization exactly as on main, and truncation
gating needs a visible stop_reason.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011pfUVhNFTRvhYGXBwoKNj6
08385015 feat(lab): claude-p auto-compact detection + coding-arm authoring parity Eli Ziff 2026-08-06 ↗ GitHub
commit body
The coding_markdown_v1 acq pilot exposed two run-killing gaps. First,
claude -p auto-compacted twice (preTokens 207,948 and 179,180) after the
model batch-whole-read all 31 docs, silently replacing served documents
with lossy summaries - invisible to the runner, and it voids the typed
context_overflow mechanism whole-read control rows depend on. Second,
the CC Write prior captured generate_docx ({filename, content}, ten
straight rejections) and the conjunctive terse refusal defeated the
model's one-key-at-a-time debugging.

- claudeP: parse system/compact_boundary stream events (real fixtures
  from the pilot session; snake_case variants covered); receipts ride
  the existing StreamChatResult.compactions rail with usage zeroed (the
  spend is already in turn envelopes) and triggerReason "provider_auto";
  system events now count as watchdog liveness (the pilot's second
  compaction ran 377s, past the 240s inactivity limit).
- lab-beaver-arm: top-level context_compaction_count/context_compactions
  metrics + loud warn on nonzero.
- localAssistantTools (CODING_PARITY only): generate_docx accepts
  'content' as the body and derives 'title' from filename/first heading,
  teaching the schema via a receipt note; the refusal now states the
  full expected contract plus received keys. Frozen arms byte-identical
  (probe: v1 keeps the terse refusal).
- upstreamMikeBenchmarkSurface: CODING_GENERATE_DOCX_TOOL names its keys
  in prose against the Write prior ("this is not a file Write"); same
  name, same schema.

Probes: compaction fixtures 7/7; plane probe parity 12/12, md 5/5,
plain 3/3; served-surface coding2/coding/v2 preflights green; suites
57/57; tsc clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011pfUVhNFTRvhYGXBwoKNj6

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-828.md from inside the repo you want the changes in.

⬇ Download capture-thread-828.md