fix(lab): requirements echo is automatic under the exposure gate; add drafting-mode arm

↗ view on GitHub · Eli Ziff · 2026-08-07 · 7b820b94

Fix A (the 47/77 reqecho run's failure): fetch_requirements was a tool call the
model could make before the authoring boundary, bumping echo_call_count to 1
and silently suppressing the coverage gate's automatic verbatim echo (the
first generate_docx refusal carried only the unread list, not the task prompt).
Now the tool is served ONLY when the echo is tool-served (the frozen markdown
treatment arms: REQUIREMENTS on, EXPOSURE off). Under the exposure gate (both
live) the echo rides the first generate_docx refusal automatically and the
prompt line describes that mechanism - no roundtrip, no way to pre-empt it.

Fix B: new coding_markdown_v5_reqecho_draft_v1 arm (MIKE_REQECHO_DRAFT_MODE=1).
Same v5 chassis + exposure echo, but the first authoring call serves the
requirements echo BEFORE any draft body is captured - nothing saved, no
refine checkpoint - so the model drafts clean against the verbatim prompt.
echo_call_count 0->1 there, so the normal coverage/refine gate runs un-echoed
on the model's next call. Contrast vs fixed-reqecho isolates echo timing.

Conformance: reqecho arms' expected resident tools drop fetch_requirements
(Glob/Grep/Read/generate_docx/Edit); armExpectedSurface uses the automatic
echo line (exposureEcho:true) and v5 tools; reqecho_draft_mode surface field;
echo_call_count>=1 gate covers both reqecho arms; receipt delta include-lists,
singleInvocationArms, and the docx-authoring list gain the draft arm.

Co-Authored-By: Claude <noreply@anthropic.com>
Repository eliziff/Beaver
Author Eli Ziff <eliasziff@gmail.com>
Authored
Parents 1b6a8900
Stats 4 files changed , +207 , -64
Part of Evaluation harness: Beaver-CAN and LegalBench-RAG adapters

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-7b820b94.md from inside the repo you want the change in.

⬇ Download capture-commit-7b820b94.md