fix(lab): requirements echo is automatic under the exposure gate; add drafting-mode arm
Fix A (the 47/77 reqecho run's failure): fetch_requirements was a tool call the model could make before the authoring boundary, bumping echo_call_count to 1 and silently suppressing the coverage gate's automatic verbatim echo (the first generate_docx refusal carried only the unread list, not the task prompt). Now the tool is served ONLY when the echo is tool-served (the frozen markdown treatment arms: REQUIREMENTS on, EXPOSURE off). Under the exposure gate (both live) the echo rides the first generate_docx refusal automatically and the prompt line describes that mechanism - no roundtrip, no way to pre-empt it. Fix B: new coding_markdown_v5_reqecho_draft_v1 arm (MIKE_REQECHO_DRAFT_MODE=1). Same v5 chassis + exposure echo, but the first authoring call serves the requirements echo BEFORE any draft body is captured - nothing saved, no refine checkpoint - so the model drafts clean against the verbatim prompt. echo_call_count 0->1 there, so the normal coverage/refine gate runs un-echoed on the model's next call. Contrast vs fixed-reqecho isolates echo timing. Conformance: reqecho arms' expected resident tools drop fetch_requirements (Glob/Grep/Read/generate_docx/Edit); armExpectedSurface uses the automatic echo line (exposureEcho:true) and v5 tools; reqecho_draft_mode surface field; echo_call_count>=1 gate covers both reqecho arms; receipt delta include-lists, singleInvocationArms, and the docx-authoring list gain the draft arm. Co-Authored-By: Claude <noreply@anthropic.com>
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | 1b6a8900 |
| Stats | 4 files changed , +207 , -64 |
| Part of | Evaluation harness: Beaver-CAN and LegalBench-RAG adapters |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-7b820b94.md
from inside the repo you want the change in.