docx-edit-bench: drive progressive tool disclosure from the runner
Arm B now ships 10 resident tools and defers 19 behind describe_tools. The provider adapter freezes its tool list when a call starts, so a domain opened mid-call cannot become callable inside it; the runner ends the call at the disclosure and continues in a new one, replaying the exchange. The replay cost is counted. A surface that defers nothing never restarts, so one path serves both arms. A call to a tool the surface has not served is refused by the runner rather than executed - the handlers dispatch on name alone, so without that guard a guessed name would make the deferral measure as free. Two arm-agnostic task fields: resident_route_exists (true for all 27 - every reference solution is a literal substitution the resident text-op tool can execute, which bounds what this bed can say about deferral) and alternative_route_domains. Task version 1 -> 2; checks unchanged, self-test still 27/27 solvable and 27/27 rejecting every wrong result. Registration amended (2026-07-31a) with the redefined arm, the measured 48.5% first-request schema saving under the sealed flag rather than the 60.5% of the unsealed catalogue, and the seven superseded receipts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | 9572d3ca |
| Stats | 9 files changed , +441 , -117 |
| Part of | SourceDoc / DOCX engine: redlines, tracked changes, editing benchmark |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-08389cf2.md
from inside the repo you want the change in.