docx-edit-bench: require an interaction class on v2 tasks
From docs/deterministic-word-actions-catalog.md, imported by the legal-skills survey: Class A is a direct UI action, B a bounded deterministic tool the model invokes but never authors, C preview-then-confirm because intent is ambiguous, D judgment that cannot be inferred from text or formatting. The survey's rule is that only Class D deserves to stay a prompt. The benchmark's corollary, now enforced by the loader: a task is a legitimate model-evaluated item only at Class C or D, and a Class A or B task must be labelled a floor task, because a deterministic tool should perform it and the model's only job is routing to that tool. v2 tasks must also declare a jurisdiction and practice area so breadth is reported rather than asserted. v1 is exempt and frozen. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | 6e7078c5 |
| Stats | 2 files changed , +32 |
| Part of | SourceDoc / DOCX engine: redlines, tracked changes, editing benchmark |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-b9c27219.md
from inside the repo you want the change in.