Register docx-edit-bench v2: build the editing half nobody has built
Step 1 surveyed what practitioners actually ask for, across docs/legal-skills-ecosystem-comparison.md and Beaver's own systemWorkflows. The headline is negative and it reshapes the design: not one of Beaver's 24 shipped workflows produces an edited document. Thirteen deliver a Markdown table, eleven a spreadsheet; six review workflows carry the identical 'Do not generate a downloadable Word document', and proofread says 'Do not rewrite the whole document.' The public skill ecosystem has the same shape. But nine assistant workflows terminate in a Recommended Change / Proposed Change / Recommended Fix / Recommended Action column - a natural-language edit instruction against a cited location. Beaver already emits the stimulus half of an editing benchmark at scale and has nothing that consumes it. That pair is v2's largest family; the survey's structural-lint list supplies a second family in the mutation form both sources leave undefined. Six families in all: apply-a-review-finding, structural lint as mutation, table-bound edits (the v1 failure neither surface could do), tasks whose only route runs through a deferred tool, multi-document, and bilingual/Quebec. Recorded honestly: the survey contains no Canadian editing ask and no bilingual grounding at all, so the Quebec and EN/FR family is grounded in Bill 96 and bilingual enactment as a deliberate extension beyond the survey, not as a finding from it. And the locally cached CUAD corpus is already page-flattened with redactions - good evidence of bad PDF extraction, poor material for clause-addressed editing. v2 adopts the catalog's Class A-D taxonomy as a scoring rule: a task is a legitimate model-evaluated item only at Class C or D, and a Class A/B task must be labelled a floor task. Real fixtures ship only with an established redistribution licence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | bf64065f |
| Stats | 1 file changed , +175 |
| Part of | SourceDoc / DOCX engine: redlines, tracked changes, editing benchmark |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-1468d284.md
from inside the repo you want the change in.