Register docx-edit-bench v2: build the editing half nobody has built

↗ view on GitHub · Eli Ziff · 2026-07-31 · 1468d284

Step 1 surveyed what practitioners actually ask for, across
docs/legal-skills-ecosystem-comparison.md and Beaver's own systemWorkflows.
The headline is negative and it reshapes the design: not one of Beaver's 24
shipped workflows produces an edited document. Thirteen deliver a Markdown
table, eleven a spreadsheet; six review workflows carry the identical 'Do not
generate a downloadable Word document', and proofread says 'Do not rewrite
the whole document.' The public skill ecosystem has the same shape.

But nine assistant workflows terminate in a Recommended Change / Proposed
Change / Recommended Fix / Recommended Action column - a natural-language
edit instruction against a cited location. Beaver already emits the stimulus
half of an editing benchmark at scale and has nothing that consumes it. That
pair is v2's largest family; the survey's structural-lint list supplies a
second family in the mutation form both sources leave undefined.

Six families in all: apply-a-review-finding, structural lint as mutation,
table-bound edits (the v1 failure neither surface could do), tasks whose only
route runs through a deferred tool, multi-document, and bilingual/Quebec.

Recorded honestly: the survey contains no Canadian editing ask and no
bilingual grounding at all, so the Quebec and EN/FR family is grounded in
Bill 96 and bilingual enactment as a deliberate extension beyond the survey,
not as a finding from it. And the locally cached CUAD corpus is already
page-flattened with redactions - good evidence of bad PDF extraction, poor
material for clause-addressed editing.

v2 adopts the catalog's Class A-D taxonomy as a scoring rule: a task is a
legitimate model-evaluated item only at Class C or D, and a Class A/B task
must be labelled a floor task. Real fixtures ship only with an established
redistribution licence.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
Repository eliziff/Beaver
Author Eli Ziff <eliasziff@gmail.com>
Authored
Parents bf64065f
Stats 1 file changed , +175
Part of SourceDoc / DOCX engine: redlines, tracked changes, editing benchmark

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-1468d284.md from inside the repo you want the change in.

⬇ Download capture-commit-1468d284.md