docx-edit-bench: additive task sets, and plumbing for real-world fixtures

↗ view on GitHub · Eli Ziff · 2026-07-31 · 5eba804e

v1's tasks, checks and fixtures are frozen so its published result stays
comparable. New tasks go in tasks-<set>.jsonl and the loader tags each task
with its set; selectTasks filters on it. Tasks also carry jurisdiction and
practice_area so breadth can be reported rather than asserted.

Adds a third fixture family, 'real', for documents cut from real sources.
It packs each source line as one paragraph verbatim rather than through the
markdown renderer, because real legal text has ALL-CAPS headings and '1.'
and '(a)' at line starts that markdown silently eats and renumbers. Every
real fixture must carry provenance - source URL, named licence, a
redistributable verdict, the required attribution, what was modified, and a
contamination note - and the manifest records it. A fixture whose licence
cannot be established does not ship.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
Repository eliziff/Beaver
Author Eli Ziff <eliasziff@gmail.com>
Authored
Parents e4fe1f94
Stats 5 files changed , +174 , -58
Part of SourceDoc / DOCX engine: redlines, tracked changes, editing benchmark

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-5eba804e.md from inside the repo you want the change in.

⬇ Download capture-commit-5eba804e.md