Stage 21 harness: retryable transport failures, stricter nav metric
The Codex backend 429s under six concurrent harness processes, and the per-cell catch wrote an error row that resume then treated as done - baking a scored zero into the arm mean for a cell the model never got to answer. Resume now keys on completed rows only, and since receipts are append-only the retry appends a second row: the report keeps the completed one, counts the error rows separately as an instrument failure rate, and reports how many cells were de-duplicated. Adds reached_by_read, derived from stored traces with no re-run. reached_any counts library_find's context window either side of every hit, which a couple of finds can blanket on a 10k document - it saturates at 1.000 and stops discriminating. reached_by_read asks whether a library_read, the model committing to a location, landed on gold. On the partial asis pairing it separates the arms where reached_any cannot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | 1e810f46 |
| Stats | 1 file changed , +82 , -8 |
| Part of | Evaluation harness: Beaver-CAN and LegalBench-RAG adapters |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-6a6edacd.md
from inside the repo you want the change in.