Stage 21 registration: agentic nav-shape A/B on LegalBench-RAG-mini

↗ view on GitHub · Eli Ziff · 2026-07-31 · fdb7fb60

Registered before the first arm call. Compares MIKE_NAV_SHAPE=legacy
(3 tools, 2,809 schema chars, 544 measured input tokens) against
=address (4 tools, 5,006 chars, 1,027 tokens) on AGENTIC retrieval:
no injected passages, the model navigates the document itself.

Bed soundness proved before any model call, both oracles 100%:
gold-coordinate oracle 1362/1362 on the normalized corpus load, and a
new surface-fidelity oracle showing the text the library surface hands
the model is byte-identical to the corpus text gold indexes into
(69/69 documents; the surface has no .txt parser, so corpus files are
ingested as DOCX with one w:p per LF line).

Two facts recorded in advance because they bound what the result can
mean. (1) The arms are separated at the SCHEMA level only: a direct
handler probe shows arm A executes at="off:N", from="end" and
library_links, and arm B executes section= and offset=. Every tool call
therefore records off-schema argument names, and a materially nonzero
rate is reported as contamination. That is a finding about backend/src,
which this stage does not touch. (2) The DOCX bed has no fixed
pagination, so arm B's page schemes are structurally inert here and
part of its +2,197 chars cannot pay for itself on this corpus.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
Repository eliziff/Beaver
Author Eli Ziff <eliasziff@gmail.com>
Authored
Parents e339f0c6
Stats 2 files changed , +1122
Part of Evaluation harness: Beaver-CAN and LegalBench-RAG adapters

Capture this commit into my fork

Download a Markdown prompt that tells Claude how to port this exact commit into your working tree. Run it via claude -p < capture-commit-fdb7fb60.md from inside the repo you want the change in.

⬇ Download capture-commit-fdb7fb60.md