Stage 21 registration: agentic nav-shape A/B on LegalBench-RAG-mini
Registered before the first arm call. Compares MIKE_NAV_SHAPE=legacy (3 tools, 2,809 schema chars, 544 measured input tokens) against =address (4 tools, 5,006 chars, 1,027 tokens) on AGENTIC retrieval: no injected passages, the model navigates the document itself. Bed soundness proved before any model call, both oracles 100%: gold-coordinate oracle 1362/1362 on the normalized corpus load, and a new surface-fidelity oracle showing the text the library surface hands the model is byte-identical to the corpus text gold indexes into (69/69 documents; the surface has no .txt parser, so corpus files are ingested as DOCX with one w:p per LF line). Two facts recorded in advance because they bound what the result can mean. (1) The arms are separated at the SCHEMA level only: a direct handler probe shows arm A executes at="off:N", from="end" and library_links, and arm B executes section= and offset=. Every tool call therefore records off-schema argument names, and a materially nonzero rate is reported as contamination. That is a finding about backend/src, which this stage does not touch. (2) The DOCX bed has no fixed pagination, so arm B's page schemes are structurally inert here and part of its +2,197 chars cannot pay for itself on this corpus. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H9ToHYJVDxfeJcJwdzrP2H
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | e339f0c6 |
| Stats | 2 files changed , +1122 |
| Part of | Evaluation harness: Beaver-CAN and LegalBench-RAG adapters |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-fdb7fb60.md
from inside the repo you want the change in.