grounding: char-stream index mode + three-way attestation calibration
build_tokengrams_index.py gains --stream chars: character codes as u16 tokens (BMP passthrough, 0xFFFE overflow, 0xFFFF doc separator) - the QUIP-canonical attestation signal on the same tokengrams base, no tokenizer dependency, no BPE boundary artifacts. Meta records the stream. calibrate_attestation_signals.py decides the H13 configuration empirically: word-trigram (interim sqlite) vs GPT-2 token n-grams (n=5,8; leading-space boundary mitigation) vs char n-grams (n=30,50) over the same labeled claims, AUC table + threshold sweeps; decision rule recorded (best AUC at matched operating points, tie-broken by boundary robustness and base-repo preference). Chars build + chained calibration launched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012AgUm5EcRKbT3duVFMbrQD
| Repository | eliziff/Beaver |
|---|---|
| Author | Eli Ziff <eliasziff@gmail.com> |
| Authored | |
| Parents | 5f65432b |
| Stats | 2 files changed , +197 , -9 |
| Part of | Legal grounding and retrieval research |
Capture this commit into my fork
Download a Markdown prompt that tells Claude how to port this
exact commit into your working tree. Run it via
claude -p < capture-commit-85b7601f.md
from inside the repo you want the change in.