mglynnhenley gives Mike a built-in lie detector

Every answer Mike writes now gets scored word by word for how likely it is to be made up.

chat-uianalytics

mglynnhenley has wired Mike to an outside service that probes the AI's output for hallucination - the polished-sounding inventions that legal AI is notorious for. After Mike finishes an answer, the service grades each word on how trustworthy it looks and sends those scores back.

The result shows up as a colour strip running through the response, so a reader can see at a glance which phrases the model was confident about and which it was effectively guessing at. It's opt-in: leave the service switched off and Mike behaves exactly as before, with no scoring and no change to the experience. There's also a built-in stand-in so the feature can be tried without paying for a live scoring service.

So what Anyone relying on AI-drafted legal text should care: a per-word confidence map turns 'trust but verify' from a slogan into something you can actually see on screen.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

5 commits from mglynnhenley/mikehasprobes, oldest first. Source extracted verbatim from the harvested git log.

SHA	Subject	Author	Date
`e7549126`	Add hallucination-probe scoring across chat + tabular review	Matilda	2026-05-06	↗ GitHub
commit body Wire Mike to a Modal-hosted, OpenAI-compatible probe service. After each Claude/Gemini response, send the completion as a prefilled assistant turn to the probe and stream per-token scores onto the existing SSE channel. Persist scores on `chat_messages.probe_scores` and `tabular_cells.probe_scores`. UI fades a heat-strip + risk badge under cells/messages as scores arrive. Also: local mock probe at /mock-probe for development without the Modal service, and a "Think" toggle on the chat input so users can opt into adaptive thinking per turn (off by default - Sonnet 4.6 was rejecting the unconditional flag). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`ef3d78ce`	Keep 000 base schema clean; probe columns live in 001/002	Matilda	2026-05-06	↗ GitHub
commit body Per review: the one-shot base schema should stay vanilla. Probe score columns are additive and belong only in 001_probe_scores.sql (tabular_cells) and 002_chat_probe_scores.sql (chat_messages), which already exist as incremental migrations. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`4dcbc056`	Consolidate probe migrations into single 001	Matilda	2026-05-06	↗ GitHub
Merge 002's chat_messages.probe_scores into 001 alongside the tabular_cells columns. One migration covers the entire probe schema extension; 002 deleted. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`d0df46d4`	Merge pull request #1 from mglynnhenley/probe-scorer	Matilda	2026-05-06	↗ GitHub
Hallucination-probe scoring for chat + tabular review
`9206a2a0`	Add inline probe highlighting and threshold slider	Matilda	2026-05-07	↗ GitHub
commit body - Render probe scores as inline background tints over the assistant text instead of a separate heat strip - Add per-user highlight threshold slider (localStorage-persisted) so low-confidence tokens can be hidden from the chat view - Switch probe scorer to non-streaming chat completions; reads scores from top-level response.scores, drops the -analyze model variant - Emit content_done SSE event so the typing indicator drops as soon as Claude finishes, while probe scores keep streaming in Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-96.md from inside the repo you want the changes in.

⬇ Download capture-thread-96.md