nwhitehouse puts the AI's reasoning on a dial

The model's chain-of-thought is now tucked behind a click, and teams can turn the deliberation down when it isn't earning its keep.

chat-uiinfrastructure

Newer AI models like Qwen3 - the open-source model running under this fork - will "think out loud" before answering, working through a problem step by step. That's useful on a knotty question and pure noise on a routine one, where it slows the reply and runs up the token bill. nwhitehouse adds a setting that controls how hard the model deliberates: full reasoning, a lighter touch, or off entirely. There's also a nudge telling the model to keep its analysis short once it has found the clause or table the user asked about.

On screen, the reasoning no longer clutters the chat. It's folded into a collapsed panel, so anyone who wants to audit how the model reached an answer can expand it, while everyone else just sees the answer.

So what Anyone running a self-hosted legal-AI deployment who's watched routine queries quietly burn through compute will want this knob.

View this fork on GitHub →

Spotted something wrong? Or know the PR text has fresher detail than the writeup above?

Commits in this thread

1 commit from nwhitehouse/mike, oldest first. Source extracted verbatim from the harvested git log.

SHA Subject Author Date

eaef8912 [feat-019] Thinking controls + collapsed reasoning UI Nick Whitehouse 2026-05-07 ↗ GitHub

SHA	Subject	Author	Date
`eaef8912`	[feat-019] Thinking controls + collapsed reasoning UI	Nick Whitehouse	2026-05-07	↗ GitHub
commit body What's in this commit: - backend/.env.example - OLAVA_THINKING_MODE (off\|low\|standard, default standard), OLAVA_MAX_TOKENS (default 8192, was 16384), OLAVA_COMPLETION_MAX_TOKENS (2048). - backend/src/lib/llm/olava.ts - Qwen3 thinking control via vLLM `chat_template_kwargs.enable_thinking`. In low/off mode also appends a /no_think hint to the system prompt. Caller-passed `enableThinking: false` forces low mode regardless of env (used by helper calls). - backend/src/lib/chatTools.ts - adds a "REASONING BUDGET: keep internal analysis brief and targeted" line to the chat system prompt as soft guidance. - backend/src/lib/research/{queryExpander,triage}.ts - non-interactive helper calls opt out of thinking (enableThinking: false) so a 5-word search-query rewrite doesn't burn 4000 tokens reasoning first. - frontend/.../AssistantMessage.tsx - thinking card collapsed by default, readable spacing, markdown-aware reasoning rendering, bounded scroll area so long reasoning doesn't dominate the message. Defaults take effect immediately on deploy. To disable Qwen reasoning entirely (snappier, no <think> block), set OLAVA_THINKING_MODE=low in the Railway env. No code change needed. Removed from earlier draft: the OLAVA_REASONING_DISPLAY_CHAR_LIMIT cap + "[Thought process truncated by display limit.]" marker. The collapsed- by-default UI handles "hide so much of the read out" without a hard backend truncation; the marker was ugly when it appeared. Backlog entries for bug-008 (assistant thinking output noisy) and feat-019 added. Rebased onto main post-Sprint-3 so feat-017's tool_call_id / tool_calls preservation in olava.ts is preserved. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

commit body

What's in this commit:
- backend/.env.example       - OLAVA_THINKING_MODE (off|low|standard,
                               default standard), OLAVA_MAX_TOKENS
                               (default 8192, was 16384),
                               OLAVA_COMPLETION_MAX_TOKENS (2048).
- backend/src/lib/llm/olava.ts - Qwen3 thinking control via vLLM
                                 `chat_template_kwargs.enable_thinking`.
                                 In low/off mode also appends a /no_think
                                 hint to the system prompt. Caller-passed
                                 `enableThinking: false` forces low mode
                                 regardless of env (used by helper calls).
- backend/src/lib/chatTools.ts - adds a "REASONING BUDGET: keep internal
                                 analysis brief and targeted" line to the
                                 chat system prompt as soft guidance.
- backend/src/lib/research/{queryExpander,triage}.ts - non-interactive
                                 helper calls opt out of thinking
                                 (enableThinking: false) so a 5-word
                                 search-query rewrite doesn't burn 4000
                                 tokens reasoning first.
- frontend/.../AssistantMessage.tsx - thinking card collapsed by default,
                                      readable spacing, markdown-aware
                                      reasoning rendering, bounded scroll
                                      area so long reasoning doesn't
                                      dominate the message.

Defaults take effect immediately on deploy. To disable Qwen reasoning
entirely (snappier, no <think> block), set OLAVA_THINKING_MODE=low in
the Railway env. No code change needed.

Removed from earlier draft: the OLAVA_REASONING_DISPLAY_CHAR_LIMIT cap +
"[Thought process truncated by display limit.]" marker. The collapsed-
by-default UI handles "hide so much of the read out" without a hard
backend truncation; the marker was ugly when it appeared.

Backlog entries for bug-008 (assistant thinking output noisy) and
feat-019 added. Rebased onto main post-Sprint-3 so feat-017's
tool_call_id / tool_calls preservation in olava.ts is preserved.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Capture this thread into my fork

Download a single Markdown prompt that tells Claude how to port every commit above into your working tree — adapting paths and structure to match your repo. Run it via claude -p < capture-thread-125.md from inside the repo you want the changes in.

⬇ Download capture-thread-125.md