Skip to content

feat(sessions): focus check — nudge a session that goes down a rabbit hole (v0.447.0) - #830

Merged
vikasprogrammer merged 1 commit into
mainfrom
feat/drift-nudge
Sep 17, 2026
Merged

vikasprogrammer merged 1 commit into
mainfrom
feat/drift-nudge

Conversation

@vikasprogrammer

Copy link
Copy Markdown
Owner

Why

Sessions go down rabbit holes and nobody can tell. The existing verdicts only see a run's shape (circling = the same action repeated, stuck = silence); a rabbit hole is busy, varied work on the wrong thing, so it reads as forward.

What

  • Trigger (free): check after 20 allowed actions, then every 20 more, at most every 8 min. Out of band, so the gate never waits.
  • Judge (Haiku):
    • It sees the original request plus later human messages, each paired with the agent proposal it answered (so "go" / "merge both" count as approval), and the transcript tail.
    • It states current_ask, then the verdict.
    • A drifted first vote is confirmed by two more; a 2-of-3 majority at confidence ≥ 0.8 is required.
  • Effect:
    • The session strip shows a new drifting verdict.
    • An advisory note goes to the agent on its next tool call (allow + additionalContext), suggesting it task_create the side-finding and return.
    • An unattended run that is still drifting at the next check gets one sessionOwner Inbox card per streak.
    • Nothing is ever blocked or stopped.
  • Backend: the workspace Anthropic key (pinned to Haiku) → OpenAI-compatible router LLM → pooled claude -p --model claude-haiku-4-5 (runClaudePrompt, extracted from the summarizer).
  • Controls:
    • Settings → Governance mode: nudge (default) · observe · off
    • per-agent driftCheck: false
    • AOS_DRIFT=0

Calibration

The replay used production prompts against 30 long instapods transcripts (read-only), 90 checkpoints. Full table in docs/drift-focus-check.md.

  • v1 flagged 8/42, mostly false positives. The main cause: <task-notification> / skill bodies were read as human messages, hiding the real redirections. Now filtered.
  • Single votes flicker on identical input, hence the confirmation votes.
  • Final: 2 genuine rabbit holes confirmed (one from the held-out half: told "the PR is done, needs a founder decision", the agent re-implemented it from scratch; 3/3 votes), 0 false positives at the 0.8 floor.

Test

  • scripts/drift-nudge-test.cjs (96 assertions, added to test:governance): parser, input building, cadence, streak outcomes, copy framing, progress precedence, and end-to-end through TerminalManager.gate against a stub Anthropic API (split votes, observe/off, opt-out, garbled judge, no card for interactive runs).
  • npm run build, web build and the full npm run test:governance pass.

🤖 Generated with Claude Code

… hole (v0.447.0)

A rabbit hole is busy, varied work on the wrong thing, so the shape-based
verdicts (circling/stuck) read it as forward. Haiku now compares a run's
recent work against the ask (as amended by later human messages) out of
band, confirms a drifted vote 2-of-3, shows `drifting` on the session strip,
parks an advisory note on the agent's next tool call, and cards the owner of
an unattended run once if it keeps drifting. Never blocks or stops anything.

Tuned on a replay of 30 long live sessions: 2 genuine rabbit holes confirmed
(one held-out), 0 false positives at the shipped floor.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vikasprogrammer
vikasprogrammer merged commit 861e7ea into main Sep 17, 2026
1 check passed
@vikasprogrammer
vikasprogrammer deleted the feat/drift-nudge branch September 17, 2026 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant