feat: the plugin-hooks session eval — the plugin's hooks in real sessions

[mauricios]

What changed and why

The plugin's hooks were rewritten five times between v1.0.2 and v1.0.6. The static hook tests run them as scripts, but nothing checked them in a real Claude Code session: that Claude Code runs them from the plugin, and that what they print reaches the session. This adds that check as a session-eval suite.

The plugin-hooks suite (evals/dynamic/fixtures/plugin-hooks/):

• project.sh builds a project on the packaged install. It has no hook scripts of its own, only config.sh and the install: packaged stamp, plus something for each hook to stop: a work branch with a spec folder, a sensitive area, a lockfile, and an env template.
• Seven cases, each driving one hook with the plugin loaded per session (--plugin-dir): session-context, guard-git, triage-first, protect-paths, careful-paths, check-env-declared, and stand-down. The last runs in a copy switched to the committed install (stand-down.setup.sh), where the plugin's hooks must do nothing.
• inspect.sh checks each run and marks it ✓ or ✘. A PostToolUse hook's message reaches Claude through the session transcript, not the output stream, so that check reads the transcript.

Runner (run-session-evals.sh), general additions:

• an optional per-case <case>.setup.sh;
• inspect.sh receives the run's output and the case name, which the existing suites ignore;
• the summary counts each run's ✓ and ✘;
• this suite defaults to Haiku, 8 turns, and $0.50 per run.

Documented in evals/dynamic/README.md, with a report in evals/dynamic/reports/2026-10-04-plugin-hooks.md.

Upgrade impact

Framework-internal.

How to verify

./evals/dynamic/run-session-evals.sh --suite plugin-hooks

It needs a signed-in CLI and costs about $0.40.

Verified / not verified

• Verified:
  • The suite through the runner: 8 of 8 checks across 7 Haiku sessions (Claude Code 2.1.286), $0.40.
  • Negative controls: every check marks ✘ when fed a run where its hook didn't act.
  • Static suites pass: 154, 81, 45, and 11.
  • The existing suites' inspect.sh ignores the new arguments.
• Not verified: Sonnet and Opus. The hooks don't depend on the model, so Haiku keeps it cheap.

Merge danger

• Evals only. Nothing in skeleton/ or the plugin changes, and a revert is safe.

🤖 Generated with Claude Code