Skip to content

fix(adapters): bound the conformance probe, honor declared timeouts, scratch cwd - #168

Merged
pacphi merged 1 commit into
mainfrom
fix/hermes-conformance-probe-timeout-cwd
Aug 18, 2026
Merged

fix(adapters): bound the conformance probe, honor declared timeouts, scratch cwd#168
pacphi merged 1 commit into
mainfrom
fix/hermes-conformance-probe-timeout-cwd

Conversation

@pacphi

@pacphi pacphi commented Aug 18, 2026

Copy link
Copy Markdown
Owner

Summary

adrianco ran a real Hermes adapter against ak host adapters conformance on alpha.42 (PR #131 follow-up) and found the activity-routing tier can't pass on a local model:

  • Bounded probe — the probe prompt ("conformance harness probe") is open-ended, so an agentic host reads it as a task and does real work (4+ minutes) instead of proving the transport. Now "Reply with exactly: OK".
  • Declared timeout honored — the outer runner budget was hardcoded to the 120s default regardless of the manifest's own declared execution.run.hook.timeoutMs, so a local-model adapter with a longer declared budget could never use it. Now honored by default; ak host adapters conformance --timeout <ms> is available as an explicit override.
  • Scratch cwdactivity-routing/primary-eligible now run their worker in a throwaway temp directory instead of the operator's process.cwd().

Also documents two things from his report that need no code fix: the nativeMcpConfig/nativeGuidance author-trap (capabilities describe what the adapter wires, not what the host can do in principle), and that AK_WORKER_CWD is advisory, not a sandbox boundary.

Held for a follow-up branch, pending his design input requested in the PR #131 comment: detection.binPath (his venv-install finding) and hook-content hash pinning (his consent-drift finding).

Test plan

  • pnpm run check (typecheck + lint + markdownlint + build + full test suite) — 1993/1998 passed, 0 failures (5 skips are win32-only)
  • New tests: bounded probe prompt reaches the real hook, scratch cwd is real+isolated+cleaned-up, explicit --timeout override reaches the worker, declared manifest timeout is honored without an override, CLI --timeout flag parses/forwards/rejects invalid input

🤖 Generated with Claude Code

…scratch cwd

adrianco's real Hermes adapter run against PR #131 (comment #5320745151)
surfaced two ergonomics bugs in the activity-routing/primary-eligible
conformance tiers:

- The probe prompt ("conformance harness probe") is open-ended, so an
  agentic host reads it as a task and goes and does real work instead of
  proving the transport. Bounded to a directive "Reply with exactly: OK".
- The outer runner budget was hardcoded to the 120s default regardless of
  the manifest's own declared execution.run.hook.timeoutMs, so a
  local-model adapter with a longer declared budget could never actually
  use it. Now honored by default, with `ak host adapters conformance
  --timeout <ms>` as an explicit override.
- Both tiers also now run their worker in a throwaway scratch directory
  instead of the operator's process.cwd() — a live, often auto-approving
  worker has no business landing wherever the operator happened to run
  the command from.

Also documents the nativeMcpConfig/nativeGuidance author-trap adrianco
hit (capabilities describe what the adapter wires, not what the host can
do) and clarifies AK_WORKER_CWD is advisory, not a sandbox boundary.

detection.binPath and hook-content hash pinning (his other two findings)
are held for a follow-up branch pending his design input requested in
the PR #131 follow-up comment.
@pacphi
pacphi merged commit 67bf24d into main Aug 18, 2026
14 checks passed
@pacphi
pacphi deleted the fix/hermes-conformance-probe-timeout-cwd branch August 18, 2026 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant