Skip to content

feat(agent-guard): add Grok PreToolUse integration - #1516

Open
KatalKavya96 wants to merge 4 commits into
apache:mainfrom
KatalKavya96:feat-grok-agent-guard
Open

KatalKavya96 wants to merge 4 commits into
apache:mainfrom
KatalKavya96:feat-grok-agent-guard

Conversation

@KatalKavya96

Copy link
Copy Markdown
Contributor

Summary

Adds native Grok PreToolUse support to tools/agent-guard while keeping the guard engine harness-neutral.

The new Grok adapter translates Grok hook input into the existing shared dispatch(command, cwd) path and returns Grok-compatible allow/deny decisions.

Refs #1416

What changed

  • Added --grok support to agent-guard
  • Added grok_main() as a thin adapter around the shared guard dispatcher
  • Added .grok/hooks/magpie-agent-guard.json
  • Handles Grok shell tools:
    • run_terminal_command
    • run_terminal_cmd
  • Returns explicit Grok decisions:
    • {"decision": "allow"}
    • {"decision": "deny", "reason": "..."}
  • Added unit and end-to-end tests for:
    • allow / deny behavior
    • shell-tool aliases
    • cwd forwarding
    • malformed and irrelevant events
    • explicit --grok routing
    • preserving the default Claude Code invocation
    • execution through the committed Grok hook

Setup lifecycle

Added Grok hook handling to setup install, upgrade, and uninstall flows.

Magpie owns only:

.grok/hooks/magpie-agent-guard.json

The lifecycle intentionally does not modify:

  • unrelated .grok/hooks/*
  • .grok/config.toml
  • Grok project-hook trust state

Hook trust remains an explicit operator decision.

Snapshot adopters derive their hook from the committed Magpie hook and only change the executable path to the .apache-magpie snapshot.

Metadata and documentation

  • Added Grok to agent-guard harness metadata
  • Added Grok to the vendor-neutrality harness vocabulary
  • Regenerated vendor-neutrality documentation
  • Updated the adapter registry from reviewer backend only to reviewer backend + action guard
  • Added the strict JSON hook file to .rat-excludes
  • Regenerated the setup-preflight fingerprint

This PR intentionally does not add full Grok spec-loop / headless harness support. That remains separate work under #1416.

Validation

Validated with:

  • tools/agent-guard: 199 tests passing
  • tools/vendor-neutrality-score: 36 tests passing
  • Ruff check and format
  • mypy
  • full workspace pytest
  • Markdown / link / placeholder checks
  • setup fingerprint checks
  • symlink and release archive checks
  • skill and tool validation
  • spec validation
  • full prek run --all-files

On macOS, the full workspace gate was run with TMPDIR=/tmp to avoid the pre-existing container-gateway AF_UNIX socket path-length issue.

AI assistance

AI assistance was used for implementation planning, repository analysis, patch review, and test/debug iteration. All changes were reviewed and validated locally before submission.

@KatalKavya96

Copy link
Copy Markdown
Contributor Author

Hi @potiuk, I’ve opened the next piece of #1416 for the Grok integration.

This PR adds native Grok PreToolUse support to agent-guard, keeps the shared guard logic harness-neutral, wires the project hook/setup lifecycle, and adds end-to-end coverage for the committed hook.

I’ve intentionally kept full Grok spec-loop / headless harness support out of this PR so the changes stay incremental and reviewable.

Would appreciate your review when you get a chance.

@github-actions github-actions Bot added substrate:analytics Tool substrate: read-only metrics / dashboards / renderers substrate:action-guard Tool substrate: deterministic pre-tool-use command guards substrate:framework-dev Tool substrate: build / validate / eval the framework itself labels Oct 5, 2026

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The adapter is tidy and mirrors the Gemini / Kiro shape, but every assumption it makes about Grok's hook API — camelCase toolName / toolInput, the run_terminal_command tool name, {"decision": …} plus exit 2 — is unsourced, and any mismatch fails open silently, so a wrong guess means the guard never fires. Please cite Grok's hook documentation and test against a captured real payload before this can count as a guard (inline on __init__.py:946).

Smaller observations

  • tools/spec-loop/specs/agent-isolation-sandbox.md lists the agent-guard harness adapters (Claude Code, OpenCode, --gemini) and isn't updated for --grok; AGENTS.md asks that "the affected specs reflect what actually shipped".
  • The install / upgrade / uninstall steps cover only the snapshot path ($GROK_WORKSPACE_ROOT/.apache-magpie/tools/...). Marketplace installs — the preferred method — have no .apache-magpie/ snapshot; please say how they get the hook.
  • If GROK_WORKSPACE_ROOT is unset, python3 "<missing>/…" exits 2 — please say what Grok does with that (block every shell call, or pass silently), once its exit-code semantics are sourced.
  • FRAMEWORK_FINGERPRINT in isolated_fingerprint.py is also changed by #1507; whichever lands second needs to regenerate it.
  • The failing prek check is a uv segfault in the symlink-lint ruff step, unrelated to this diff; a re-run should clear it.

This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. After you've
addressed the points above and pushed an update, an Apache Magpie
maintainer — a real person — will take the next look
at the PR. The findings cite the project's review criteria;
if you think one of them is mis-applied, please reply on the
PR and a maintainer will weigh in.

More on how Apache Magpie handles maintainer review:
CONTRIBUTING.md.

Comment thread tools/agent-guard/src/agent_guard/__init__.py Outdated
Comment thread tools/agent-guard/src/agent_guard/__init__.py
Comment thread tools/agent-guard/tests/test_grok.py Outdated
Comment thread plugins/magpie-setup/skills/setup/install.md Outdated
@potiuk potiuk added family:tools tools/* family:setup setup-* skills and removed substrate:analytics Tool substrate: read-only metrics / dashboards / renderers labels Oct 5, 2026
@github-actions github-actions Bot added the substrate:analytics Tool substrate: read-only metrics / dashboards / renderers label Oct 5, 2026
@KatalKavya96
KatalKavya96 force-pushed the feat-grok-agent-guard branch from b501aeb to 59eab13 Compare October 8, 2026 17:48
@github-actions github-actions Bot added family:docs Docs, MISSION.md, READMEs family:ci .github workflows, prek, validators capability:platform Framework / agent substrate skills (install, verify, doctor, override, status, setup bootstrap) labels Oct 8, 2026
@KatalKavya96

Copy link
Copy Markdown
Contributor Author

@potiuk Thanks for the review — I’ve pushed the follow-up changes in 59eab13.

The Grok guard is now based on the documented and observed hook contract rather than assumptions:

  • switched the hook matcher to the documented Bash matcher
  • verified against Grok Build 1.0.46 (2765805b9442)
  • captured a real PreToolUse payload from that runtime and use it verbatim in the regression test
  • removed the speculative run_terminal_cmd alias; the observed concrete shell tool is run_terminal_command
  • documented the Grok contract, version, trust boundary, lifecycle, marketplace behavior, and fail-open behavior in docs/adapters/grok.md
  • aligned allow/deny handling with the documented semantics: silent exit 0 for allow and JSON deny reason + exit 2 for denial
  • added --grok to the affected action-guard spec without adding Grok spec-loop/headless/sandbox support
  • documented how marketplace installs use the existing Claude-compatible magpie-agent-guard plugin directly, while source/snapshot installs use the project hook
  • documented that a broken/missing launcher fails open, so verification requires a real live denial
  • reflowed the Grok documentation blocks to semantic line breaks
  • rebased and regenerated the vendor-neutrality output, setup fingerprint, and generated skill metadata

I also tested it live in Grok:

  • grok inspect --json showed the trusted project PreToolUse hook with matcher Bash
  • pwd was allowed
  • git commit --no-verify --dry-run was blocked by the agent guard before execution

The full local prek suite and focused agent-guard tests are green. CI is currently running on the updated branch.

@KatalKavya96
KatalKavya96 requested a review from potiuk October 9, 2026 05:20

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @KatalKavya96. This is the follow-up I asked for: the contract is now sourced from xAI's hooks docs, the fixture is a verbatim capture from Grok Build 1.0.46, and the deny shape ({"decision": "deny", ...} plus exit 2) matches the documented contract. The project-hook path looks right.

Two claims in docs/adapters/grok.md are still unverified, and both are about the guard failing silently, which is the failure this adapter has to rule out.

Major — the marketplace path probably never reaches grok_main()

The doc says the existing magpie-agent-guard plugin works under Grok as-is, because the no-argument dispatcher sees GROK_HOOK_EVENT. But that plugin's hook command is python3 "${CLAUDE_PLUGIN_ROOT}/tools/agent-guard/src/agent_guard/__init__.py" (plugins/magpie-agent-guard/.claude-plugin/plugin.json), and xAI's plugin docs only say plugin hooks receive GROK_PLUGIN_ROOT and GROK_PLUGIN_DATA. If Grok does not also set CLAUDE_PLUGIN_ROOT, the path becomes /tools/agent-guard/..., python3 exits 2 with "can't open file", and the guard never runs. Please either verify the marketplace install live on Grok 1.0.46 (a known denial, not just an allow) and say so in the doc, or make the plugin command fall back, for example ${CLAUDE_PLUGIN_ROOT:-$GROK_PLUGIN_ROOT}.

Major — "a broken launcher fails open" needs evidence

python3 exits 2 for a missing script, and xAI's hooks page says "exit code 2 denies". So a broken launcher (unset GROK_WORKSPACE_ROOT, a moved snapshot, the plugin-root case above) may not fail open as the doc states. It may block every shell command instead. The same page also says "only an explicit deny blocks". Either way, please check what 1.0.46 actually does with a bare exit 2 and no JSON, and document the observed behaviour instead of the assumed one. (This was the open question from my last review.)

Smaller observations

  • Grok also reads enabled Claude Code plugins, and this repo's .claude/settings.json enables magpie-agent-guard. So in this checkout, Grok runs the guard twice (plugin plus .grok/hooks/magpie-agent-guard.json), which contradicts the "exactly one hook" check in Verify. Please say which one this repo should keep, or document the overlap.
  • The branch conflicts with main in isolated_fingerprint.py (the #1507 overlap I mentioned) and needs a rebase, then a regenerated FRAMEWORK_FINGERPRINT.

This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. After you've
addressed the points above and pushed an update, an Apache Magpie
maintainer — a real person — will take the next look
at the PR. The findings cite the project's review criteria;
if you think one of them is mis-applied, please reply on the
PR and a maintainer will weigh in.

More on how Apache Magpie handles maintainer review:
Contributing guide.

Comment thread docs/adapters/grok.md Outdated
Comment thread docs/adapters/grok.md Outdated
Comment thread docs/adapters/grok.md Outdated
@KatalKavya96
KatalKavya96 force-pushed the feat-grok-agent-guard branch from 59eab13 to e3b07f7 Compare October 10, 2026 02:45
@KatalKavya96

Copy link
Copy Markdown
Contributor Author

@potiuk Addressed the remaining review concerns.

For the marketplace/plugin path, magpie-agent-guard now registers through hooks/hooks.json and packages the guard runtime as real files inside the plugin instead of relying on the escaping symlink. I verified this on Grok Build 1.0.50: the plugin reports provides.hooks: true, the installed runtime resolves correctly, /hooks shows the active hook, and git commit --no-verify --dry-run is blocked by the actual Magpie agent-guard[no-verify] policy.

I also rechecked the failure semantics. A bare PreToolUse hook that exits 2 with no JSON blocks the shell tool call on Grok 1.0.50, and a missing Python hook target that exits 2 is blocked as well. The docs now distinguish that from malformed/unrelated input handled inside grok_main(), which deliberately returns 0 and remains fail-open.

I kept the original payload provenance separate: the captured run_terminal_command payload is from Grok Build 1.0.46, while marketplace loading, packaged-runtime execution, real policy denial, and exit-2 behavior were verified on 1.0.50.

Full prek run --all-files is green after the changes.

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @KatalKavya96 — this round closes the two major points. The plugin hook now falls back to GROK_PLUGIN_ROOT, the marketplace path is verified with a real agent-guard[no-verify] denial on Grok 1.0.50, and the exit-2 behaviour is now observed and documented rather than assumed: a broken launcher blocks, while malformed input stays fail-open inside grok_main(). Packaging the guard as real files, byte-checked against tools/agent-guard by check-family-plugins.py, is a sound way around the escaping symlink.

Two things are left before this can be approved.

  • Two Magpie hooks in this repository. This repo enables magpie-agent-guard in .claude/settings.json, which Grok also reads, and also commits .grok/hooks/magpie-agent-guard.json. So a Grok session here runs the guard twice — the overlap your own Verify step tells operators to warn about. Please either drop one of the two for this repo, or document why this repo keeps both. (The open thread on docs/adapters/grok.md.)
  • Claude Code is now on the new hook too. The plugin's hook moved from the manifest to hooks/hooks.json, and its command now goes through ${GROK_PLUGIN_ROOT:-${CLAUDE_PLUGIN_ROOT}}. That changes the path every Claude Code user takes. Please confirm one live denial on Claude Code with the rebuilt plugin, as you did for Grok.

Not blocking:

  • Adapter selection by environment variable. With no flag, cli() routes to grok_main() whenever GROK_HOOK_EVENT is set, and the plugin's hook command prefers GROK_PLUGIN_ROOT. If either variable is ever inherited into a Claude Code hook process, the guard either silently allows everything (grok_main() ignores Claude's Bash payload) or blocks every call (an unreachable path exits 2). Selecting the adapter from the payload's shape (toolName in Grok's event, tool_name: "Bash" in Claude's) would remove that dependency.
  • Nits in docs/adapters/grok.md: "supported in this PR" won't read right once merged, and the SPDX comment appears twice.

This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. After you've
addressed the points above and pushed an update, an Apache Magpie
maintainer — a real person — will take the next look
at the PR. The findings cite the project's review criteria;
if you think one of them is mis-applied, please reply on the
PR and a maintainer will weigh in.

More on how Apache Magpie handles maintainer review:
Contributing guide.

Comment thread docs/adapters/grok.md Outdated
Comment thread docs/adapters/grok.md Outdated
Add a native Grok hook adapter that translates PreToolUse events into the shared agent-guard dispatcher. Wire the project hook and snapshot setup lifecycle without modifying Grok trust state or unrelated configuration.

Register Grok in harness metadata and vendor-neutrality reporting, with end-to-end coverage for the committed hook.

Refs apache#1416
@KatalKavya96
KatalKavya96 force-pushed the feat-grok-agent-guard branch from a26a3f9 to 5bff83a Compare October 10, 2026 07:22
@KatalKavya96

Copy link
Copy Markdown
Contributor Author

@potiuk Addressed the remaining review items in 5bff83a.

  • Removed the duplicate active Grok project hook from this repository and kept the project-hook shape as an inactive setup template. grok inspect --json now shows a single active Magpie action-guard hook from the enabled plugin.
  • Live-verified the rebuilt shared plugin on Claude Code 2.1.287: git commit --no-verify --dry-run was blocked during PreToolUse:Bash by agent-guard[no-verify].
  • Replaced GROK_HOOK_EVENT-based no-argument routing with payload-shape routing, including regression coverage that an inherited Grok env var cannot misroute a Claude payload.
  • Fixed the adapter wording and duplicate SPDX nit.
  • Regenerated the packaged runtime and isolated setup fingerprint.

Full TMPDIR=/tmp prek run --all-files is green.

@KatalKavya96
KatalKavya96 requested a review from potiuk October 10, 2026 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capability:platform Framework / agent substrate skills (install, verify, doctor, override, status, setup bootstrap) family:ci .github workflows, prek, validators family:docs Docs, MISSION.md, READMEs family:setup setup-* skills family:tools tools/* substrate:action-guard Tool substrate: deterministic pre-tool-use command guards substrate:analytics Tool substrate: read-only metrics / dashboards / renderers substrate:framework-dev Tool substrate: build / validate / eval the framework itself

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants