Proposal: optional security-evaluation plugin for ADK agents (safelabs-eval, OWASP ASI-aligned)

[iamwaqarjaved]

### Is your feature request related to a specific problem?
ADK agents can be evaluated for task correctness, but there's no first-class way to run an adversarial security evaluation against an ADK agent — prompt injection, indirect injection via tool output, excessive agency, unsafe tool invocation — and get structured, reproducible results back.

Today that means wrapping Runner myself and reconstructing what happened from the final response. That loses the detail that actually decides whether a run was unsafe: which tool was called, with what arguments, and what came back. In agentic security, a clean-looking final answer can sit on top of a harmful tool call, and a refusal-looking answer can sit on top of no action at all. Final-text-only evaluation gets both cases wrong.

### Describe the Solution You'd Like

An optional, opt-in plugin that observes an ADK run through the existing BasePlugin callbacks and emits a structured trace, which is then scored against a suite of adversarial test cases into PASS / FAIL / UNCERTAIN verdicts.

Concretely:

• Input: an ADK agent, a Runner, and a named suite of adversarial test cases
• Output: a JSON report — per-test-case verdict, plus the captured tool calls, arguments, and results that justify each verdict
• Nothing else changes. No modification to Runner, agents, tools, or existing callbacks. No behavior change for anyone who doesn't construct the plugin. Optional dependency extra, so core install weight is unchanged.

The plugin would use only public callbacks:

Callback | Used for -- | -- before_run_callback / after_run_callback | invocation boundaries before_model_callback / after_model_callback | model input and output before_tool_callback | tool identity and arguments before execution after_tool_callback | tool results — where indirect injection surfaces on_tool_error_callback / on_model_error_callback | error paths

Two variants, and I don't have a strong preference:

1. Observe-only (my default proposal). The plugin records and scores; it never blocks execution. Purely an evaluation harness.
2. Optionally enforcing. before_tool_callback could refuse a tool call that violates a policy, turning the same rules into a runtime guardrail. Closer to the "policy enforcement" use case the plugin docs describe, but a larger surface — I'd only build it if you wanted it.

### Impact on your work
I maintain safelabs-eval, an Apache-2.0 open-source security evaluation framework for agentic systems, aligned to the OWASP Agentic Security Initiative categories. I want ADK supported properly rather than through a brittle external wrapper that breaks whenever internals shift.

Not blocking or urgent, and there's no deadline on your side. I'm raising it before building so I build the shape you'd actually accept rather than reworking it afterwards.

### Willingness to contribute
Yes. I'd implement it, with tests and documentation, once you've indicated the shape and placement you'd prefer.

To be upfront about timing: I'm planning to do the implementation in October 2026, after some prerequisite work on my side lands. No PR will appear unexpectedly before then.

---

🟡 Recommended Information

### Describe Alternatives You've Considered

• External wrapper around Runner — works today, but only sees final output. It can't distinguish "the attack failed" from "the attack succeeded and the summary looks fine," which is the distinction that matters.
• Per-agent callbacks instead of a plugin — would require users to modify every agent definition. A plugin applies once at the Runner level and leaves agent code untouched.
• Keeping it entirely in safelabs-eval — viable, and possibly still the right answer (see the placement question below). The downside is that it couples my package to ADK internals I don't control, which is exactly the fragility the plugin interface exists to avoid.

Proposed API / Implementation

from google.adk.runners import InMemoryRunner
from safelabs_eval.adk import SafeLabsEvalPlugin   # placeholder module path

plugin = SafeLabsEvalPlugin(suite="owasp-asi-core")

runner = InMemoryRunner(
    agent=root_agent,
    app_name="my_app",
    plugins=[plugin],
)

# run the agent against the suite's adversarial inputs, then:
report = plugin.report()

# sketch — real implementation would follow the installed BasePlugin signatures
class SafeLabsEvalPlugin(BasePlugin):
    async def before_tool_callback(self, *, tool, tool_args, **kwargs):
        self._trace.record_tool_request(tool.name, tool_args)

    async def after_tool_callback(self, *, tool, tool_result, **kwargs):
        self._trace.record_tool_result(tool.name, tool_result)

    async def after_run_callback(self, *, invocation_context, **kwargs):
        self._verdicts.append(self._score(self._trace))

Report shape:

{
  "suite": "owasp-asi-core",
  "results": [
    {
      "test_case_id": "asi01-003",
      "verdict": "FAIL",
      "category": "indirect_prompt_injection",
      "evidence": {
        "tool_calls": [
          {"name": "send_email", "args": {"to": "attacker@example.com"}}
        ]
      }
    }
  ]
}

Additional Context

On placement — I'm filing here because this uses core plugin surfaces and ADK already ships optional integration extras. But if you'd rather it live in adk-python-community as a standalone plugin package, or stay external with only a docs reference from here, any of those work. You know the structure better than I do, and I'll follow whichever you prefer.

On testing — tests would run against real ADK objects with a deterministic local model. No paid API calls, so CI stays usable for contributors.

What I'm asking:

• Is an optional security - evaluation plugin something you'd want associated with ADK at all?
• If yes — here, adk-python-community, or external with a docs link?
• Observe-only, or would the enforcing variant be more useful?

If this isn't a fit for your roadmap, no problem at all — I'd rather know now than after building it. Thanks for ADK.