BigQueryAgentAnalyticsPlugin records error-bearing tool results as TOOL_COMPLETED with status OK

[roanny]

Describe the bug

BigQueryAgentAnalyticsPlugin writes TOOL_ERROR (with status = 'ERROR' and error_message) only from on_tool_error_callback, that is, only when a tool raises and no plugin registered ahead of it answers the error. after_tool_callback writes TOOL_COMPLETED with status = 'OK' for whatever the tool returned, without looking at it. Two common failures therefore land as successes:

1. An MCP tool whose server answers isError: true. McpTool returns the CallToolResult as a result dict (_dump_mcp_model, which restores isError) instead of raising, which is the MCP protocol's shape for a tool-level failure. The row is TOOL_COMPLETED, status = 'OK', error_message = NULL; the failure only shows in content.result.isError.
2. A tool that raises while ReflectAndRetryToolPlugin is registered ahead of the analytics plugin. PluginManager._run_callbacks stops at the first plugin that returns a value, so the reflect-and-retry response becomes the tool's result and reaches after_tool_callback: TOOL_COMPLETED, status = 'OK', with the error in content.result.error_details. ToolboxToolset tools raise on failure, so a Toolbox error lands here.

An audit, report or alert built on status, error_message or TOOL_ERROR reads both failures as successes.

To Reproduce

google-adk 2.9.0; no BigQuery is needed, because the rows are captured by replacing _log_event. The MCP case uses a FunctionTool that returns the dict McpTool returns for isError: true — the plugin never inspects the result, so the recorded row is the same.

import asyncio
from unittest.mock import AsyncMock

from google.adk.agents.llm_agent import LlmAgent
from google.adk.apps import App
from google.adk.models.base_llm import BaseLlm
from google.adk.models.llm_response import LlmResponse
from google.adk.plugins import ReflectAndRetryToolPlugin
from google.adk.plugins.bigquery_agent_analytics_plugin import BigQueryAgentAnalyticsPlugin
from google.adk.runners import InMemoryRunner
from google.adk.tools import FunctionTool
from google.genai import types


def mcp_style_failure() -> dict:
    return {"content": [{"type": "text", "text": "Git reset to remote failed."}], "isError": True}


def raising_tool() -> str:
    raise RuntimeError("MCP request failed with code 403: Access Denied")


def build(with_reflect: bool):
    rows = []
    analytics = BigQueryAgentAnalyticsPlugin(project_id="p", dataset_id="d", table_id="events")

    async def record(event_type, callback_context, raw_content=None, is_truncated=False, event_data=None):
        if event_type.startswith("TOOL_"):
            rows.append((event_type, event_data.status if event_data else "OK", (raw_content or {}).get("tool")))

    analytics._log_event = record
    analytics._ensure_started = AsyncMock(return_value="ok")
    analytics.flush = AsyncMock()
    turns = [[types.Part(function_call=types.FunctionCall(name="mcp_style_failure", args={}, id="c0"))]]
    if with_reflect:  # without a plugin that answers the error, the raise ends the run
        turns.append([types.Part(function_call=types.FunctionCall(name="raising_tool", args={}, id="c1"))])
    turns.append([types.Part(text="done")])

    class Scripted(BaseLlm):
        async def generate_content_async(self, llm_request, stream=False):
            yield LlmResponse(content=types.Content(role="model", parts=turns.pop(0)))

    agent = LlmAgent(name="repro", model=Scripted(model="scripted"), instruction="x",
                     tools=[FunctionTool(mcp_style_failure), FunctionTool(raising_tool)])
    plugins = ([ReflectAndRetryToolPlugin(max_retries=3)] if with_reflect else []) + [analytics]
    return InMemoryRunner(app=App(name="repro", root_agent=agent, plugins=plugins)), rows


async def main():
    for with_reflect in (False, True):
        runner, rows = build(with_reflect)
        await runner.session_service.create_session(app_name="repro", user_id="u", session_id="s")
        async for _ in runner.run_async(user_id="u", session_id="s",
                                        new_message=types.Content(role="user", parts=[types.Part(text="go")])):
            pass
        print(f"ReflectAndRetryToolPlugin ahead={with_reflect}:", rows)


asyncio.run(main())

Output:

ReflectAndRetryToolPlugin ahead=False: [('TOOL_STARTING', 'OK', 'mcp_style_failure'), ('TOOL_COMPLETED', 'OK', 'mcp_style_failure')]
ReflectAndRetryToolPlugin ahead=True: [('TOOL_STARTING', 'OK', 'mcp_style_failure'), ('TOOL_COMPLETED', 'OK', 'mcp_style_failure'), ('TOOL_STARTING', 'OK', 'raising_tool'), ('TOOL_COMPLETED', 'OK', 'raising_tool')]

Expected behavior

An error-bearing result is recorded as a failure — TOOL_ERROR, or status = 'ERROR' with a non-null error_message — while the tool's result still reaches the model unchanged. Two shapes ADK itself produces would cover both cases above: an MCP result with isError: true, and a ReflectAndRetryToolPlugin response (response_type == REFLECT_AND_RETRY_RESPONSE_TYPE). A way for an application to classify a result (for example a callable in BigQueryLoggerConfig returning a status and a message) would also cover application-specific shapes.

Related, minor

_get_tool_origin classifies McpTool, TransferToAgentTool, AgentTool and FunctionTool, and returns UNKNOWN for everything else. The tools of ADK's own ToolboxToolset (toolbox_adk.ToolboxTool, a BaseTool) and the SkillToolset tools therefore land as UNKNOWN, so the table cannot separate BigQuery calls from skill loads by origin.

Desktop

• OS: macOS
• Python version: 3.12
• ADK version: 2.9.0. On main (checked 2026-09-13) the plugin still has no reference to isError or to the Toolbox.

Model Information: not applicable; the reproduction uses a scripted BaseLlm.

Additional context

Searched existing issues for BigQueryAgentAnalyticsPlugin isError, BigQueryAgentAnalyticsPlugin TOOL_ERROR, tool_origin UNKNOWN and ToolboxToolset analytics and found none. The nearest precedent is #5073 (closed), a tool_origin classification gap for RemoteA2aAgent. We currently work around it with a subclass that classifies the result in after_tool_callback and writes the row through the plugin's _log_event, which depends on private helpers.

[Zsanz3]

I'd like to take a minimal fix for this.

Approach: In BigQueryAgentAnalyticsPlugin.after_tool_callback, when the tool result carries isError / error_details, emit TOOL_ERROR (same classification as on_tool_error_callback) instead of TOOL_COMPLETED/OK. Keep the diff tiny and add a focused unit test with a mocked _log_event. Not expanding into ReflectAndRetry.

Feedback welcome before I open the PR.

[llalitkumarrr]

Hello @haiyuan-eng-google,

Could you please take a look at this?