fix: exclude thought parts from response match scoring

[MrCapricornLiu]

The response-match evaluator counts text parts marked thought=True as final-answer text. A correct answer can lose points because of its thought summary, and a wrong answer can receive credit for a word mentioned only in that summary. Exclude thought parts from both actual and reference text, preserving the existing tokenization and empty-response behavior.

Reproduction and expected behavior

On ADK 2.8.0 / main (2284c581), Linux, Python 3.12.13, run the added regression cases in:

pytest tests/unittests/evaluation/test_final_response_match_v1.py -q

The added cases fail on the original implementation and pass with this change. No LiteLLM or live model endpoint is required; Runner verification uses a scripted BaseLlm response.

Testing Plan

Type checking matches the unmodified upstream baseline (831 existing errors; no new errors).

• Relevant unit tests on current main (da10185e): 74 passed.
• The Runner reproduction passes on current main. A wheel built against 2284c581 was also verified in a clean environment.
• Earlier full-suite validation on 2284c581, Python 3.10–3.14: tests passed with the two GC-introspection cases run in separate processes (14,248 passing cases on Python 3.12). One unrelated MCP stdio teardown case was deselected after it also failed to finish on the unmodified base. The default uninterrupted tox run did not complete.

Runner setup and reproduction output

Save the following as repro.py, then run python repro.py after installing the locally built wheel. Install rouge-score as well.

import asyncio
import json
from google.adk.agents import Agent
from google.adk.artifacts.in_memory_artifact_service import InMemoryArtifactService
from google.adk.models.base_llm import BaseLlm
from google.adk.models.llm_request import LlmRequest
from google.adk.models.llm_response import LlmResponse
from google.adk.runners import Runner
from google.adk.sessions.in_memory_session_service import InMemorySessionService
from google.genai import types
from pydantic import BaseModel, Field

class ScriptedModel(BaseLlm):
    model: str = "local-repro"
    responses: list[types.Content]
    requests: list[LlmRequest] = Field(default_factory=list)

    async def generate_content_async(self, llm_request, stream=False):
        index = len(self.requests)
        self.requests.append(llm_request.model_copy(deep=True))
        yield LlmResponse(content=self.responses[index])

def response(*parts):
    return types.Content(role="model", parts=list(parts))

async def run_agent(model, tools=(), artifacts=None, setup=None):
    sessions = InMemorySessionService()
    session = await sessions.create_session(app_name="repro", user_id="user")
    if setup:
        await setup(session)
    async with Runner(app_name="repro", agent=Agent(name="repro_agent", model=model, tools=list(tools)), session_service=sessions, artifact_service=artifacts) as runner:
        events = [event async for event in runner.run_async(user_id="user", session_id=session.id, new_message=types.Content(role="user", parts=[types.Part(text="Run the check.")]))]
        stored = await sessions.get_session(app_name="repro", user_id="user", session_id=session.id)
    return events, stored

from google.adk.evaluation.evaluation_generator import EvaluationGenerator
from google.adk.evaluation.eval_metrics import EvalMetric
from google.adk.evaluation.final_response_match_v1 import RougeEvaluator

async def main():
    for visible, score in [("Paris", 1.0), ("London", 0.0)]:
        model = ScriptedModel(responses=[response(types.Part(text="Consider Paris, the capital of France.", thought=True), types.Part(text=visible))])
        _, session = await run_agent(model)
        actual = EvaluationGenerator.convert_events_to_eval_invocations(session.events)
        assert len(actual) == 1
        assert actual[0].final_response.parts[0].thought is True
        expected = actual[0].model_copy(update={"final_response": response(types.Part(text="Paris"))})
        result = RougeEvaluator(EvalMetric(metric_name="response_match_score", threshold=0.5)).evaluate_invocations(actual, [expected])
        assert result.overall_score == score, result
        print("PASS: Runner -> stored events -> eval invocation -> ROUGE, visible answer", visible, "score", result.overall_score)

asyncio.run(main())

PASS: Runner -> stored events -> eval invocation -> ROUGE, visible answer Paris score 1.0
PASS: Runner -> stored events -> eval invocation -> ROUGE, visible answer London score 0.0

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.