ApigeeLlm: streaming tool-call arguments parsed per-chunk, crashes on any provider that fragments them across SSE events

[timothyvdwerf]

Describe the bug

ApigeeLlm's streaming tool-call accumulator parses each raw arguments SSE delta as a standalone JSON document, instead of concatenating the raw string across chunks and parsing once. This crashes with a JSONDecodeError the moment a provider streams arguments across more than one chunk, which is normal OpenAI streaming behavior (arguments stream token by token, the same way message content does).

Location: google/adk/models/apigee_llm.py, _upsert_tool_call:

func = tool_call.get('function', {})
args_delta = func.get('arguments', '')
if args_delta:
    args = _json_utils.safe_json_loads(
        args_delta, context=f'tool call arguments: {args_delta}'
    )

args_delta here is one SSE chunk's fragment of the arguments string, not the full accumulated string. Parsing it as JSON on its own only works if the upstream provider happens to send the whole arguments value in a single chunk.

google/adk/models/lite_llm.py has already been through several rounds of fixing exactly this class of bug (see #5008, #5896, #5897, #5756, #6716), including a dedicated brace-counting accumulator for streamed tool-call arguments. That fix was never ported to apigee_llm.py, which still has the naive per-chunk parse.

To Reproduce

Point ApigeeLlm (or any OpenAI-compatible chat/completions backend that streams tool-call arguments incrementally) at a model that emits a function call, with stream=True. Capture the raw SSE delta.tool_calls[].function.arguments fragments — they arrive like:

chunk 1: args_delta=''
chunk 2: args_delta='{"'
chunk 3: args_delta='name'
chunk 4: args_delta='":"'
chunk 5: args_delta='Tim'
...

_upsert_tool_call calls json.loads('{"') on chunk 2 alone, which raises:

json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 2 (char 1)

Surfaced end-to-end in adk web as:

errorCode: "ValueError"
errorMessage: "Invalid JSON in tool call arguments: {": Unterminated string starting at: line 1 column 2 (char 1)"

Reproduced against google-adk 2.9.2 (latest on PyPI at time of filing) using gpt-5.6-luna through an Apigee-fronted OpenAI-compatible gateway with api_type=CHAT_COMPLETIONS. A Vertex-backed model behind the same ApigeeLlm instance does not trigger this, because that particular upstream happens to send the whole arguments string in one chunk — this is incidental to that provider, not something ApigeeLlm should rely on.

Expected behavior

Streamed tool-call arguments accumulate as a raw string across all chunks for a given index, and get parsed as JSON once, after the stream for that tool call completes — matching the approach already implemented in lite_llm.py.

Environment

• google-adk 2.9.2
• Python 3.12

[palthepusaisahith]

I reproduced this on v2.9.2 (dafa8e952a57e8ee613008dc6b8a32acf69b853b) using a synthetic streamed Chat Completions response with {"city":"Hyderabad"} split across multiple function.arguments chunks.

On v2.9.2, the first incomplete fragment is parsed immediately and raises:

ValueError: Invalid JSON in tool call arguments: {"city":": Unterminated string ...

However, I could not reproduce this on current main (9625b06c9a1be6b9ecd85225657690ae5c0e9d3e).

The behavior appears to have been fixed by commit f44978013378ecba9e3cfc0c955b7890ceb7951b (“stream partial function-call arguments across model adapters”). The Apigee streaming path now buffers argument fragments by tool-call index and parses the joined payload when constructing the final function call.

I also ran the same synthetic repro directly at f449780, where it passes.

So this looks reproducible on v2.9.2 but already resolved on current main.