diff --git a/docs/specs/004-python-function-calling-loop.md b/docs/specs/004-python-function-calling-loop.md
new file mode 100644
index 00000000000..ef33f154de5
--- /dev/null
+++ b/docs/specs/004-python-function-calling-loop.md
@@ -0,0 +1,545 @@
+---
+status: proposed
+contact: eavanvalkenburg
+date: 2026-07-27
+deciders: eavanvalkenburg
+---
+
+# Python function-calling loop contract and validation matrix
+
+## Scope
+
+This specification defines the required behavior and validation coverage for the Python function-calling loop.
+It covers:
+
+- normal local function execution;
+- streaming and non-streaming response aggregation;
+- tool approval request and resume;
+- approved, rejected, mixed, and replayed approval rounds;
+- reasoning content and opaque reasoning signatures bound to function calls;
+- history persistence and service-side continuation;
+- error, user-input, middleware-termination, and loop-limit paths;
+- provider and transport serialization of function calls and results.
+
+The primary implementation is in `python/packages/core/agent_framework/_tools.py`. History replay behavior in
+`python/packages/core/agent_framework/_sessions.py`, provider serializers, hosting packages, and UI transports are
+part of the same contract when they carry function-call loop content.
+
+## Change sensitivity
+
+This code is high risk. Small changes can produce duplicate side effects, orphaned calls or results, invalid
+provider histories, invisible streaming results, stale approval authority, or loops that never terminate.
+Dropping reasoning content that a service binds to a tool call can also make an otherwise balanced call/result
+transcript invalid.
+
+Any change to the function-calling loop or its approval/history/serialization paths must:
+
+1. identify every affected row in the scenario matrix below;
+2. add or update the corresponding regression tests;
+3. validate streaming updates, streaming finalization, and non-streaming output where applicable;
+4. validate both model-bound history and caller-visible responses;
+5. run the full core package tests plus every affected provider or transport package;
+6. run source typing, test typing, and syntax checks for every affected package;
+7. receive extra review focused on call/result pairing, exactly-once execution, and history replay.
+
+A passing narrow regression test is not sufficient evidence for changes in this area.
+
+### Contribution ownership
+
+Issues involving this code must not be picked up by external contributors without first checking with the Agent
+Framework core team. The core team must confirm the intended behavior, affected scenario-matrix rows, ownership
+across core/providers/transports, and the required validation scope before implementation starts.
+
+## Flow diagrams and code map
+
+### Main function-calling flow
+
+The main control flow deliberately has separate streaming and non-streaming methods. They share policy helpers, but
+their output mechanics differ: one returns an aggregated `ChatResponse`; the other yields `ChatResponseUpdate`
+items and is finalized by `ResponseStream`.
+
+The diagrams use only the generic distinction between **local tools**, which Agent Framework executes, and
+**hosted-service tools**, whose calls and approval decisions are owned by a remote service. Provider-specific wire
+formats and regression tests appear later in the scenario matrix.
+
+```mermaid
+flowchart TD
+ Entry["FunctionInvocationLayer.get_response(...)"]
+ Setup["Prepare middleware, options, session, budget state,
and execute_function_calls partial"]
+ Enabled{"Function invocation enabled?"}
+ Direct["Delegate directly to super().get_response(...)"]
+ Mode{"stream?"}
+ NonStream["_get_response_with_function_invocation(...)"]
+ Stream["_stream_response_with_function_invocation(...)"]
+ Resolve["_resolve_approval_responses(...)
runs once before the model-iteration loop"]
+ ApprovalAction{"approval action"}
+ Immediate["Return/yield terminal result or user-input request
without another model call"]
+ ApprovalPolicy["Record approval executions;
apply stop/function-call-limit policy"]
+ Model["Call super_get_response(...)
response may contain reasoning + function_call"]
+ Process["_process_model_function_calls(...)"]
+ FunctionAction{"function-processing action"}
+ Execute["_execute_function_calls(...)"]
+ Try["_try_execute_function_calls(...)"]
+ Single["_execute_single_function_call(...)"]
+ Handle["_handle_function_call_results(...)"]
+ PostCallPolicy["Record executions; apply error/function-call-limit policy;
reset required tool choice"]
+ Advance["_prepare_messages_for_next_iteration(...)"]
+ More{"iteration budget remains?"}
+ Final["Final model call with tool_choice = none
and deterministic fallback if needed"]
+ Output["Return ChatResponse or complete ResponseStream"]
+
+ Entry --> Setup --> Enabled
+ Enabled -- no --> Direct
+ Enabled -- yes --> Mode
+ Mode -- no --> NonStream
+ Mode -- yes --> Stream
+ NonStream --> Resolve
+ Stream --> Resolve
+ Resolve --> ApprovalAction
+ ApprovalAction -- return --> Immediate --> Output
+ ApprovalAction -- stop --> ApprovalPolicy
+ ApprovalAction -- continue --> ApprovalPolicy
+ ApprovalPolicy --> More
+ Model --> Process
+ Process --> Execute --> Try --> Single --> Handle --> FunctionAction
+ FunctionAction -- return --> Output
+ FunctionAction -- stop --> PostCallPolicy
+ FunctionAction -- continue --> PostCallPolicy
+ PostCallPolicy --> Advance
+ Advance --> More
+ More -- yes --> Model
+ More -- no --> Final --> Output
+```
+
+Code-reading landmarks:
+
+- `get_response(...)` owns setup and selects the response mode.
+- `_get_response_with_function_invocation(...)` owns non-streaming aggregation.
+- `_stream_response_with_function_invocation(...)` owns streamed emission/finalization.
+- `_resolve_approval_responses(...)` handles only inbound approval decisions.
+- `_process_model_function_calls(...)` handles only calls from a completed model response.
+- `_try_execute_function_calls(...)` decides approval/declaration/execution behavior for a batch.
+- `_replace_approval_contents_with_results(...)` is the occurrence-aware approval transcript normalizer.
+
+### Approval pause and resume
+
+```mermaid
+sequenceDiagram
+ participant Caller
+ participant History as HistoryProvider
+ participant Layer as FunctionInvocationLayer
+ participant Tool
+ participant Model
+
+ Caller->>Layer: Initial user request
+ Layer->>Model: Messages + tools
+ Model-->>Layer: reasoning content + function_call
+ Layer->>Layer: Tool requires approval
+ Layer-->>Caller: function_call + function_approval_request
+
+ Caller->>Layer: function_approval_response
+ Layer->>Layer: Copy caller-owned messages
+ Layer->>Layer: _resolve_approval_responses(...)
+
+ alt approved
+ Layer->>Tool: Execute exactly once
+ Tool-->>Layer: result or exception
+ Layer->>Layer: Create terminal function_result
+ else rejected
+ Layer->>Layer: Create synthetic rejection function_result
+ end
+
+ Layer-->>Caller: Terminal result message/update
+
+ alt tool requests more user input
+ Layer-->>Caller: User-input request with assistant role
+ else middleware terminates
+ Layer-->>Caller: Termination result
+ else error limit reached
+ Layer->>Model: Normalized reasoning/call/result history, tools disabled
+ Model-->>Layer: Final assistant response
+ Layer-->>Caller: Final assistant response
+ else continue normally
+ Layer->>Model: Normalized reasoning/call/result history
+ Model-->>Layer: Final assistant response or another function_call
+ Layer-->>Caller: Final assistant response / continued loop
+ end
+
+ Layer-->>History: Persist caller input + returned response
+ Note over History: Later model replay filters approval request/response wrappers
+```
+
+The terminal result is caller-visible in both modes. The private normalized message copy is model-visible. The
+original caller input and earlier response remain unchanged.
+
+### Reasoning-bound function-call groups
+
+Some hosted services bind reasoning content or an opaque reasoning signature to the function call that follows it.
+For those services, reasoning is not optional decoration; it is part of the provider-valid function-call group.
+
+```mermaid
+flowchart TD
+ Response["Assistant response:
reasoning content + function_call"]
+ Group["One logical reasoning/function-call group"]
+ Owner{"local or hosted-service tool?"}
+ Local["Local execution"]
+ Hosted["Hosted service owns tool execution/state"]
+ Result["Terminal function_result or hosted result"]
+ Continuation{"continuation mode"}
+ Stateless["Stateless or framework-history replay"]
+ Replayable{"reasoning payload/signature
is replayable?"}
+ Replay["Replay reasoning + call + result atomically"]
+ Reject["Fail before the service call;
do not send a lossy transcript"]
+ Service["Hosted-service continuation"]
+ Reference["Reference service-stored reasoning/call;
send only the new result or approval decision"]
+ Compact{"compaction needed?"}
+ Atomic["Keep or exclude the complete
reasoning/call/result group"]
+ Caller["Caller-visible response retains reasoning
with the function-call turn"]
+
+ Response --> Group --> Owner
+ Group --> Caller
+ Owner -- local --> Local --> Result
+ Owner -- hosted service --> Hosted --> Result
+ Result --> Compact
+ Compact -- yes --> Atomic --> Continuation
+ Compact -- no --> Continuation
+ Continuation -- stateless / local history --> Stateless --> Replayable
+ Replayable -- yes --> Replay
+ Replayable -- no --> Reject
+ Continuation -- service-managed --> Service --> Reference
+```
+
+The generic contract is:
+
+- reasoning content remains ordered immediately before or alongside the function call it explains;
+- a terminal result does not replace or discard the reasoning/call portion of the active group;
+- stateless replay includes the service-required reasoning payload or opaque signature;
+- if required reasoning cannot be reconstructed, the adapter fails before sending invalid or lossy history;
+- service-managed continuation may rely on the hosted service's stored reasoning/call items and send only new
+ outputs or approval decisions;
+- compaction keeps or removes the entire reasoning/call/result group atomically.
+
+In the code, core response aggregation preserves reasoning `Content` items, compaction annotations bind reasoning to
+the tool-call group, and provider adapters serialize or reconstruct the provider-specific reasoning representation.
+
+### Approval correlation, replay, and reused ids
+
+`call_id` is not globally unique forever. The normalizer therefore tracks open logical occurrences in transcript
+order instead of keeping one global result per id.
+
+```mermaid
+flowchart TD
+ Scan["Scan normalized messages in order"]
+ Kind{"content type"}
+ Call["function_call:
open a call occurrence"]
+ Request["function_approval_request"]
+ Bind{"unbound call occurrence
with same call_id?"}
+ BindExisting["Bind request id to existing occurrence
and remove wrapper"]
+ Duplicate{"same request identity
already restored?"}
+ DropDuplicate["Remove replayed duplicate wrapper"]
+ Restore["Restore embedded function_call
as a new occurrence"]
+ Placeholder["function_result with APPROVAL_PENDING:
attach placeholder to open occurrence"]
+ Completed["terminal function_result:
close earliest open occurrence"]
+ Response["function_approval_response"]
+ Pending{"response still pending?"}
+ RemoveOld["Remove already-resolved historical response"]
+ Decision{"approved?"}
+ Approved["Pop next execution result for this call_id"]
+ Rejected["Create synthetic rejection result"]
+ HasPlaceholder{"occurrence has placeholder?"}
+ Replace["Replace placeholder and remove response wrapper"]
+ ReplaceResponse["Replace response wrapper with terminal content"]
+ Close["Close occurrence; append terminal content
to resumed response"]
+ Next["Continue scan"]
+
+ Scan --> Kind
+ Kind -- function_call --> Call --> Next
+ Kind -- approval request --> Request --> Bind
+ Bind -- yes --> BindExisting --> Next
+ Bind -- no --> Duplicate
+ Duplicate -- yes --> DropDuplicate --> Next
+ Duplicate -- no --> Restore --> Next
+ Kind -- pending placeholder --> Placeholder --> Next
+ Kind -- terminal result --> Completed --> Next
+ Kind -- approval response --> Response --> Pending
+ Pending -- no --> RemoveOld --> Next
+ Pending -- yes --> Decision
+ Decision -- yes --> Approved --> HasPlaceholder
+ Decision -- no --> Rejected --> HasPlaceholder
+ HasPlaceholder -- yes --> Replace --> Close --> Next
+ HasPlaceholder -- no --> ReplaceResponse --> Close --> Next
+ Next --> Kind
+```
+
+This flow corresponds to `_ApprovalCallOccurrence`, `_collect_approval_responses(...)`, and
+`_replace_approval_contents_with_results(...)`.
+
+### History and service-side continuation
+
+```mermaid
+flowchart LR
+ Store["History backing store
(may retain approval wrappers for audit)"]
+ Load{"HistoryProvider.load_messages?"}
+ Filter["_filter_approval_control_messages(...)"]
+ Context["SessionContext model history:
function_call + terminal function_result"]
+ Current["Current caller input:
new function_approval_response"]
+ Layer["FunctionInvocationLayer private copy"]
+ Local{"local or hosted-service approval?"}
+ LocalResult["Execute locally and normalize to function_result"]
+ Hosted["Hosted-service adapter"]
+ StoredRequest["Prior service-issued approval request"]
+ NewResponse["Current hosted approval decision"]
+ Skip["Do not replay the stored request inline"]
+ Send["Send the approval decision exactly once"]
+ Later["Later turn"]
+ Manual["Manual-history caller"]
+
+ Store --> Load
+ Load -- yes --> Filter --> Context --> Layer
+ Load -- no --> Layer
+ Current --> Layer
+ Layer --> Local
+ Local -- local --> LocalResult --> Later
+ Local -- hosted service --> Hosted
+ StoredRequest --> Hosted --> Skip
+ NewResponse --> Hosted --> Send --> Later
+ Later --> Store
+ Manual -. owns equivalent filtering .-> Layer
+```
+
+When `load_messages=False`, no history is replayed and the history filter is intentionally not invoked. Callers
+that manually replay messages own the equivalent rule: do not resend an approval response after its terminal result.
+
+## Normative contract
+
+### Function calls and results
+
+- Every actionable local `function_call` produces exactly one terminal `function_result`, unless execution pauses
+ for a new user-input request.
+- Parallel calls retain model order in the returned transcript.
+- Reused `call_id` values are correlated by logical occurrence, not one global value per id.
+- A completed function call/result pair is inert on later turns.
+- Informational-only and declaration-only calls are not executed as local tools.
+
+### Reasoning-bound calls
+
+- Reasoning content or opaque reasoning metadata that a service binds to a function call is part of the same logical
+ group as that call and its terminal result.
+- Active function loops preserve the reasoning content, function call, function result, and final assistant output
+ in caller-visible responses.
+- Framework-managed/stateless replay includes the service-required reasoning representation before the paired call.
+- Service-managed continuation may omit inline reasoning/call items only when the hosted service already owns them.
+- Missing non-reconstructable reasoning fails explicitly before a provider request instead of silently dropping the
+ content.
+- Compaction preserves or excludes the complete reasoning/call/result group atomically.
+
+### Approval request and resume
+
+- A tool that requires approval does not execute before an approved response.
+- An approved tool executes exactly once.
+- A rejected tool executes zero times and produces one synthetic rejection `function_result` using the original
+ function `call_id`.
+- The resumed response contains the newly resolved approved and rejected terminal results before any final assistant
+ message.
+- Streaming yields the same logical result content and ordering as non-streaming output and
+ `ResponseStream.get_final_response()`.
+- The function invocation layer normalizes a private copy of caller messages. It must not mutate the caller's
+ approval `Message`, approval `Content`, or an earlier returned response.
+- Approval-time `UserInputRequiredException` and `MiddlewareTermination` return immediately without another model
+ call.
+
+### Approval control content
+
+- `function_approval_request` and `function_approval_response` are control-plane contents, not durable model
+ transcript items.
+- A current hosted approval response must be sent once on the immediate resume request.
+- A server-issued approval request must not be replayed inline during service-side continuation.
+- History providers may retain approval control contents in their backing store for audit, but base history replay
+ filters them before later model calls.
+- Callers that manually own and replay message history without a loading `HistoryProvider` must likewise omit a
+ previously submitted approval response from later continuation requests.
+
+### History and continuation
+
+- Model-bound history contains one function call/result pair per completed logical occurrence.
+- Append-only history must not replay stale approval request/response wrappers to the model.
+- Framework-managed and service-managed continuation must preserve the same logical call/result transcript.
+- A terminal result consumes the corresponding approval authority in explicit stateless replay.
+
+## Scenario-to-test matrix
+
+### Normal function invocation
+
+| Scenario | Required invariant | Primary regression test |
+|---|---|---|
+| Single non-streaming call | Call, result, and final assistant message are returned in order. | `packages/core/tests/core/test_function_invocation_logic.py::test_base_client_with_function_calling` |
+| String input | Flexible string input follows the same loop behavior. | `test_base_client_with_function_calling_string_input` |
+| Multiple sequential rounds | Each round retains one call/result pair. | `test_base_client_with_function_calling_resets` |
+| Streaming call | Call chunks, one result update, and final text are emitted in order. | `test_base_client_with_streaming_function_calling` |
+| Reasoning-bound call | Finalized output retains reasoning, function call, function result, and final text. | `test_streaming_function_calling_response_includes_reasoning_and_tool_results` |
+| Calls across response messages | Every actionable call is executed once. | `test_base_client_executes_function_calls_across_multiple_response_messages` |
+| Parallel calls | Results retain the corresponding call ids and execution count. | `test_max_function_calls_limits_parallel_invocations`, `test_streaming_multiple_function_calls_parallel_execution` |
+| Informational-only call | The call is returned but not executed or approved. | `test_informational_only_function_call_is_not_invoked`, `test_informational_only_function_call_does_not_request_approval`, `test_streaming_informational_only_function_call_is_not_invoked` |
+| Declaration-only call | The call is surfaced as user input and is not executed; streaming arguments appear once while finalized request metadata remains available. | `test_declaration_only_tool`, `test_streaming_declaration_only_tool_preserves_metadata_without_duplicate_arguments` |
+| Function invocation disabled | The client bypasses the invocation loop without losing invocation kwargs. | `test_function_invocation_config_enabled_false`, `test_function_invocation_config_enabled_false_preserves_invocation_kwargs`, `test_streaming_function_invocation_config_enabled_false` |
+| Runtime tool changes | Added tools become available on the next iteration and retain approval behavior. | `test_add_tools_available_next_iteration`, `test_add_tools_with_approval_required_tool` |
+
+### Approval pause and resume
+
+| Scenario | Required invariant | Primary regression test |
+|---|---|---|
+| Initial approval request | Assistant response contains the original call and approval request; tool does not execute. | `test_approval_requests_in_assistant_message`, `test_streaming_approval_request_generated`, `test_streaming_approval_requests_in_assistant_message` |
+| Approved non-streaming resume | Result precedes final text; tool executes once; inputs remain unchanged. | `packages/core/tests/core/test_harness_tool_approval.py::test_approval_resume_returns_result_without_mutating_inputs[non-streaming-approved]` |
+| Rejected non-streaming resume | Rejection result precedes final text; tool executes zero times; inputs remain unchanged. | `test_approval_resume_returns_result_without_mutating_inputs[non-streaming-rejected]` |
+| Approved streaming resume | Result update precedes final text and final response matches non-streaming shape. | `test_approval_resume_returns_result_without_mutating_inputs[streaming-approved]`, `test_streaming_approval_resume_yields_terminal_result_before_model_text[approved]` |
+| Rejected streaming resume | Rejection result update precedes final text and tool executes zero times. | `test_approval_resume_returns_result_without_mutating_inputs[streaming-rejected]`, `test_streaming_approval_resume_yields_terminal_result_before_model_text[rejected]` |
+| Mixed approved/rejected batch | Every call gets one correctly correlated terminal result. | `packages/core/tests/core/test_function_invocation_logic.py::test_rejected_approval` |
+| Persisted approval replay | Resume executes with the prior call available. | `test_persisted_approval_messages_replay_correctly` |
+| Hosted approval pass-through | Hosted requests/responses are not processed as local calls. | `test_hosted_tool_approval_response`, `test_hosted_mcp_approval_response_passthrough`, `test_mixed_local_and_hosted_approval_flow` |
+| Approval-time user input | Every user-input request from one approved execution returns in order with assistant role and no extra model call; the execution consumes one call-budget unit. | `packages/core/tests/core/test_harness_tool_approval.py::test_approval_resume_returns_all_user_input_requests_without_another_model_call`, `packages/core/tests/core/test_function_invocation_logic.py::test_approval_resume_user_input_counts_toward_function_call_budget` |
+| Approval-time middleware termination | Terminal result returns with no extra model call in either response mode. | `packages/core/tests/core/test_function_invocation_logic.py::test_approval_resume_honors_middleware_termination` |
+| Approval re-entry after iteration budget | Pending approved calls resolve once even when prior model calls consumed `max_iterations`. | `packages/core/tests/core/test_harness_tool_approval.py::test_auto_approval_resolves_after_iteration_budget_is_exhausted` |
+| Approval resume with reasoning | Model-bound resume history retains reasoning before the call and terminal result in both modes. | `packages/core/tests/core/test_harness_tool_approval.py::test_approval_resume_replays_reasoning_with_function_call_group` |
+
+### Approval correlation and replay
+
+| Scenario | Required invariant | Primary regression test |
+|---|---|---|
+| Result matching without placeholders | Results match calls by id even when the result list is reordered. | `test_replace_approval_contents_with_results_uses_result_call_ids_without_placeholders` |
+| Reused id after completion | A later round with the same id creates a second valid pair. | `test_replace_approval_contents_with_results_allows_reused_call_id_after_completion` |
+| Replayed approval wrapper | A duplicated wrapper does not restore another function call. | `test_replace_approval_contents_with_results_deduplicates_replayed_approval_request` |
+| Historical resolved response plus new round | The old response is removed from normalized input and is not converted into a rejection result. | `test_replace_approval_contents_with_results_ignores_already_resolved_response` |
+| Multiple reused-id rounds | Approved and rejected rounds retain separate call/result occurrences. | `test_replace_approval_contents_with_results_correlates_reused_call_id_occurrences` |
+| Multi-content result with reused id | Every content produced by one execution stays with that approval occurrence and cannot bleed into the next reused-id round. | `test_replace_approval_contents_with_results_keeps_multi_content_group_with_reused_call_id` |
+| Follow-up request closes one occurrence | A user-input follow-up consumes only the preceding approval authority and leaves a later reused-id response pending. | `test_collect_approval_responses_consumes_matching_follow_up_request_occurrence` |
+| Reused-id placeholders | Placeholder results consume approved results by occurrence. | `test_replace_approval_contents_with_results_correlates_reused_call_id_placeholders` |
+| Rejected placeholder | Rejection replaces the pending placeholder instead of adding a second result. | `test_replace_approval_contents_with_results_replaces_rejected_placeholder` |
+| Results reordered with placeholders | Results still match the correct call ids. | `test_replace_approval_contents_with_results_uses_result_call_ids_for_placeholders` |
+| Missing result call id | A malformed result does not steal another approval's result. | `test_replace_approval_contents_with_results_skips_results_without_call_id` |
+| Empty approval message cleanup | Fully consumed approval messages are removed from normalized model input. | `test_replace_approval_contents_with_results_prunes_emptied_messages` |
+| Later stateless turn | A prior terminal approval response cannot execute again. | `test_resolved_approval_response_is_inert_on_later_stateless_turn` |
+| Duplicate function-call prevention | Approval normalization does not create a second call for one round. | `test_no_duplicate_function_calls_after_approval_processing` |
+| Rejection call id | Rejection result uses the function call id, not only the approval id. | `test_rejection_result_uses_function_call_id` |
+
+### Mixed batches and approval middleware
+
+| Scenario | Required invariant | Primary regression test |
+|---|---|---|
+| Safe and approval-required calls in one batch | Hidden safe calls replay only with the matching visible approval. | `packages/core/tests/core/test_harness_tool_approval.py::test_mixed_batch_hides_already_approved_request_until_approval_replay` |
+| Restored approval state | Serialized `ToolApprovalState` restores mixed-batch behavior. | `test_mixed_batch_accepts_restored_tool_approval_state` |
+| Unrelated turn before approval | Hidden calls do not execute on an unrelated turn. | `test_hidden_mixed_batch_requests_do_not_replay_on_unrelated_turn` |
+| Multiple abandoned batches | Hidden calls replay only for the matching batch. | `test_hidden_mixed_batch_requests_replay_only_for_matching_visible_approval` |
+| Queued approvals | One unresolved approval is surfaced per run without premature execution. | `test_tool_approval_middleware_queues_multiple_approval_requests`, `test_tool_approval_middleware_queues_streamed_approval_requests` |
+| Middleware state plus hidden core state | State saves do not discard hidden mixed-batch calls. | `test_tool_approval_middleware_preserves_hidden_mixed_batch_requests` |
+| Auto-approval callback | Callback receives the original function call and executes the approved set once. | `test_tool_approval_middleware_auto_approval_rule_receives_function_call` |
+| Shared call budget | Auto-approved re-entry does not reset `max_function_calls`, and every executed approval group counts even when it pauses for input. | `test_tool_approval_middleware_auto_approved_loops_share_function_call_budget`, `test_approval_resume_user_input_counts_toward_function_call_budget` |
+| Standing tool rule | Tool-level approval applies only to later matching tools. | `test_tool_approval_middleware_always_approve_tool_rule` |
+| Hosted server boundary | Standing approval does not cross `server_label`. | `test_tool_approval_middleware_standing_rules_include_hosted_server_boundary` |
+| Argument-scoped rule | Exact arguments are required; empty arguments are not tool-wide. | `test_tool_approval_middleware_always_approve_tool_with_arguments_rule`, `test_tool_approval_middleware_empty_arguments_rule_is_not_tool_wide` |
+| Provider-injected approval tool | A tool added during `before_run` defers to in-run resolution, executes once, and emits one result. | `packages/ag-ui/tests/ag_ui/test_endpoint.py::test_endpoint_agent_approval_deferred_provider_tool_executes` |
+
+### Errors, control flow, and limits
+
+| Scenario | Required invariant | Primary regression test |
+|---|---|---|
+| Rejected execution | Rejection is a normal terminal result, not an exception to the caller. | `test_unapproved_tool_execution_raises_exception` |
+| Approved tool exception | Generic and detailed error modes preserve one result and one execution. | `test_approved_function_call_with_error_without_detailed_errors`, `test_approved_function_call_with_error_with_detailed_errors` |
+| Approved validation error | Validation failure returns one result without invoking the function body. | `test_approved_function_call_with_validation_error` |
+| Approved success | Successful approved execution returns one result. | `test_approved_function_call_successful_execution` |
+| Consecutive error cap | Error threshold stops repeated failures, submits collected results, and makes only the required final no-tool model call. | `test_function_invocation_config_max_consecutive_errors`, `test_streaming_function_invocation_config_max_consecutive_errors`, `test_approval_resume_error_limit_forces_final_no_tool_response` |
+| Unknown call handling | Configured false returns an error result; configured true raises. | `test_function_invocation_config_terminate_on_unknown_calls_false`, `test_function_invocation_config_terminate_on_unknown_calls_true`, streaming equivalents |
+| Middleware termination | Normal non-approval loop stops without a second model call. | `test_terminate_loop_single_function_call`, `test_terminate_loop_multiple_function_calls_one_terminates`, `test_terminate_loop_streaming_single_function_call` |
+| Maximum iterations | No orphan calls; a final no-tool response or deterministic fallback is returned. | `test_max_iterations_limit`, `test_max_iterations_no_orphaned_function_calls`, `test_max_iterations_makes_final_toolchoice_none_call`, `test_max_iterations_blank_final_fallback_synthesizes_message`, streaming equivalents |
+| Maximum function calls | Parallel overshoot is bounded after the batch; every executed result group counts even without a `function_result`; blank final responses get fallback content. | `test_max_function_calls_limits_parallel_invocations`, `test_max_function_calls_single_calls_per_iteration`, `test_user_input_request_multiple_contents_propagate`, `test_approval_resume_user_input_counts_toward_function_call_budget`, `test_max_function_calls_blank_final_fallback_synthesizes_message`, streaming equivalent |
+| Provider tool content after an active limit | A call or approval request returned despite `tool_choice="none"` is removed in both response modes while metadata-only streaming updates remain visible. | `test_function_invocation_limit_drops_unexecutable_tool_content`, `test_streaming_function_invocation_limit_drops_unexecutable_tool_content`, `test_streaming_function_invocation_limit_preserves_metadata_after_tool_content_is_dropped` |
+| Conversation continuation | Conversation id updates between iterations and is cleared on stop where required. | `test_conversation_id_updated_in_options_between_tool_iterations`, `test_function_invocation_stop_clears_conversation_id_non_stream`, `test_streaming_function_invocation_stop_clears_conversation_id` |
+
+### History and provider serialization
+
+| Scenario | Required invariant | Primary regression test |
+|---|---|---|
+| Append-only history replay | Resolved approval wrappers do not reach a later model call; one call/result pair remains. | `packages/core/tests/core/test_harness_tool_approval.py::test_approval_resume_filters_resolved_control_items_from_file_history` |
+| Non-history provider plus session | Local history is still auto-injected for approval resume. | `packages/core/tests/core/test_agents.py::test_non_history_context_provider_still_injects_inmemory` |
+| Service-side approval decision | Stored request is skipped; current approved or rejected response is sent. | `packages/openai/tests/openai/test_openai_chat_client.py::test_prepare_messages_strips_approval_request_but_keeps_response_under_storage` |
+| OpenAI approval serialization | Approval id and decision serialize to `mcp_approval_response`. | `test_prepare_message_for_openai_with_function_approval_response`, `test_prepare_content_for_opentool_approval_response`, `test_function_approval_response_with_mcp_tool_call` |
+| OpenAI end-to-end hosted approval | Hosted request parses, response sends, and continuation completes. | `test_end_to_end_mcp_approval_flow` |
+| Stored function call/result | Service-side storage drops server-issued calls but keeps new outputs. | `test_prepare_options_with_conversation_id_strips_server_issued_items`, `test_prepare_messages_for_openai_full_conversation_with_reasoning` |
+| Stateless reasoning replay | Replay reconstructs reasoning, call, and result together; missing required reasoning fails before the request. | `test_tool_loop_store_false_replays_encrypted_reasoning_group`, `test_stateless_request_rejects_non_replayable_reasoning_bound_mcp_output`, `test_prepare_messages_for_openai_full_conversation_with_reasoning` |
+| Opaque reasoning signature replay | Provider-specific opaque reasoning metadata is captured and restored on reconstructed calls. | `packages/gemini/tests/test_gemini_client.py::test_function_call_part_captures_thought_signature_as_reasoning_content`, `test_reconstructed_function_call_replays_thought_signature_from_reasoning_content` |
+| Chat Completions approval wrappers | Framework approval wrappers are not sent as chat messages. | `packages/openai/tests/openai/test_openai_chat_completion_client.py` approval serialization tests |
+| AG-UI approval result event | Approved result emits once with content and persists in snapshot. | `packages/ag-ui/tests/ag_ui/test_approval_result_event.py::test_approval_resume_emits_tool_call_result`, `test_approval_resume_result_has_content`, `test_approval_resume_snapshot_replaces_approval_payload_with_tool_result`, `test_approval_resume_zero_updates_emits_tool_result` |
+| AG-UI rejection/mixed decision | Transport emits only the events defined for approved and rejected calls without duplicates. | `test_rejection_does_not_emit_tool_call_result`, `test_mixed_approve_reject_emits_only_approved_tool_result`, `test_resolve_approval_responses_returns_only_approved` |
+| AG-UI approval-time follow-up | The full grouped user-input pause remains in message history and emits no synthetic `TOOL_CALL_RESULT`. | `test_resolve_approval_responses_preserves_follow_up_user_input_group` |
+| AG-UI approval execution failure | A grouped executor failure becomes one deterministic terminal error result for the approved call. | `test_resolve_approval_responses_returns_failure_when_grouped_execution_raises` |
+| AG-UI no-approval path | Ordinary tool results do not gain an extra approval result event. | `test_no_approval_no_extra_tool_result` |
+| AG-UI `confirm_changes` snapshot | The synthetic confirmation result is replaced only by the result for its original function call; rejection and missing-result fallbacks contain no approval payload. | `packages/ag-ui/tests/ag_ui/test_confirm_changes_snapshot.py` |
+| AG-UI malformed `confirm_changes` metadata | Non-list tool-call metadata and malformed argument JSON are ignored without guessing a target call. | `test_confirm_changes_target_ignores_non_list_tool_calls`, `test_confirm_changes_target_rejects_malformed_arguments_json` |
+| Compaction pair integrity | Adjacent and non-adjacent function call/result groups remain atomic without pairing ambiguous or out-of-order ids. | `packages/core/tests/core/test_compaction.py::test_group_annotations_keep_tool_call_and_tool_result_atomic`, `test_group_annotations_include_reasoning_in_tool_call_group`, `test_group_annotations_pair_nonadjacent_function_result_by_call_id`, `test_group_annotations_pair_multiple_nonadjacent_results_with_declaration`, `test_group_annotations_do_not_pair_ambiguous_duplicate_call_ids` |
+
+## Remaining required coverage gap
+
+This service-owned scenario remains outside the core and adapter coverage in this specification:
+
+| Gap | Tracking |
+|---|---|
+| Service-owned `previous_response_id` continuation cannot execute a terminal approval again on a later turn. | #6851 |
+
+Do not mark this row covered by transcript-shape tests; it needs a service-continuation regression that proves the
+dispatch path is not entered again.
+
+## Minimum validation commands
+
+Run from `python/` for any core function-loop change:
+
+```bash
+uv run poe test -P core
+uv run poe syntax -P core
+uv run poe pyright -P core
+uv run poe test-typing -P core
+```
+
+Also run every affected package. Common approval-loop changes require:
+
+```bash
+uv run poe test -P openai
+uv run poe syntax -P openai
+uv run poe pyright -P openai
+uv run poe test-typing -P openai
+uv run poe test -P ag-ui
+uv run --directory packages/foundry_hosting poe test
+```
+
+Run focused regression files first while iterating, but do not substitute them for the full package commands above.
+
+## Review checklist
+
+Before accepting an update, reviewers must confirm:
+
+- the changed behavior is represented in this specification;
+- the matrix names a regression test for every affected scenario;
+- approved tools cannot execute twice;
+- rejected tools cannot execute;
+- no call or result becomes orphaned or duplicated;
+- call/result matching does not assume `call_id` is globally unique forever;
+- reasoning content or opaque signatures remain in the same logical group as the paired call/result, or replay fails
+ explicitly before sending a lossy provider request;
+- caller messages and previous responses remain immutable;
+- streaming updates and final response agree with non-streaming output;
+- history replay does not reintroduce approval authority;
+- service-side continuation sends new decisions and omits stored requests;
+- full package, syntax, source typing, and test typing checks were run.
+
+## Related issues and pull requests
+
+- #7241 / #7243 — approval-resolution result streaming
+- #7267 / #7271 and #7304 / #7326 — replayed calls and reused ids
+- #7125 / #7133 — service-side approval response serialization
+- #7043 / #7091 — provider-injected approval execution
+- #6828 / #7316 and #6909 — AG-UI result persistence and ordering
+- #6851 — duplicate side effects after approval continuation
+- #7212 / #7244 — call/result compaction integrity
+- #7045 / #7334 — post-limit streaming orphans
+- #6973 / #7110 — duplicated declaration-only streaming arguments
+- #6963 / #7095 — opaque reasoning-signature replay
+- #6074 / #7233 — reasoning-paired tool-call replay
+- #6450 / #6794 — provider message and tool-result serialization
diff --git a/python/AGENTS.md b/python/AGENTS.md
index b99f0ec7dad..70c9e7f00b3 100644
--- a/python/AGENTS.md
+++ b/python/AGENTS.md
@@ -6,6 +6,8 @@ Instructions for AI coding agents working in the Python codebase.
- [DEV_SETUP.md](DEV_SETUP.md) - Development environment setup and available poe tasks
- [CODING_STANDARD.md](CODING_STANDARD.md) - Coding standards, docstring format, and performance guidelines
- [samples/SAMPLE_GUIDELINES.md](samples/SAMPLE_GUIDELINES.md) - Sample structure and guidelines
+- [Python function-calling loop specification](../docs/specs/004-python-function-calling-loop.md) - Required
+ behavior, scenario-to-test mapping, remaining coverage gap, and extra validation for function-loop changes
**Agent Skills** (`.github/skills/`) — detailed, task-specific instructions loaded on demand:
- `python-development` — coding standards, type annotations, docstrings, logging, performance
@@ -48,6 +50,16 @@ When preparing a PR description:
Run `uv run poe` from the `python/` directory to see available commands. See [DEV_SETUP.md](DEV_SETUP.md) for detailed usage.
+## Function-Calling Loop Changes
+
+Changes to the Python function-calling loop, approval resume behavior, function-call history, provider
+serialization, or transport result handling must follow
+[the function-calling loop specification](../docs/specs/004-python-function-calling-loop.md). This area requires
+extra validation because small changes can duplicate side effects, orphan call/result pairs, replay stale approval
+authority, or make streaming and non-streaming behavior diverge. Update the specification and its scenario-to-test
+mapping whenever coverage or behavior changes. External contributors must check with the Agent Framework core team
+before picking up issues in this area.
+
## Project Structure
```
diff --git a/python/packages/ag-ui/AGENTS.md b/python/packages/ag-ui/AGENTS.md
index 4c58c746d2d..20d852b1481 100644
--- a/python/packages/ag-ui/AGENTS.md
+++ b/python/packages/ag-ui/AGENTS.md
@@ -29,6 +29,8 @@ AG-UI protocol integration for building agent UIs with the AG-UI standard.
- Multimodal user inputs support both legacy (`text`, `binary`) and draft-style (`image`, `audio`, `video`, `document`) shapes.
- Interrupted runs complete with `RUN_FINISHED.outcome.type == "interrupt"` and canonical `outcome.interrupts`; do not document or add new flows that depend on the legacy top-level `RUN_FINISHED.interrupt` field.
- `Interrupt` and `ResumeEntry` come from the `ag-ui-protocol` package (`ag_ui.core`), not from an Agent Framework-specific interrupt model.
+- Approval-time execution preserves each call's complete result group. Follow-up user-input requests remain in the
+ resumed messages, while `TOOL_CALL_RESULT` events are emitted only for terminal `function_result` contents.
- SSE keepalive is endpoint-owned transport behavior configured through
`add_agent_framework_fastapi_endpoint(keepalive_seconds=...)`. It emits SSE comments only; do not add `PING`,
`HEARTBEAT`, or `KEEPALIVE` AG-UI events, and do not add runner-level keepalive settings.
diff --git a/python/packages/ag-ui/agent_framework_ag_ui/_agent_run.py b/python/packages/ag-ui/agent_framework_ag_ui/_agent_run.py
index 84820bba408..d9de44e23c9 100644
--- a/python/packages/ag-ui/agent_framework_ag_ui/_agent_run.py
+++ b/python/packages/ag-ui/agent_framework_ag_ui/_agent_run.py
@@ -40,9 +40,10 @@
from agent_framework._tools import (
_ALREADY_APPROVED_APPROVAL_REQUEST_GROUPS_KEY, # type: ignore
_collect_approval_responses, # type: ignore
+ _get_tool_map, # type: ignore
_replace_approval_contents_with_results, # type: ignore
_TOOL_APPROVAL_STATE_KEY, # type: ignore
- _try_execute_function_calls, # type: ignore
+ _try_execute_function_call_groups, # type: ignore
normalize_function_invocation_configuration,
)
from agent_framework._types import ResponseStream
@@ -1306,10 +1307,19 @@ async def _resolve_approval_responses(
approved_responses = validated
rejected_responses = validated_rejected
- approved_function_results: list[Any] = []
+ approved_function_result_groups: list[list[Content]] = []
- # Execute approved tool calls
- if approved_responses and tools:
+ # Partition approved responses into static (execute now) and deferred (execute during run)
+ tool_map = _get_tool_map(tools) if tools else {}
+ static_approved: list[Content] = []
+
+ for approval in approved_responses:
+ tool_name = approval.function_call.name if approval.function_call else None
+ if tool_name in tool_map:
+ static_approved.append(approval)
+
+ # Execute only statically-available approved tool calls
+ if static_approved and tools:
client = getattr(agent, "client", None)
config = normalize_function_invocation_configuration(getattr(client, "function_invocation_configuration", None))
middleware_pipeline = FunctionMiddlewarePipeline(
@@ -1319,36 +1329,31 @@ async def _resolve_approval_responses(
# Filter out AG-UI-specific kwargs that should not be passed to tool execution
tool_kwargs = {k: v for k, v in run_kwargs.items() if k != "options"}
try:
- results, _ = await _try_execute_function_calls(
+ approved_function_result_groups, _ = await _try_execute_function_call_groups(
custom_args=tool_kwargs,
- attempt_idx=0,
- function_calls=approved_responses,
+ function_calls=static_approved,
tools=tools,
middleware_pipeline=middleware_pipeline,
config=config,
)
- approved_function_results = list(results)
except Exception as e:
logger.exception("Failed to execute approved tool calls; injecting error results: %s", e)
- approved_function_results = []
+ approved_function_result_groups = []
- # Build results for approved responses (used for TOOL_CALL_RESULT event emission)
+ # Normalize one group per static approval and collect only terminal results for TOOL_CALL_RESULT events.
+ # Deferred provider-injected approvals are left in messages for ToolApprovalMiddleware to process.
+ replacement_groups: list[list[Content]] = []
approved_results: list[Content] = []
- for idx, approval in enumerate(approved_responses):
- if (
- idx < len(approved_function_results)
- and getattr(approved_function_results[idx], "type", None) == "function_result"
- ):
- approved_results.append(approved_function_results[idx])
- continue
- # Get call_id from function_call if present, otherwise use approval.id
- func_call = approval.function_call
- call_id = (func_call.call_id if func_call else None) or approval.id or ""
- approved_results.append(
- Content.from_function_result(call_id=call_id, result="Error: Tool call invocation failed.")
- )
+ for idx, approval in enumerate(static_approved):
+ result_group = approved_function_result_groups[idx] if idx < len(approved_function_result_groups) else []
+ if not result_group:
+ func_call = approval.function_call
+ call_id = (func_call.call_id if func_call else None) or approval.id or ""
+ result_group = [Content.from_function_result(call_id=call_id, result="Error: Tool call invocation failed.")]
+ replacement_groups.append(result_group)
+ approved_results.extend(content for content in result_group if content.type == "function_result")
- _replace_approval_contents_with_results(messages, fcc_todo, approved_results)
+ _replace_approval_contents_with_results(messages, fcc_todo, replacement_groups)
# Post-process: Convert user messages with function_result content to proper tool messages.
# After _replace_approval_contents_with_results, approved tool calls have their results
@@ -1401,6 +1406,39 @@ def _convert_approval_results_to_tool_messages(messages: list[Message]) -> None:
messages[:] = result
+def _confirm_changes_target_call_id(
+ snapshot_messages: list[dict[str, Any]],
+ confirm_call_id: str,
+ approval_payload: Mapping[str, Any],
+) -> str | None:
+ explicit_call_id = approval_payload.get("function_call_id")
+ if explicit_call_id:
+ return str(explicit_call_id)
+
+ for snapshot_message in snapshot_messages:
+ if normalize_agui_role(snapshot_message.get("role", "")) != "assistant":
+ continue
+ tool_calls = snapshot_message.get("tool_calls") or snapshot_message.get("toolCalls")
+ if not isinstance(tool_calls, list):
+ continue
+ for tool_call in tool_calls:
+ if not isinstance(tool_call, Mapping) or str(tool_call.get("id") or "") != confirm_call_id:
+ continue
+ function = tool_call.get("function")
+ if not isinstance(function, Mapping) or function.get("name") != "confirm_changes":
+ return None
+ arguments = function.get("arguments")
+ if isinstance(arguments, str):
+ try:
+ arguments = json.loads(arguments)
+ except json.JSONDecodeError:
+ return None
+ if isinstance(arguments, Mapping) and arguments.get("function_call_id"):
+ return str(arguments["function_call_id"])
+ return None
+ return None
+
+
def _clean_resolved_approvals_from_snapshot(
snapshot_messages: list[dict[str, Any]],
resolved_messages: list[Message],
@@ -1434,9 +1472,6 @@ def _clean_resolved_approvals_from_snapshot(
)
result_by_call_id[str(content.call_id)] = result_text
- if not result_by_call_id:
- return
-
for snap_msg in snapshot_messages:
if normalize_agui_role(snap_msg.get("role", "")) != "tool":
continue
@@ -1455,12 +1490,23 @@ def _clean_resolved_approvals_from_snapshot(
# Find matching tool result by toolCallId
tool_call_id = snap_msg.get("toolCallId") or snap_msg.get("tool_call_id") or ""
replacement = result_by_call_id.get(str(tool_call_id))
- if replacement is not None:
- snap_msg["content"] = replacement
- logger.info(
- "Replaced approval payload in snapshot for tool_call_id=%s with actual result",
- tool_call_id,
+ if replacement is None:
+ target_call_id = _confirm_changes_target_call_id(
+ snapshot_messages,
+ str(tool_call_id),
+ parsed,
)
+ if target_call_id is None:
+ continue
+ if parsed.get("accepted"):
+ replacement = result_by_call_id.get(target_call_id, "Changes confirmed and applied successfully.")
+ else:
+ replacement = "Changes declined."
+ snap_msg["content"] = replacement
+ logger.info(
+ "Replaced approval payload in snapshot for tool_call_id=%s with resolved content",
+ tool_call_id,
+ )
def _snapshot_tool_call_ids(message: Mapping[str, Any]) -> list[str]:
diff --git a/python/packages/ag-ui/tests/ag_ui/test_approval_result_event.py b/python/packages/ag-ui/tests/ag_ui/test_approval_result_event.py
index 84482ce1888..361cc7ad5cf 100644
--- a/python/packages/ag-ui/tests/ag_ui/test_approval_result_event.py
+++ b/python/packages/ag-ui/tests/ag_ui/test_approval_result_event.py
@@ -524,6 +524,90 @@ async def test_resolve_approval_responses_returns_only_approved() -> None:
assert "rejected" in str(rejection_results[0].result).lower()
+async def test_resolve_approval_responses_preserves_follow_up_user_input_group() -> None:
+ """Approval-time follow-up requests stay grouped and do not emit a synthetic tool result."""
+ from agent_framework import Message
+ from agent_framework.exceptions import UserInputRequiredException
+
+ from agent_framework_ag_ui._agent_run import _resolve_approval_responses
+
+ def request_consent() -> str:
+ raise UserInputRequiredException(
+ contents=[
+ Content.from_oauth_consent_request(consent_link="https://example.com/consent-1"),
+ Content.from_oauth_consent_request(consent_link="https://example.com/consent-2"),
+ ]
+ )
+
+ consent_tool = FunctionTool(
+ name="request_consent",
+ description="Request two consent steps",
+ func=request_consent,
+ approval_mode="always_require",
+ )
+ function_call = Content.from_function_call(call_id="call_consent", name="request_consent", arguments="{}")
+ approval_request = Content.from_function_approval_request(id="approval_consent", function_call=function_call)
+ messages: list[Any] = [
+ Message(role="assistant", contents=[approval_request]),
+ Message(role="user", contents=[approval_request.to_function_approval_response(approved=True)]),
+ ]
+ agent = StubAgent(updates=[], default_options={"tools": [consent_tool]})
+
+ results = await _resolve_approval_responses(messages, [consent_tool], agent, {})
+
+ follow_up_requests = [content for message in messages for content in message.contents if content.user_input_request]
+ assert results == []
+ assert [request.consent_link for request in follow_up_requests] == [
+ "https://example.com/consent-1",
+ "https://example.com/consent-2",
+ ]
+ assert not [content for message in messages for content in message.contents if content.type == "function_result"]
+ assert any(message.role == "assistant" and message.contents == follow_up_requests for message in messages)
+
+
+async def test_resolve_approval_responses_returns_failure_when_grouped_execution_raises(
+ monkeypatch: Any,
+) -> None:
+ """A grouped-execution failure produces one deterministic result for the approved call."""
+ from agent_framework import Message
+
+ from agent_framework_ag_ui._agent_run import _resolve_approval_responses
+
+ async def fail_grouped_execution(**kwargs: Any) -> tuple[list[list[Content]], bool]:
+ del kwargs
+ raise RuntimeError("execution failed")
+
+ monkeypatch.setattr(
+ "agent_framework_ag_ui._agent_run._try_execute_function_call_groups",
+ fail_grouped_execution,
+ )
+ weather_tool = _make_weather_tool()
+ function_call = Content.from_function_call(
+ call_id="call_execution_failure",
+ name="get_weather",
+ arguments='{"city": "Seattle"}',
+ )
+ approval_request = Content.from_function_approval_request(
+ id="approval_execution_failure",
+ function_call=function_call,
+ )
+ messages: list[Any] = [
+ Message(role="assistant", contents=[approval_request]),
+ Message(role="user", contents=[approval_request.to_function_approval_response(approved=True)]),
+ ]
+ agent = StubAgent(updates=[], default_options={"tools": [weather_tool]})
+
+ results = await _resolve_approval_responses(messages, [weather_tool], agent, {})
+
+ assert len(results) == 1
+ assert results[0].type == "function_result"
+ assert results[0].call_id == "call_execution_failure"
+ assert results[0].result == "Error: Tool call invocation failed."
+ assert [
+ content.result for message in messages for content in message.contents if content.type == "function_result"
+ ] == ["Error: Tool call invocation failed."]
+
+
class TestApprovalToolResultDisplayChannel:
"""Approved tools using ``state_update(..., tool_result=...)`` must route the
display payload to the UI event while ``flow.tool_results`` still receives
diff --git a/python/packages/ag-ui/tests/ag_ui/test_confirm_changes_snapshot.py b/python/packages/ag-ui/tests/ag_ui/test_confirm_changes_snapshot.py
new file mode 100644
index 00000000000..2183aace473
--- /dev/null
+++ b/python/packages/ag-ui/tests/ag_ui/test_confirm_changes_snapshot.py
@@ -0,0 +1,156 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+from __future__ import annotations
+
+import json
+from typing import Any
+
+from agent_framework import Content, Message
+
+from agent_framework_ag_ui._agent_run import (
+ _clean_resolved_approvals_from_snapshot,
+ _confirm_changes_target_call_id,
+)
+
+
+def _confirm_snapshot(
+ *,
+ original_call_id: str,
+ confirm_call_id: str,
+ accepted: bool,
+) -> list[dict[str, Any]]:
+ return [
+ {
+ "role": "assistant",
+ "content": "",
+ "tool_calls": [
+ {
+ "id": original_call_id,
+ "type": "function",
+ "function": {"name": "apply_changes", "arguments": "{}"},
+ },
+ {
+ "id": confirm_call_id,
+ "type": "function",
+ "function": {
+ "name": "confirm_changes",
+ "arguments": json.dumps({"function_call_id": original_call_id}),
+ },
+ },
+ ],
+ },
+ {
+ "role": "tool",
+ "toolCallId": confirm_call_id,
+ "content": json.dumps({"accepted": accepted, "steps": []}),
+ },
+ ]
+
+
+def test_confirm_changes_target_ignores_non_list_tool_calls() -> None:
+ snapshot_messages: list[dict[str, Any]] = [
+ {
+ "role": "assistant",
+ "tool_calls": {"id": "call_confirm"},
+ }
+ ]
+
+ target_call_id = _confirm_changes_target_call_id(snapshot_messages, "call_confirm", {})
+
+ assert target_call_id is None
+
+
+def test_confirm_changes_target_rejects_malformed_arguments_json() -> None:
+ snapshot_messages: list[dict[str, Any]] = [
+ {
+ "role": "assistant",
+ "tool_calls": [
+ {
+ "id": "call_confirm",
+ "type": "function",
+ "function": {
+ "name": "confirm_changes",
+ "arguments": "{not-json",
+ },
+ }
+ ],
+ }
+ ]
+
+ target_call_id = _confirm_changes_target_call_id(snapshot_messages, "call_confirm", {})
+
+ assert target_call_id is None
+
+
+def test_confirm_changes_snapshot_uses_original_call_id_with_multiple_results() -> None:
+ snapshot_messages = _confirm_snapshot(
+ original_call_id="call_target",
+ confirm_call_id="call_confirm",
+ accepted=True,
+ )
+ resolved_messages = [
+ Message(
+ role="tool",
+ contents=[
+ Content.from_function_result(call_id="call_old", result="old result"),
+ Content.from_function_result(call_id="call_target", result="target result"),
+ ],
+ )
+ ]
+
+ _clean_resolved_approvals_from_snapshot(snapshot_messages, resolved_messages)
+
+ assert snapshot_messages[1]["content"] == "target result"
+
+
+def test_confirm_changes_snapshot_accepts_explicit_payload_call_id() -> None:
+ snapshot_messages: list[dict[str, Any]] = [
+ {
+ "role": "tool",
+ "toolCallId": "call_confirm",
+ "content": json.dumps({"accepted": True, "function_call_id": "call_target"}),
+ }
+ ]
+ resolved_messages = [
+ Message(
+ role="tool",
+ contents=[Content.from_function_result(call_id="call_target", result="target result")],
+ )
+ ]
+
+ _clean_resolved_approvals_from_snapshot(snapshot_messages, resolved_messages)
+
+ assert snapshot_messages[0]["content"] == "target result"
+
+
+def test_confirm_changes_snapshot_cleans_rejection_without_results() -> None:
+ snapshot_messages = _confirm_snapshot(
+ original_call_id="call_target",
+ confirm_call_id="call_confirm",
+ accepted=False,
+ )
+
+ _clean_resolved_approvals_from_snapshot(snapshot_messages, [])
+
+ assert snapshot_messages[1]["content"] == "Changes declined."
+
+
+def test_confirm_changes_snapshot_does_not_join_unrelated_results_when_target_is_missing() -> None:
+ snapshot_messages = _confirm_snapshot(
+ original_call_id="call_target",
+ confirm_call_id="call_confirm",
+ accepted=True,
+ )
+ resolved_messages = [
+ Message(
+ role="tool",
+ contents=[
+ Content.from_function_result(call_id="call_old_1", result="old result 1"),
+ Content.from_function_result(call_id="call_old_2", result="old result 2"),
+ ],
+ )
+ ]
+
+ _clean_resolved_approvals_from_snapshot(snapshot_messages, resolved_messages)
+
+ assert snapshot_messages[1]["content"] == "Changes confirmed and applied successfully."
diff --git a/python/packages/ag-ui/tests/ag_ui/test_endpoint.py b/python/packages/ag-ui/tests/ag_ui/test_endpoint.py
index 303a307d91e..8c9ae87b064 100644
--- a/python/packages/ag-ui/tests/ag_ui/test_endpoint.py
+++ b/python/packages/ag-ui/tests/ag_ui/test_endpoint.py
@@ -5233,3 +5233,199 @@ def resolve_scope(request: AGUIRequest) -> str:
runner._resolve_workflow("thread-1", "tenant-b") # pyright: ignore[reportPrivateUsage]
is created_workflows[1]
)
+
+
+async def test_endpoint_agent_approval_defers_provider_injected_tools() -> None:
+ """Approving provider-injected tools should not produce rejection errors.
+
+ When approval responses include tools not in the static tool map,
+ they should be deferred (not executed by transport) rather than
+ producing "Tool call invocation was rejected" errors.
+
+ This prevents the scenario where approval mode loops with tool invocation failures.
+ Fixes issue #7043.
+ """
+ executed_tools: list[str] = []
+
+ def static_tool() -> str:
+ executed_tools.append("static")
+ return "static result"
+
+ # Create approval responses: one for static tool, one for provider tool (not in static list)
+ static_approval = Content.from_function_approval_response(
+ id="static_id",
+ approved=True,
+ function_call=Content.from_function_call(
+ call_id="call_static",
+ name="static_tool",
+ arguments="{}",
+ ),
+ )
+ provider_approval = Content.from_function_approval_response(
+ id="provider_id",
+ approved=True,
+ function_call=Content.from_function_call(
+ call_id="call_provider",
+ name="provider_file_access", # Not in agent's tool list
+ arguments="{}",
+ ),
+ )
+
+ # Stub agent that returns the approval responses embedded in a message
+ agent = StubAgent(
+ updates=[
+ AgentResponseUpdate(contents=[static_approval, provider_approval], role="user"),
+ ]
+ )
+
+ app = FastAPI()
+ wrapped_agent = AgentFrameworkAgent(agent=agent, require_confirmation=False)
+ add_agent_framework_fastapi_endpoint(app, wrapped_agent, path="/approval")
+ client = TestClient(app)
+
+ # Submit approval responses (static and provider-injected)
+ response = client.post(
+ "/approval",
+ json={
+ "runId": "run-approval",
+ "threadId": "thread-approval",
+ "messages": [
+ {
+ "role": "user",
+ "content": [
+ {
+ "type": "function_approval_response",
+ "id": "static_id",
+ "approved": True,
+ "function_call": {
+ "call_id": "call_static",
+ "name": "static_tool",
+ "arguments": "{}",
+ },
+ },
+ {
+ "type": "function_approval_response",
+ "id": "provider_id",
+ "approved": True,
+ "function_call": {
+ "call_id": "call_provider",
+ "name": "provider_file_access",
+ "arguments": "{}",
+ },
+ },
+ ],
+ }
+ ],
+ },
+ )
+
+ assert response.status_code == 200
+ events = _decode_sse_events(response)
+ response_text = json.dumps(events)
+
+ # Verify no rejection error for provider tool
+ assert "Tool call invocation was rejected" not in response_text, (
+ "Deferred provider tool should not produce rejection error"
+ )
+
+
+async def test_endpoint_agent_approval_deferred_provider_tool_executes(streaming_chat_client_stub) -> None:
+ """A provider-injected tool approved via AG-UI executes in-run instead of being rejected.
+
+ Regression for #7043. A tool registered by a context provider during ``before_run`` is
+ absent from the transport's static tool map, so ``_resolve_approval_responses`` must defer
+ it (not execute or reject it) and leave it for the in-run ``ToolApprovalMiddleware`` to run.
+ This drives the full pause -> approve -> resume flow with a real provider-injected tool and
+ asserts the approved side effect actually happens without any rejection/failure result.
+
+ The deferred tool result must still be returned to AG-UI exactly once.
+ """
+ side_effects: list[str] = []
+ state = {"phase": "pause"}
+
+ def provider_write() -> str:
+ side_effects.append("wrote")
+ return "wrote to disk"
+
+ provider_tool = FunctionTool(
+ name="provider_write",
+ description="Write to disk (provider-injected)",
+ func=provider_write,
+ approval_mode="always_require",
+ )
+
+ class ToolInjectingProvider(ContextProvider):
+ """Registers a tool during before_run, mimicking FileAccessProvider/CodeInterpreterProvider."""
+
+ async def before_run(self, *, agent, session, context, state) -> None: # type: ignore[override] # pyrefly: ignore # ty: ignore
+ del agent, session, state
+ context.extend_tools(self.source_id, [provider_tool])
+
+ async def stream_fn(
+ messages: list[Message],
+ options: dict[str, Any],
+ **kwargs: Any,
+ ) -> AsyncIterator[ChatResponseUpdate]:
+ del options, kwargs
+ if state["phase"] == "pause":
+ yield ChatResponseUpdate(
+ contents=[Content.from_function_call(call_id="call_provider", name="provider_write", arguments="{}")],
+ role="assistant",
+ )
+ return
+ yield ChatResponseUpdate(contents=[Content.from_text(text="Done.")], role="assistant")
+
+ # provider_write is intentionally NOT in the static tools list -- it is only injected via before_run.
+ agent = Agent(
+ name="test_agent",
+ instructions="Test",
+ client=streaming_chat_client_stub(stream_fn),
+ tools=[],
+ middleware=[ToolApprovalMiddleware()],
+ context_providers=[ToolInjectingProvider(source_id="tool_injector")],
+ )
+ app = FastAPI()
+ add_agent_framework_fastapi_endpoint(
+ app,
+ AgentFrameworkAgent(agent=agent, require_confirmation=False),
+ path="/approval",
+ )
+ client = TestClient(app)
+
+ # Pause: the harness surfaces the provider-injected tool for approval, nothing executes yet.
+ pause_response = client.post(
+ "/approval",
+ json={
+ "runId": "run-pause",
+ "threadId": "thread-provider",
+ "messages": [{"role": "user", "content": "Write something"}],
+ },
+ )
+ assert pause_response.status_code == 200
+ pause_finished = [event for event in _decode_sse_events(pause_response) if event.get("type") == "RUN_FINISHED"]
+ assert [interrupt["id"] for interrupt in _run_finished_interrupts(pause_finished[-1])] == ["call_provider"]
+ assert side_effects == []
+
+ # Resume with approval: the deferred provider tool runs during agent.run.
+ state["phase"] = "resume"
+ resume_response = client.post(
+ "/approval",
+ json={
+ "runId": "run-resume",
+ "threadId": "thread-provider",
+ "messages": [],
+ "resume": [{"interruptId": "call_provider", "status": "resolved", "payload": {"accepted": True}}],
+ },
+ )
+ assert resume_response.status_code == 200
+ resume_events = _decode_sse_events(resume_response)
+ resume_text = json.dumps(resume_events)
+
+ # The approved provider tool actually executed -- its side effect fired.
+ assert side_effects == ["wrote"]
+ tool_results = [event for event in resume_events if event.get("type") == "TOOL_CALL_RESULT"]
+ assert [(event["toolCallId"], event["content"]) for event in tool_results] == [("call_provider", "wrote to disk")]
+ # And it was neither rejected nor reported as a transport failure (the #7043 bug).
+ assert "Tool call invocation was rejected" not in resume_text
+ assert "Tool call invocation failed" not in resume_text
+ assert not [event for event in resume_events if event.get("type") == "RUN_ERROR"]
diff --git a/python/packages/core/AGENTS.md b/python/packages/core/AGENTS.md
index 7ebfaa59f44..4f745cb26ee 100644
--- a/python/packages/core/AGENTS.md
+++ b/python/packages/core/AGENTS.md
@@ -145,6 +145,26 @@ agent_framework/
available, approval requests for known non-approval-required tools are treated as already approved, hidden, stored
in session state keyed to the visible approval request ids from that batch, and reinjected only when that visible
approval flow resumes.
+- Approval resume is an immutable response boundary: the function invocation layer normalizes a private copy of
+ caller messages, returns approved and rejected terminal results in the resumed response (and stream) before any
+ final assistant message, and does not mutate the caller's approval `Message` or the earlier approval-request
+ response.
+- Approval/result correlation is occurrence-aware. A `call_id` may be reused after a completed round, so approval
+ normalization matches ordered call occurrences and consumes approved results per occurrence rather than using one
+ global result per `call_id`. All contents produced by one execution remain one result group and are consumed
+ together, including multiple user-input requests.
+- Function-call budget accounting counts one unit per executed result group, not per emitted `function_result`, so
+ executions that pause for user input still consume `max_function_calls`.
+- Declaration-only streamed calls emit their arguments only from the provider stream. The function layer sends a
+ metadata-only follow-up (`arguments=None`) so `id` and `user_input_request` survive final aggregation without
+ duplicating arguments.
+- `function_approval_request` and `function_approval_response` are control-plane contents. History providers may
+ retain them in their backing store for audit, but the base `HistoryProvider.before_run` filters them from later
+ model replay; the normalized model transcript contains the function call and one terminal function result.
+ Providers configured with `load_messages=False` do not replay history, so this filter is intentionally not invoked.
+- Reasoning content or opaque reasoning metadata bound to a function call is part of the same logical group as the
+ call and terminal result. Function-loop replay and compaction must preserve that group atomically; adapters should
+ fail before a stateless request when required reasoning cannot be reconstructed.
### Agent Loop (`_harness/_loop.py`)
- **`AgentLoopMiddleware`** - `AgentMiddleware` that re-runs an agent in a loop by calling `call_next()` repeatedly (the pipeline re-reads `context.messages` each time). One configurable class covers two patterns: a required user `should_continue` predicate (sync or async, the first positional/keyword arg), and a chat-client judge built via the `.with_judge(...)` factory (a second chat client decides whether the original request was answered; loops while it is *not*, using a `JudgeVerdict` structured-output response — internally just an async `should_continue` predicate). The constructor covers the predicate pattern directly; only the judge has a convenience classmethod factory (`.with_judge(judge_client, ...)`) that forwards to `__init__`. Supports both streaming and non-streaming runs. By default a non-streaming run returns an aggregated `AgentResponse` containing every iteration's messages plus the injected `next_message` "nudge" messages (as `user` messages); set `return_final_only=True` to return only the last iteration's response. Streaming runs always yield each iteration's updates and emit the injected nudge messages as `user` updates between iterations (the `return_final_only` flag has no effect on streaming, and the final response reflects the last iteration; `MiddlewareTermination` is handled cleanly). `should_continue` is required; other constructor args are optional: `max_iterations` (safety cap; defaults to `DEFAULT_MAX_ITERATIONS`=10, explicit `None`→unbounded, positive int caps; `.with_judge` uses `DEFAULT_JUDGE_MAX_ITERATIONS`=5 as its default), `next_message` (defaults to a short "continue" nudge), `return_final_only`, and `additional_instructions` (an extra `system` message injected ahead of the input before the agent runs — becomes part of the original messages so it survives `fresh_context` resets and persists via a session). The judge is configured only through `.with_judge` (`judge_client`/`instructions`/`criteria`), not the constructor, and its `reasoning` is fed back to the agent as the next iteration's input; the judge forwards the original request messages and the agent's latest response messages verbatim so multi-modal content is preserved. `criteria` (a `list[str]`) is both injected as the agent's `additional_instructions` and rendered into the judge instructions wherever the `{{criteria}}` placeholder (`CRITERIA_PLACEHOLDER`) appears (`DEFAULT_JUDGE_INSTRUCTIONS` ends with it; custom `instructions` may include it, and it is stripped when no criteria are given). The `should_continue`/`next_message` callables are invoked with keyword args (`iteration`, `last_result`, `messages`, `original_messages`, `session`, `agent`, `progress`, `feedback`) and may be sync or async; declare only what you need plus `**kwargs`. `should_continue` may return a plain `bool` or a `(bool, str | None)` tuple whose second item is feedback surfaced to `next_message`/`record_feedback` via the `feedback` kwarg (the judge uses this to relay its `reasoning`). Stop precedence per iteration is `max_iterations` → `should_continue`, evaluated before `record_feedback` so the feedback is available to it.
diff --git a/python/packages/core/agent_framework/_compaction.py b/python/packages/core/agent_framework/_compaction.py
index 59abb10a468..485d2c93ce9 100644
--- a/python/packages/core/agent_framework/_compaction.py
+++ b/python/packages/core/agent_framework/_compaction.py
@@ -101,6 +101,18 @@ def _is_reasoning_only_assistant(message: Message) -> bool:
return all(content.type == "text_reasoning" for content in message.contents)
+def _function_call_ids(message: Message) -> set[str]:
+ if message.role != "assistant":
+ return set()
+ return {content.call_id for content in message.contents if content.type == "function_call" and content.call_id}
+
+
+def _function_result_ids(message: Message) -> set[str]:
+ if message.role != "tool":
+ return set()
+ return {content.call_id for content in message.contents if content.type == "function_result" and content.call_id}
+
+
def _ensure_message_ids(
messages: list[Message], *, id_offset: int = 0, reserved_ids: Iterable[str] | None = None
) -> None:
@@ -126,6 +138,68 @@ def _group_id_for(message: Message, group_index: int) -> str:
return f"group_index_{group_index}"
+def _link_function_call_result_spans(messages: Sequence[Message], spans: list[dict[str, Any]]) -> None:
+ """Link non-adjacent function results to their unique declaration group."""
+ if len(spans) < 2:
+ return
+
+ declaration_spans: dict[str, set[int]] = {}
+ result_ids_by_span: list[set[str]] = []
+ for span_index, span in enumerate(spans):
+ start_index = int(span["start_index"])
+ end_index = int(span["end_index"])
+ declared_ids: set[str] = set()
+ result_ids: set[str] = set()
+ for message in messages[start_index : end_index + 1]:
+ declared_ids.update(_function_call_ids(message))
+ result_ids.update(_function_result_ids(message))
+ for call_id in declared_ids:
+ declaration_spans.setdefault(call_id, set()).add(span_index)
+ result_ids_by_span.append(result_ids)
+ if not declaration_spans or not any(result_ids_by_span):
+ return
+
+ parents = list(range(len(spans)))
+
+ def find(span_index: int) -> int:
+ while parents[span_index] != span_index:
+ parents[span_index] = parents[parents[span_index]]
+ span_index = parents[span_index]
+ return span_index
+
+ def union(left: int, right: int) -> bool:
+ left_root = find(left)
+ right_root = find(right)
+ if left_root == right_root:
+ return False
+ earlier_root = min(left_root, right_root)
+ later_root = max(left_root, right_root)
+ parents[later_root] = earlier_root
+ return True
+
+ linked = False
+ for result_span_index, result_ids in enumerate(result_ids_by_span):
+ for call_id in result_ids:
+ matches = declaration_spans.get(call_id)
+ if matches is None or len(matches) != 1:
+ continue
+ declaration_span_index = next(iter(matches))
+ if declaration_span_index < result_span_index and union(result_span_index, declaration_span_index):
+ linked = True
+ if not linked:
+ return
+
+ has_reasoning_by_root: dict[int, bool] = {}
+ for span_index, span in enumerate(spans):
+ root = find(span_index)
+ has_reasoning_by_root[root] = has_reasoning_by_root.get(root, False) or bool(span["has_reasoning"])
+
+ for span_index, span in enumerate(spans):
+ root = find(span_index)
+ span["group_id"] = spans[root]["group_id"]
+ span["has_reasoning"] = has_reasoning_by_root[root]
+
+
def group_messages(
messages: list[Message], *, id_offset: int = 0, reserved_ids: Iterable[str] | None = None
) -> list[dict[str, Any]]:
@@ -145,6 +219,7 @@ def group_messages(
Returns:
Ordered list of lightweight span dicts with keys:
``group_id``, ``kind``, ``start_index``, ``end_index``, ``has_reasoning``.
+ Non-contiguous function-call declaration and result spans share a group id.
"""
_ensure_message_ids(messages, id_offset=id_offset, reserved_ids=reserved_ids)
spans: list[dict[str, Any]] = []
@@ -249,6 +324,7 @@ def group_messages(
i += 1
group_index += 1
+ _link_function_call_result_spans(messages, spans)
return spans
@@ -434,6 +510,38 @@ def _reannotation_start(messages: Sequence[Message], index: int) -> int:
return previous_index
+def _function_pair_reannotation_start(messages: Sequence[Message], start_index: int) -> int:
+ result_ids: set[str] = set()
+ for message in messages[start_index:]:
+ result_ids.update(_function_result_ids(message))
+ if not result_ids:
+ return start_index
+
+ declaration_indices: dict[str, set[int]] = {}
+ for index, message in enumerate(messages):
+ for call_id in _function_call_ids(message):
+ declaration_indices.setdefault(call_id, set()).add(index)
+
+ matching_indices: list[int] = []
+ for call_id in result_ids:
+ indices = declaration_indices.get(call_id)
+ if indices is None or len(indices) != 1:
+ continue
+ declaration_index = next(iter(indices))
+ if declaration_index < start_index:
+ matching_indices.append(declaration_index)
+ if not matching_indices:
+ return start_index
+
+ earliest_index = min(matching_indices)
+ declaration_group_id = _group_id(messages[earliest_index])
+ if declaration_group_id is None:
+ return earliest_index
+ while earliest_index > 0 and _group_id(messages[earliest_index - 1]) == declaration_group_id:
+ earliest_index -= 1
+ return earliest_index
+
+
def annotate_message_groups(
messages: list[Message],
*,
@@ -444,7 +552,8 @@ def annotate_message_groups(
"""Annotate message groups while reusing existing annotations when possible.
By default, the function re-annotates only the suffix that contains new
- messages and keeps previously annotated prefixes untouched. When a
+ messages and keeps previously annotated prefixes untouched. A newly added
+ function result expands that suffix back to its unique declaration. When a
``tokenizer`` is provided, token-count annotations are also populated
incrementally.
"""
@@ -466,18 +575,29 @@ def annotate_message_groups(
start_index = min(candidate_starts)
start_index = _reannotation_start(messages, start_index)
+ start_index = _function_pair_reannotation_start(messages, start_index)
- # Continue group indices from the preserved prefix when only re-annotating a suffix.
- group_index_offset = 0
- if start_index > 0:
- previous_group_index = _group_index(messages[start_index - 1])
- if previous_group_index is not None:
- group_index_offset = previous_group_index + 1
+ # Linked groups can be non-contiguous, so the last prefix message does not
+ # necessarily carry the highest group index.
+ prefix_group_indices = [
+ group_index for message in messages[:start_index] if (group_index := _group_index(message)) is not None
+ ]
+ group_index_offset = max(prefix_group_indices, default=-1) + 1
reserved_ids = {message.message_id for message in messages[:start_index] if message.message_id}
spans = group_messages(messages[start_index:], id_offset=start_index, reserved_ids=reserved_ids)
- for span_index, span in enumerate(spans):
+ span_counts_by_group_id: dict[str, int] = {}
+ for span in spans:
+ group_id = str(span["group_id"])
+ span_counts_by_group_id[group_id] = span_counts_by_group_id.get(group_id, 0) + 1
+ linked_group_ids = {group_id for group_id, count in span_counts_by_group_id.items() if count > 1}
+
+ group_indices: dict[str, int] = {}
+ grouped_messages: dict[str, list[Message]] = {}
+ for span in spans:
group_id = str(span["group_id"])
+ if group_id not in group_indices:
+ group_indices[group_id] = group_index_offset + len(group_indices)
kind = _coerce_group_kind(span["kind"])
if kind is None:
raise ValueError(f"Unexpected group kind in span: {span['kind']}")
@@ -490,12 +610,19 @@ def annotate_message_groups(
message,
group_id=group_id,
kind=kind,
- index=group_index_offset + span_index,
+ index=group_indices[group_id],
has_reasoning=has_reasoning,
)
message.additional_properties.setdefault(EXCLUDED_KEY, False)
+ if group_id in linked_group_ids:
+ grouped_messages.setdefault(group_id, []).append(message)
if tokenizer is not None and _token_count(message) is None:
_write_token_count(message, tokenizer.count_tokens(_serialize_message(message)))
+
+ for group in grouped_messages.values():
+ if any(not message.additional_properties.get(EXCLUDED_KEY, False) for message in group):
+ for message in group:
+ message.additional_properties[EXCLUDED_KEY] = False
return _ordered_group_ids_from_annotations(messages)
diff --git a/python/packages/core/agent_framework/_sessions.py b/python/packages/core/agent_framework/_sessions.py
index 9a6dab9a7c1..7836cfdfe1f 100644
--- a/python/packages/core/agent_framework/_sessions.py
+++ b/python/packages/core/agent_framework/_sessions.py
@@ -477,6 +477,26 @@ async def after_run(
"""
+def _filter_approval_control_messages(messages: Sequence[Message]) -> list[Message]:
+ """Remove approval request/response wrappers from history before model replay."""
+ filtered_messages: list[Message] = []
+ for message in messages:
+ filtered_contents = [
+ content
+ for content in message.contents
+ if content.type not in {"function_approval_request", "function_approval_response"}
+ ]
+ if not filtered_contents:
+ continue
+ if len(filtered_contents) == len(message.contents):
+ filtered_messages.append(message)
+ continue
+ filtered_message = copy.copy(message)
+ filtered_message.contents = filtered_contents
+ filtered_messages.append(filtered_message)
+ return filtered_messages
+
+
class HistoryProvider(ContextProvider):
"""Base class for conversation history storage providers.
@@ -579,7 +599,7 @@ async def before_run(
state: dict[str, Any],
) -> None:
"""Load history into context. Skipped by the agent when load_messages=False."""
- history = await self.get_messages(context.session_id, state=state)
+ history = _filter_approval_control_messages(await self.get_messages(context.session_id, state=state))
context.extend_messages(self, history)
async def after_run(
diff --git a/python/packages/core/agent_framework/_tools.py b/python/packages/core/agent_framework/_tools.py
index 5d783cb0368..7437795b44e 100644
--- a/python/packages/core/agent_framework/_tools.py
+++ b/python/packages/core/agent_framework/_tools.py
@@ -4,11 +4,13 @@
import asyncio
import contextvars
+import copy
import inspect
import json
import logging
import sys
import typing
+from collections import deque
from collections.abc import (
AsyncIterable,
Awaitable,
@@ -18,6 +20,7 @@
Sequence,
)
from contextlib import suppress
+from dataclasses import dataclass
from functools import partial, wraps
from time import perf_counter, time_ns
from typing import (
@@ -98,6 +101,7 @@
"Function invocation limit reached before a final answer could be produced."
)
_USER_VISIBLE_CONTENT_TYPES: Final[set[str]] = {"data", "uri", "error", "hosted_file", "hosted_vector_store"}
+_UNEXECUTABLE_TOOL_CONTENT_TYPES: Final[set[str]] = {"function_call", "function_approval_request"}
ApprovalMode: TypeAlias = Literal["always_require", "never_require"]
ChatClientT = TypeVar("ChatClientT", bound="SupportsChatGetResponse[Any]")
ResponseModelBoundT = TypeVar("ResponseModelBoundT", bound=BaseModel)
@@ -1399,6 +1403,31 @@ def normalize_function_invocation_configuration(
return normalized
+def _function_execution_error_result(
+ function_call: Content,
+ tool_name: str,
+ exception: Exception,
+ config: FunctionInvocationConfiguration,
+) -> Content:
+ from ._types import Content
+
+ logger.warning(
+ "Function '%s' raised an exception; returning an error result to the model. "
+ "Set include_detailed_errors=True for the full detail. Exception: %r",
+ tool_name,
+ exception,
+ )
+ message = "Error: Function failed."
+ if config.get("include_detailed_errors", False):
+ message = f"{message} Exception: {exception}"
+ return Content.from_function_result(
+ call_id=function_call.call_id, # type: ignore[arg-type]
+ result=message,
+ exception=str(exception),
+ additional_properties=function_call.additional_properties,
+ )
+
+
async def _auto_invoke_function(
function_call_content: Content,
custom_args: dict[str, Any] | None = None,
@@ -1406,8 +1435,6 @@ async def _auto_invoke_function(
config: FunctionInvocationConfiguration,
tool_map: dict[str, FunctionTool],
invocation_session: AgentSession | None = None,
- sequence_index: int | None = None,
- request_index: int | None = None,
middleware_pipeline: FunctionMiddlewarePipeline | None = None,
live_tools: list[ToolTypes] | None = None,
) -> Content:
@@ -1421,8 +1448,6 @@ async def _auto_invoke_function(
config: The function invocation configuration.
tool_map: A mapping of tool names to FunctionTool instances.
invocation_session: The agent session for this invocation, if any.
- sequence_index: The index of the function call in the sequence.
- request_index: The index of the request iteration.
middleware_pipeline: Optional middleware pipeline to apply during execution.
live_tools: The live, mutable tools list for the current agent run, exposed on
the FunctionInvocationContext so tools can add/remove tools at runtime.
@@ -1441,7 +1466,6 @@ async def _auto_invoke_function(
# this function is called. This function only handles the actual execution of approved,
# non-declaration-only functions.
- tool: FunctionTool | None = None
approval_response: Content | None = None
if function_call_content.type == "function_call":
@@ -1535,19 +1559,7 @@ async def _auto_invoke_function(
except UserInputRequiredException:
raise
except Exception as exc:
- logger.warning(
- f"Function '{tool.name}' raised an exception; returning an error result to the "
- f"model. Set include_detailed_errors=True for the full detail. Exception: {exc!r}"
- )
- message = "Error: Function failed."
- if config.get("include_detailed_errors", False):
- message = f"{message} Exception: {exc}"
- return Content.from_function_result(
- call_id=function_call_content.call_id, # type: ignore[arg-type]
- result=message,
- exception=str(exc),
- additional_properties=function_call_content.additional_properties,
- )
+ return _function_execution_error_result(function_call_content, tool.name, exc, config)
# Execute through middleware pipeline if available
middleware_context = FunctionInvocationContext(
function=tool,
@@ -1611,49 +1623,94 @@ async def final_function_handler(context_obj: Any) -> Any:
except UserInputRequiredException:
raise
except Exception as exc:
- logger.warning(
- f"Function '{tool.name}' raised an exception; returning an error result to the "
- f"model. Set include_detailed_errors=True for the full detail. Exception: {exc!r}"
- )
- message = "Error: Function failed."
- if config.get("include_detailed_errors", False):
- message = f"{message} Exception: {exc}"
- return Content.from_function_result(
- call_id=function_call_content.call_id, # type: ignore[arg-type]
- result=message,
- exception=str(exc),
- additional_properties=function_call_content.additional_properties,
- )
+ return _function_execution_error_result(function_call_content, tool.name, exc, config)
def _get_tool_map(
tools: ToolTypes | Callable[..., Any] | Sequence[ToolTypes | Callable[..., Any]],
) -> dict[str, FunctionTool]:
- tool_list: dict[str, FunctionTool] = {}
- for tool_item in _ensure_unique_tool_names(tools):
- if isinstance(tool_item, FunctionTool):
- tool_list[tool_item.name] = tool_item
- return tool_list
+ return {
+ tool_item.name: tool_item
+ for tool_item in _ensure_unique_tool_names(tools)
+ if isinstance(tool_item, FunctionTool)
+ }
def _is_actionable_function_call(content: Content) -> bool:
return content.type == "function_call" and not content.informational_only
-async def _try_execute_function_calls(
+def _underlying_function_call(content: Content) -> Content:
+ if content.type == "function_approval_response" and content.function_call is not None:
+ return content.function_call
+ return content
+
+
+async def _execute_single_function_call(
+ function_call: Content,
+ *,
+ custom_args: dict[str, Any],
+ config: FunctionInvocationConfiguration,
+ tool_map: dict[str, FunctionTool],
+ invocation_session: AgentSession | None,
+ middleware_pipeline: FunctionMiddlewarePipeline | None,
+ live_tools: list[ToolTypes] | None,
+) -> tuple[list[Content], bool]:
+ from ._middleware import MiddlewareTermination
+ from ._types import Content
+
+ try:
+ result = await _auto_invoke_function(
+ function_call_content=function_call,
+ custom_args=custom_args,
+ tool_map=tool_map,
+ invocation_session=invocation_session,
+ middleware_pipeline=middleware_pipeline,
+ config=config,
+ live_tools=live_tools,
+ )
+ return [result], False
+ except MiddlewareTermination as exc:
+ if isinstance(exc.result, Content):
+ return [exc.result], True
+ source_function_call = _underlying_function_call(function_call)
+ return [
+ Content.from_function_result(
+ call_id=source_function_call.call_id, # type: ignore[arg-type]
+ result=exc.result,
+ )
+ ], True
+ except UserInputRequiredException as exc:
+ source_function_call = _underlying_function_call(function_call)
+ call_id = source_function_call.call_id
+ propagated_contents = [item for item in exc.contents if isinstance(item, Content)] if exc.contents else []
+ for item in propagated_contents:
+ item.call_id = call_id
+ if not item.id:
+ item.id = call_id
+ if propagated_contents:
+ return propagated_contents, False
+ return [
+ Content.from_function_result(
+ call_id=call_id, # type: ignore[arg-type]
+ result="Tool requires user input but no request details were provided.",
+ exception="UserInputRequiredException",
+ )
+ ], False
+
+
+async def _try_execute_function_call_groups(
custom_args: dict[str, Any],
- attempt_idx: int,
function_calls: Sequence[Content],
tools: ToolTypes | Callable[..., Any] | Sequence[ToolTypes | Callable[..., Any]],
config: FunctionInvocationConfiguration,
invocation_session: AgentSession | None = None,
- middleware_pipeline: Any = None,
-) -> tuple[Sequence[Content], bool]:
- """Execute multiple function calls concurrently.
+ middleware_pipeline: FunctionMiddlewarePipeline | None = None,
+) -> tuple[list[list[Content]], bool]:
+ """Execute multiple function calls concurrently while preserving per-call result groups.
Args:
custom_args: Custom arguments to pass to each function.
- attempt_idx: The index of the current attempt iteration.
function_calls: A sequence of FunctionCallContent to execute.
tools: The tools available for execution.
config: Configuration for function invocation.
@@ -1662,10 +1719,8 @@ async def _try_execute_function_calls(
Returns:
A tuple of:
- - A list of Content containing the results of each function call,
- or the approval requests if any function requires approval,
- or the original function calls if any are declaration only.
- - Always False; termination via middleware is no longer supported.
+ - One ordered content group per function call.
+ - True when function middleware requested loop termination.
"""
from ._types import Content
@@ -1675,67 +1730,64 @@ async def _try_execute_function_calls(
if function_call.type == "function_approval_response" or _is_actionable_function_call(function_call)
]
if not function_calls:
- return ([], False)
+ return [], False
tool_map = _get_tool_map(tools)
# The live tools list (when tools is the run-local list) is exposed on the
# FunctionInvocationContext so tools can add/remove tools during the run.
live_tools: list[ToolTypes] | None = cast("list[ToolTypes]", tools) if isinstance(tools, list) else None
- approval_tools = {tool_name for tool_name, tool in tool_map.items() if tool.approval_mode == "always_require"}
+ approval_tool_names = {tool_name for tool_name, tool in tool_map.items() if tool.approval_mode == "always_require"}
logger.debug(
"_try_execute_function_calls: tool_map keys=%s, approval_tools=%s",
list(tool_map.keys()),
- approval_tools,
+ approval_tool_names,
)
- declaration_only = {tool_name for tool_name, tool in tool_map.items() if tool.declaration_only}
- configured_additional_tools = config.get("additional_tools") or []
- additional_tool_names = {tool.name for tool in configured_additional_tools}
- # check if any are calling functions that need approval
- # if so, we return approval request for all
- approval_needed = False
- declaration_only_flag = False
- for fcc in function_calls:
- fcc_name = getattr(fcc, "name", None)
+ declaration_only_tool_names = {tool_name for tool_name, tool in tool_map.items() if tool.declaration_only}
+ additional_tool_names = {tool.name for tool in config.get("additional_tools") or []}
+ actionable_calls = [
+ function_call for function_call in function_calls if _is_actionable_function_call(function_call)
+ ]
+ requires_approval = False
+ has_declaration_only_call = False
+ # A user-input pause takes precedence over unknown-call termination in mixed batches.
+ for function_call in actionable_calls:
+ function_name = function_call.name
logger.debug(
"Checking function call: type=%s, name=%s, in approval_tools=%s",
- fcc.type,
- fcc_name,
- fcc_name in approval_tools,
+ function_call.type,
+ function_name,
+ function_name in approval_tool_names,
)
- if _is_actionable_function_call(fcc) and fcc.name in approval_tools:
- logger.debug("Approval needed for function: %s", fcc.name)
- approval_needed = True
+ if function_name in approval_tool_names:
+ logger.debug("Approval needed for function: %s", function_name)
+ requires_approval = True
break
- if _is_actionable_function_call(fcc) and (fcc.name in declaration_only or fcc.name in additional_tool_names):
- declaration_only_flag = True
+ if function_name in declaration_only_tool_names or function_name in additional_tool_names:
+ has_declaration_only_call = True
break
- if (
- config.get("terminate_on_unknown_calls", False)
- and _is_actionable_function_call(fcc)
- and fcc.name not in tool_map
- ):
- raise KeyError(f'Error: Requested function "{fcc.name}" not found.')
- if approval_needed:
+ if config.get("terminate_on_unknown_calls", False) and function_name not in tool_map:
+ raise KeyError(f'Error: Requested function "{function_name}" not found.')
+ if requires_approval:
# approval can only be needed for Function Call Content, not Approval Responses.
logger.debug("Returning visible function_approval_request contents and storing already-approved requests")
visible_requests: list[Content] = []
already_approved_requests: list[Content] = []
- for fcc in function_calls:
- if fcc.type != "function_call":
+ for function_call in function_calls:
+ if function_call.type != "function_call":
continue
approval_request = Content.from_function_approval_request(
- id=fcc.call_id, # type: ignore[arg-type]
- function_call=fcc,
+ id=function_call.call_id, # type: ignore[arg-type]
+ function_call=function_call,
)
- tool_name = fcc.name
+ tool_name = function_call.name
if tool_name is None:
visible_requests.append(approval_request)
continue
tool = tool_map.get(tool_name)
if (
- tool_name in approval_tools
+ tool_name in approval_tool_names
or tool is None
- or tool_name in declaration_only
+ or tool_name in declaration_only_tool_names
or tool_name in additional_tool_names
):
visible_requests.append(approval_request)
@@ -1749,115 +1801,88 @@ async def _try_execute_function_calls(
visible_requests,
already_approved_requests,
)
- return (visible_requests, False)
- if declaration_only_flag:
+ return [[request] for request in visible_requests], False
+ if has_declaration_only_call:
# return the declaration only tools to the user, since we cannot execute them.
# Mark as user_input_request so AgentExecutor emits request_info events and pauses the workflow.
declaration_only_calls: list[Content] = []
- for fcc in function_calls:
- if fcc.type == "function_call":
- fcc.user_input_request = True
- fcc.id = fcc.call_id
- declaration_only_calls.append(fcc)
- return (declaration_only_calls, False)
-
- # Run all function calls concurrently, handling MiddlewareTermination
- from ._middleware import MiddlewareTermination
-
- extra_user_input_contents: list[Content] = []
-
- async def invoke_with_termination_handling(
- function_call: Content,
- seq_idx: int,
- ) -> tuple[Content, bool]:
- """Invoke function and catch MiddlewareTermination, returning (result, should_terminate)."""
- try:
- result = await _auto_invoke_function(
- function_call_content=function_call,
- custom_args=custom_args,
- tool_map=tool_map,
- invocation_session=invocation_session,
- sequence_index=seq_idx,
- request_index=attempt_idx,
- middleware_pipeline=middleware_pipeline,
- config=config,
- live_tools=live_tools,
- )
- return (result, False)
- except MiddlewareTermination as exc:
- # Middleware requested termination - return result as Content
- # exc.result may already be a Content (set by _auto_invoke_function) or raw value
- if isinstance(exc.result, Content):
- return (exc.result, True)
- result_content = Content.from_function_result(
- call_id=function_call.call_id, # type: ignore[arg-type]
- result=exc.result,
- )
- return (result_content, True)
- except UserInputRequiredException as exc:
- if exc.contents:
- propagated: list[Content] = []
- for item in exc.contents:
- if isinstance(item, Content):
- item.call_id = function_call.call_id
- if not item.id:
- item.id = function_call.call_id
- propagated.append(item)
- if propagated:
- extra_user_input_contents.extend(propagated[1:])
- return (propagated[0], False)
- return (
- Content.from_function_result(
- call_id=function_call.call_id, # type: ignore[arg-type]
- result="Tool requires user input but no request details were provided.",
- exception="UserInputRequiredException",
- ),
- False,
- )
+ for function_call in function_calls:
+ if function_call.type == "function_call":
+ function_call.user_input_request = True
+ function_call.id = function_call.call_id
+ declaration_only_calls.append(function_call)
+ return [[function_call] for function_call in declaration_only_calls], False
# Create each task inside a copied context so the active agent span is
# preserved for every parallel tool invocation.
execution_tasks = [
contextvars.copy_context().run(
asyncio.create_task,
- invoke_with_termination_handling(function_call, seq_idx),
+ _execute_single_function_call(
+ function_call,
+ custom_args=custom_args,
+ config=config,
+ tool_map=tool_map,
+ invocation_session=invocation_session,
+ middleware_pipeline=middleware_pipeline,
+ live_tools=live_tools,
+ ),
)
- for seq_idx, function_call in enumerate(function_calls)
+ for function_call in function_calls
]
execution_results = await asyncio.gather(*execution_tasks)
- # Unpack results - each is (Content, terminate_flag)
- contents: list[Content] = [result[0] for result in execution_results]
- contents.extend(extra_user_input_contents)
- # If any function requested termination, terminate the loop
- should_terminate = any(result[1] for result in execution_results)
- return (contents, should_terminate)
+ should_terminate = any(terminate for _, terminate in execution_results)
+ return [result_contents for result_contents, _ in execution_results], should_terminate
+
+
+@dataclass
+class _FunctionExecutionBatch:
+ """Results from one ordered batch of function-call executions."""
+
+ result_groups: list[list[Content]]
+ should_terminate: bool = False
+
+ @property
+ def contents(self) -> list[Content]:
+ """Flatten the ordered result groups for ordinary response processing."""
+ return [content for result_group in self.result_groups for content in result_group]
+
+ @property
+ def had_errors(self) -> bool:
+ """Whether any execution produced an error result."""
+ return any(
+ content.exception is not None
+ for result_group in self.result_groups
+ for content in result_group
+ if content.type == "function_result"
+ )
async def _execute_function_calls(
*,
custom_args: dict[str, Any],
- attempt_idx: int,
function_calls: list[Content],
- tool_options: dict[str, Any] | None,
+ options: dict[str, Any] | None,
config: FunctionInvocationConfiguration,
invocation_session: AgentSession | None = None,
- middleware_pipeline: Any = None,
-) -> tuple[list[Content], bool, bool]:
- tools = _extract_tools(tool_options)
+ middleware_pipeline: FunctionMiddlewarePipeline | None = None,
+) -> _FunctionExecutionBatch:
+ tools = _extract_tools(options)
if not tools:
- return [], False, False
- results, should_terminate = await _try_execute_function_calls(
+ return _FunctionExecutionBatch(result_groups=[])
+ result_groups, should_terminate = await _try_execute_function_call_groups(
custom_args=custom_args,
- attempt_idx=attempt_idx,
function_calls=function_calls,
tools=tools,
invocation_session=invocation_session,
middleware_pipeline=middleware_pipeline,
config=config,
)
- had_errors = any(fcr.exception is not None for fcr in results if fcr.type == "function_result")
- return list(results), should_terminate, had_errors
+ return _FunctionExecutionBatch(
+ result_groups=result_groups,
+ should_terminate=should_terminate,
+ )
def _update_conversation_id(
@@ -1923,7 +1948,16 @@ def _response_has_visible_content(response: ChatResponse[Any]) -> bool:
return False
+def _drop_unexecutable_tool_contents_from_response(response: ChatResponse[Any]) -> None:
+ for message in response.messages:
+ if any(content.type in _UNEXECUTABLE_TOOL_CONTENT_TYPES for content in message.contents):
+ message.contents = [
+ content for content in message.contents if content.type not in _UNEXECUTABLE_TOOL_CONTENT_TYPES
+ ]
+
+
def _ensure_function_invocation_limit_fallback_response(response: ChatResponse[Any]) -> ChatResponse[Any]:
+ _drop_unexecutable_tool_contents_from_response(response)
if _response_has_visible_content(response):
return response
@@ -1948,6 +1982,28 @@ def _function_invocation_limit_fallback_update() -> ChatResponseUpdate:
)
+def _update_has_meaningful_metadata(update: ChatResponseUpdate) -> bool:
+ return any((
+ update.author_name is not None,
+ update.response_id is not None,
+ update.message_id is not None,
+ update.conversation_id is not None,
+ update.model is not None,
+ update.created_at is not None,
+ update.finish_reason is not None,
+ update.continuation_token is not None,
+ bool(update.additional_properties),
+ update.raw_representation is not None,
+ ))
+
+
+def _drop_unexecutable_tool_contents_from_update(update: ChatResponseUpdate) -> ChatResponseUpdate | None:
+ if not any(content.type in _UNEXECUTABLE_TOOL_CONTENT_TYPES for content in update.contents):
+ return update
+ update.contents = [content for content in update.contents if content.type not in _UNEXECUTABLE_TOOL_CONTENT_TYPES]
+ return update if update.contents or _update_has_meaningful_metadata(update) else None
+
+
def _extract_tools(
options: dict[str, Any] | None,
) -> ToolTypes | Callable[..., Any] | Sequence[ToolTypes | Callable[..., Any]] | None:
@@ -1959,9 +2015,7 @@ def _extract_tools(
Returns:
ToolTypes | Callable[..., Any] | Sequence[ToolTypes | Callable[..., Any]] | None
"""
- if options and isinstance(options, dict):
- return options.get("tools")
- return None
+ return options.get("tools") if options else None
def _is_hosted_tool_approval(content: Any) -> bool:
@@ -2027,7 +2081,7 @@ def _store_already_approved_approval_requests(
return
existing_groups = state.get(_ALREADY_APPROVED_APPROVAL_REQUEST_GROUPS_KEY)
- pending_groups: list[Any] = list(cast(Iterable[Any], existing_groups)) if isinstance(existing_groups, list) else []
+ pending_groups = list(cast(list[Any], existing_groups)) if isinstance(existing_groups, list) else []
pending_groups.append({
"approval_request_ids": visible_ids,
"approval_requests": [request.to_dict() for request in already_approved_requests],
@@ -2048,24 +2102,23 @@ def _pop_already_approved_approval_responses(
raw_groups = state.get(_ALREADY_APPROVED_APPROVAL_REQUEST_GROUPS_KEY, [])
if not isinstance(raw_groups, list):
return []
+ typed_groups = cast(list[Any], raw_groups)
responses: list[Content] = []
remaining_groups: list[Any] = []
- raw_group_items = list(cast(Iterable[Any], raw_groups))
- for raw_group in raw_group_items:
+ for raw_group in typed_groups:
if not isinstance(raw_group, Mapping):
continue
group = cast(Mapping[str, Any], raw_group)
raw_ids = group.get("approval_request_ids")
- raw_group_ids: Iterable[Any] = cast(Iterable[Any], raw_ids) if isinstance(raw_ids, list) else ()
- group_ids = {str(item) for item in raw_group_ids}
+ group_ids: set[str] = {str(item) for item in cast(list[Any], raw_ids)} if isinstance(raw_ids, list) else set()
if group_ids.isdisjoint(approval_response_ids):
remaining_groups.append(raw_group)
continue
raw_requests = group.get("approval_requests")
if not isinstance(raw_requests, list):
continue
- for raw_request in list(cast(Iterable[Any], raw_requests)):
+ for raw_request in cast(list[Any], raw_requests):
request = _content_from_state(raw_request)
if request is None or request.type != "function_approval_request":
continue
@@ -2085,15 +2138,36 @@ def _collect_approval_responses(
Hosted tool approvals (e.g. MCP) are excluded because they must be
forwarded to the API as-is rather than processed locally.
"""
- from ._types import Message
-
- fcc_todo: dict[str, Content] = {}
- for msg in messages:
- for content in msg.contents if isinstance(msg, Message) else []:
- # Collect BOTH approved and rejected responses, but skip hosted tool approvals
+ approval_responses: list[Content] = []
+ pending_by_call_id: dict[str, deque[Content]] = {}
+ resolved_response_ids: set[int] = set()
+ for message in messages:
+ for content in message.contents:
if content.type == "function_approval_response" and not _is_hosted_tool_approval(content):
- fcc_todo[content.id] = content # type: ignore[attr-defined, index]
- return fcc_todo
+ function_call = content.function_call
+ if function_call is None or function_call.call_id is None:
+ continue
+ approval_responses.append(content)
+ pending_by_call_id.setdefault(function_call.call_id, deque()).append(content)
+ continue
+ if content.call_id is None:
+ continue
+ is_terminal_result = content.type == "function_result" and not _is_approval_placeholder_result(content)
+ is_follow_up_request = content.user_input_request and content.type not in {
+ "function_approval_request",
+ "function_approval_response",
+ }
+ if not (is_terminal_result or is_follow_up_request):
+ continue
+ pending_responses = pending_by_call_id.get(content.call_id)
+ if pending_responses:
+ resolved_response_ids.add(id(pending_responses.popleft()))
+
+ return {
+ content.id: content
+ for content in approval_responses
+ if id(content) not in resolved_response_ids and content.id is not None
+ }
def _is_approval_placeholder_result(content: Content) -> bool:
@@ -2102,340 +2176,465 @@ def _is_approval_placeholder_result(content: Content) -> bool:
return isinstance(result, str) and "[APPROVAL_PENDING]" in result
+@dataclass
+class _ApprovalCallOccurrence:
+ function_call: Content
+ approval_id: str | None = None
+ placeholder_message: Message | None = None
+ placeholder_content: Content | None = None
+ closed: bool = False
+
+
def _replace_approval_contents_with_results(
messages: list[Message],
- fcc_todo: dict[str, Content],
- approved_function_results: list[Content],
-) -> None:
+ pending_approval_responses: dict[str, Content],
+ approved_function_result_groups: list[list[Content]],
+) -> list[Content]:
"""Replace approval request/response contents with function call/result contents in-place.
Also replaces placeholder tool results (marked with [APPROVAL_PENDING]) with actual results.
+
+ Returns:
+ The terminal contents produced while resolving the approval responses, in response order.
"""
from ._types import (
Content,
)
- # Match results back to approvals by actual call_id instead of relying on
- # approval/result iteration order.
- result_by_call_id: dict[str, Content] = {}
- for approved_result in approved_function_results:
- if approved_result.call_id is not None and approved_result.call_id not in result_by_call_id:
- result_by_call_id[approved_result.call_id] = approved_result
-
- # Track which call_ids had their placeholders replaced
- placeholders_replaced: set[str] = set()
-
- # Collect *pending* function call IDs across all messages to avoid duplicates. The
- # function call and its approval request are frequently carried in separate messages
- # (e.g. when a hosting layer replays them as separate items on an approval round trip),
- # so scoping this per-message would let the same call_id be restored twice and leave
- # the copy without a result unanswered.
- #
- # Calls that already carry a real result are excluded: reusing a call_id for a later
- # invocation is supported, and a completed pair must not suppress the fresh request —
- # that would drop the new call and attach its result to the old one. Placeholder
- # results still count as pending, since the call they answer is the one being restored.
- answered_call_ids = {
- content.call_id
- for msg in messages
- for content in msg.contents
- if content.type == "function_result" and content.call_id and not _is_approval_placeholder_result(content)
- }
- existing_call_ids = {
- content.call_id
- for msg in messages
- for content in msg.contents
- if content.type == "function_call" and content.call_id and content.call_id not in answered_call_ids
- }
+ result_groups_by_call_id: dict[str, deque[list[Content]]] = {}
+ for result_group in approved_function_result_groups:
+ call_id = next((result.call_id for result in result_group if result.call_id is not None), None)
+ if call_id is not None:
+ result_groups_by_call_id.setdefault(call_id, deque()).append(result_group)
+
+ occurrences_by_call_id: dict[str, list[_ApprovalCallOccurrence]] = {}
+ occurrences_by_approval_id: dict[str, list[_ApprovalCallOccurrence]] = {}
+ seen_approval_requests: set[tuple[str, str, str | None, str]] = set()
+ placeholder_replacements: list[tuple[Message, Content, list[Content]]] = []
+ resolved_contents: list[Content] = []
+
+ def find_open_occurrence(call_id: str, *, require_unbound: bool = False) -> _ApprovalCallOccurrence | None:
+ for occurrence in occurrences_by_call_id.get(call_id, []):
+ if occurrence.closed:
+ continue
+ if require_unbound and occurrence.approval_id is not None:
+ continue
+ return occurrence
+ return None
+
+ def find_approval_occurrence(approval_id: str) -> _ApprovalCallOccurrence | None:
+ for occurrence in occurrences_by_approval_id.get(approval_id, []):
+ if not occurrence.closed:
+ return occurrence
+ return None
for msg in messages:
- # Track approval requests that should be removed (duplicates)
contents_to_remove: list[int] = []
+ replacement_groups_by_index: dict[int, list[Content]] = {}
for content_idx, content in enumerate(msg.contents):
- if content.type == "function_approval_request":
- # Skip hosted tool approvals — they must pass through to the API unchanged
+ if content.type == "function_call" and content.call_id:
+ occurrences_by_call_id.setdefault(content.call_id, []).append(
+ _ApprovalCallOccurrence(function_call=content)
+ )
+ elif content.type == "function_approval_request":
if _is_hosted_tool_approval(content):
continue
- # Don't add the function call if it already exists (would create duplicate)
- if content.function_call is not None and content.function_call.call_id in existing_call_ids:
- # Just mark for removal - the function call already exists
+ if content.function_call is None or content.function_call.call_id is None or content.id is None:
+ continue
+ call_id = content.function_call.call_id
+ occurrence = find_open_occurrence(call_id, require_unbound=True)
+ request_identity = (
+ content.id,
+ call_id,
+ content.function_call.name,
+ str(content.function_call.arguments),
+ )
+ if occurrence is None and request_identity in seen_approval_requests:
contents_to_remove.append(content_idx)
- elif content.function_call is not None:
- # Put back the function call content only if it doesn't exist
+ continue
+ seen_approval_requests.add(request_identity)
+ if occurrence is None:
+ occurrence = _ApprovalCallOccurrence(
+ function_call=content.function_call,
+ approval_id=content.id,
+ )
+ occurrences_by_call_id.setdefault(call_id, []).append(occurrence)
msg.contents[content_idx] = content.function_call
- if content.function_call.call_id:
- existing_call_ids.add(content.function_call.call_id)
+ else:
+ occurrence.approval_id = content.id
+ contents_to_remove.append(content_idx)
+ occurrences_by_approval_id.setdefault(content.id, []).append(occurrence)
elif content.type == "function_approval_response":
- # Skip hosted tool approvals — they must pass through to the API unchanged
if _is_hosted_tool_approval(content):
continue
- if content.function_call is None or content.function_call.call_id is None:
+ if content.function_call is None or content.function_call.call_id is None or content.id is None:
+ continue
+ if content.id not in pending_approval_responses:
+ contents_to_remove.append(content_idx)
continue
call_id = content.function_call.call_id
- if content.approved and content.id in fcc_todo:
- # Check if we already replaced a placeholder for this call_id
- if call_id in placeholders_replaced:
- # Placeholder was replaced - just remove the approval response
- contents_to_remove.append(content_idx)
- else:
- # No placeholder - replace approval response with result directly
- # This handles the original approval_mode="always_require" case
- replacement_result = result_by_call_id.get(call_id)
- if replacement_result is not None:
- msg.contents[content_idx] = replacement_result
- msg.role = "tool"
+ occurrence = find_approval_occurrence(content.id)
+ if occurrence is None:
+ occurrence = find_open_occurrence(call_id)
+ replacements: list[Content] | None
+ if content.approved:
+ call_result_groups = result_groups_by_call_id.get(call_id)
+ replacements = call_result_groups.popleft() if call_result_groups else None
else:
- # Create a "not approved" result for rejected calls
- # Use function_call.call_id (the function's ID), not content.id (approval's ID)
- msg.contents[content_idx] = Content.from_function_result(
- call_id=content.function_call.call_id,
- result="Error: Tool call invocation was rejected by user.",
- )
- msg.role = "tool"
+ replacements = [
+ Content.from_function_result(
+ call_id=call_id,
+ result="Error: Tool call invocation was rejected by user.",
+ additional_properties=content.function_call.additional_properties,
+ )
+ ]
+ if not replacements:
+ continue
+ if (
+ occurrence is not None
+ and occurrence.placeholder_message is not None
+ and occurrence.placeholder_content is not None
+ ):
+ placeholder_replacements.append((
+ occurrence.placeholder_message,
+ occurrence.placeholder_content,
+ replacements,
+ ))
+ contents_to_remove.append(content_idx)
+ else:
+ replacement_groups_by_index[content_idx] = replacements
+ if occurrence is not None:
+ occurrence.closed = True
+ resolved_contents.extend(replacements)
elif content.type == "function_result":
- # Check if this is a placeholder result that should be replaced
- if _is_approval_placeholder_result(content) and content.call_id in result_by_call_id:
- # Replace placeholder with actual result
- msg.contents[content_idx] = result_by_call_id[content.call_id]
- placeholders_replaced.add(content.call_id)
-
- # Remove contents marked for removal (in reverse order to preserve indices)
- for idx in reversed(contents_to_remove):
- msg.contents.pop(idx)
-
- # Second pass: Remove messages that are now empty after content removal
- # We need to iterate in reverse to safely remove by index
+ if content.call_id is None:
+ continue
+ occurrence = find_open_occurrence(content.call_id)
+ if occurrence is None:
+ continue
+ if _is_approval_placeholder_result(content):
+ occurrence.placeholder_message = msg
+ occurrence.placeholder_content = content
+ else:
+ occurrence.closed = True
+
+ if replacement_groups_by_index:
+ msg.role = (
+ "assistant"
+ if any(
+ replacement.user_input_request
+ for replacements in replacement_groups_by_index.values()
+ for replacement in replacements
+ )
+ else "tool"
+ )
+ if contents_to_remove or replacement_groups_by_index:
+ removed_indexes = set(contents_to_remove)
+ updated_contents: list[Content] = []
+ for idx, existing in enumerate(msg.contents):
+ if idx in removed_indexes:
+ continue
+ replacements = replacement_groups_by_index.get(idx)
+ if replacements is not None:
+ updated_contents.extend(replacements)
+ else:
+ updated_contents.append(existing)
+ msg.contents = updated_contents
+
+ for placeholder_message, placeholder_content, replacements in placeholder_replacements:
+ for idx, existing in enumerate(placeholder_message.contents):
+ if existing is placeholder_content:
+ placeholder_message.contents[idx : idx + 1] = replacements
+ break
+
messages_to_remove: list[int] = []
for msg_idx, msg in enumerate(messages):
if not msg.contents:
messages_to_remove.append(msg_idx)
for msg_idx in reversed(messages_to_remove):
messages.pop(msg_idx)
-
-
-def _get_result_hooks_from_stream(stream: Any) -> list[Callable[[Any], Any]]:
- inner_stream = getattr(stream, "_inner_stream", None)
- if inner_stream is None:
- inner_source = getattr(stream, "_inner_stream_source", None)
- if inner_source is not None:
- inner_stream = inner_source
- if inner_stream is None:
- inner_stream = stream
- return list(getattr(inner_stream, "_result_hooks", []))
+ return resolved_contents
def _extract_function_calls(response: ChatResponse) -> list[Content]:
- function_results = {
- item.call_id
- for message in response.messages
- for item in message.contents
- if item.type == "function_result" and item.call_id
- }
+ completed_call_ids: set[str] = set()
seen_call_ids: set[str] = set()
- function_calls: list[Content] = []
+ candidate_calls: list[Content] = []
for message in response.messages:
for item in message.contents:
- if not _is_actionable_function_call(item):
+ if item.type == "function_result" and item.call_id:
+ completed_call_ids.add(item.call_id)
continue
- if item.call_id and item.call_id in function_results:
+ if not _is_actionable_function_call(item):
continue
if item.call_id and item.call_id in seen_call_ids:
continue
if item.call_id:
seen_call_ids.add(item.call_id)
- function_calls.append(item)
- return function_calls
+ candidate_calls.append(item)
+ return [
+ function_call
+ for function_call in candidate_calls
+ if not function_call.call_id or function_call.call_id not in completed_call_ids
+ ]
+
+
+def _prepend_function_call_messages(response: ChatResponse, function_call_messages: list[Message]) -> None:
+ response.messages[:0] = function_call_messages
-def _prepend_fcc_messages(response: ChatResponse, fcc_messages: list[Message]) -> None:
- if not fcc_messages:
+def _copy_messages_for_function_invocation(messages: Any) -> list[Message]:
+ from ._types import normalize_messages
+
+ copied_messages: list[Message] = []
+ for message in normalize_messages(messages):
+ copied_message = copy.copy(message)
+ copied_message.contents = list(message.contents)
+ copied_message.additional_properties = dict(message.additional_properties)
+ copied_messages.append(copied_message)
+ return copied_messages
+
+
+def _function_call_limit_reached(total_function_calls: int, max_function_calls: int | None) -> bool:
+ return max_function_calls is not None and total_function_calls >= max_function_calls
+
+
+def _update_consecutive_error_count(
+ errors_in_a_row: int,
+ *,
+ had_errors: bool,
+ max_errors: int,
+) -> tuple[int, bool]:
+ if not had_errors:
+ return 0, False
+ errors_in_a_row += 1
+ reached_error_limit = errors_in_a_row >= max_errors
+ if reached_error_limit:
+ logger.warning(
+ "Maximum consecutive function call errors reached (%d). Stopping further function calls for this request.",
+ max_errors,
+ )
+ return errors_in_a_row, reached_error_limit
+
+
+def _disable_tools_at_function_call_limit(
+ options: dict[str, Any],
+ total_function_calls: int,
+ max_function_calls: int | None,
+) -> None:
+ if not _function_call_limit_reached(total_function_calls, max_function_calls):
return
- for msg in reversed(fcc_messages):
- response.messages.insert(0, msg)
+ logger.info(
+ "Maximum function calls reached (%d/%d). Stopping further function calls for this request.",
+ total_function_calls,
+ max_function_calls,
+ )
+ options["tool_choice"] = "none"
-class FunctionRequestResult(TypedDict, total=False):
- """Result of processing function requests.
+def _record_function_calls(
+ budget_state: dict[str, Any],
+ total_function_calls: int,
+ function_call_count: int,
+) -> int:
+ total_function_calls += function_call_count
+ budget_state["total_function_calls"] = total_function_calls
+ return total_function_calls
- Attributes:
- action: The action to take ("return", "continue", or "stop").
- errors_in_a_row: The number of consecutive errors encountered.
- result_message: The message containing function call results, if any.
- update_role: The role to update for the next message, if any.
- function_call_results: The list of function call results, if any.
- function_call_count: The number of function calls executed in this processing step.
- """
- action: Literal["return", "continue", "stop"]
+def _reset_required_tool_choice(options: dict[str, Any]) -> None:
+ tool_choice = options.get("tool_choice")
+ required_mode = isinstance(tool_choice, Mapping) and cast(Mapping[str, Any], tool_choice).get("mode") == "required"
+ if tool_choice == "required" or required_mode:
+ options["tool_choice"] = None
+
+
+def _prepare_messages_for_next_iteration(prepared_messages: list[Message], response: ChatResponse[Any]) -> None:
+ if response.conversation_id is None:
+ prepared_messages.extend(response.messages)
+ return
+ prepared_messages[:] = response.messages[-1:]
+
+
+@dataclass
+class _FunctionProcessingResult:
+ """Control data produced while resolving or executing function calls."""
+
errors_in_a_row: int
- result_message: Message | None
- update_role: Literal["assistant", "tool"] | None
- function_call_results: list[Content] | None
- function_call_count: int
+ action: Literal["return", "continue", "stop"] = "continue"
+ function_call_count: int = 0
+ response_message: Message | None = None
+ streaming_update: ChatResponseUpdate | None = None
+
+
+_FunctionCallExecutor: TypeAlias = Callable[..., Awaitable[_FunctionExecutionBatch]]
def _handle_function_call_results(
*,
response: ChatResponse,
- function_call_results: list[Content],
- fcc_messages: list[Message],
+ execution_results: list[Content],
+ function_call_count: int,
+ function_call_messages: list[Message] | None,
errors_in_a_row: int,
had_errors: bool,
max_errors: int,
-) -> FunctionRequestResult:
- from ._types import Message
+) -> _FunctionProcessingResult:
+ """Append execution results to the response and determine the next loop action."""
+ from ._types import ChatResponseUpdate, Message
if any(
- fccr.type in {"function_approval_request", "function_call"} or fccr.user_input_request
- for fccr in function_call_results
+ result.type in {"function_approval_request", "function_call"} or result.user_input_request
+ for result in execution_results
):
# Only add items that aren't already in the message (e.g. function_approval_request wrappers).
# Declaration-only function_call items are already present from the LLM response.
- new_items = [fccr for fccr in function_call_results if fccr.type != "function_call"]
+ new_items = [result for result in execution_results if result.type != "function_call"]
if new_items:
if response.messages and response.messages[0].role == "assistant":
response.messages[0].contents.extend(new_items)
else:
response.messages.append(Message(role="assistant", contents=new_items))
- return {
- "action": "return",
- "errors_in_a_row": errors_in_a_row,
- "result_message": None,
- "update_role": "assistant",
- "function_call_results": None,
- }
+ streaming_items: list[Content] = []
+ for result in execution_results:
+ if result.type == "function_call":
+ metadata_only_result = copy.copy(result)
+ metadata_only_result.arguments = None
+ streaming_items.append(metadata_only_result)
+ else:
+ streaming_items.append(result)
+ return _FunctionProcessingResult(
+ errors_in_a_row=errors_in_a_row,
+ action="return",
+ function_call_count=function_call_count,
+ streaming_update=ChatResponseUpdate(contents=streaming_items, role="assistant"),
+ )
- if had_errors:
- errors_in_a_row += 1
- reached_error_limit = errors_in_a_row >= max_errors
- if reached_error_limit:
- logger.warning(
- "Maximum consecutive function call errors reached (%d). "
- "Stopping further function calls for this request.",
- max_errors,
- )
- else:
- errors_in_a_row = 0
- reached_error_limit = False
+ errors_in_a_row, reached_error_limit = _update_consecutive_error_count(
+ errors_in_a_row,
+ had_errors=had_errors,
+ max_errors=max_errors,
+ )
- result_message = Message(role="tool", contents=function_call_results)
- response.messages.append(result_message)
- fcc_messages.extend(response.messages)
- return {
- "action": "stop" if reached_error_limit else "continue",
- "errors_in_a_row": errors_in_a_row,
- "result_message": result_message,
- "update_role": "tool",
- "function_call_results": None,
- }
+ response.messages.append(Message(role="tool", contents=execution_results))
+ if function_call_messages is not None:
+ function_call_messages.extend(response.messages)
+ return _FunctionProcessingResult(
+ errors_in_a_row=errors_in_a_row,
+ action="stop" if reached_error_limit else "continue",
+ function_call_count=function_call_count,
+ streaming_update=ChatResponseUpdate(contents=execution_results, role="tool"),
+ )
-async def _process_function_requests(
+async def _resolve_approval_responses(
*,
- response: ChatResponse | None,
- prepped_messages: list[Message] | None,
- tool_options: dict[str, Any] | None,
- attempt_idx: int,
- fcc_messages: list[Message] | None,
+ prepared_messages: list[Message],
+ options: dict[str, Any] | None,
errors_in_a_row: int,
max_errors: int,
- execute_function_calls: Callable[..., Awaitable[tuple[list[Content], bool, bool]]],
+ execute_function_calls: _FunctionCallExecutor,
invocation_session: AgentSession | None = None,
-) -> FunctionRequestResult:
- from ._types import Message
-
- if prepped_messages is not None:
- explicit_approval_response_ids = {
- content.id
- for message in prepped_messages
- if isinstance(message, Message)
- for content in message.contents
- if content.type == "function_approval_response" and content.id
- }
- already_approved_responses = _pop_already_approved_approval_responses(
- invocation_session,
- explicit_approval_response_ids,
+) -> _FunctionProcessingResult:
+ """Resolve inbound approval responses before the next model call."""
+ from ._types import ChatResponseUpdate, Message
+
+ explicit_approval_response_ids = {
+ content.id
+ for message in prepared_messages
+ for content in message.contents
+ if content.type == "function_approval_response" and content.id
+ }
+ already_approved_responses = _pop_already_approved_approval_responses(
+ invocation_session,
+ explicit_approval_response_ids,
+ )
+ if already_approved_responses:
+ prepared_messages.append(Message(role="user", contents=already_approved_responses))
+ pending_approval_responses = _collect_approval_responses(prepared_messages)
+ if not pending_approval_responses:
+ return _FunctionProcessingResult(errors_in_a_row=errors_in_a_row)
+
+ responses_to_execute = [response for response in pending_approval_responses.values() if response.approved]
+ execution_result_groups: list[list[Content]] = []
+ should_terminate = False
+ reached_error_limit = False
+ if responses_to_execute:
+ execution = await execute_function_calls(
+ function_calls=responses_to_execute,
+ options=options,
)
- if already_approved_responses:
- prepped_messages.append(Message(role="user", contents=already_approved_responses))
- fcc_todo = _collect_approval_responses(prepped_messages)
- if not fcc_todo:
- fcc_todo = {}
- if fcc_todo:
- approved_responses = [resp for resp in fcc_todo.values() if resp.approved]
- approved_function_results: list[Content] = []
- should_terminate = False
- if approved_responses:
- results, should_terminate, had_errors = await execute_function_calls(
- attempt_idx=attempt_idx,
- function_calls=approved_responses,
- tool_options=tool_options,
- )
- approved_function_results = list(results)
- if had_errors:
- errors_in_a_row += 1
- if errors_in_a_row >= max_errors:
- logger.warning(
- "Maximum consecutive function call errors reached (%d). "
- "Stopping further function calls for this request.",
- max_errors,
- )
- _replace_approval_contents_with_results(prepped_messages, fcc_todo, approved_function_results)
- executed_count = sum(1 for r in approved_function_results if r.type == "function_result")
- # Continue to call chat client with updated messages (containing function results)
- # so it can generate the final response
- return {
- "action": "return" if should_terminate else "continue",
- "errors_in_a_row": errors_in_a_row,
- "result_message": None,
- "update_role": None,
- "function_call_results": None,
- "function_call_count": executed_count,
- }
+ execution_result_groups = execution.result_groups
+ should_terminate = execution.should_terminate
+ errors_in_a_row, reached_error_limit = _update_consecutive_error_count(
+ errors_in_a_row,
+ had_errors=execution.had_errors,
+ max_errors=max_errors,
+ )
+ terminal_contents = _replace_approval_contents_with_results(
+ prepared_messages,
+ pending_approval_responses,
+ execution_result_groups,
+ )
+ executed_function_count = len(execution_result_groups)
+ requires_user_input = any(
+ result.type == "function_call" or result.user_input_request for result in terminal_contents
+ )
+ response_role: Literal["assistant", "tool"] | None = None
+ if terminal_contents:
+ response_role = "assistant" if requires_user_input else "tool"
+ terminal_message = Message(role=response_role, contents=terminal_contents) if response_role else None
+ streaming_update = (
+ ChatResponseUpdate(contents=terminal_contents, role=response_role) if response_role is not None else None
+ )
+ action: Literal["return", "continue", "stop"] = "continue"
+ if should_terminate or requires_user_input:
+ action = "return"
+ elif reached_error_limit:
+ action = "stop"
+ return _FunctionProcessingResult(
+ errors_in_a_row=errors_in_a_row,
+ action=action,
+ function_call_count=executed_function_count,
+ response_message=terminal_message,
+ streaming_update=streaming_update,
+ )
- if response is None or fcc_messages is None:
- return {
- "action": "continue",
- "errors_in_a_row": errors_in_a_row,
- "result_message": None,
- "update_role": None,
- "function_call_results": None,
- "function_call_count": 0,
- }
- tools = _extract_tools(tool_options)
+async def _process_model_function_calls(
+ *,
+ response: ChatResponse,
+ options: dict[str, Any] | None,
+ function_call_messages: list[Message] | None,
+ errors_in_a_row: int,
+ max_errors: int,
+ execute_function_calls: _FunctionCallExecutor,
+) -> _FunctionProcessingResult:
+ """Execute function calls from a newly completed model response."""
+ tools = _extract_tools(options)
function_calls = _extract_function_calls(response)
if not (function_calls and tools):
- _prepend_fcc_messages(response, fcc_messages)
- return {
- "action": "return",
- "errors_in_a_row": errors_in_a_row,
- "result_message": None,
- "update_role": None,
- "function_call_results": None,
- "function_call_count": 0,
- }
+ if function_call_messages is not None:
+ _prepend_function_call_messages(response, function_call_messages)
+ return _FunctionProcessingResult(errors_in_a_row=errors_in_a_row, action="return")
- function_call_results, should_terminate, had_errors = await execute_function_calls(
- attempt_idx=attempt_idx,
+ execution = await execute_function_calls(
function_calls=function_calls,
- tool_options=tool_options,
+ options=options,
)
- result = _handle_function_call_results(
+ processing_result = _handle_function_call_results(
response=response,
- function_call_results=function_call_results,
- fcc_messages=fcc_messages,
+ execution_results=execution.contents,
+ function_call_count=len(execution.result_groups),
+ function_call_messages=function_call_messages,
errors_in_a_row=errors_in_a_row,
- had_errors=had_errors,
+ had_errors=execution.had_errors,
max_errors=max_errors,
)
- result["function_call_results"] = list(function_call_results)
- result["function_call_count"] = sum(1 for r in function_call_results if r.type == "function_result")
- # If middleware requested termination, change action to return
- if should_terminate:
- result["action"] = "return"
- return result
+ if execution.should_terminate:
+ processing_result.action = "return"
+ return processing_result
OptionsCoT = TypeVar(
@@ -2458,23 +2657,23 @@ def __init__(
) -> None:
from ._middleware import categorize_middleware
- middleware_list = categorize_middleware(middleware)
- self.function_middleware: list[FunctionMiddlewareTypes] = list(middleware_list["function"])
+ categorized_middleware = categorize_middleware(middleware)
+ self.function_middleware: list[FunctionMiddlewareTypes] = list(categorized_middleware["function"])
self._cached_function_middleware_pipeline: FunctionMiddlewarePipeline | None = None
self.function_invocation_configuration = normalize_function_invocation_configuration(
function_invocation_configuration
)
- if (chat_middleware := (middleware_list["chat"] or None)) is not None:
+ if (chat_middleware := (categorized_middleware["chat"] or None)) is not None:
kwargs["middleware"] = chat_middleware
super().__init__(**kwargs)
def _get_function_middleware_pipeline(
self,
- middleware: Sequence[FunctionMiddlewareTypes],
+ runtime_middleware: Sequence[FunctionMiddlewareTypes],
) -> FunctionMiddlewarePipeline:
from ._middleware import FunctionMiddlewarePipeline
- effective_middleware = [*self.function_middleware, *middleware]
+ effective_middleware = [*self.function_middleware, *runtime_middleware]
if self._cached_function_middleware_pipeline is not None and self._cached_function_middleware_pipeline.matches(
effective_middleware
):
@@ -2483,6 +2682,290 @@ def _get_function_middleware_pipeline(
self._cached_function_middleware_pipeline = FunctionMiddlewarePipeline(*effective_middleware)
return self._cached_function_middleware_pipeline
+ async def _get_response_with_function_invocation(
+ self,
+ *,
+ super_get_response: Callable[..., Any],
+ messages: Sequence[Message],
+ options: dict[str, Any],
+ request_kwargs: dict[str, Any],
+ compaction_strategy: CompactionStrategy | None,
+ tokenizer: TokenizerProtocol | None,
+ execute_function_calls: _FunctionCallExecutor,
+ invocation_session: AgentSession | None,
+ budget_state: dict[str, Any],
+ max_errors: int,
+ ) -> ChatResponse[Any]:
+ """Run the non-streaming function invocation loop."""
+ from ._types import ChatResponse, add_usage_details
+
+ errors_in_a_row = 0
+ total_function_calls = int(budget_state.get("total_function_calls", 0) or 0)
+ max_function_calls = self.function_invocation_configuration.get("max_function_calls")
+ prepared_messages = _copy_messages_for_function_invocation(messages)
+ function_call_messages: list[Message] = []
+ response: ChatResponse[Any] | None = None
+ aggregated_usage: UsageDetails | None = None
+ max_iterations = self.function_invocation_configuration.get("max_iterations", DEFAULT_MAX_ITERATIONS)
+ attempt_start = int(budget_state.get("attempt_count", 0) or 0)
+
+ approval_processing = await _resolve_approval_responses(
+ prepared_messages=prepared_messages,
+ options=options,
+ errors_in_a_row=errors_in_a_row,
+ max_errors=max_errors,
+ execute_function_calls=execute_function_calls,
+ invocation_session=invocation_session,
+ )
+ if approval_processing.response_message is not None:
+ function_call_messages.append(approval_processing.response_message)
+ errors_in_a_row = approval_processing.errors_in_a_row
+ total_function_calls = _record_function_calls(
+ budget_state,
+ total_function_calls,
+ approval_processing.function_call_count,
+ )
+ if approval_processing.action == "return":
+ response = ChatResponse(messages=list(function_call_messages))
+ response.usage_details = aggregated_usage
+ return _clear_internal_conversation_id(response)
+ if approval_processing.action == "stop":
+ options["tool_choice"] = "none"
+ else:
+ _disable_tools_at_function_call_limit(options, total_function_calls, max_function_calls)
+
+ for attempt_idx in range(attempt_start, max_iterations):
+ budget_state["attempt_count"] = attempt_idx + 1
+ response = cast(
+ ChatResponse[Any],
+ await super_get_response(
+ messages=prepared_messages,
+ stream=False,
+ options=options,
+ compaction_strategy=compaction_strategy,
+ tokenizer=tokenizer,
+ client_kwargs=request_kwargs,
+ ),
+ )
+ if options.get("tool_choice") == "none" and _function_call_limit_reached(
+ total_function_calls, max_function_calls
+ ):
+ response = _ensure_function_invocation_limit_fallback_response(response)
+ aggregated_usage = add_usage_details(aggregated_usage, response.usage_details)
+ _update_continuation_state(
+ request_kwargs,
+ response,
+ session=invocation_session,
+ options=options,
+ )
+
+ function_processing = await _process_model_function_calls(
+ response=response,
+ options=options,
+ function_call_messages=function_call_messages,
+ errors_in_a_row=errors_in_a_row,
+ max_errors=max_errors,
+ execute_function_calls=execute_function_calls,
+ )
+ total_function_calls = _record_function_calls(
+ budget_state,
+ total_function_calls,
+ function_processing.function_call_count,
+ )
+ if function_processing.action == "return":
+ response.usage_details = aggregated_usage
+ return _clear_internal_conversation_id(response)
+ if function_processing.action == "stop":
+ options["tool_choice"] = "none"
+ else:
+ _disable_tools_at_function_call_limit(options, total_function_calls, max_function_calls)
+ errors_in_a_row = function_processing.errors_in_a_row
+ _reset_required_tool_choice(options)
+ _prepare_messages_for_next_iteration(prepared_messages, response)
+
+ if response is not None:
+ logger.info(
+ "Maximum iterations reached (%d). Requesting final response without tools.",
+ max_iterations,
+ )
+ options["tool_choice"] = "none"
+ response = cast(
+ ChatResponse[Any],
+ await super_get_response(
+ messages=prepared_messages,
+ stream=False,
+ options=options,
+ compaction_strategy=compaction_strategy,
+ tokenizer=tokenizer,
+ client_kwargs=request_kwargs,
+ ),
+ )
+ response = _ensure_function_invocation_limit_fallback_response(response)
+ aggregated_usage = add_usage_details(aggregated_usage, response.usage_details)
+ _update_continuation_state(
+ request_kwargs,
+ response,
+ session=invocation_session,
+ options=options,
+ )
+ response.usage_details = aggregated_usage
+ _prepend_function_call_messages(response, function_call_messages)
+ return _clear_internal_conversation_id(response)
+
+ async def _stream_response_with_function_invocation(
+ self,
+ *,
+ super_get_response: Callable[..., Any],
+ messages: Sequence[Message],
+ options: dict[str, Any],
+ request_kwargs: dict[str, Any],
+ compaction_strategy: CompactionStrategy | None,
+ tokenizer: TokenizerProtocol | None,
+ execute_function_calls: _FunctionCallExecutor,
+ invocation_session: AgentSession | None,
+ budget_state: dict[str, Any],
+ max_errors: int,
+ ) -> AsyncIterable[ChatResponseUpdate]:
+ """Run the streaming function invocation loop."""
+ errors_in_a_row = 0
+ total_function_calls = int(budget_state.get("total_function_calls", 0) or 0)
+ max_function_calls = self.function_invocation_configuration.get("max_function_calls")
+ prepared_messages = _copy_messages_for_function_invocation(messages)
+ response: ChatResponse[Any] | None = None
+ max_iterations = self.function_invocation_configuration.get("max_iterations", DEFAULT_MAX_ITERATIONS)
+ attempt_start = int(budget_state.get("attempt_count", 0) or 0)
+
+ approval_processing = await _resolve_approval_responses(
+ prepared_messages=prepared_messages,
+ options=options,
+ errors_in_a_row=errors_in_a_row,
+ max_errors=max_errors,
+ execute_function_calls=execute_function_calls,
+ invocation_session=invocation_session,
+ )
+ errors_in_a_row = approval_processing.errors_in_a_row
+ total_function_calls = _record_function_calls(
+ budget_state,
+ total_function_calls,
+ approval_processing.function_call_count,
+ )
+ if approval_processing.streaming_update is not None:
+ yield approval_processing.streaming_update
+ if approval_processing.action == "return":
+ return
+ if approval_processing.action == "stop":
+ options["tool_choice"] = "none"
+ else:
+ _disable_tools_at_function_call_limit(options, total_function_calls, max_function_calls)
+
+ for attempt_idx in range(attempt_start, max_iterations):
+ budget_state["attempt_count"] = attempt_idx + 1
+ inner_stream = cast(
+ "ResponseStream[ChatResponseUpdate, ChatResponse[Any]]",
+ super_get_response(
+ messages=prepared_messages,
+ stream=True,
+ options=options,
+ compaction_strategy=compaction_strategy,
+ tokenizer=tokenizer,
+ client_kwargs=request_kwargs,
+ ),
+ )
+ await inner_stream
+ drop_unexecutable_calls = options.get("tool_choice") == "none" and _function_call_limit_reached(
+ total_function_calls,
+ max_function_calls,
+ )
+ async for update in inner_stream:
+ if drop_unexecutable_calls:
+ update = _drop_unexecutable_tool_contents_from_update(update)
+ if update is None:
+ continue
+ yield update
+
+ response = await inner_stream.get_final_response()
+ response_had_visible_content = _response_has_visible_content(response)
+ function_call_limit_reached = options.get("tool_choice") == "none" and _function_call_limit_reached(
+ total_function_calls, max_function_calls
+ )
+ if function_call_limit_reached:
+ response = _ensure_function_invocation_limit_fallback_response(response)
+ _update_continuation_state(
+ request_kwargs,
+ response,
+ session=invocation_session,
+ options=options,
+ )
+
+ if not any(
+ item.type == "function_approval_request" or _is_actionable_function_call(item)
+ for message in response.messages
+ for item in message.contents
+ ):
+ if function_call_limit_reached and not response_had_visible_content:
+ yield _function_invocation_limit_fallback_update()
+ return
+
+ function_processing = await _process_model_function_calls(
+ response=response,
+ options=options,
+ function_call_messages=None,
+ errors_in_a_row=errors_in_a_row,
+ max_errors=max_errors,
+ execute_function_calls=execute_function_calls,
+ )
+ errors_in_a_row = function_processing.errors_in_a_row
+ total_function_calls = _record_function_calls(
+ budget_state,
+ total_function_calls,
+ function_processing.function_call_count,
+ )
+ if function_processing.streaming_update is not None:
+ yield function_processing.streaming_update
+ if function_processing.action == "stop":
+ options["tool_choice"] = "none"
+ elif function_processing.action != "continue":
+ return
+ else:
+ _disable_tools_at_function_call_limit(options, total_function_calls, max_function_calls)
+ _reset_required_tool_choice(options)
+ _prepare_messages_for_next_iteration(prepared_messages, response)
+
+ if response is not None:
+ logger.info(
+ "Maximum iterations reached (%d). Requesting final response without tools.",
+ max_iterations,
+ )
+ options["tool_choice"] = "none"
+ final_inner_stream = cast(
+ "ResponseStream[ChatResponseUpdate, ChatResponse[Any]]",
+ super_get_response(
+ messages=prepared_messages,
+ stream=True,
+ options=options,
+ compaction_strategy=compaction_strategy,
+ tokenizer=tokenizer,
+ client_kwargs=request_kwargs,
+ ),
+ )
+ await final_inner_stream
+ async for update in final_inner_stream:
+ update = _drop_unexecutable_tool_contents_from_update(update)
+ if update is None:
+ continue
+ yield update
+ final_response = await final_inner_stream.get_final_response()
+ final_response_had_visible_content = _response_has_visible_content(final_response)
+ final_response = _ensure_function_invocation_limit_fallback_response(final_response)
+ _update_continuation_state(
+ request_kwargs,
+ final_response,
+ session=invocation_session,
+ options=options,
+ )
+ if not final_response_had_visible_content:
+ yield _function_invocation_limit_fallback_update()
+
@overload
def get_response(
self,
@@ -2540,29 +3023,32 @@ def get_response(
from ._middleware import categorize_middleware
from ._types import (
ChatResponse,
- ChatResponseUpdate,
ResponseStream,
- add_usage_details,
)
- super_get_response = super().get_response # type: ignore[misc]
- effective_client_kwargs = dict(client_kwargs) if client_kwargs is not None else {}
+ super_get_response = cast(
+ Callable[..., Any],
+ super().get_response, # pyright: ignore[reportAttributeAccessIssue, reportUnknownMemberType]
+ )
+ request_kwargs = dict(client_kwargs) if client_kwargs is not None else {}
if middleware is not None:
- existing = effective_client_kwargs.get("middleware", [])
- effective_client_kwargs["middleware"] = [
+ existing_middleware = request_kwargs.get("middleware", [])
+ request_kwargs["middleware"] = [
*(
- existing
- if isinstance(existing, Sequence) and not isinstance(existing, (str, bytes))
- else [existing]
+ existing_middleware
+ if isinstance(existing_middleware, Sequence) and not isinstance(existing_middleware, (str, bytes))
+ else [existing_middleware]
),
*middleware,
]
- runtime_middleware = categorize_middleware(effective_client_kwargs.pop("middleware", []))
+ categorized_runtime_middleware = categorize_middleware(request_kwargs.pop("middleware", []))
- function_middleware_pipeline = self._get_function_middleware_pipeline(runtime_middleware["function"])
- if runtime_middleware["chat"]:
- effective_client_kwargs["middleware"] = runtime_middleware["chat"]
- raw_budget_state = effective_client_kwargs.pop(_FUNCTION_INVOCATION_BUDGET_STATE_KEY, None)
+ function_middleware_pipeline = self._get_function_middleware_pipeline(
+ categorized_runtime_middleware["function"]
+ )
+ if categorized_runtime_middleware["chat"]:
+ request_kwargs["middleware"] = categorized_runtime_middleware["chat"]
+ raw_budget_state = request_kwargs.pop(_FUNCTION_INVOCATION_BUDGET_STATE_KEY, None)
budget_state: dict[str, Any] = (
cast(dict[str, Any], raw_budget_state) if isinstance(raw_budget_state, dict) else {}
)
@@ -2576,7 +3062,7 @@ def get_response(
additional_function_arguments.update(cast(Mapping[str, Any], additional_opts))
from ._sessions import AgentSession as _AgentSession
- raw_session = effective_client_kwargs.get("session")
+ raw_session = request_kwargs.get("session")
invocation_session = raw_session if isinstance(raw_session, _AgentSession) else None
execute_function_calls = partial(
_execute_function_calls,
@@ -2585,22 +3071,20 @@ def get_response(
invocation_session=invocation_session,
middleware_pipeline=function_middleware_pipeline,
)
- filtered_kwargs = dict(effective_client_kwargs)
-
# Make options mutable so we can update conversation_id during function invocation loop
mutable_options: dict[str, Any] = dict(options) if options else {}
# Remove additional_function_arguments from options passed to underlying chat client
# It's for tool invocation only and not recognized by chat service APIs
mutable_options.pop("additional_function_arguments", None)
if not self.function_invocation_configuration.get("enabled", True):
- return super_get_response( # type: ignore[no-any-return]
+ return super_get_response(
messages=messages,
stream=stream,
options=mutable_options,
compaction_strategy=compaction_strategy,
tokenizer=tokenizer,
function_invocation_kwargs=function_invocation_kwargs,
- client_kwargs=filtered_kwargs,
+ client_kwargs=request_kwargs,
)
# Establish a single, run-local mutable tools list so that tools can add or remove
# tools during the run (progressive tool exposure). A fresh list is created via
@@ -2610,349 +3094,35 @@ def get_response(
if mutable_options.get("tools"):
mutable_options["tools"] = normalize_tools(mutable_options["tools"])
if not stream:
-
- async def _get_response() -> ChatResponse[Any]:
- nonlocal mutable_options
- nonlocal filtered_kwargs
- errors_in_a_row: int = 0
- total_function_calls = int(budget_state.get("total_function_calls", 0) or 0)
- max_function_calls: int | None = self.function_invocation_configuration.get("max_function_calls")
- prepped_messages = list(messages)
- fcc_messages: list[Message] = []
- response: ChatResponse[Any] | None = None
- aggregated_usage: UsageDetails | None = None
-
- loop_enabled = self.function_invocation_configuration.get("enabled", True)
- max_iterations = self.function_invocation_configuration.get("max_iterations", DEFAULT_MAX_ITERATIONS)
- attempt_start = int(budget_state.get("attempt_count", 0) or 0)
- for attempt_idx in range(attempt_start, max_iterations if loop_enabled else 0):
- budget_state["attempt_count"] = attempt_idx + 1
- approval_result = await _process_function_requests(
- response=None,
- prepped_messages=prepped_messages,
- tool_options=mutable_options,
- attempt_idx=attempt_idx,
- fcc_messages=None,
- errors_in_a_row=errors_in_a_row,
- max_errors=max_errors,
- execute_function_calls=execute_function_calls,
- invocation_session=invocation_session,
- )
- if approval_result.get("action") == "stop":
- response = ChatResponse(messages=prepped_messages)
- break
- errors_in_a_row = approval_result.get("errors_in_a_row", errors_in_a_row)
- total_function_calls += approval_result.get("function_call_count", 0)
- budget_state["total_function_calls"] = total_function_calls
- if max_function_calls is not None and total_function_calls >= max_function_calls:
- logger.info(
- "Maximum function calls reached (%d/%d). Stopping further function calls for this request.",
- total_function_calls,
- max_function_calls,
- )
- mutable_options["tool_choice"] = "none"
-
- response = cast(
- ChatResponse[Any],
- await super_get_response(
- messages=prepped_messages,
- stream=False,
- options=mutable_options,
- compaction_strategy=compaction_strategy,
- tokenizer=tokenizer,
- client_kwargs=filtered_kwargs,
- ),
- )
- if (
- mutable_options.get("tool_choice") == "none"
- and max_function_calls is not None
- and total_function_calls >= max_function_calls
- ):
- response = _ensure_function_invocation_limit_fallback_response(response)
- aggregated_usage = add_usage_details(aggregated_usage, response.usage_details)
- _update_continuation_state(
- filtered_kwargs,
- response,
- session=invocation_session,
- options=mutable_options,
- )
-
- if response.conversation_id is not None:
- prepped_messages = []
-
- result = await _process_function_requests(
- response=response,
- prepped_messages=None,
- tool_options=mutable_options,
- attempt_idx=attempt_idx,
- fcc_messages=fcc_messages,
- errors_in_a_row=errors_in_a_row,
- max_errors=max_errors,
- execute_function_calls=execute_function_calls,
- invocation_session=invocation_session,
- )
- if result.get("action") == "return":
- response.usage_details = aggregated_usage
- return _clear_internal_conversation_id(response)
- total_function_calls += result.get("function_call_count", 0)
- budget_state["total_function_calls"] = total_function_calls
- if result.get("action") == "stop":
- # Error threshold reached: force a final non-tool turn so
- # function_call_output items are submitted before exit.
- mutable_options["tool_choice"] = "none"
- elif max_function_calls is not None and total_function_calls >= max_function_calls:
- # Best-effort limit: checked after each batch of parallel calls completes,
- # so the current batch always runs to completion even if it overshoots.
- logger.info(
- "Maximum function calls reached (%d/%d). Stopping further function calls for this request.",
- total_function_calls,
- max_function_calls,
- )
- mutable_options["tool_choice"] = "none"
- errors_in_a_row = result.get("errors_in_a_row", errors_in_a_row)
-
- # When tool_choice is 'required', reset tool_choice after one iteration to avoid infinite loops
- if mutable_options.get("tool_choice") == "required" or (
- isinstance(mutable_options.get("tool_choice"), dict)
- and mutable_options.get("tool_choice", {}).get("mode") == "required"
- ):
- mutable_options["tool_choice"] = None # reset to default for next iteration
-
- if response.conversation_id is not None:
- # For conversation-based APIs, the server already has the function call message.
- # Only send the new function result message (added by _handle_function_call_results).
- prepped_messages.clear()
- if response.messages:
- prepped_messages.append(response.messages[-1])
- else:
- prepped_messages.extend(response.messages)
- continue
-
- # Loop exhausted all iterations (or function invocation disabled).
- # Make a final model call with tool_choice="none" so the model
- # produces a plain text answer instead of leaving orphaned
- # function_call items without matching results.
- if response is not None and self.function_invocation_configuration.get("enabled", True):
- logger.info(
- "Maximum iterations reached (%d). Requesting final response without tools.",
- self.function_invocation_configuration.get("max_iterations", DEFAULT_MAX_ITERATIONS),
- )
- mutable_options["tool_choice"] = "none"
- response = cast(
- ChatResponse[Any],
- await super_get_response(
- messages=prepped_messages,
- stream=False,
- options=mutable_options,
- compaction_strategy=compaction_strategy,
- tokenizer=tokenizer,
- client_kwargs=filtered_kwargs,
- ),
- )
- response = _ensure_function_invocation_limit_fallback_response(response)
- aggregated_usage = add_usage_details(aggregated_usage, response.usage_details)
- _update_continuation_state(
- filtered_kwargs,
- response,
- session=invocation_session,
- options=mutable_options,
- )
- response.usage_details = aggregated_usage
- if fcc_messages:
- for msg in reversed(fcc_messages):
- response.messages.insert(0, msg)
- return _clear_internal_conversation_id(response)
-
- return _get_response()
-
- response_format = mutable_options.get("response_format") if mutable_options else None
- stream_result_hooks: list[Callable[[ChatResponse], Any]] = []
-
- async def _stream() -> AsyncIterable[ChatResponseUpdate]:
- nonlocal filtered_kwargs
- nonlocal mutable_options
- nonlocal stream_result_hooks
- errors_in_a_row: int = 0
- total_function_calls = int(budget_state.get("total_function_calls", 0) or 0)
- max_function_calls: int | None = self.function_invocation_configuration.get("max_function_calls")
- prepped_messages = list(messages)
- fcc_messages: list[Message] = []
- response: ChatResponse[Any] | None = None
-
- loop_enabled = self.function_invocation_configuration.get("enabled", True)
- max_iterations = self.function_invocation_configuration.get("max_iterations", DEFAULT_MAX_ITERATIONS)
- attempt_start = int(budget_state.get("attempt_count", 0) or 0)
- for attempt_idx in range(attempt_start, max_iterations if loop_enabled else 0):
- budget_state["attempt_count"] = attempt_idx + 1
- approval_result = await _process_function_requests(
- response=None,
- prepped_messages=prepped_messages,
- tool_options=mutable_options,
- attempt_idx=attempt_idx,
- fcc_messages=None,
- errors_in_a_row=errors_in_a_row,
- max_errors=max_errors,
- execute_function_calls=execute_function_calls,
- invocation_session=invocation_session,
- )
- errors_in_a_row = approval_result.get("errors_in_a_row", errors_in_a_row)
- total_function_calls += approval_result.get("function_call_count", 0)
- budget_state["total_function_calls"] = total_function_calls
- if max_function_calls is not None and total_function_calls >= max_function_calls:
- logger.info(
- "Maximum function calls reached (%d/%d). Stopping further function calls for this request.",
- total_function_calls,
- max_function_calls,
- )
- mutable_options["tool_choice"] = "none"
- if approval_result.get("action") == "stop":
- mutable_options["tool_choice"] = "none"
- return
-
- inner_stream = cast(
- ResponseStream[ChatResponseUpdate, ChatResponse[Any]],
- super_get_response(
- messages=prepped_messages,
- stream=True,
- options=mutable_options,
- compaction_strategy=compaction_strategy,
- tokenizer=tokenizer,
- client_kwargs=filtered_kwargs,
- ),
- )
- await inner_stream
- # Collect result hooks from the inner stream to run later
- stream_result_hooks[:] = _get_result_hooks_from_stream(inner_stream)
-
- # Yield updates from the inner stream, letting it collect them
- async for update in inner_stream:
- yield update
-
- # Get the finalized response from the inner stream
- # This triggers the inner stream's finalizer and result hooks
- response = await inner_stream.get_final_response()
- response_had_visible_content = _response_has_visible_content(response)
- function_call_limit_reached = (
- mutable_options.get("tool_choice") == "none"
- and max_function_calls is not None
- and total_function_calls >= max_function_calls
- )
- if function_call_limit_reached:
- response = _ensure_function_invocation_limit_fallback_response(response)
- _update_continuation_state(
- filtered_kwargs,
- response,
- session=invocation_session,
- options=mutable_options,
- )
-
- if not any(
- item.type == "function_approval_request" or _is_actionable_function_call(item)
- for msg in response.messages
- for item in msg.contents
- ):
- if function_call_limit_reached and not response_had_visible_content:
- yield _function_invocation_limit_fallback_update()
- return
-
- if response.conversation_id is not None:
- prepped_messages = []
-
- result = await _process_function_requests(
- response=response,
- prepped_messages=None,
- tool_options=mutable_options,
- attempt_idx=attempt_idx,
- fcc_messages=fcc_messages,
- errors_in_a_row=errors_in_a_row,
- max_errors=max_errors,
- execute_function_calls=execute_function_calls,
- invocation_session=invocation_session,
- )
- errors_in_a_row = result.get("errors_in_a_row", errors_in_a_row)
- total_function_calls += result.get("function_call_count", 0)
- budget_state["total_function_calls"] = total_function_calls
- if role := result.get("update_role"):
- yield ChatResponseUpdate(
- contents=result.get("function_call_results") or [],
- role=role,
- )
- if result.get("action") == "stop":
- # Error threshold reached: submit collected function_call_output
- # items once more with tools disabled.
- mutable_options["tool_choice"] = "none"
- elif result.get("action") != "continue":
- return
- elif max_function_calls is not None and total_function_calls >= max_function_calls:
- # Best-effort limit: checked after each batch of parallel calls completes,
- # so the current batch always runs to completion even if it overshoots.
- logger.info(
- "Maximum function calls reached (%d/%d). Stopping further function calls for this request.",
- total_function_calls,
- max_function_calls,
- )
- mutable_options["tool_choice"] = "none"
-
- # When tool_choice is 'required', reset the tool_choice after one iteration to avoid infinite loops
- if mutable_options.get("tool_choice") == "required" or (
- isinstance(mutable_options.get("tool_choice"), dict)
- and mutable_options.get("tool_choice", {}).get("mode") == "required"
- ):
- mutable_options["tool_choice"] = None # reset to default for next iteration
-
- if response.conversation_id is not None:
- # For conversation-based APIs, the server already has the function call message.
- # Only send the new function result message (the last one added by _handle_function_call_results).
- prepped_messages.clear()
- if response.messages:
- prepped_messages.append(response.messages[-1])
- else:
- prepped_messages.extend(response.messages)
- continue
-
- # Loop exhausted all iterations (or function invocation disabled).
- # Make a final model call with tool_choice="none" so the model
- # produces a plain text answer instead of leaving orphaned
- # function_call items without matching results.
- if response is not None and self.function_invocation_configuration.get("enabled", True):
- logger.info(
- "Maximum iterations reached (%d). Requesting final response without tools.",
- self.function_invocation_configuration.get("max_iterations", DEFAULT_MAX_ITERATIONS),
- )
- mutable_options["tool_choice"] = "none"
- final_inner_stream = cast(
- ResponseStream[ChatResponseUpdate, ChatResponse[Any]],
- super_get_response(
- messages=prepped_messages,
- stream=True,
- options=mutable_options,
- compaction_strategy=compaction_strategy,
- tokenizer=tokenizer,
- client_kwargs=filtered_kwargs,
- ),
- )
- await final_inner_stream
- async for update in final_inner_stream:
- yield update
- # Finalize the inner stream to trigger its hooks
- final_response = await final_inner_stream.get_final_response()
- final_response_had_visible_content = _response_has_visible_content(final_response)
- final_response = _ensure_function_invocation_limit_fallback_response(final_response)
- _update_continuation_state(
- filtered_kwargs,
- final_response,
- session=invocation_session,
+ return self._get_response_with_function_invocation(
+ super_get_response=super_get_response,
+ messages=messages,
options=mutable_options,
+ request_kwargs=request_kwargs,
+ compaction_strategy=compaction_strategy,
+ tokenizer=tokenizer,
+ execute_function_calls=execute_function_calls,
+ invocation_session=invocation_session,
+ budget_state=budget_state,
+ max_errors=max_errors,
)
- if not final_response_had_visible_content:
- yield _function_invocation_limit_fallback_update()
- def _finalize(updates: Sequence[ChatResponseUpdate]) -> ChatResponse[Any]:
- # Note: stream_result_hooks are already run via inner stream's get_final_response()
- # We don't need to run them again here
- return ChatResponse.from_updates(updates, output_format_type=response_format)
-
- return ResponseStream(_stream(), finalizer=_finalize)
+ response_format = mutable_options.get("response_format")
+ return ResponseStream(
+ self._stream_response_with_function_invocation(
+ super_get_response=super_get_response,
+ messages=messages,
+ options=mutable_options,
+ request_kwargs=request_kwargs,
+ compaction_strategy=compaction_strategy,
+ tokenizer=tokenizer,
+ execute_function_calls=execute_function_calls,
+ invocation_session=invocation_session,
+ budget_state=budget_state,
+ max_errors=max_errors,
+ ),
+ finalizer=partial(ChatResponse.from_updates, output_format_type=response_format),
+ )
# Alias for the @tool decorator, used by security tools and samples
diff --git a/python/packages/core/agent_framework/_types.py b/python/packages/core/agent_framework/_types.py
index b09471d541e..7a0a982ba8e 100644
--- a/python/packages/core/agent_framework/_types.py
+++ b/python/packages/core/agent_framework/_types.py
@@ -1558,6 +1558,8 @@ def _add_function_call_content(self, other: Content) -> Content:
call_id=self_call_id,
name=getattr(self, "name", None) or getattr(other, "name", None),
arguments=arguments,
+ id=self.id or other.id,
+ user_input_request=self.user_input_request or other.user_input_request,
exception=getattr(self, "exception", None) or getattr(other, "exception", None),
informational_only=getattr(self, "informational_only", False)
or getattr(other, "informational_only", False),
diff --git a/python/packages/core/tests/core/test_compaction.py b/python/packages/core/tests/core/test_compaction.py
index 4507d39946f..b8a4b9e96fd 100644
--- a/python/packages/core/tests/core/test_compaction.py
+++ b/python/packages/core/tests/core/test_compaction.py
@@ -12,6 +12,7 @@
GROUP_ANNOTATION_KEY,
GROUP_HAS_REASONING_KEY,
GROUP_ID_KEY,
+ GROUP_INDEX_KEY,
GROUP_KIND_KEY,
GROUP_TOKEN_COUNT_KEY,
SUMMARIZED_BY_SUMMARY_ID_KEY,
@@ -113,6 +114,14 @@ def _group_kind(message: Message) -> str | None:
return value if isinstance(value, str) else None
+def _group_index(message: Message) -> int | None:
+ annotation = message.additional_properties.get(GROUP_ANNOTATION_KEY)
+ if not isinstance(annotation, dict):
+ return None
+ value = annotation.get(GROUP_INDEX_KEY)
+ return value if isinstance(value, int) else None
+
+
def _group_has_reasoning(message: Message) -> bool | None:
annotation = message.additional_properties.get(GROUP_ANNOTATION_KEY)
if not isinstance(annotation, dict):
@@ -186,6 +195,107 @@ def test_group_annotations_handle_same_message_reasoning_and_function_calls() ->
assert _group_has_reasoning(messages[1]) is True
+def test_group_annotations_pair_nonadjacent_function_result_by_call_id() -> None:
+ messages = [
+ _assistant_reasoning_and_function_calls("c1"),
+ Message(role="assistant", contents=["approval completed"]),
+ _tool_result("c1", "ok"),
+ ]
+
+ annotate_message_groups(messages)
+
+ call_group = _group_id(messages[0])
+ assert call_group is not None
+ assert _group_id(messages[2]) == call_group
+ assert _group_index(messages[2]) == _group_index(messages[0])
+ assert _group_has_reasoning(messages[2]) is True
+ assert _group_id(messages[1]) != call_group
+
+
+def test_group_annotations_pair_multiple_nonadjacent_results_with_declaration() -> None:
+ messages = [
+ _assistant_reasoning_and_function_calls("c1", "c2"),
+ Message(role="assistant", contents=["first approval"]),
+ _tool_result("c1", "ok1"),
+ Message(role="assistant", contents=["second approval"]),
+ _tool_result("c2", "ok2"),
+ ]
+
+ annotate_message_groups(messages)
+
+ call_group = _group_id(messages[0])
+ assert call_group is not None
+ assert _group_id(messages[2]) == call_group
+ assert _group_id(messages[4]) == call_group
+ assert _group_index(messages[2]) == _group_index(messages[0])
+ assert _group_index(messages[4]) == _group_index(messages[0])
+
+
+def test_group_annotations_merge_declaration_groups_for_combined_result_message() -> None:
+ messages = [
+ _assistant_function_call("c1"),
+ Message(role="assistant", contents=["between calls"]),
+ _assistant_function_call("c2"),
+ Message(role="assistant", contents=["approval completed"]),
+ Message(
+ role="tool",
+ contents=[
+ Content.from_function_result(call_id="c1", result="ok1"),
+ Content.from_function_result(call_id="c2", result="ok2"),
+ ],
+ ),
+ ]
+
+ annotate_message_groups(messages)
+
+ call_group = _group_id(messages[0])
+ assert call_group is not None
+ assert _group_id(messages[2]) == call_group
+ assert _group_id(messages[4]) == call_group
+ assert _group_index(messages[2]) == _group_index(messages[0])
+ assert _group_index(messages[4]) == _group_index(messages[0])
+
+
+def test_group_annotations_leave_unmatched_result_separate_from_pending_call() -> None:
+ messages = [
+ _assistant_function_call("pending"),
+ Message(role="assistant", contents=["waiting"]),
+ _tool_result("unknown", "result"),
+ ]
+
+ annotate_message_groups(messages)
+
+ assert _group_id(messages[0]) != _group_id(messages[2])
+
+
+def test_group_annotations_do_not_pair_result_before_declaration() -> None:
+ messages = [
+ _tool_result("late", "result"),
+ Message(role="assistant", contents=["between"]),
+ _assistant_function_call("late"),
+ ]
+
+ annotate_message_groups(messages)
+
+ assert _group_id(messages[0]) != _group_id(messages[2])
+
+
+def test_group_annotations_do_not_pair_ambiguous_duplicate_call_ids() -> None:
+ messages = [
+ _assistant_function_call("duplicate"),
+ Message(role="assistant", contents=["between declarations"]),
+ _assistant_function_call("duplicate"),
+ Message(role="assistant", contents=["before result"]),
+ _tool_result("duplicate", "result"),
+ ]
+
+ annotate_message_groups(messages)
+
+ result_group = _group_id(messages[4])
+ assert result_group != _group_id(messages[0])
+ assert result_group != _group_id(messages[2])
+
+
async def test_sliding_window_keeps_reasoning_and_mcp_call_atomic() -> None:
messages = [
Message(role="system", contents=["system"]),
@@ -258,6 +368,64 @@ def test_extend_compaction_messages_preserves_existing_annotations_and_tokens()
assert isinstance(_token_count(messages[1]), int)
+def test_extend_compaction_messages_pairs_nonadjacent_result_incrementally() -> None:
+ tokenizer = CharacterEstimatorTokenizer()
+ messages = [
+ _assistant_function_call("c4"),
+ Message(role="assistant", contents=["approval completed"]),
+ ]
+ annotate_message_groups(messages, tokenizer=tokenizer)
+ call_group = _group_id(messages[0])
+ intervening_group = _group_id(messages[1])
+
+ extend_compaction_messages(messages, [_tool_result("c4", "ok")], tokenizer=tokenizer)
+
+ assert _group_id(messages[0]) == call_group
+ assert _group_id(messages[1]) == intervening_group
+ assert _group_id(messages[2]) == call_group
+ assert _group_index(messages[2]) == _group_index(messages[0])
+ assert isinstance(_token_count(messages[2]), int)
+
+ append_compaction_message(
+ messages,
+ Message(role="assistant", contents=["final answer"]),
+ tokenizer=tokenizer,
+ )
+
+ assert _group_id(messages[3]) not in {call_group, intervening_group}
+ assert _group_index(messages[3]) == 2
+
+
+def test_extend_compaction_messages_reincludes_excluded_declaration_for_new_result() -> None:
+ messages = [
+ _assistant_function_call("c5"),
+ Message(role="assistant", contents=["approval pending"]),
+ ]
+ annotate_message_groups(messages)
+ messages[0].additional_properties[EXCLUDED_KEY] = True
+
+ extend_compaction_messages(messages, [_tool_result("c5", "ok")])
+
+ assert messages[0].additional_properties[EXCLUDED_KEY] is False
+ assert messages[2].additional_properties[EXCLUDED_KEY] is False
+ assert _group_id(messages[2]) == _group_id(messages[0])
+
+
+def test_extend_compaction_messages_preserves_adjacent_duplicate_call_pair() -> None:
+ messages = [
+ _assistant_function_call("duplicate"),
+ Message(role="assistant", contents=["between declarations"]),
+ _assistant_function_call("duplicate"),
+ ]
+ annotate_message_groups(messages)
+
+ extend_compaction_messages(messages, [_tool_result("duplicate", "result")])
+
+ result_group = _group_id(messages[3])
+ assert result_group != _group_id(messages[0])
+ assert result_group == _group_id(messages[2])
+
+
def test_append_compaction_message_annotates_new_message() -> None:
messages = [Message(role="user", contents=["hello"])]
annotate_message_groups(messages)
@@ -383,6 +551,35 @@ async def test_truncation_strategy_keeps_latest_group_when_it_exceeds_target() -
assert included_messages(messages) == messages
+async def test_truncation_strategy_keeps_nonadjacent_tool_pair_atomic() -> None:
+ tokenizer = CharacterEstimatorTokenizer()
+ messages = [
+ Message(role="user", contents=["original request " + "x" * 1600]),
+ _assistant_function_call("call-1"),
+ Message(role="assistant", contents=["intervening approval traffic " + "y" * 1600]),
+ _tool_result("call-1", "result"),
+ Message(role="user", contents=["follow up " + "z" * 400]),
+ ]
+ strategy = TruncationStrategy(
+ max_n=600,
+ compact_to=520,
+ tokenizer=tokenizer,
+ )
+ annotate_message_groups(messages, tokenizer=tokenizer)
+
+ changed = await strategy(messages)
+
+ assert changed is True
+ projected = included_messages(messages)
+ declared = {
+ content.call_id for message in projected for content in message.contents if content.type == "function_call"
+ }
+ results = {
+ content.call_id for message in projected for content in message.contents if content.type == "function_result"
+ }
+ assert declared == results
+
+
def test_truncation_strategy_validates_token_targets() -> None:
try:
TruncationStrategy(max_n=3, compact_to=4)
diff --git a/python/packages/core/tests/core/test_function_invocation_logic.py b/python/packages/core/tests/core/test_function_invocation_logic.py
index ae01d82a226..263f2ed6370 100644
--- a/python/packages/core/tests/core/test_function_invocation_logic.py
+++ b/python/packages/core/tests/core/test_function_invocation_logic.py
@@ -2,7 +2,7 @@
import asyncio
from collections.abc import AsyncIterable, Awaitable, Callable, Sequence
-from typing import Any
+from typing import Any, Literal
import pytest
@@ -96,6 +96,54 @@ def _finalize(updates: Sequence[ChatResponseUpdate]) -> ChatResponse:
chat_client_base._get_streaming_response = _get_streaming_response
+def _post_limit_tool_content(
+ content_type: Literal["function_call", "function_approval_request"],
+) -> Content:
+ function_call = Content.from_function_call(call_id="call_2", name="lookup", arguments='{"key": "b"}')
+ if content_type == "function_call":
+ return function_call
+ return Content.from_function_approval_request(id="approval_2", function_call=function_call)
+
+
+def _force_tool_content_tool_choice_none_stream(
+ chat_client_base: Any,
+ *,
+ content: Content,
+ additional_properties: dict[str, Any] | None = None,
+ finish_reason: Literal["stop", "length", "tool_calls", "content_filter"] | None = None,
+) -> None:
+ original_get_streaming_response = chat_client_base._get_streaming_response
+
+ def _get_streaming_response(
+ *,
+ messages: Sequence[Message],
+ options: dict[str, Any],
+ **kwargs: Any,
+ ) -> ResponseStream[ChatResponseUpdate, ChatResponse]:
+ if options.get("tool_choice") != "none":
+ return original_get_streaming_response(messages=messages, options=options, **kwargs)
+
+ updates = (
+ ChatResponseUpdate(
+ contents=[content],
+ role="assistant",
+ additional_properties=additional_properties,
+ finish_reason=finish_reason,
+ ),
+ )
+
+ async def _stream() -> AsyncIterable[ChatResponseUpdate]:
+ for update in updates:
+ yield update
+
+ def _finalize(stream_updates: Sequence[ChatResponseUpdate]) -> ChatResponse:
+ return ChatResponse.from_updates(stream_updates, output_format_type=options.get("response_format"))
+
+ return ResponseStream(_stream(), finalizer=_finalize)
+
+ chat_client_base._get_streaming_response = _get_streaming_response
+
+
async def test_base_client_with_function_calling(chat_client_base: SupportsChatGetResponse):
exec_counter = 0
@@ -946,15 +994,23 @@ def func_rejected(arg1: str) -> str:
all_messages = response.messages + [Message(role="user", contents=[approved_response, rejected_response])]
# Call get_response which will process the approvals
- await chat_client_base.get_response(
+ resumed_response = await chat_client_base.get_response(
all_messages, options={"tool_choice": "auto", "tools": [func_approved, func_rejected]}
)
+ assert [[content.type for content in message.contents] for message in resumed_response.messages] == [
+ ["function_result", "function_result"],
+ ["text"],
+ ]
+ assert all_messages[-1].role == "user"
+ assert [content.type for content in all_messages[-1].contents] == [
+ "function_approval_response",
+ "function_approval_response",
+ ]
# Verify the approval/rejection was processed correctly
- # Find the results in the input messages (modified in-place)
approved_result = None
rejected_result = None
- for msg in all_messages:
+ for msg in resumed_response.messages:
for content in msg.contents:
if content.type == "function_result":
if content.call_id == "1":
@@ -974,7 +1030,7 @@ def func_rejected(arg1: str) -> str:
# Verify that messages with FunctionResultContent have role="tool"
# This ensures the message format is correct for OpenAI's API
- for msg in all_messages:
+ for msg in resumed_response.messages:
for content in msg.contents:
if content.type == "function_result":
assert msg.role == "tool", f"Message with FunctionResultContent must have role='tool', got '{msg.role}'"
@@ -1069,6 +1125,70 @@ def func_with_approval(arg1: str) -> str:
assert exec_counter == 1
+async def test_resolved_approval_response_is_inert_on_later_stateless_turn(
+ chat_client_base: SupportsChatGetResponse,
+) -> None:
+ """A terminal result must consume approval authority in an explicitly replayed transcript."""
+ exec_counter = 0
+
+ @tool(name="guarded_stateless_tool", approval_mode="always_require")
+ def guarded_stateless_tool() -> str:
+ nonlocal exec_counter
+ exec_counter += 1
+ return "approved result"
+
+ chat_client_base.run_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ ChatResponse(
+ messages=Message(
+ role="assistant",
+ contents=[
+ Content.from_function_call(
+ call_id="call_guarded_stateless",
+ name="guarded_stateless_tool",
+ arguments="{}",
+ )
+ ],
+ )
+ ),
+ ChatResponse(messages=Message(role="assistant", contents=["done"])),
+ ChatResponse(messages=Message(role="assistant", contents=["later"])),
+ ]
+
+ first_response = await chat_client_base.get_response(
+ [Message(role="user", contents=["run guarded"])],
+ options={"tools": [guarded_stateless_tool]},
+ )
+ approval_request = next(
+ content
+ for message in first_response.messages
+ for content in message.contents
+ if content.type == "function_approval_request"
+ )
+ approval_message = Message(
+ role="user",
+ contents=[approval_request.to_function_approval_response(approved=True)],
+ )
+ second_response = await chat_client_base.get_response(
+ [Message(role="user", contents=["run guarded"]), *first_response.messages, approval_message],
+ options={"tools": [guarded_stateless_tool]},
+ )
+ assert exec_counter == 1
+
+ later_response = await chat_client_base.get_response(
+ [
+ Message(role="user", contents=["run guarded"]),
+ *first_response.messages,
+ approval_message,
+ *second_response.messages,
+ Message(role="user", contents=["unrelated later turn"]),
+ ],
+ options={"tools": [guarded_stateless_tool]},
+ )
+
+ assert later_response.text == "later"
+ assert exec_counter == 1
+
+
async def test_no_duplicate_function_calls_after_approval_processing(chat_client_base: SupportsChatGetResponse):
"""Processing approval should not create duplicate function calls in messages."""
@@ -1146,11 +1266,14 @@ def func_with_approval(arg1: str) -> str:
)
all_messages = response1.messages + [Message(role="user", contents=[rejection_response])]
- await chat_client_base.get_response(all_messages, options={"tool_choice": "auto", "tools": [func_with_approval]})
+ resumed_response = await chat_client_base.get_response(
+ all_messages,
+ options={"tool_choice": "auto", "tools": [func_with_approval]},
+ )
# Find the rejection result
rejection_result = next(
- (content for msg in all_messages for content in msg.contents if content.type == "function_result"),
+ (content for msg in resumed_response.messages for content in msg.contents if content.type == "function_result"),
None,
)
@@ -2305,6 +2428,33 @@ def test_is_hosted_tool_approval_without_server_label():
assert _is_hosted_tool_approval("not a content") is False
+def test_collect_approval_responses_consumes_matching_follow_up_request_occurrence() -> None:
+ """A follow-up user-input request resolves only the preceding approval occurrence for a reused call id."""
+ from agent_framework._tools import _collect_approval_responses
+
+ _, _, first_response = _build_approved_tool_roundtrip(
+ call_id="call_reused",
+ approval_id="approval_1",
+ tool_name="guarded_tool",
+ )
+ _, _, second_response = _build_approved_tool_roundtrip(
+ call_id="call_reused",
+ approval_id="approval_2",
+ tool_name="guarded_tool",
+ )
+ follow_up_request = Content.from_oauth_consent_request(consent_link="https://example.com/consent")
+ follow_up_request.call_id = "call_reused"
+ messages = [
+ Message(role="user", contents=[first_response]),
+ Message(role="assistant", contents=[follow_up_request]),
+ Message(role="user", contents=[second_response]),
+ ]
+
+ pending_responses = _collect_approval_responses(messages)
+
+ assert pending_responses == {"approval_2": second_response}
+
+
def test_replace_approval_contents_with_results_uses_result_call_ids_without_placeholders() -> None:
from agent_framework._tools import _collect_approval_responses, _replace_approval_contents_with_results
@@ -2324,8 +2474,8 @@ def test_replace_approval_contents_with_results_uses_result_call_ids_without_pla
messages,
_collect_approval_responses(messages),
[
- Content.from_function_result(call_id="call_2", result="second result"),
- Content.from_function_result(call_id="call_1", result="first result"),
+ [Content.from_function_result(call_id="call_2", result="second result")],
+ [Content.from_function_result(call_id="call_1", result="first result")],
],
)
@@ -2363,7 +2513,7 @@ def test_replace_approval_contents_with_results_allows_reused_call_id_after_comp
_replace_approval_contents_with_results(
messages,
_collect_approval_responses(messages),
- [Content.from_function_result(call_id="call_reused", result="second output")],
+ [[Content.from_function_result(call_id="call_reused", result="second output")]],
)
function_calls = [c for m in messages for c in m.contents if c.type == "function_call"]
@@ -2374,6 +2524,247 @@ def test_replace_approval_contents_with_results_allows_reused_call_id_after_comp
("call_reused", "second output"),
]
+
+def test_replace_approval_contents_with_results_deduplicates_replayed_approval_request() -> None:
+ """A replayed wrapper must not restore another copy of an already-present function call."""
+ from agent_framework._tools import _collect_approval_responses, _replace_approval_contents_with_results
+
+ function_call, request, response = _build_approved_tool_roundtrip(
+ call_id="call_1",
+ approval_id="approval_1",
+ tool_name="guarded_tool",
+ )
+ replayed_request = Content.from_dict(request.to_dict())
+ messages = [
+ Message(role="assistant", contents=[function_call]),
+ Message(role="assistant", contents=[request]),
+ Message(role="assistant", contents=[replayed_request]),
+ Message(role="user", contents=[response]),
+ ]
+
+ _replace_approval_contents_with_results(
+ messages,
+ _collect_approval_responses(messages),
+ [[Content.from_function_result(call_id="call_1", result="done")]],
+ )
+
+ calls = [content for message in messages for content in message.contents if content.type == "function_call"]
+ results = [content for message in messages for content in message.contents if content.type == "function_result"]
+ assert [content.call_id for content in calls] == ["call_1"]
+ assert [(content.call_id, content.result) for content in results] == [("call_1", "done")]
+
+
+def test_replace_approval_contents_with_results_ignores_already_resolved_response() -> None:
+ """Historical approval responses with terminal results must not become rejection results."""
+ from agent_framework._tools import _collect_approval_responses, _replace_approval_contents_with_results
+
+ old_call, old_request, old_response = _build_approved_tool_roundtrip(
+ call_id="call_reused",
+ approval_id="approval_old",
+ tool_name="guarded_tool",
+ )
+ new_call, new_request, new_response = _build_approved_tool_roundtrip(
+ call_id="call_reused",
+ approval_id="approval_new",
+ tool_name="guarded_tool",
+ )
+ messages = [
+ Message(role="assistant", contents=[old_call, old_request]),
+ Message(role="user", contents=[old_response]),
+ Message(role="tool", contents=[Content.from_function_result(call_id="call_reused", result="old result")]),
+ Message(role="assistant", contents=[new_call, new_request]),
+ Message(role="user", contents=[new_response]),
+ ]
+
+ _replace_approval_contents_with_results(
+ messages,
+ _collect_approval_responses(messages),
+ [[Content.from_function_result(call_id="call_reused", result="new result")]],
+ )
+
+ assert not [
+ content
+ for message in messages
+ for content in message.contents
+ if content.type in {"function_approval_request", "function_approval_response"}
+ ]
+ results = [content for message in messages for content in message.contents if content.type == "function_result"]
+ assert [(content.call_id, content.result) for content in results] == [
+ ("call_reused", "old result"),
+ ("call_reused", "new result"),
+ ]
+
+
+def test_replace_approval_contents_with_results_correlates_reused_call_id_occurrences() -> None:
+ """Each approval round must retain its own call and terminal result when call ids are reused."""
+ from agent_framework._tools import _collect_approval_responses, _replace_approval_contents_with_results
+
+ completed_call = Content.from_function_call(call_id="call_reused", name="run_skill_script", arguments="{}")
+ completed_result = Content.from_function_result(call_id="call_reused", result="first output")
+ replayed_call, approved_request, approved_response = _build_approved_tool_roundtrip(
+ call_id="call_reused", approval_id="approval_2", tool_name="run_skill_script"
+ )
+ rejected_call, rejected_request, rejected_response = _build_approved_tool_roundtrip(
+ call_id="call_reused", approval_id="approval_3", tool_name="run_skill_script"
+ )
+ rejected_response.approved = False
+
+ messages = [
+ Message(role="assistant", contents=[completed_call]),
+ Message(role="tool", contents=[completed_result]),
+ Message(role="assistant", contents=[replayed_call]),
+ Message(role="assistant", contents=[approved_request]),
+ Message(role="user", contents=[approved_response]),
+ Message(role="assistant", contents=[rejected_call]),
+ Message(role="assistant", contents=[rejected_request]),
+ Message(role="user", contents=[rejected_response]),
+ ]
+
+ _replace_approval_contents_with_results(
+ messages,
+ _collect_approval_responses(messages),
+ [[Content.from_function_result(call_id="call_reused", result="second output")]],
+ )
+
+ calls = [content for message in messages for content in message.contents if content.type == "function_call"]
+ results = [content for message in messages for content in message.contents if content.type == "function_result"]
+ assert [content.call_id for content in calls] == ["call_reused", "call_reused", "call_reused"]
+ assert [(content.call_id, content.result) for content in results] == [
+ ("call_reused", "first output"),
+ ("call_reused", "second output"),
+ ("call_reused", "Error: Tool call invocation was rejected by user."),
+ ]
+
+
+def test_replace_approval_contents_with_results_keeps_multi_content_group_with_reused_call_id() -> None:
+ """All contents from one execution stay with its approval occurrence when call ids are reused."""
+ from agent_framework._tools import _collect_approval_responses, _replace_approval_contents_with_results
+
+ first_call, first_request, first_response = _build_approved_tool_roundtrip(
+ call_id="call_reused",
+ approval_id="approval_1",
+ tool_name="run_skill_script",
+ )
+ second_call, second_request, second_response = _build_approved_tool_roundtrip(
+ call_id="call_reused",
+ approval_id="approval_2",
+ tool_name="run_skill_script",
+ )
+ first_user_requests = [
+ Content.from_oauth_consent_request(consent_link="https://example.com/consent-1"),
+ Content.from_oauth_consent_request(consent_link="https://example.com/consent-2"),
+ ]
+ for request in first_user_requests:
+ request.call_id = "call_reused"
+ second_result = Content.from_function_result(call_id="call_reused", result="second output")
+ messages = [
+ Message(role="assistant", contents=[first_call, first_request]),
+ Message(role="user", contents=[first_response]),
+ Message(role="assistant", contents=[second_call, second_request]),
+ Message(role="user", contents=[second_response]),
+ ]
+
+ resolved_contents = _replace_approval_contents_with_results(
+ messages,
+ _collect_approval_responses(messages),
+ [first_user_requests, [second_result]],
+ )
+
+ user_requests = [content for message in messages for content in message.contents if content.user_input_request]
+ results = [content for message in messages for content in message.contents if content.type == "function_result"]
+ assert [request.consent_link for request in user_requests] == [
+ "https://example.com/consent-1",
+ "https://example.com/consent-2",
+ ]
+ assert results == [second_result]
+ assert resolved_contents == [*first_user_requests, second_result]
+
+
+def test_replace_approval_contents_with_results_correlates_reused_call_id_placeholders() -> None:
+ """Placeholder replacement must consume approved results by approval occurrence, not call id."""
+ from agent_framework._tools import _collect_approval_responses, _replace_approval_contents_with_results
+
+ completed_call = Content.from_function_call(call_id="call_reused", name="run_skill_script", arguments="{}")
+ completed_result = Content.from_function_result(call_id="call_reused", result="first output")
+ second_call, second_request, second_response = _build_approved_tool_roundtrip(
+ call_id="call_reused", approval_id="approval_2", tool_name="run_skill_script"
+ )
+ third_call, third_request, third_response = _build_approved_tool_roundtrip(
+ call_id="call_reused", approval_id="approval_3", tool_name="run_skill_script"
+ )
+
+ messages = [
+ Message(role="assistant", contents=[completed_call]),
+ Message(role="tool", contents=[completed_result]),
+ Message(role="assistant", contents=[second_call, second_request]),
+ Message(
+ role="tool",
+ contents=[Content.from_function_result(call_id="call_reused", result="[APPROVAL_PENDING] second")],
+ ),
+ Message(role="user", contents=[second_response]),
+ Message(role="assistant", contents=[third_call, third_request]),
+ Message(
+ role="tool",
+ contents=[Content.from_function_result(call_id="call_reused", result="[APPROVAL_PENDING] third")],
+ ),
+ Message(role="user", contents=[third_response]),
+ ]
+
+ _replace_approval_contents_with_results(
+ messages,
+ _collect_approval_responses(messages),
+ [
+ [Content.from_function_result(call_id="call_reused", result="second output")],
+ [Content.from_function_result(call_id="call_reused", result="third output")],
+ ],
+ )
+
+ calls = [content for message in messages for content in message.contents if content.type == "function_call"]
+ results = [content for message in messages for content in message.contents if content.type == "function_result"]
+ assert [content.call_id for content in calls] == ["call_reused", "call_reused", "call_reused"]
+ assert [(content.call_id, content.result) for content in results] == [
+ ("call_reused", "first output"),
+ ("call_reused", "second output"),
+ ("call_reused", "third output"),
+ ]
+
+
+def test_replace_approval_contents_with_results_replaces_rejected_placeholder() -> None:
+ """A rejected call should replace its pending placeholder instead of adding a second result."""
+ from agent_framework._tools import _collect_approval_responses, _replace_approval_contents_with_results
+
+ function_call, request, _ = _build_approved_tool_roundtrip(
+ call_id="call_rejected",
+ approval_id="approval_rejected",
+ tool_name="guarded_tool",
+ )
+ rejection = request.to_function_approval_response(approved=False)
+ messages = [
+ Message(role="assistant", contents=[function_call, request]),
+ Message(
+ role="tool",
+ contents=[
+ Content.from_function_result(
+ call_id="call_rejected",
+ result="[APPROVAL_PENDING] guarded_tool",
+ )
+ ],
+ ),
+ Message(role="user", contents=[rejection]),
+ ]
+
+ resolved_contents = _replace_approval_contents_with_results(
+ messages,
+ _collect_approval_responses(messages),
+ [],
+ )
+
+ results = [content for message in messages for content in message.contents if content.type == "function_result"]
+ assert len(results) == 1
+ assert results[0].result == "Error: Tool call invocation was rejected by user."
+ assert resolved_contents == results
+
+
def test_replace_approval_contents_with_results_uses_result_call_ids_for_placeholders() -> None:
from agent_framework._tools import _collect_approval_responses, _replace_approval_contents_with_results
@@ -2400,8 +2791,8 @@ def test_replace_approval_contents_with_results_uses_result_call_ids_for_placeho
messages,
_collect_approval_responses(messages),
[
- Content.from_function_result(call_id="call_2", result="second result"),
- Content.from_function_result(call_id="call_1", result="first result"),
+ [Content.from_function_result(call_id="call_2", result="second result")],
+ [Content.from_function_result(call_id="call_1", result="first result")],
],
)
@@ -2433,8 +2824,10 @@ def test_replace_approval_contents_with_results_skips_results_without_call_id()
messages,
_collect_approval_responses(messages),
[
- Content.from_function_result(call_id=None, result="ignored result"), # type: ignore[arg-type] # pyrefly: ignore[bad-argument-type] # ty: ignore[invalid-argument-type]
- Content.from_function_result(call_id="call_1", result="first result"),
+ [
+ Content.from_function_result(call_id=None, result="ignored result") # type: ignore[arg-type] # pyrefly: ignore[bad-argument-type] # ty: ignore[invalid-argument-type]
+ ],
+ [Content.from_function_result(call_id="call_1", result="first result")],
],
)
@@ -2478,8 +2871,8 @@ def test_replace_approval_contents_with_results_prunes_emptied_messages() -> Non
messages,
_collect_approval_responses(messages),
[
- Content.from_function_result(call_id="call_1", result="first result"),
- Content.from_function_result(call_id="call_2", result="second result"),
+ [Content.from_function_result(call_id="call_1", result="first result")],
+ [Content.from_function_result(call_id="call_2", result="second result")],
],
)
@@ -2590,13 +2983,16 @@ def test_func(arg1: str) -> str:
all_messages = response1.messages + [Message(role="user", contents=[rejection_response])]
# This should handle the rejection gracefully (not raise ToolException to user)
- await chat_client_base.get_response(all_messages, options={"tool_choice": "auto", "tools": [test_func]})
+ resumed_response = await chat_client_base.get_response(
+ all_messages,
+ options={"tool_choice": "auto", "tools": [test_func]},
+ )
# Should have a rejection result
rejection_result = next(
(
content
- for msg in all_messages
+ for msg in resumed_response.messages
for content in msg.contents
if content.type == "function_result" and "rejected" in (content.result or content.exception or "").lower()
),
@@ -2649,7 +3045,10 @@ def error_func(arg1: str) -> str:
all_messages = response1.messages + [Message(role="user", contents=[approval_response])]
# Execute the approved function (which will error)
- await chat_client_base.get_response(all_messages, options={"tool_choice": "auto", "tools": [error_func]})
+ resumed_response = await chat_client_base.get_response(
+ all_messages,
+ options={"tool_choice": "auto", "tools": [error_func]},
+ )
# Should have executed the function
assert exec_counter == 1
@@ -2658,7 +3057,7 @@ def error_func(arg1: str) -> str:
error_result = next(
(
content
- for msg in all_messages
+ for msg in resumed_response.messages
for content in msg.contents
if content.type == "function_result" and content.exception is not None
),
@@ -2714,7 +3113,10 @@ def error_func(arg1: str) -> str:
all_messages = response1.messages + [Message(role="user", contents=[approval_response])]
# Execute the approved function (which will error)
- await chat_client_base.get_response(all_messages, options={"tool_choice": "auto", "tools": [error_func]})
+ resumed_response = await chat_client_base.get_response(
+ all_messages,
+ options={"tool_choice": "auto", "tools": [error_func]},
+ )
# Should have executed the function
assert exec_counter == 1
@@ -2723,7 +3125,7 @@ def error_func(arg1: str) -> str:
error_result = next(
(
content
- for msg in all_messages
+ for msg in resumed_response.messages
for content in msg.contents
if content.type == "function_result" and content.exception is not None
),
@@ -2779,7 +3181,10 @@ def typed_func(arg1: int) -> str: # Expects int, not str
all_messages = response1.messages + [Message(role="user", contents=[approval_response])]
# Execute the approved function (which will fail validation)
- await chat_client_base.get_response(all_messages, options={"tool_choice": "auto", "tools": [typed_func]})
+ resumed_response = await chat_client_base.get_response(
+ all_messages,
+ options={"tool_choice": "auto", "tools": [typed_func]},
+ )
# Should NOT have executed the function (validation failed before execution)
assert exec_counter == 0
@@ -2788,7 +3193,7 @@ def typed_func(arg1: int) -> str: # Expects int, not str
error_result = next(
(
content
- for msg in all_messages
+ for msg in resumed_response.messages
for content in msg.contents
if content.type == "function_result" and content.exception is not None
),
@@ -2837,7 +3242,10 @@ def success_func(arg1: str) -> str:
all_messages = response1.messages + [Message(role="user", contents=[approval_response])]
# Execute the approved function
- await chat_client_base.get_response(all_messages, options={"tool_choice": "auto", "tools": [success_func]})
+ resumed_response = await chat_client_base.get_response(
+ all_messages,
+ options={"tool_choice": "auto", "tools": [success_func]},
+ )
# Should have executed successfully
assert exec_counter == 1
@@ -2846,7 +3254,7 @@ def success_func(arg1: str) -> str:
success_result = next(
(
content
- for msg in all_messages
+ for msg in resumed_response.messages
for content in msg.contents
if content.type == "function_result" and content.exception is None
),
@@ -2907,6 +3315,68 @@ async def test_declaration_only_tool(chat_client_base: SupportsChatGetResponse):
assert len(function_results) == 0
+@pytest.mark.parametrize(
+ "argument_chunks",
+ [
+ ['{"location":', '"Seattle"}'],
+ ['{"location":"Seattle"}'],
+ ],
+ ids=["split-arguments", "single-chunk"],
+)
+async def test_streaming_declaration_only_tool_preserves_metadata_without_duplicate_arguments(
+ chat_client_base: SupportsChatGetResponse,
+ argument_chunks: list[str],
+) -> None:
+ """Declaration-only streaming emits metadata once without replaying already-streamed arguments."""
+ from agent_framework import FunctionTool
+
+ declaration_tool = FunctionTool(
+ name="get_weather",
+ func=None,
+ description="Get the weather",
+ input_model={"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]},
+ )
+ chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ [
+ ChatResponseUpdate(
+ contents=[
+ Content.from_function_call(
+ call_id="call_weather",
+ name="get_weather",
+ arguments=arguments,
+ )
+ ],
+ role="assistant",
+ )
+ for arguments in argument_chunks
+ ]
+ ]
+
+ stream = chat_client_base.get_response(
+ [Message(role="user", contents=["What's the weather in Seattle?"])],
+ options={"tool_choice": "auto", "tools": [declaration_tool]},
+ stream=True,
+ )
+ metadata_updates: list[tuple[Any, str | None]] = []
+ async for update in stream:
+ for content in update.contents:
+ if content.type == "function_call" and content.call_id == "call_weather" and content.user_input_request:
+ metadata_updates.append((content.arguments, content.id))
+ final_response = await stream.get_final_response()
+ function_calls = [
+ content
+ for message in final_response.messages
+ for content in message.contents
+ if content.type == "function_call" and content.call_id == "call_weather"
+ ]
+
+ assert metadata_updates == [(None, "call_weather")]
+ assert len(function_calls) == 1
+ assert function_calls[0].arguments == '{"location":"Seattle"}'
+ assert function_calls[0].user_input_request is True
+ assert function_calls[0].id == "call_weather"
+
+
async def test_multiple_function_calls_parallel_execution(chat_client_base: SupportsChatGetResponse):
"""Test that multiple function calls are executed in parallel."""
import asyncio
@@ -3284,6 +3754,172 @@ def lookup_func(key: str) -> str:
assert updates[-1].text == _EXPECTED_FUNCTION_INVOCATION_LIMIT_FALLBACK_TEXT
+@pytest.mark.parametrize("limit_type", ["max_function_calls", "max_iterations"])
+@pytest.mark.parametrize("content_type", ["function_call", "function_approval_request"])
+async def test_function_invocation_limit_drops_unexecutable_tool_content(
+ chat_client_base: SupportsChatGetResponse,
+ limit_type: str,
+ content_type: Literal["function_call", "function_approval_request"],
+) -> None:
+ """Tool content returned after the active limit is not exposed without a terminal result."""
+ _force_blank_tool_choice_none_fallback(
+ chat_client_base,
+ final_contents=[_post_limit_tool_content(content_type)],
+ )
+ exec_counter = 0
+
+ @tool(name="lookup", approval_mode="never_require")
+ def lookup_func(key: str) -> str:
+ nonlocal exec_counter
+ exec_counter += 1
+ return f"Value for {key}"
+
+ chat_client_base.run_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ ChatResponse(
+ messages=Message(
+ role="assistant",
+ contents=[Content.from_function_call(call_id="call_1", name="lookup", arguments='{"key": "a"}')],
+ )
+ )
+ ]
+ if limit_type == "max_function_calls":
+ chat_client_base.function_invocation_configuration["max_iterations"] = 10 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ chat_client_base.function_invocation_configuration["max_function_calls"] = 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ else:
+ chat_client_base.function_invocation_configuration["max_iterations"] = 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+
+ response = await chat_client_base.get_response(
+ [Message(role="user", contents=["look up key"])],
+ options={"tool_choice": "auto", "tools": [lookup_func]},
+ )
+
+ function_call_ids = {
+ content.call_id
+ for message in response.messages
+ for content in message.contents
+ if content.type == "function_call"
+ }
+ approval_request_ids = {
+ content.id
+ for message in response.messages
+ for content in message.contents
+ if content.type == "function_approval_request"
+ }
+ assert exec_counter == 1
+ assert function_call_ids == {"call_1"}
+ assert not approval_request_ids
+ assert response.messages[-1].text == _EXPECTED_FUNCTION_INVOCATION_LIMIT_FALLBACK_TEXT
+
+
+@pytest.mark.parametrize("limit_type", ["max_function_calls", "max_iterations"])
+@pytest.mark.parametrize("content_type", ["function_call", "function_approval_request"])
+async def test_streaming_function_invocation_limit_drops_unexecutable_tool_content(
+ chat_client_base: SupportsChatGetResponse,
+ limit_type: str,
+ content_type: Literal["function_call", "function_approval_request"],
+) -> None:
+ """Tool content returned after the active limit is not exposed without a terminal result."""
+ _force_tool_content_tool_choice_none_stream(
+ chat_client_base,
+ content=_post_limit_tool_content(content_type),
+ )
+ exec_counter = 0
+
+ @tool(name="lookup", approval_mode="never_require")
+ def lookup_func(key: str) -> str:
+ nonlocal exec_counter
+ exec_counter += 1
+ return f"Value for {key}"
+
+ chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ [
+ ChatResponseUpdate(
+ contents=[Content.from_function_call(call_id="call_1", name="lookup", arguments='{"key": "a"}')],
+ role="assistant",
+ ),
+ ],
+ ]
+ if limit_type == "max_function_calls":
+ chat_client_base.function_invocation_configuration["max_iterations"] = 10 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ chat_client_base.function_invocation_configuration["max_function_calls"] = 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ else:
+ chat_client_base.function_invocation_configuration["max_iterations"] = 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+
+ updates = [
+ update
+ async for update in chat_client_base.get_response(
+ [Message(role="user", contents=["look up key"])],
+ options={"tool_choice": "auto", "tools": [lookup_func]},
+ stream=True,
+ )
+ ]
+
+ function_call_ids = {
+ content.call_id for update in updates for content in update.contents if content.type == "function_call"
+ }
+ function_result_ids = {
+ content.call_id for update in updates for content in update.contents if content.type == "function_result"
+ }
+ approval_request_ids = {
+ content.id for update in updates for content in update.contents if content.type == "function_approval_request"
+ }
+ assert exec_counter == 1
+ assert function_call_ids == {"call_1"}
+ assert function_result_ids == {"call_1"}
+ assert not approval_request_ids
+ assert updates[-1].text == _EXPECTED_FUNCTION_INVOCATION_LIMIT_FALLBACK_TEXT
+
+
+@pytest.mark.parametrize("limit_type", ["max_function_calls", "max_iterations"])
+@pytest.mark.parametrize("content_type", ["function_call", "function_approval_request"])
+async def test_streaming_function_invocation_limit_preserves_metadata_after_tool_content_is_dropped(
+ chat_client_base: SupportsChatGetResponse,
+ limit_type: str,
+ content_type: Literal["function_call", "function_approval_request"],
+) -> None:
+ """Metadata-only chunks survive when their unexecutable function call content is removed."""
+ _force_tool_content_tool_choice_none_stream(
+ chat_client_base,
+ content=_post_limit_tool_content(content_type),
+ additional_properties={"provider_metadata": "keep"},
+ finish_reason="stop",
+ )
+
+ @tool(name="lookup", approval_mode="never_require")
+ def lookup_func(key: str) -> str:
+ return f"Value for {key}"
+
+ chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ [
+ ChatResponseUpdate(
+ contents=[Content.from_function_call(call_id="call_1", name="lookup", arguments='{"key": "a"}')],
+ role="assistant",
+ ),
+ ],
+ ]
+ if limit_type == "max_function_calls":
+ chat_client_base.function_invocation_configuration["max_iterations"] = 10 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ chat_client_base.function_invocation_configuration["max_function_calls"] = 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ else:
+ chat_client_base.function_invocation_configuration["max_iterations"] = 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+
+ updates = [
+ update
+ async for update in chat_client_base.get_response(
+ [Message(role="user", contents=["look up key"])],
+ options={"tool_choice": "auto", "tools": [lookup_func]},
+ stream=True,
+ )
+ ]
+ metadata_updates = [
+ update
+ for update in updates
+ if update.additional_properties == {"provider_metadata": "keep"} and update.finish_reason == "stop"
+ ]
+ assert len(metadata_updates) == 1
+ assert metadata_updates[0].contents == []
+
+
async def test_streaming_function_invocation_config_enabled_false(chat_client_base: SupportsChatGetResponse):
"""Test that setting enabled=False disables function invocation in streaming mode."""
exec_counter = 0
@@ -3818,6 +4454,315 @@ async def process(self, context: FunctionInvocationContext, next_handler: Callab
raise MiddlewareTermination
+@pytest.mark.parametrize("approved", [True, False], ids=["approved", "rejected"])
+async def test_streaming_approval_resume_yields_terminal_result_before_model_text(
+ chat_client_base: SupportsChatGetResponse,
+ approved: bool,
+) -> None:
+ """The streaming invocation layer emits one terminal result before continuing to model text."""
+ calls = 0
+
+ @tool(name="guarded_stream_tool", approval_mode="always_require")
+ def guarded_stream_tool() -> str:
+ nonlocal calls
+ calls += 1
+ return "approved output"
+
+ function_call = Content.from_function_call(
+ call_id="call_stream_approval",
+ name="guarded_stream_tool",
+ arguments="{}",
+ )
+ chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ [ChatResponseUpdate(role="assistant", contents=[function_call])],
+ [ChatResponseUpdate(role="assistant", contents=[Content.from_text("done")])],
+ ]
+ first_stream = chat_client_base.get_response(
+ [Message(role="user", contents=["run guarded"])],
+ stream=True,
+ options={"tools": [guarded_stream_tool]},
+ )
+ first_updates = [update async for update in first_stream]
+ approval_request = next(
+ content
+ for update in first_updates
+ for content in update.contents
+ if content.type == "function_approval_request"
+ )
+
+ resumed_stream = chat_client_base.get_response(
+ [Message(role="user", contents=[approval_request.to_function_approval_response(approved=approved)])],
+ stream=True,
+ options={"tools": [guarded_stream_tool]},
+ )
+ resumed_updates = [update async for update in resumed_stream]
+ resumed_response = await resumed_stream.get_final_response()
+
+ assert [[content.type for content in update.contents] for update in resumed_updates] == [
+ ["function_result"],
+ ["text"],
+ ]
+ terminal_result = resumed_updates[0].contents[0]
+ assert terminal_result.call_id == "call_stream_approval"
+ assert terminal_result.result == (
+ "approved output" if approved else "Error: Tool call invocation was rejected by user."
+ )
+ assert [[content.type for content in message.contents] for message in resumed_response.messages] == [
+ ["function_result"],
+ ["text"],
+ ]
+ assert calls == int(approved)
+
+
+@pytest.mark.parametrize("streaming", [False, True], ids=["non-streaming", "streaming"])
+async def test_approval_resume_honors_middleware_termination(
+ chat_client_base: SupportsChatGetResponse,
+ streaming: bool,
+) -> None:
+ """Approval-time termination should return its result without another model call."""
+ from agent_framework._tools import _FUNCTION_INVOCATION_BUDGET_STATE_KEY
+
+ @tool(name="guarded_termination_tool", approval_mode="always_require")
+ def guarded_termination_tool() -> str:
+ return "should not execute"
+
+ function_call = Content.from_function_call(
+ call_id="call_termination",
+ name="guarded_termination_tool",
+ arguments="{}",
+ )
+ budget_state: dict[str, int] = {}
+ if streaming:
+ chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ [ChatResponseUpdate(role="assistant", contents=[function_call])],
+ [ChatResponseUpdate(role="assistant", contents=[Content.from_text("unexpected model call")])],
+ ]
+ first_stream = chat_client_base.get_response(
+ [Message(role="user", contents=["run guarded"])],
+ stream=True,
+ options={"tools": [guarded_termination_tool]},
+ )
+ first_updates = [update async for update in first_stream]
+ approval_request = next(
+ content
+ for update in first_updates
+ for content in update.contents
+ if content.type == "function_approval_request"
+ )
+ resumed_stream = chat_client_base.get_response(
+ [Message(role="user", contents=[approval_request.to_function_approval_response(approved=True)])],
+ stream=True,
+ options={"tools": [guarded_termination_tool]},
+ client_kwargs={
+ "middleware": [TerminateLoopMiddleware()],
+ _FUNCTION_INVOCATION_BUDGET_STATE_KEY: budget_state,
+ },
+ )
+ resumed_updates = [update async for update in resumed_stream]
+ resumed_response = await resumed_stream.get_final_response()
+ assert [[content.type for content in update.contents] for update in resumed_updates] == [["function_result"]]
+ assert len(chat_client_base.streaming_responses) == 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ else:
+ chat_client_base.run_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ ChatResponse(messages=Message(role="assistant", contents=[function_call])),
+ ChatResponse(messages=Message(role="assistant", contents=["unexpected model call"])),
+ ]
+ first_response = await chat_client_base.get_response(
+ [Message(role="user", contents=["run guarded"])],
+ options={"tools": [guarded_termination_tool]},
+ )
+ approval_request = next(
+ content
+ for message in first_response.messages
+ for content in message.contents
+ if content.type == "function_approval_request"
+ )
+ resumed_response = await chat_client_base.get_response(
+ [Message(role="user", contents=[approval_request.to_function_approval_response(approved=True)])],
+ options={"tools": [guarded_termination_tool]},
+ client_kwargs={
+ "middleware": [TerminateLoopMiddleware()],
+ _FUNCTION_INVOCATION_BUDGET_STATE_KEY: budget_state,
+ },
+ )
+ assert len(chat_client_base.run_responses) == 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+
+ assert [[content.type for content in message.contents] for message in resumed_response.messages] == [
+ ["function_result"]
+ ]
+ assert resumed_response.messages[0].contents[0].result == "terminated by middleware"
+ assert budget_state["total_function_calls"] == 1
+
+
+@pytest.mark.parametrize("streaming", [False, True], ids=["non-streaming", "streaming"])
+async def test_approval_resume_user_input_counts_toward_function_call_budget(
+ chat_client_base: SupportsChatGetResponse,
+ streaming: bool,
+) -> None:
+ """An approved execution that pauses for user input still consumes one function-call budget unit."""
+ from agent_framework._tools import _FUNCTION_INVOCATION_BUDGET_STATE_KEY
+ from agent_framework.exceptions import UserInputRequiredException
+
+ calls = 0
+
+ @tool(name="guarded_input_tool", approval_mode="always_require")
+ def guarded_input_tool() -> str:
+ nonlocal calls
+ calls += 1
+ raise UserInputRequiredException(
+ contents=[
+ Content.from_oauth_consent_request(consent_link="https://example.com/consent-1"),
+ Content.from_oauth_consent_request(consent_link="https://example.com/consent-2"),
+ ]
+ )
+
+ chat_client_base.function_invocation_configuration["max_function_calls"] = 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ function_call = Content.from_function_call(
+ call_id="call_user_input_budget",
+ name="guarded_input_tool",
+ arguments="{}",
+ )
+ budget_state: dict[str, int] = {}
+ if streaming:
+ chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ [ChatResponseUpdate(role="assistant", contents=[function_call])],
+ [ChatResponseUpdate(role="assistant", contents=[Content.from_text("unexpected model call")])],
+ ]
+ first_stream = chat_client_base.get_response(
+ [Message(role="user", contents=["run guarded"])],
+ stream=True,
+ options={"tools": [guarded_input_tool]},
+ )
+ first_updates = [update async for update in first_stream]
+ approval_request = next(
+ content
+ for update in first_updates
+ for content in update.contents
+ if content.type == "function_approval_request"
+ )
+ resumed_stream = chat_client_base.get_response(
+ [Message(role="user", contents=[approval_request.to_function_approval_response(approved=True)])],
+ stream=True,
+ options={"tools": [guarded_input_tool]},
+ client_kwargs={_FUNCTION_INVOCATION_BUDGET_STATE_KEY: budget_state},
+ )
+ resumed_updates = [update async for update in resumed_stream]
+ resumed_response = await resumed_stream.get_final_response()
+ assert [[content.type for content in update.contents] for update in resumed_updates] == [
+ ["oauth_consent_request", "oauth_consent_request"]
+ ]
+ assert len(chat_client_base.streaming_responses) == 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ else:
+ chat_client_base.run_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ ChatResponse(messages=Message(role="assistant", contents=[function_call])),
+ ChatResponse(messages=Message(role="assistant", contents=["unexpected model call"])),
+ ]
+ first_response = await chat_client_base.get_response(
+ [Message(role="user", contents=["run guarded"])],
+ options={"tools": [guarded_input_tool]},
+ )
+ approval_request = next(
+ content
+ for message in first_response.messages
+ for content in message.contents
+ if content.type == "function_approval_request"
+ )
+ resumed_response = await chat_client_base.get_response(
+ [Message(role="user", contents=[approval_request.to_function_approval_response(approved=True)])],
+ options={"tools": [guarded_input_tool]},
+ client_kwargs={_FUNCTION_INVOCATION_BUDGET_STATE_KEY: budget_state},
+ )
+ assert len(chat_client_base.run_responses) == 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+
+ assert calls == 1
+ assert budget_state["total_function_calls"] == 1
+ user_input_requests = [
+ content for message in resumed_response.messages for content in message.contents if content.user_input_request
+ ]
+ assert [content.consent_link for content in user_input_requests] == [
+ "https://example.com/consent-1",
+ "https://example.com/consent-2",
+ ]
+
+
+@pytest.mark.parametrize("streaming", [False, True], ids=["non-streaming", "streaming"])
+async def test_approval_resume_error_limit_forces_final_no_tool_response(
+ chat_client_base: SupportsChatGetResponse,
+ streaming: bool,
+) -> None:
+ """Approval-time errors should submit the result once, then request a final no-tool answer."""
+ calls = 0
+
+ @tool(name="guarded_error_tool", approval_mode="always_require")
+ def guarded_error_tool() -> str:
+ nonlocal calls
+ calls += 1
+ raise ValueError("approval failure")
+
+ chat_client_base.function_invocation_configuration["max_consecutive_errors_per_request"] = 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ function_call = Content.from_function_call(
+ call_id="call_error",
+ name="guarded_error_tool",
+ arguments="{}",
+ )
+ if streaming:
+ chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ [ChatResponseUpdate(role="assistant", contents=[function_call])],
+ [ChatResponseUpdate(role="assistant", contents=[Content.from_text("unused")])],
+ ]
+ first_stream = chat_client_base.get_response(
+ [Message(role="user", contents=["run guarded"])],
+ stream=True,
+ options={"tools": [guarded_error_tool]},
+ )
+ first_updates = [update async for update in first_stream]
+ approval_request = next(
+ content
+ for update in first_updates
+ for content in update.contents
+ if content.type == "function_approval_request"
+ )
+ resumed_stream = chat_client_base.get_response(
+ [Message(role="user", contents=[approval_request.to_function_approval_response(approved=True)])],
+ stream=True,
+ options={"tools": [guarded_error_tool]},
+ )
+ resumed_updates = [update async for update in resumed_stream]
+ resumed_response = await resumed_stream.get_final_response()
+ assert [[content.type for content in update.contents] for update in resumed_updates] == [
+ ["function_result"],
+ ["text"],
+ ]
+ else:
+ chat_client_base.run_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ ChatResponse(messages=Message(role="assistant", contents=[function_call])),
+ ChatResponse(messages=Message(role="assistant", contents=["unused"])),
+ ]
+ first_response = await chat_client_base.get_response(
+ [Message(role="user", contents=["run guarded"])],
+ options={"tools": [guarded_error_tool]},
+ )
+ approval_request = next(
+ content
+ for message in first_response.messages
+ for content in message.contents
+ if content.type == "function_approval_request"
+ )
+ resumed_response = await chat_client_base.get_response(
+ [Message(role="user", contents=[approval_request.to_function_approval_response(approved=True)])],
+ options={"tools": [guarded_error_tool]},
+ )
+
+ assert calls == 1
+ assert chat_client_base.call_count == 2 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ assert [[content.type for content in message.contents] for message in resumed_response.messages] == [
+ ["function_result"],
+ ["text"],
+ ]
+ assert resumed_response.messages[0].contents[0].exception == "approval failure"
+ assert resumed_response.text == "I broke out of the function invocation loop..."
+
+
async def test_terminate_loop_single_function_call(chat_client_base: SupportsChatGetResponse):
"""Test that terminate_loop=True exits the function calling loop after single function call."""
exec_counter = 0
@@ -4329,6 +5274,7 @@ def delegate_tool(task: str) -> str:
async def test_user_input_request_multiple_contents_propagate(chat_client_base: SupportsChatGetResponse):
"""Test that multiple user_input_request items in a single exception all propagate to the parent response."""
+ from agent_framework._tools import _FUNCTION_INVOCATION_BUDGET_STATE_KEY
from agent_framework.exceptions import UserInputRequiredException
@tool(name="multi_request_tool", approval_mode="never_require")
@@ -4358,10 +5304,13 @@ def multi_request(task: str) -> str:
)
)
]
+ chat_client_base.function_invocation_configuration["max_function_calls"] = 1 # type: ignore[attr-defined] # ty: ignore[unresolved-attribute]
+ budget_state: dict[str, int] = {}
response = await chat_client_base.get_response(
[Message(role="user", contents=["do something"])],
options={"tool_choice": "auto", "tools": [multi_request]},
+ client_kwargs={_FUNCTION_INVOCATION_BUDGET_STATE_KEY: budget_state},
)
user_requests = [
@@ -4377,6 +5326,7 @@ def multi_request(task: str) -> str:
"https://example.com/consent2",
"https://example.com/consent3",
}
+ assert budget_state["total_function_calls"] == 1
async def test_user_input_request_empty_contents_returns_fallback(chat_client_base: SupportsChatGetResponse):
diff --git a/python/packages/core/tests/core/test_harness_tool_approval.py b/python/packages/core/tests/core/test_harness_tool_approval.py
index f530483db65..120a35e891f 100644
--- a/python/packages/core/tests/core/test_harness_tool_approval.py
+++ b/python/packages/core/tests/core/test_harness_tool_approval.py
@@ -2,6 +2,13 @@
from __future__ import annotations
+import warnings
+from collections.abc import MutableSequence
+from pathlib import Path
+from typing import Any
+
+import pytest
+
from agent_framework import (
DEFAULT_TOOL_APPROVAL_SOURCE_ID,
Agent,
@@ -9,6 +16,7 @@
ChatResponse,
ChatResponseUpdate,
Content,
+ FileHistoryProvider,
Message,
ToolApprovalMiddleware,
ToolApprovalState,
@@ -16,6 +24,7 @@
create_always_approve_tool_with_arguments_response,
tool,
)
+from agent_framework._feature_stage import ExperimentalWarning
from .conftest import MockBaseChatClient
@@ -31,6 +40,320 @@ def _function_call(request: Content) -> Content:
return request.function_call
+@pytest.mark.parametrize("approved", [True, False], ids=["approved", "rejected"])
+@pytest.mark.parametrize("streaming", [False, True], ids=["non-streaming", "streaming"])
+async def test_approval_resume_returns_result_without_mutating_inputs(
+ chat_client_base: MockBaseChatClient,
+ approved: bool,
+ streaming: bool,
+) -> None:
+ """Approval resume should return its terminal result without changing caller-owned messages."""
+ calls = 0
+
+ @tool(name="guarded_tool", approval_mode="always_require")
+ def guarded_tool() -> str:
+ nonlocal calls
+ calls += 1
+ return "approved result"
+
+ agent = Agent(client=chat_client_base, tools=[guarded_tool])
+ session = AgentSession(session_id=f"immutable-approval-{streaming}-{approved}")
+ function_call = Content.from_function_call(call_id="call_guarded", name="guarded_tool", arguments="{}")
+
+ if streaming:
+ chat_client_base.streaming_responses = [[ChatResponseUpdate(role="assistant", contents=[function_call])]]
+ first_stream = agent.run("run guarded", stream=True, session=session)
+ first_updates = [update async for update in first_stream]
+ first_response = await first_stream.get_final_response()
+ approval_request = next(content for update in first_updates for content in update.user_input_requests)
+ else:
+ chat_client_base.run_responses = [ChatResponse(messages=Message(role="assistant", contents=[function_call]))]
+ first_response = await agent.run("run guarded", session=session)
+ approval_request = first_response.user_input_requests[0]
+
+ approval_response = approval_request.to_function_approval_response(approved=approved)
+ approval_message = Message(role="user", contents=[approval_response])
+
+ if streaming:
+ chat_client_base.streaming_responses = [
+ [ChatResponseUpdate(role="assistant", contents=[Content.from_text("done")])]
+ ]
+ second_stream = agent.run(approval_message, stream=True, session=session)
+ second_updates = [update async for update in second_stream]
+ second_response = await second_stream.get_final_response()
+ assert [[content.type for content in update.contents] for update in second_updates] == [
+ ["function_result"],
+ ["text"],
+ ]
+ else:
+ chat_client_base.run_responses = [ChatResponse(messages=Message(role="assistant", contents=["done"]))]
+ second_response = await agent.run(approval_message, session=session)
+
+ assert [[content.type for content in message.contents] for message in second_response.messages] == [
+ ["function_result"],
+ ["text"],
+ ]
+ result = second_response.messages[0].contents[0]
+ assert result.call_id == "call_guarded"
+ assert result.result == ("approved result" if approved else "Error: Tool call invocation was rejected by user.")
+ assert calls == int(approved)
+ assert approval_message.role == "user"
+ assert approval_message.contents == [approval_response]
+ assert [[content.type for content in message.contents] for message in first_response.messages] == [
+ ["function_call", "function_approval_request"]
+ ]
+
+
+@pytest.mark.parametrize("streaming", [False, True], ids=["non-streaming", "streaming"])
+async def test_approval_resume_replays_reasoning_with_function_call_group(
+ chat_client_base: MockBaseChatClient,
+ streaming: bool,
+) -> None:
+ """Reasoning bound to a function call must be replayed with its terminal result."""
+
+ @tool(name="reasoning_tool", approval_mode="always_require")
+ def reasoning_tool() -> str:
+ return "approved result"
+
+ captured_calls: list[list[tuple[str, list[tuple[str, str | None, str | None, str | None]]]]] = []
+
+ def capture(messages: MutableSequence[Message]) -> None:
+ captured_calls.append([
+ (
+ message.role,
+ [(content.type, content.id, content.call_id, content.protected_data) for content in message.contents],
+ )
+ for message in messages
+ ])
+
+ if streaming:
+ original_get_streaming_response = chat_client_base._get_streaming_response
+
+ def capture_streaming_messages(
+ *,
+ messages: MutableSequence[Message],
+ options: dict[str, Any],
+ **kwargs: Any,
+ ) -> Any:
+ capture(messages)
+ return original_get_streaming_response(messages=messages, options=options, **kwargs)
+
+ chat_client_base._get_streaming_response = capture_streaming_messages # type: ignore[method-assign] # ty: ignore[invalid-assignment]
+ else:
+ original_get_non_streaming_response = chat_client_base._get_non_streaming_response
+
+ async def capture_non_streaming_messages(
+ *,
+ messages: MutableSequence[Message],
+ options: dict[str, Any],
+ **kwargs: Any,
+ ) -> ChatResponse:
+ capture(messages)
+ return await original_get_non_streaming_response(messages=messages, options=options, **kwargs)
+
+ chat_client_base._get_non_streaming_response = capture_non_streaming_messages # type: ignore[method-assign] # ty: ignore[invalid-assignment]
+
+ agent = Agent(client=chat_client_base, tools=[reasoning_tool])
+ session = AgentSession(session_id=f"approval-reasoning-{streaming}")
+ reasoning = Content.from_text_reasoning(
+ id="reasoning_1",
+ text="I need to run the tool",
+ protected_data="encrypted-reasoning",
+ additional_properties={"status": "completed"},
+ )
+ function_call = Content.from_function_call(
+ call_id="call_reasoning",
+ name="reasoning_tool",
+ arguments="{}",
+ )
+
+ if streaming:
+ chat_client_base.streaming_responses = [
+ [
+ ChatResponseUpdate(role="assistant", contents=[reasoning]),
+ ChatResponseUpdate(role="assistant", contents=[function_call]),
+ ],
+ [ChatResponseUpdate(role="assistant", contents=[Content.from_text("done")])],
+ ]
+ first_stream = agent.run("run with reasoning", stream=True, session=session)
+ first_updates = [update async for update in first_stream]
+ first_response = await first_stream.get_final_response()
+ approval_request = next(content for update in first_updates for content in update.user_input_requests)
+ resumed_stream = agent.run(
+ approval_request.to_function_approval_response(approved=True),
+ stream=True,
+ session=session,
+ )
+ _ = [update async for update in resumed_stream]
+ resumed_response = await resumed_stream.get_final_response()
+ else:
+ chat_client_base.run_responses = [
+ ChatResponse(messages=Message(role="assistant", contents=[reasoning, function_call])),
+ ChatResponse(messages=Message(role="assistant", contents=["done"])),
+ ]
+ first_response = await agent.run("run with reasoning", session=session)
+ resumed_response = await agent.run(
+ first_response.user_input_requests[0].to_function_approval_response(approved=True),
+ session=session,
+ )
+
+ replayed_contents = [content for _, contents in captured_calls[1] for content in contents]
+ replayed_types = [content_type for content_type, _, _, _ in replayed_contents]
+ reasoning_index = replayed_types.index("text_reasoning")
+ call_index = replayed_types.index("function_call")
+ result_index = replayed_types.index("function_result")
+
+ assert reasoning_index < call_index < result_index
+ assert replayed_contents[reasoning_index] == (
+ "text_reasoning",
+ "reasoning_1",
+ None,
+ "encrypted-reasoning",
+ )
+ assert "function_approval_request" not in replayed_types
+ assert "function_approval_response" not in replayed_types
+ assert any(content.type == "text_reasoning" for content in first_response.messages[0].contents)
+ assert [[content.type for content in message.contents] for message in resumed_response.messages] == [
+ ["function_result"],
+ ["text"],
+ ]
+
+
+async def test_approval_resume_filters_resolved_control_items_from_file_history(
+ chat_client_base: MockBaseChatClient,
+ tmp_path: Path,
+) -> None:
+ """Resolved approval wrappers should not be replayed from append-only history."""
+ calls = 0
+
+ @tool(name="guarded_history_tool", approval_mode="always_require")
+ def guarded_history_tool() -> str:
+ nonlocal calls
+ calls += 1
+ return "approved result"
+
+ with warnings.catch_warnings():
+ warnings.simplefilter("ignore", ExperimentalWarning)
+ history_provider = FileHistoryProvider(tmp_path)
+ agent = Agent(
+ client=chat_client_base,
+ tools=[guarded_history_tool],
+ context_providers=[history_provider],
+ )
+ session = AgentSession(session_id="approval-file-history")
+ chat_client_base.run_responses = [
+ ChatResponse(
+ messages=Message(
+ role="assistant",
+ contents=[
+ Content.from_function_call(
+ call_id="call_guarded_history",
+ name="guarded_history_tool",
+ arguments="{}",
+ )
+ ],
+ )
+ )
+ ]
+ first_response = await agent.run("run guarded", session=session)
+ chat_client_base.run_responses = [ChatResponse(messages=Message(role="assistant", contents=["done"]))]
+ await agent.run(
+ first_response.user_input_requests[0].to_function_approval_response(approved=True),
+ session=session,
+ )
+
+ captured_types: list[list[str]] = []
+ original_get_response = chat_client_base._get_non_streaming_response
+
+ async def capture_messages(
+ *,
+ messages: MutableSequence[Message],
+ options: dict[str, Any],
+ **kwargs: Any,
+ ) -> ChatResponse:
+ captured_types.extend([[content.type for content in message.contents] for message in messages])
+ return await original_get_response(messages=messages, options=options, **kwargs)
+
+ chat_client_base._get_non_streaming_response = capture_messages # type: ignore[method-assign] # ty: ignore[invalid-assignment]
+ chat_client_base.run_responses = [ChatResponse(messages=Message(role="assistant", contents=["later"]))]
+ await agent.run("unrelated later turn", session=session)
+
+ flattened_types = [content_type for message_types in captured_types for content_type in message_types]
+ assert flattened_types.count("function_call") == 1
+ assert flattened_types.count("function_result") == 1
+ assert "function_approval_request" not in flattened_types
+ assert "function_approval_response" not in flattened_types
+ assert calls == 1
+
+
+@pytest.mark.parametrize("streaming", [False, True], ids=["non-streaming", "streaming"])
+async def test_approval_resume_returns_all_user_input_requests_without_another_model_call(
+ chat_client_base: MockBaseChatClient,
+ streaming: bool,
+) -> None:
+ """All user input requested during approved execution should return before another model call."""
+ from agent_framework.exceptions import UserInputRequiredException
+
+ calls = 0
+
+ @tool(name="oauth_tool", approval_mode="always_require")
+ def oauth_tool() -> str:
+ nonlocal calls
+ calls += 1
+ raise UserInputRequiredException(
+ contents=[
+ Content.from_oauth_consent_request(consent_link="https://example.com/consent-1"),
+ Content.from_oauth_consent_request(consent_link="https://example.com/consent-2"),
+ Content.from_oauth_consent_request(consent_link="https://example.com/consent-3"),
+ ]
+ )
+
+ agent = Agent(client=chat_client_base, tools=[oauth_tool])
+ session = AgentSession(session_id=f"approval-user-input-{streaming}")
+ function_call = Content.from_function_call(call_id="call_oauth", name="oauth_tool", arguments="{}")
+
+ if streaming:
+ chat_client_base.streaming_responses = [
+ [ChatResponseUpdate(role="assistant", contents=[function_call])],
+ [ChatResponseUpdate(role="assistant", contents=[Content.from_text("unexpected model call")])],
+ ]
+ first_stream = agent.run("run oauth", stream=True, session=session)
+ first_updates = [update async for update in first_stream]
+ approval_request = next(content for update in first_updates for content in update.user_input_requests)
+ resumed_stream = agent.run(
+ approval_request.to_function_approval_response(approved=True),
+ stream=True,
+ session=session,
+ )
+ resumed_updates = [update async for update in resumed_stream]
+ resumed_response = await resumed_stream.get_final_response()
+ assert len(chat_client_base.streaming_responses) == 1
+ assert [content.consent_link for update in resumed_updates for content in update.user_input_requests] == [
+ "https://example.com/consent-1",
+ "https://example.com/consent-2",
+ "https://example.com/consent-3",
+ ]
+ else:
+ chat_client_base.run_responses = [
+ ChatResponse(messages=Message(role="assistant", contents=[function_call])),
+ ChatResponse(messages=Message(role="assistant", contents=["unexpected model call"])),
+ ]
+ first_response = await agent.run("run oauth", session=session)
+ resumed_response = await agent.run(
+ first_response.user_input_requests[0].to_function_approval_response(approved=True),
+ session=session,
+ )
+ assert len(chat_client_base.run_responses) == 1
+
+ assert [content.consent_link for content in resumed_response.user_input_requests] == [
+ "https://example.com/consent-1",
+ "https://example.com/consent-2",
+ "https://example.com/consent-3",
+ ]
+ assert resumed_response.messages[0].role == "assistant"
+ assert calls == 1
+
+
async def test_mixed_batch_hides_already_approved_request_until_approval_replay(
chat_client_base: MockBaseChatClient,
) -> None:
@@ -482,6 +805,54 @@ def auto_approve_budgeted_tool(function_call: Content) -> bool:
assert calls == 1
+@pytest.mark.parametrize("streaming", [False, True], ids=["non-streaming", "streaming"])
+async def test_auto_approval_resolves_after_iteration_budget_is_exhausted(
+ chat_client_base: MockBaseChatClient,
+ streaming: bool,
+) -> None:
+ """Approval resolution must run before the model-iteration budget is checked."""
+ calls = 0
+
+ @tool(name="last_iteration_tool", approval_mode="always_require")
+ def last_iteration_tool() -> str:
+ nonlocal calls
+ calls += 1
+ return "executed"
+
+ chat_client_base.function_invocation_configuration["max_iterations"] = 1
+ agent = Agent(
+ client=chat_client_base,
+ tools=[last_iteration_tool],
+ middleware=[ToolApprovalMiddleware(auto_approval_rules=[lambda function_call: True])],
+ )
+ session = AgentSession(session_id=f"approval-iteration-budget-{streaming}")
+ function_call = Content.from_function_call(
+ call_id="call_last_iteration",
+ name="last_iteration_tool",
+ arguments="{}",
+ )
+
+ if streaming:
+ chat_client_base.streaming_responses = [
+ [ChatResponseUpdate(role="assistant", contents=[function_call])],
+ [ChatResponseUpdate(role="assistant", contents=[Content.from_text("unused")])],
+ ]
+ response_stream = agent.run("run once", stream=True, session=session)
+ updates = [update async for update in response_stream]
+ response = await response_stream.get_final_response()
+ assert any(content.type == "function_result" for update in updates for content in update.contents)
+ else:
+ chat_client_base.run_responses = [
+ ChatResponse(messages=Message(role="assistant", contents=[function_call])),
+ ChatResponse(messages=Message(role="assistant", contents=["unused"])),
+ ]
+ response = await agent.run("run once", session=session)
+
+ assert calls == 1
+ assert any(content.type == "function_result" for message in response.messages for content in message.contents)
+ assert response.text == "I broke out of the function invocation loop..."
+
+
async def test_tool_approval_middleware_queues_streamed_approval_requests(
chat_client_base: MockBaseChatClient,
) -> None:
diff --git a/python/packages/core/tests/core/test_types.py b/python/packages/core/tests/core/test_types.py
index 1f38005cdb7..61a1b0c0683 100644
--- a/python/packages/core/tests/core/test_types.py
+++ b/python/packages/core/tests/core/test_types.py
@@ -571,6 +571,15 @@ def test_function_call_content_add_merging_and_errors():
c = a + b
assert c.informational_only is True
+ # control metadata is preserved when a metadata-only update follows argument chunks
+ metadata = Content.from_function_call(call_id="1", name="f", arguments=None)
+ metadata.id = "1"
+ metadata.user_input_request = True
+ c = c + metadata
+ assert c.arguments == '{"x":1}'
+ assert c.id == "1"
+ assert c.user_input_request is True
+
# incompatible argument types
a = Content.from_function_call(call_id="1", name="f", arguments="abc")
b = Content.from_function_call(call_id="1", name="f", arguments={"y": 2})
diff --git a/python/packages/openai/AGENTS.md b/python/packages/openai/AGENTS.md
index 752919fa28e..1dd22ad0c97 100644
--- a/python/packages/openai/AGENTS.md
+++ b/python/packages/openai/AGENTS.md
@@ -24,6 +24,11 @@ agent_framework_openai/
All clients follow the Raw + Full-Featured pattern (e.g., `RawOpenAIChatClient` + `OpenAIChatClient`).
+For Responses API continuation with service-side storage, a prior `function_approval_request` is server-issued and
+must not be replayed inline, while the new `function_approval_response` is always serialized as
+`mcp_approval_response` so the user's approved or rejected decision reaches the service. Applications that manually
+replay message history must not send that same approval response again on later turns.
+
The generic OpenAI clients support both OpenAI and Azure OpenAI routing. Precedence is:
explicit Azure inputs (`credential`, `azure_endpoint`, `api_version`) → OpenAI API key
(`OPENAI_API_KEY`) → Azure environment fallback (`AZURE_OPENAI_*`).
diff --git a/python/packages/openai/agent_framework_openai/_chat_client.py b/python/packages/openai/agent_framework_openai/_chat_client.py
index e35ff3500a6..92cfd6671de 100644
--- a/python/packages/openai/agent_framework_openai/_chat_client.py
+++ b/python/packages/openai/agent_framework_openai/_chat_client.py
@@ -1684,7 +1684,7 @@ def _prepare_message_for_openai(
)
if function_call:
all_messages.append(function_call)
- case "function_approval_response" | "function_approval_request":
+ case "function_approval_request":
if request_uses_service_side_storage:
continue
prepared = self._prepare_content_for_openai(
@@ -1694,6 +1694,14 @@ def _prepare_message_for_openai(
)
if prepared:
all_messages.append(prepared)
+ case "function_approval_response":
+ prepared = self._prepare_content_for_openai(
+ message.role,
+ content,
+ replays_local_storage=replays_local_storage,
+ )
+ if prepared:
+ all_messages.append(prepared)
case "mcp_server_tool_call" | "mcp_server_tool_result":
# Hosted MCP call/result contents serialize as a single
# top-level mcp_call input item; the result side emits an
diff --git a/python/packages/openai/tests/openai/test_openai_chat_client.py b/python/packages/openai/tests/openai/test_openai_chat_client.py
index f9921bcd09b..182c86594f0 100644
--- a/python/packages/openai/tests/openai/test_openai_chat_client.py
+++ b/python/packages/openai/tests/openai/test_openai_chat_client.py
@@ -7762,9 +7762,9 @@ def test_prepare_messages_keeps_function_call_without_storage() -> None:
assert output_item["call_id"] == "call_1"
-def test_prepare_messages_strips_approval_items_under_storage() -> None:
- """Approval request/response items also carry server-issued IDs and must be stripped under
- storage. Without storage they are kept (#3295)."""
+@pytest.mark.parametrize("approved", [True, False], ids=["approved", "rejected"])
+def test_prepare_messages_strips_approval_request_but_keeps_response_under_storage(approved: bool) -> None:
+ """Stored requests are not replayed, but the new approval decision must reach the service."""
client = OpenAIChatClient(model="test-model", api_key="test-key")
function_call = Content.from_function_call(
@@ -7777,7 +7777,7 @@ def test_prepare_messages_strips_approval_items_under_storage() -> None:
function_call=function_call,
)
approval_response = Content.from_function_approval_response(
- approved=True,
+ approved=approved,
id="approval_req_1",
function_call=function_call,
)
@@ -7789,7 +7789,9 @@ def test_prepare_messages_strips_approval_items_under_storage() -> None:
storage_on = client._prepare_messages_for_openai(messages, request_uses_service_side_storage=True)
storage_on_types = [item.get("type") for item in storage_on]
assert "mcp_approval_request" not in storage_on_types
- assert "mcp_approval_response" not in storage_on_types
+ assert storage_on_types == ["mcp_approval_response"]
+ assert storage_on[0]["approval_request_id"] == "approval_req_1"
+ assert storage_on[0]["approve"] is approved
storage_off = client._prepare_messages_for_openai(messages, request_uses_service_side_storage=False)
storage_off_types = [item.get("type") for item in storage_off]