Skip to content

Python: Fix Gemini harness tool declarations - #7322

Merged
eavanvalkenburg merged 4 commits into
microsoft:mainfrom
scarab-systems:scarab-systems/gemini-harness-tools
Jul 28, 2026
Merged

Python: Fix Gemini harness tool declarations#7322
eavanvalkenburg merged 4 commits into
microsoft:mainfrom
scarab-systems:scarab-systems/gemini-harness-tools

Conversation

@scarab-systems

Copy link
Copy Markdown
Contributor

Motivation & Context

Fixes #6954.

Gemini Developer API calls can combine native Gemini tools, such as Google Search grounding, with Agent Framework function tools. That mixed-tool path needs Gemini's server-side tool invocation reporting enabled so native tool calls are returned alongside function calls.

Function tool schemas also need to be forwarded through the Gemini SDK's JSON Schema field so richer Agent Framework tool schemas are preserved instead of being coerced into the SDK Schema subset.

Disclosure: This PR was prepared with AI assistance under human direction and review.

Description & Review Guide

  • What are the major changes?
    • Pass FunctionTool schemas through parameters_json_schema when building Gemini function declarations.
    • Enable include_server_side_tool_invocations for Gemini Developer API requests that mix native Gemini tools with function declarations.
    • Preserve existing tool_choice / function-calling configuration when adding that Developer API flag.
    • Add Gemini client tests for JSON Schema pass-through, mixed native/function tool config, preserved tool_choice, and Vertex AI behavior.
  • What is the impact of these changes?
    • The failing Gemini harness mixed-tool scenario gets the native tool-call reporting it needs while keeping Vertex AI behavior unchanged.
  • What do you want reviewers to focus on?
    • Whether the Developer API-only condition is the right scope for enabling server-side tool invocation reporting on mixed native/function tool requests.

Related Issue

Fixes #6954

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) -- a workflow keeps the label and title prefix in sync automatically.

Forward Agent Framework FunctionTool JSON Schemas to the Gemini SDK parameters_json_schema field and enable Developer API server-side tool invocation reporting when native Gemini tools are mixed with function declarations.

Preserves Vertex AI behavior and existing function-calling tool_choice config.

Validation:

- uv run --directory python poe check -P gemini

- uv run --directory python poe build -P gemini

- uv run --directory python poe test -A -m 'not integration'

- uv run --directory python pytest packages/gemini/tests/test_gemini_client.py -q -m integration (8 skipped: credential-gated)
Copilot AI review requested due to automatic review settings July 26, 2026 03:12
@agent-framework-automation agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Jul 26, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes Gemini (Developer API) tool-calling interoperability in the Python Gemini client so harness scenarios can combine native Gemini tools (e.g., Google Search grounding) with Agent Framework FunctionTools without request-prep validation failures or Gemini API rejections.

Changes:

  • Forward full FunctionTool JSON Schema via types.FunctionDeclaration.parameters_json_schema (instead of coercing into the SDK’s parameters Schema subset).
  • When using the Gemini Developer API and the request mixes native Gemini tools with function declarations, set tool_config.include_server_side_tool_invocations = True while preserving any existing function-calling configuration (tool_choice).
  • Add/extend Gemini client tests covering JSON Schema pass-through, mixed native/function tools behavior, tool_choice preservation, and Vertex AI non-impact.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
python/packages/gemini/agent_framework_gemini/_chat_client.py Updates tool declaration building to use JSON Schema passthrough and conditionally enables Developer API server-side tool invocation reporting for mixed native/function tool requests.
python/packages/gemini/tests/test_gemini_client.py Adds tests verifying JSON Schema forwarding, server-side tool invocation flag behavior for Developer API mixed-tool requests, tool_choice preservation, and unchanged Vertex AI behavior.

@scarab-systems
scarab-systems marked this pull request as ready for review July 26, 2026 03:16
@eavanvalkenburg

Copy link
Copy Markdown
Member

@scarab-systems please check the failing tests!

@scarab-systems

scarab-systems commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

@scarab-systems please check the failing tests!

on it now! one moment - done
also updating branch.

Use typing_extensions.TypedDict for the Gemini JSON Schema test helper so Pydantic can build the model on Python 3.11.

This keeps the CI fix scoped to the failing test compatibility issue without changing Gemini client behavior.
@github-actions

Copy link
Copy Markdown
Contributor

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/gemini/agent_framework_gemini
   _chat_client.py4681995%402, 719, 740–741, 759–760, 769–770, 773–774, 806, 813, 938, 949, 1138–1139, 1143, 1154–1155
TOTAL45595447990% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
9414 34 💤 0 ❌ 0 🔥 2m 26s ⏱️

@eavanvalkenburg
eavanvalkenburg added this pull request to the merge queue Jul 28, 2026
Merged via the queue into microsoft:main with commit 3daa3b2 Jul 28, 2026
37 checks passed
@scarab-systems
scarab-systems deleted the scarab-systems/gemini-harness-tools branch July 28, 2026 18:03
scarab-systems added a commit to scarab-systems/scarab-field-lab that referenced this pull request Jul 28, 2026
Record the merged microsoft/agent-framework#7322 outcome for issue #6954.

Update the Field Lab index and merged contribution summary to reflect 17 merged public contributions, including the second accepted Microsoft Agent Framework contribution.

Validation: live GitHub status confirmed merged; GitHub Markdown render passed for changed records; changed public links returned 200; git diff --staged --check passed; staged leakage scan found no private/local process terms.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: Gemini chat client can't call the harness tool set

4 participants