Skip to content

perf(agentgateway): parallelize LoB MCP tool listing across fragments - #322

Open
wangzhengzhuo05 wants to merge 1 commit into
SAP:mainfrom
wangzhengzhuo05:perf/parallelize-mcp-tool-listing
Open

perf(agentgateway): parallelize LoB MCP tool listing across fragments#322
wangzhengzhuo05 wants to merge 1 commit into
SAP:mainfrom
wangzhengzhuo05:perf/parallelize-mcp-tool-listing

Conversation

@wangzhengzhuo05

Copy link
Copy Markdown

Fixes #301

What / Why

get_mcp_tools_lob fetched MCP tools from destination fragments sequentially — N fragments × ~3s network round-trip each. Concurrent fetching makes total time ~max(fragment time) instead of the sum (e.g. 14 fragments ≈ 3s instead of ≈44s).

How

  • Replaced the sequential for fragment ... await list_server_tools loop with asyncio.gather(*coros, return_exceptions=True).
  • URL-missing fragments are still skipped (with warning) at scheduling time; failing fragments are isolated via return_exceptions=True + _log_mcp_server_error — same continuation semantics as before.
  • Results are zipped back in fragment order (gather returns in input order), so tool ordering is unchanged.
  • Pre-fetch ORD filter, post-fetch names filter, and logging are untouched.

Tests

  • test_lists_fragments_concurrently: 3 fragments × 0.05s sleeps complete in < 2× per-call (sequential would be 3×). Mutation check: reverting to a sequential loop makes this test fail (verified).
  • test_failure_in_one_fragment_does_not_fail_others: 2nd fragment raises; other fragments' tools still returned.
  • test_preserves_fragment_order: completion order is inverted via varied latencies; returned tools still follow fragment order.

Local: 168 passed (test_lob + test_agw_client + test_customer).

AI-generated code disclosure

This change was implemented with the assistance of AI (OpenCode + muse-spark) and reviewed by a human maintainer process; per SAP GenAI guidelines the code has been verified for correctness, security, and license compatibility.

@wangzhengzhuo05
wangzhengzhuo05 requested a review from a team as a code owner September 9, 2026 13:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parallelize fragment connections inside list_mcp_tools so all fragments are handled concurrently

1 participant