[Tracking] Dynamic MCP Tool Loading — Implementation Roadmap

[edelauna]

Tech Brief: Dynamic MCP Tool Loading

Problem

Zoo Code currently loads all MCP tool definitions (name, description, full JSON inputSchema) into the LLM context on every task. This is implemented in getMcpServerTools() (src/core/prompts/tools/native-tools/mcp_server.ts) which runs eagerly at task-build time.

With a modest MCP setup (e.g. GitHub MCP with ~27 tools) this costs ~18k tokens before the conversation begins. At 7 servers, users report 67k tokens consumed — 33% of a 200k budget — on tool definitions alone. As the MCP ecosystem grows this only gets worse.

A user on a model with a 128-tool hard limit (e.g. GPT-5.4 via OpenAI) hits API failures with just a few robust MCP servers active.

This is a known pain point across MCP-supporting tools:

• MCP tool loading causes context pollution - need lazy loading or tool consolidation. microsoft/vscode#282699
• Lazy-load MCP tool definitions to reduce context usage anthropics/claude-code#11364

Answer to "does Zoo Code do dynamic or static tool loading?"

Static. McpHub.connectToServer() calls tools/list on connect and caches full definitions. Every task injects all definitions into the context via getMcpServerTools(). There is no lazy loading or dynamic discovery today.

Proposed Solution

A layered approach: first add scoping primitives so users can reduce the candidate pool, then add a ranked dynamic loader that injects only the most relevant tools at task start and lets the model request more via an mcp_load tool.

Architecture

Task start
  └─ getMcpServerTools()
       └─ (#571) onlyAllow filter per server
       └─ (#572) blockedMcpServers filter per mode
       └─ existing allowedMcpServers filter (already implemented)
  └─ if scopedTools.length > threshold (#578)
       └─ ToolRouter.attachInitial(taskText, scopedTools, initialK)  ← #577
            └─ EmbeddingsRanker (#576) or Bm25Ranker (#575) fallback
       └─ API tools = nativeTools + attachedMcpTools + mcp_load  ← #579
  └─ rolling window: turns 1–N re-rank and union into attachedMcpToolNames  ← #580

Ranker tiers

Tier	Implementation	When used
Tier 1	EmbeddingsRanker — cosine similarity on provider vectors	When CodeIndexManager has an embedding provider configured
Tier 2	Bm25Ranker — pure-JS BM25 (k1=1.5, b=0.75)	Always available; automatic fallback if Tier 1 fails

Rolling window (union-only)

For the first N turns (default 7), ranking re-runs against the latest user message and unions results into attachedMcpToolNames. Tools are only ever appended — never removed — preserving prompt cache stability. After turn N the set is frozen; the model uses mcp_load to fetch additional tools.

Stories

Wave 0 — Scoping Primitives (independent, no prereqs)

Issue	Description
#571	onlyAllow per-server field (complement to disabledTools)
#572	blockedMcpServers per-mode field (allowedMcpServers allowlist already done)
#573	Override-able tool descriptions per server
#574	Detailed context-usage panel showing per-section token cost

Wave 1 — Ranker + Dynamic Loader

Issue	Description	Depends on
#575	Bm25Ranker (pure-JS, always available)
#576	EmbeddingsRanker with BM25 fallback	#575
#577	ToolRouter singleton: index lifecycle and search	#575, #576
#578	Threshold gate + Task.attachedMcpToolNames	#577
#579	mcp_load native tool	#578
#580	Rolling-search window across first N turns	#578, #579

Wave 2 — Polish

Issue	Description	Depends on
#581	Replace TooManyToolsWarning with DynamicToolsStatus	#578
#582	Telemetry for dynamic tool loading	#578, #579, #580
#583	Settings UI + documentation	#578, #579, #580

Delivery Order

#571, #572, #573, #574 (all parallel, no prereqs)

#575 ──► #576 ──► #577 ──► #578 ──► #579 ──► #580
                                └──────────────────► #581
                                                     #582
                                                     #583

Wave 0 issues can land at any time and are good first-contributor issues.
The #575 → #576 → #577 → #578 → #579 chain is the critical path.

Key Design Decisions

Decision	Choice	Rationale
Ranker fallback	BM25 always available; embeddings when provider configured	No setup cost for most users
Union-only growth	Tools only appended to attachedMcpToolNames, never removed	Preserves prompt cache stability across turns
Rolling window	Re-rank for first N turns, then freeze	Handles topic drift in early turns without unbounded growth
Kill-switch	searchTurns: 0 disables dynamic loading	Simple opt-out without removing infrastructure
mcp_load escape hatch	Model-initiated tool attachment	Lets the model recover from a bad initial ranking

Scope Boundaries

In scope: VS Code extension only.

Out of scope:

• Standalone CLI experience
• Automatic tool eviction (tools are union-only within a task)
• Per-tool embedding caching across restarts (in-memory only for v1)

[DScoNOIZ]

@edelauna

This is an excellent and long-overdue decision. I fully support it. If you implement this, it will be a huge improvement.

In fact, I'm planning to do something very similar with skills. I want to take the skill management and self-improvement mechanisms from Nous Research Hermes Agent and integrate them into ZooCode. I'm also considering bringing in long-term memory capabilities and expanding them by combining the best ideas and practices from both Hermes Agent and OpenFang.

My overall vision is that ZooCode should first be equipped with highly advanced native persistent memory, skill management, MCP intellectual management, and skill self-improvement capabilities. I also think ZooCode should eventually be able to enable and disable installed skills and MCP servers autonomously.

For example, imagine having 250 installed skills and 100 MCP servers. ZooCode could automatically search through them, determine which ones are most suitable for a specific task, and dynamically enable or disable them as needed.

I have a lot of plans and ideas around this. Later on, I would also like to integrate ZooCode more deeply with either Hermes Agent or OpenFang, although I haven't fully decided which direction to take yet.

My current concept is that Hermes Agent or OpenFang would serve as the primary continuously thinking and planning core. We would assign high-level objectives to it through the standard ZooCode interface, and this always-running core would then orchestrate and direct ZooCode as the execution layer.

For example, ZooCode could have a simple toggle in the chat interface. Depending on its position, users could either communicate directly with the "director" (Hermes Agent) or with ZooCode itself to guide and refine execution.

At the moment, that's roughly how I envision it.

In this architecture, Hermes Agent would function as the reasoning and coordination engine. It could continuously evaluate progress, search for alternative approaches, gather new information, and adjust ZooCode's actions accordingly.

A typical workflow might look like this:

We give a high-level objective to Hermes Agent, and it begins working toward that goal by issuing instructions to ZooCode. While ZooCode is executing tasks, Hermes Agent continues thinking about the problem, researching information on the internet, evaluating better strategies, and exploring alternative solutions.

In other words, the planning core and the execution layer operate in parallel. The core continuously performs background reasoning and information gathering while ZooCode handles execution, allowing it to refine and adjust ZooCode's actions in real time as new insights become available.