|
| 1 | +--- |
| 2 | +title: Add an internal MCP |
| 3 | +description: "Connect an MCP server from your private network with an outbound tunnel. Set up caller identity, team access, and a highly available deployment." |
| 4 | +--- |
| 5 | + |
| 6 | +import { Callout } from "@/mdx/components"; |
| 7 | +import TunnelDiagram from "@/features/docs/internal-mcp/TunnelDiagram.astro"; |
| 8 | +import "@/features/docs/internal-mcp/internal-mcp.css"; |
| 9 | + |
| 10 | +<div className="internal-mcp-guide"> |
| 11 | + |
| 12 | +Connect an MCP server in your own infrastructure to the Speakeasy AI control plane. A tunnel agent runs beside your server and connects outbound to Speakeasy. Your team gets a hosted MCP endpoint with authentication and access controls. The server stays inside your network without a public IP, inbound firewall rule, or public ingress. |
| 13 | + |
| 14 | +Use a tunnel for internal APIs, operational tools, or services that need private access to databases and other systems. For high-traffic services, run multiple replicas behind one endpoint. See [High availability](#high-availability). |
| 15 | + |
| 16 | +## How the tunnel works |
| 17 | + |
| 18 | +The agent opens the connection from inside the private network. Speakeasy uses that connection to send MCP requests back to the server. |
| 19 | + |
| 20 | +<TunnelDiagram /> |
| 21 | + |
| 22 | +1. The tunnel agent authenticates with a tunnel key and opens an outbound, TLS-protected WebSocket to Speakeasy. |
| 23 | +2. An MCP client connects to your hosted endpoint. Speakeasy authenticates the caller and checks their access to the server and its tools. |
| 24 | +3. Speakeasy sends requests over the existing connection. The agent forwards them to your configured private MCP URL and streams responses back along the same path. Multiple requests share the connection. |
| 25 | + |
| 26 | +Your MCP server uses the standard Streamable HTTP transport. It can use any framework and keep its existing authentication. Servers that only support stdio need an HTTP adapter before they can be tunneled. The agent's destination is fixed at startup; a client cannot choose another address on your network. |
| 27 | + |
| 28 | +The hosted endpoint is reachable by MCP clients over the internet and requires authentication when its visibility is Private. To also restrict that client-facing endpoint to your network, combine the tunnel with [Tailscale private access](/docs/ai-control-plane/distribute/tailscale). |
| 29 | + |
| 30 | +## Create the source |
| 31 | + |
| 32 | +You need an MCP server with a Streamable HTTP endpoint, a host that can reach it, and a plan with [tunneled MCP servers](/docs/ai-control-plane/distribute/mcp-servers/tunneling#plan-limits). Creating and managing a tunnel requires `mcp:write` access to the project; the default Admin role includes it. |
| 33 | + |
| 34 | +1. Open **Connect > Sources** and choose **Add Source > Tunneled MCP Server**. |
| 35 | +2. Name the source. If your server has a protected resource identifier, enter it in **Resource identifier (optional)**, for example `https://mcp.internal.example.com/mcp`. This identifies the server for upstream credentials and the assertion audience. Speakeasy never connects to this address. |
| 36 | +3. Create the source and save the tunnel key in your secret manager. It is shown once and cannot be retrieved later. |
| 37 | +4. Under **Tunnel endpoint**, choose **Existing server** and enter the private MCP endpoint the agent will reach. Choose **New server** to try a sample hello-world server instead. |
| 38 | +5. Copy the generated Docker, Kubernetes, or CLI setup snippet and start the agent. |
| 39 | + |
| 40 | +Speakeasy creates a linked MCP server and a default endpoint for the source. Once the agent connects, open that server's **Inspect** tab to connect and list its tools. Use the hosted endpoint in your MCP client, then make a tool call to check the full path. |
| 41 | + |
| 42 | +Keep the server's visibility **Private** for internal services. [Grant access](#grant-users-and-roles-access) before distributing the endpoint to your team. |
| 43 | + |
| 44 | +## Deploy the agent |
| 45 | + |
| 46 | +Run the agent wherever you can keep an outbound WebSocket connection open and reach the MCP server. For example: |
| 47 | + |
| 48 | +| Environment | Deployment | |
| 49 | +| --- | --- | |
| 50 | +| Kubernetes, including EKS and GKE | A container beside your MCP server in the same Pod, or a Deployment pointing at an in-cluster Service. | |
| 51 | +| AWS | An ECS task or EC2 process with access to the server in your VPC. | |
| 52 | +| Google Cloud | A GKE workload or Compute Engine process with access to the private endpoint. | |
| 53 | +| Docker | A container on the same network as the MCP server. | |
| 54 | +| A VM, on-premises host, or laptop | The agent container or `gram tunnel run`, managed by your process supervisor. | |
| 55 | + |
| 56 | +Allow outbound HTTPS/WebSocket traffic to the gateway URL in your generated snippet, normally on port 443. The agent also needs network access to its configured MCP endpoint. Run it as a long-running process and account for your platform's connection timeouts and instance shutdown policies. |
| 57 | + |
| 58 | +### Agent configuration |
| 59 | + |
| 60 | +The agent reads environment variables: |
| 61 | + |
| 62 | +| Variable | Required | Value | |
| 63 | +| --- | --- | --- | |
| 64 | +| `TUNNEL_GATEWAY_URL` | Yes | The TLS gateway URL from your setup snippet. | |
| 65 | +| `TUNNEL_KEY` | Yes | The secret issued when you created the source. Multiple agent replicas can share it. | |
| 66 | +| `TUNNEL_LOCAL_MCP_URL` | Yes | The MCP endpoint the agent can reach, such as `http://internal-mcp:3000/mcp`. | |
| 67 | +| `TUNNEL_SERVICE_VERSION` | Yes | The version of your MCP service, recorded with the connection. | |
| 68 | +| `TUNNEL_METADATA` | No | A JSON object of string values, up to 1,024 bytes, for deployment metadata. | |
| 69 | + |
| 70 | +The gateway URL must use `wss://` or `https://` outside local development. The private MCP endpoint can use HTTP or HTTPS. If your MCP server's OAuth endpoints live on another origin, place a local reverse proxy in front of both: the agent forwards MCP and OAuth traffic to its configured origin. |
| 71 | + |
| 72 | +### Docker example |
| 73 | + |
| 74 | +Suppose your MCP container is named `internal-mcp`, listens on port 3000, and belongs to the Docker network `mcp-network`. Create a local `tunnel.env` file using the gateway URL and key from the dashboard: |
| 75 | + |
| 76 | +```dotenv |
| 77 | +TUNNEL_GATEWAY_URL=<GATEWAY_URL_FROM_SETUP> |
| 78 | +TUNNEL_KEY=<YOUR_TUNNEL_KEY> |
| 79 | +TUNNEL_LOCAL_MCP_URL=http://internal-mcp:3000/mcp |
| 80 | +TUNNEL_SERVICE_VERSION=1.0.0 |
| 81 | +``` |
| 82 | + |
| 83 | +Keep this file out of version control and restrict access to it. Start the agent: |
| 84 | + |
| 85 | +```bash |
| 86 | +docker run -d --name mcp-tunnel \ |
| 87 | + --restart unless-stopped \ |
| 88 | + --network mcp-network \ |
| 89 | + --env-file tunnel.env \ |
| 90 | + ghcr.io/speakeasy-api/gram-tunnel-agent:latest |
| 91 | +``` |
| 92 | + |
| 93 | +`localhost` inside the agent container refers to that container, so use the MCP container's network name. For production, pin the agent image to a released version or digest. Inject `TUNNEL_KEY` from your platform's secret manager. |
| 94 | + |
| 95 | +### Kubernetes |
| 96 | + |
| 97 | +The dashboard generates a Secret and a Deployment for an existing server, or a sample server with an agent. Set `TUNNEL_LOCAL_MCP_URL` to the MCP Service's internal address, such as `http://internal-mcp.default.svc.cluster.local:3000/mcp`. |
| 98 | + |
| 99 | +For a server that keeps MCP session state in memory, run an agent beside each MCP container in the same Pod and point it at `http://127.0.0.1:3000/mcp`. This keeps a tunnel connection attached to a specific server replica. The [HA deployment pattern](#high-availability) below covers replica placement and session recovery. |
| 100 | + |
| 101 | +## Grant users and roles access |
| 102 | + |
| 103 | +Speakeasy enforces access before forwarding requests through the tunnel. Configure access in the control plane even if your MCP server does not implement user authentication itself. |
| 104 | + |
| 105 | +1. Open **Organization settings > Secure > Roles & Permissions** as an organization admin. |
| 106 | +2. Create a role for the people who should use this server. Add an `mcp:connect` grant scoped to the specific MCP server. Use **Specific tools** to limit the grant to selected tools. |
| 107 | +3. Assign the role to the intended users on the **Members** tab. When Directory Sync is enabled, assign roles through your identity provider. |
| 108 | +4. Open the MCP server's **Team Access** tab to review the resulting grants. Test with a member account that has the intended roles. |
| 109 | + |
| 110 | +If **Specific tools** is empty, connect from the server's **Inspect** tab first to record its tool metadata. |
| 111 | + |
| 112 | +<Callout type="info"> |
| 113 | + Permissions from all of a user's roles add together. The default Member role grants broad MCP read and connect access, and `mcp:read` and `mcp:write` also imply connect access. A narrow custom role does not remove an existing broad grant. To restrict a server to selected users, review those existing grants and adjust the baseline roles before assigning scoped access. Changes to a system role affect everyone who holds it. |
| 114 | +</Callout> |
| 115 | + |
| 116 | +See [Roles and permissions](/docs/ai-control-plane/org-admin/roles-and-permissions) for scope inheritance and directory-managed roles. An upstream server can also apply its own policy using [signed caller identity](#signed-caller-identity). |
| 117 | + |
| 118 | +## Signed caller identity |
| 119 | + |
| 120 | +Private tunneled servers receive a short-lived JWT on each forwarded request from a supported authenticated caller: |
| 121 | + |
| 122 | +```http |
| 123 | +SPEAKEASY_AUTHZ: <JWT> |
| 124 | +``` |
| 125 | + |
| 126 | +The header contains the JWT without a `Bearer` prefix. Your server can ignore this extra header and work unchanged. Upstream OAuth credentials continue to use `Authorization`. |
| 127 | + |
| 128 | +### JWT contract |
| 129 | + |
| 130 | +The protected JWT header contains `alg=RS256`, `typ=speakeasy-authz+jwt`, and a `kid` identifying the signing key. The `kid` is the public key's RFC 7638 SHA-256 thumbprint. |
| 131 | + |
| 132 | +| Claim | Value | |
| 133 | +| --- | --- | |
| 134 | +| `version` | `1`. | |
| 135 | +| `iss` | `https://tunnel.speakeasy.com`. | |
| 136 | +| `aud` | The destination's saved resource identifier, or `tunneled-mcp-server:<TUNNELED_MCP_SERVER_ID>` if it is unset. | |
| 137 | +| `sub` | A stable, typed principal ID: `user:<USER_ID>`, `api_key:<API_KEY_ID>`, or `agent:<AGENT_ID>`. | |
| 138 | +| `principal_type` | `user`, `api_key`, or `agent`, matching the subject. | |
| 139 | +| `email` | The human caller's profile email. Absent for API keys and agents. | |
| 140 | +| `iat`, `exp` | Issuance and expiry in Unix seconds. Valid for at most 60 seconds, and capped by the source credential's expiry where available. | |
| 141 | +| `jti` | A unique identifier for this assertion. Each forwarded request or retry gets a fresh assertion. | |
| 142 | +| `organization_id`, `project_id` | The destination organization and project. | |
| 143 | +| `mcp_server_id` | The MCP server serving the request. | |
| 144 | +| `tunneled_mcp_server_id` | The underlying tunneled source. | |
| 145 | +| `purpose` | `mcp_request` for runtime requests, or `mcp_discovery` during authenticated consent discovery. | |
| 146 | +| `allowed_methods` | Present for consent discovery; describes the methods Speakeasy permits in that context. | |
| 147 | + |
| 148 | +The audience matches the saved resource identifier exactly, including trailing slashes and escaped characters. In a gateway with several servers, it identifies the selected destination. A caller cannot override it with a request parameter. If you change the saved identifier, update your verifier and reconnect any upstream OAuth credentials associated with the old resource. |
| 149 | + |
| 150 | +Use `sub` as the stable identity key because email addresses can change. Subject IDs come from the Speakeasy control plane, not the external IdP. The contract does not include role names, IdP groups, or an `email_verified` claim. API keys and agents have their own identities; their subjects do not identify their human owners. |
| 151 | + |
| 152 | +### Verify the assertion and refresh keys |
| 153 | + |
| 154 | +The issuer is the Speakeasy AI control plane. Its public keys are available at: |
| 155 | + |
| 156 | +```text |
| 157 | +https://tunnel.speakeasy.com/.well-known/jwks.json |
| 158 | +``` |
| 159 | + |
| 160 | +If your server uses these claims to identify a caller: |
| 161 | + |
| 162 | +1. Use a JWT library to select the RSA signing key by `kid` from this fixed JWKS URL. Do not follow key URLs or issuers supplied by a token. |
| 163 | +2. Verify the signature with an explicit RS256 allowlist. Require `typ=speakeasy-authz+jwt`, `version=1`, `iss=https://tunnel.speakeasy.com`, and the exact audience you expect for your server. |
| 164 | +3. Require `iat` and `exp`. Reject expired or future-dated tokens, and enforce a maximum 60-second lifetime with at most five seconds of clock tolerance. |
| 165 | +4. Apply your own access policy to the verified identity. If your policy binds a specific organization, project, or server, also check the corresponding ID claims. |
| 166 | + |
| 167 | +The JWKS endpoint supports GET, HEAD, ETag, and conditional GET. It advertises a five-minute cache lifetime (`Cache-Control: public, max-age=300, must-revalidate`). On an unknown `kid`, refresh from that fixed URL once before rejecting the assertion. Signing keys rotate approximately every 180 days. |
| 168 | + |
| 169 | +Verify each request whose claims you use. An initialization assertion covers only that request. A response admitted before expiry can continue streaming afterwards. Use `jti` to correlate an assertion with a request; it does not prevent replay before expiry. Keep assertions out of application logs, tool arguments, and error responses. |
| 170 | + |
| 171 | +## High availability |
| 172 | + |
| 173 | +Run multiple tunnel agents with the same tunnel key to serve one source. Each agent opens its own connection. Speakeasy routes traffic across live connections and keeps established MCP sessions attached to the agent serving them. |
| 174 | + |
| 175 | +<TunnelDiagram variant="ha" /> |
| 176 | + |
| 177 | +HA deployments have sustained healthy traffic at 300 requests per second. That observation does not establish a per-replica capacity limit or guarantee throughput. Size your deployment for your tools' latency, concurrency, and streaming behavior. |
| 178 | + |
| 179 | +For production: |
| 180 | + |
| 181 | +- Run at least two agent and MCP server replicas. Place replicas on different nodes, and across availability zones when your infrastructure supports it. |
| 182 | +- For stateful servers, pair an agent with each MCP process, for example in the same Kubernetes Pod or ECS task. Point each agent at its paired server. If agents instead connect through a shared Service or load balancer, the MCP tier must preserve session routing or share its state. |
| 183 | +- Use readiness checks for the MCP service, restart failed processes, and retain enough spare capacity to lose a replica. In Kubernetes, use topology spread or pod anti-affinity and a disruption budget appropriate to your replica count. |
| 184 | +- Test rolling updates and the loss of an agent, server, and node under representative traffic. Include long-lived streams and slow tools in the test. |
| 185 | + |
| 186 | +### Session recovery |
| 187 | + |
| 188 | +The agent reconnects automatically when its connection drops, with jittered backoff from half a second up to 30 seconds. A healthy connection for 30 seconds resets the backoff. |
| 189 | + |
| 190 | +Routing affinity does not replicate an MCP server's in-memory state. If the agent serving a session disappears, that session can fail and the client must initialize a new one. Plan for interrupted streams during failures or deployments. Retry mutating tool calls only when the tool's semantics or an idempotency key make that safe. |
| 191 | + |
| 192 | +### Rate limits and capacity |
| 193 | + |
| 194 | +For private, authenticated traffic, enforce any application-specific per-user quotas in your MCP server or its local proxy. Use the verified, stable `sub` claim as the quota key, scoped by tenant when quotas are tenant-specific. Monitor concurrent requests and tool latency when setting capacity; a long-running stream occupies capacity even at a low request rate. |
| 195 | + |
| 196 | +If you publish a tunneled server for anonymous use, open its **Settings > Anonymous Rate Limit** section and set **Requests per second** and **Burst**. All anonymous callers and MCP methods share one token bucket for that source. Requests over the limit receive HTTP 429 with `Retry-After`. These controls limit aggregate anonymous traffic; they do not set per-user quotas for private access. The dashboard shows the effective limits, including defaults when fields are blank. |
| 197 | + |
| 198 | +Public access requires both source-level permission and Public visibility on the MCP server. It allows anonymous access to the exposed tools and does not carry signed caller identity. See [Public visibility](/docs/ai-control-plane/distribute/mcp-servers/tunneling#public-visibility) before enabling it. |
| 199 | + |
| 200 | +## Monitor and maintain the connection |
| 201 | + |
| 202 | +The dashboard shows connected tunnel agents, heartbeats, service versions, and active streams. Check both the tunnel connection and the health of the MCP service behind it. A connected agent alone does not prove that a tool call will succeed. |
| 203 | + |
| 204 | +Use [tool logs](/docs/ai-control-plane/observe/tool-logs) to investigate calls, errors, and latency. Enable tool I/O logging to record arguments and results. For longer retention, export the `tool_call_logs` data source to an OTLP/HTTP destination and set retention there. See [OpenTelemetry exports](/docs/ai-control-plane/observe/opentelemetry) for destination configuration and sensitive-data controls. |
| 205 | + |
| 206 | +To replace a lost or compromised tunnel key, rotate it in the dashboard and update every replica. Rotation invalidates the previous key and disconnects agents using it, so plan for clients to reconnect. This key authenticates the agent; it is separate from the JWT signing keys published through JWKS. |
| 207 | + |
| 208 | +</div> |
0 commit comments