Skip to content

fix: bound graph rebuilds of the public RPC and release its connections before external calls - #534

Merged
eigmax merged 2 commits into
devfrom
audit/26-27-rpc-bounds
Nov 17, 2026
Merged

eigmax merged 2 commits into
devfrom
audit/26-27-rpc-bounds

Conversation

@KSlashh

@KSlashh KSlashh commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

Audit items M26 and M27. Open against dev.

Graph rebuilds (M26). /v1/graphs/{id}/tx and /v1/graphs/{id}/txn take no auth and have no cache or rate limit, and each request did the full graph work twice: once in load_validated_graph_definition, which rebuilds the graph to check its pre-signatures, and once more in the handler's from_simplified. Both ran on the tokio workers while the request held a pool connection.

load_validated_graph_definition is split into a database read (read_graph_definition) and a CPU check (StoredGraphDefinition::build). build returns the graph it already rebuilt. Both endpoints now:

  • read the definition and release the connection;
  • run build on the blocking pool, at most 2 at a time (GRAPH_BUILDS);
  • use the returned graph instead of rebuilding it.

Other callers of load_validated_graph_definition are unchanged.

Connections during external calls (M27). The pool has 10 connections, shared with the engine. get_instances, get_instance, get_graph, get_graphs and the two endpoints above kept their connection through the esplora and Goat calls, so plain GET requests could take every connection; engine and inbox writes then timed out and /v1/health returned 503. Each of them now releases its connection after its last query.

Review notes

  • A request waiting for a build slot holds no connection and no raw definition: the slot is taken before the definition is read.
  • No separate RPC pool: once requests hold connections only for their queries, the shared pool is not held across external calls.
  • The debug endpoints (send-challenge, send-verifier-challenge) still use the old path; they are behind rpc-debug-endpoints.

Tests: the public build rejects an unsigned graph with the same error as the validated load, and the read returns nothing for an unknown graph and fails for a mismatched instance. Node tests, workspace clippy with -D warnings (with and without rpc-debug-endpoints) and fmt pass.

The test coverage and validation notes above are from the zkm PR notes. Branch scope and diff formatting were checked locally; builds and tests were not rerun during publication.

let Some(stored) = stored else {
return Ok(None);
};
Ok(Some(tokio::task::spawn_blocking(move || stored.build()).await??))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Keep the semaphore permit inside the blocking build

_permit is owned by the HTTP handler future. When the client disconnects, that future is dropped and the permit is released, but an already-running spawn_blocking closure continues executing. Repeating requests followed by disconnects therefore bypasses GRAPH_BUILDS = 2, allowing unauthenticated clients to accumulate expensive graph builds and consume CPU/memory.

I reproduced this with a loopback axum HTTP/1.1 server using this exact load_public_graph body and a gated substitute for stored.build(): start two requests, wait until both builds are running, disconnect both clients, then send a third request. The third build starts while the original two are still running. The regression asserting at most two concurrent builds fails.

Please move the acquired permit into the spawn_blocking closure so it remains held until the actual build finishes, even if the awaiting handler is cancelled. Add a regression covering client disconnects while builds are running.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in be1a57d. The permit now moves into the spawn_blocking task, so a request that is dropped keeps it until the build finishes. Added a regression: two builds running, both requests aborted, no permit is available until the builds end.

@eigmax
eigmax force-pushed the audit/26-27-rpc-bounds branch from be1a57d to 4aefc26 Compare November 16, 2026 07:50
@eigmax
eigmax merged commit 63a8c31 into dev Nov 17, 2026
4 checks passed
@eigmax
eigmax deleted the audit/26-27-rpc-bounds branch November 17, 2026 02:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants