fix: warm RPC providers at bootstrap to close the cold-start detectNetwork race - #91
fix: warm RPC providers at bootstrap to close the cold-start detectNetwork race#91shan8851 wants to merge 4 commits into
Conversation
…twork race The shared docker-test composite marks the container ready as soon as /health-check (no RPC dependency) returns 200, then vitest's fileParallelism fires every test file's first request concurrently. Each first request per (network, version) tuple constructs a fresh StaticJsonRpcProvider, whose maiden detectNetwork() call schedules a real JSON-RPC round trip on the next tick; in a fresh container network namespace that cold DNS+TLS+RPC round trip can exceed the steady-state 2-attempt retry budget, causing a 404 on exit-payload-state-sync.test.ts (3x on run 31686169287). Warm the same POSClient cache used by request handlers during server bootstrap, before opening the listen socket, so the detectNetwork() round trip fires during boot instead of on the first real request. Bounded by a 5s per-tuple timeout and wrapped so a failed or slow RPC never blocks /health-check from coming up.
Pre-existing lint failure from the 2.0.0 changelog render; blocks marking #91 ready. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d client cache on failure The rc.0 proof run (31690399999) showed the fire-and-forget warm-up was insufficient: it logged "server has started" right after init(), but the state-sync test still failed with noNetwork 3s later, with 2 retries failing 1ms apart against an already-rejected cached network promise. ethers v5's StaticJsonRpcProvider only *schedules* detectNetwork() via setTimeout(0) in its constructor, so awaiting init() resolving proves detection started, not that it succeeded. warmMaticClients now drives an actual awaited round-trip through the same cached POSClient instance request handlers reuse — exitUtil.rootChain's getLastChildBlock(), the same call isBlockIncluded() makes on the request path — retrying up to 3 times with a 500ms backoff. Each retry evicts the client from clientCache first and re-inits from scratch, since a resolved init() followed by a failing probe means the provider may be sitting on bad state from the same race, and re-probing the same instance risks repeating it. The overall per-tuple timeout is raised from 5s to 8s to give the retries room while staying well under boot-liveness budgets.
|
The rc.0 proof run (31690399999) showed the fire-and-forget warm-up was insufficient: Pushed |
ci-trigger.yml already used the eRPC URL; the release gate was the only consumer of the dead rpc-amoy.polygon.technology hostname, which is why CI passed while every release-tag run failed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Closing in favour of #92: the release-gate failures were entirely caused by docker-release-trigger.yml's AMOY_RPC pointing at the retired rpc-amoy.polygon.technology hostname (no DNS record) — proven by rc proof runs: rc.1 (warm-up alone) failed, rc.3 (env fix alone, no warm-up) passed the gate. The bootstrap warm-up is therefore unnecessary; branch preserved for the record — the ethers v5 cold-detectNetwork/poisoned-cache analysis in this PR's history is accurate, just not the cause here. |
1 participant
Closes the cold-start race that blocked the Docker release gate: run 31686169287 failed 3x on
src/test/exit-payload-state-sync.test.tsbecause the docker-test composite marks the container ready as soon as/health-checkreturns 200, then vitest fires every test file's first request concurrently against a fresh container — racing each(network, version)tuple's maidendetectNetwork()call against the steady-state 2-attempt retry budget.This warms the same
POSClientcache used by request handlers during server bootstrap, before the listen socket opens, so the cold DNS+TLS+RPC round trip happens during boot instead of on the first real request. Bounded by a per-tuple timeout and wrapped so a failed or slow RPC never blocks/health-checkfrom coming up. Unblocks the 2.0.x release — 2.0.0's image never built due to this gate.