Filed from the 2026-08-25 release of 0.1.75, which carried a production
dispatch fix. This test blocked it.
The failure
FAIL src/cli/teammate-mcp.test.ts > Factory teammate MCP
> exposes discover and bounded ask to the spawned worker
through its injected MCP server
Error: Test timed out in 5000ms.
Test Files 1 failed | 109 passed | 1 skipped (111)
Tests 1 failed | 2254 passed | 1 skipped (2256)
Publish run 32821387417. Because Test is step 7, every later step —
Resolve version, Create release tag, Publish, Verify final release state — was skipped. One test held back the whole release.
It is a flake, and here is the proof rather than an assertion
Same commit 952d450e, two runs:
| run |
Test step |
32821387417 |
failure — timed out in 5000ms |
32821900947 |
success |
A fail and a pass at the same commit is the pair required before calling
anything a flake; "it passed on rerun" alone would not be. The identical tree
had also already passed CI twice — on main at 423e293b and on
release/v0.1.75 at 763710b2.
Why it matters more than a normal flake
It sits on the release critical path. publish.yml runs the suite itself,
so this test can block shipping a fix even when the change under test is
unrelated — which is exactly what happened, during an active dispatch outage.
Origin
src/cli/teammate-mcp.test.ts arrived in 92a6c24 — #178, "Who can I ask?".
It is not covered by the existing flake issues #342 (shared timing substrate
in factory.test.ts), #328 or #353.
Suspected shape, not asserted
A 5000ms default testTimeout on a test that stands up an injected MCP server
and exercises a spawned-worker round trip. Under a loaded CI runner, server
start plus round trip can plausibly exceed 5s without anything being wrong. That
is a hypothesis from the failure mode and the test's name — I have not profiled
it, and it should be measured rather than assumed. In particular, do NOT simply
raise the timeout until it stops failing: that hides a real race just as
effectively as it fixes a slow setup.
What done looks like
- Establish which it is — slow setup, or a genuine race in the injected MCP
server / spawned-worker handshake. Measure the actual distribution rather
than inferring from one timeout.
- If slow setup: an explicit, justified timeout for this test with the measured
headroom stated, not a global bump.
- If a race: fix the race. A raised timeout would convert a reproducible
failure into a rare one, which is worse.
- Either way, the test must be deterministic enough that a release is never
blocked by it again. size the fixture to the mechanism's boundary applies:
name the boundary first.
Filed from the 2026-08-25 release of
0.1.75, which carried a productiondispatch fix. This test blocked it.
The failure
Publish run
32821387417. BecauseTestis step 7, every later step —Resolve version,Create release tag,Publish,Verify final release state— was skipped. One test held back the whole release.It is a flake, and here is the proof rather than an assertion
Same commit
952d450e, two runs:3282138741732821900947A fail and a pass at the same commit is the pair required before calling
anything a flake; "it passed on rerun" alone would not be. The identical tree
had also already passed CI twice — on
mainat423e293band onrelease/v0.1.75at763710b2.Why it matters more than a normal flake
It sits on the release critical path.
publish.ymlruns the suite itself,so this test can block shipping a fix even when the change under test is
unrelated — which is exactly what happened, during an active dispatch outage.
Origin
src/cli/teammate-mcp.test.tsarrived in92a6c24— #178, "Who can I ask?".It is not covered by the existing flake issues #342 (shared timing substrate
in
factory.test.ts), #328 or #353.Suspected shape, not asserted
A 5000ms default
testTimeouton a test that stands up an injected MCP serverand exercises a spawned-worker round trip. Under a loaded CI runner, server
start plus round trip can plausibly exceed 5s without anything being wrong. That
is a hypothesis from the failure mode and the test's name — I have not profiled
it, and it should be measured rather than assumed. In particular, do NOT simply
raise the timeout until it stops failing: that hides a real race just as
effectively as it fixes a slow setup.
What done looks like
server / spawned-worker handshake. Measure the actual distribution rather
than inferring from one timeout.
headroom stated, not a global bump.
failure into a rare one, which is worse.
blocked by it again.
size the fixture to the mechanism's boundaryapplies:name the boundary first.