Skip to content

test(oidc): make the Kubernetes token-rotation check deterministic - #2436

Merged
rdimitrov merged 1 commit into
mainfrom
fix/oidc-rotation-test-flake
Oct 20, 2026
Merged

rdimitrov merged 1 commit into
mainfrom
fix/oidc-rotation-test-flake

Conversation

@rdimitrov

Copy link
Copy Markdown
Member

Why

TestKubernetesValidatorDerivesIssuerAndAuthenticatesEndpoints failed in CI with Ready sent "***" after rotation, want the rotated token (Test (non-race draft) on #2435, which doesn't touch authn/). It passed 200 race-enabled runs locally.

The test checked the last Authorization header the fixture saw on /keys. toolhive-core v0.0.51 registers the JWKS URL with WithWaitReady(false), which queues an asynchronous first fetch, and also runs a synchronous Refresh. That background request can arrive after Ready, still carrying the pre-rotation token, and overwrite the record. This is a race in what the test observes, not a product bug: every request re-reads the token file when it is sent.

What

  • The fixture can now make /keys accept only one bearer and answer 401 to any other.
  • The test makes /keys accept only Bearer rotated, then asserts that Ready fails before the token file is rewritten and succeeds after. That proves Ready re-reads the file, whenever background fetches land. A background fetch that gets the 401 is harmless.

Testing

  • go test -race -count=300 of the test at GOMAXPROCS=1 and at GOMAXPROCS=2, plus go test -race ./... in authn/oidc.
  • Mutation: writing the rotated token to a different file makes the test fail with Ready after rotation: ... status 401.
  • GOOS=linux task lint.

🤖 Generated with Claude Code

The test asserted that the last Authorization header seen on /keys was the
rotated token. toolhive-core registers the JWKS URL with an asynchronous
first fetch alongside its synchronous one, so that background request can
land after Ready with the pre-rotation token and overwrite the record.

Assert on Ready's own result instead: once /keys accepts only the rotated
token, Ready fails before the token file is rewritten and succeeds after,
which holds however background fetches interleave.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rdimitrov
rdimitrov merged commit 0d02211 into main Oct 20, 2026
34 checks passed
@rdimitrov
rdimitrov deleted the fix/oidc-rotation-test-flake branch October 20, 2026 10:59
@toolhive-release-app toolhive-release-app Bot mentioned this pull request Oct 23, 2026
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant