Skip to content

Run the node signal adapters on every replica, not only the leader - #11

Merged
kylan11 merged 1 commit into
mitis-cloud:mainfrom
0xmichalis:fix/signal-adapters-every-replica
Oct 19, 2026
Merged

kylan11 merged 1 commit into
mitis-cloud:mainfrom
0xmichalis:fix/signal-adapters-every-replica

Conversation

@0xmichalis

Copy link
Copy Markdown
Contributor

Summary

  • The cordon and taint adapters are controllers, so controller-runtime runs them on the leader only. Every replica serves the eviction webhook, though, and standInReady consults that replica's own in-memory signal.Registry. On a replica without the lease the registry never holds cordon or taint dooms. With replicaCount: 2, the production recommendation, an eviction served by the standby is admitted while the only ready stand-in sits on another cordoned or tainted node (a node pool upgrade or a multi-node drain), where the leader would correctly keep holding.
  • Set NeedLeaderElection: false on both adapters. They only read nodes from the shared cache and write the in-memory registry, so running them on every replica is safe and adds no API writes.
  • Docs: README Safety and how-it-works Failure behaviour now say every replica watches node signals.
  • Out of scope: doom signals from the eviction webhook itself stay per replica, so the leader learns of an eviction-only drainer from the retries it serves.

Verification

  • New envtest spec starts a second manager whose leader lease is held by another identity. On main its registry never sees a cordoned or a tainted node (the spec times out). With this change both appear, and the manager never becomes leader.
  • make test lint build and the ARGO_ROLLOUTS_CRD= "Optional Rollout API" run: passed, no generated-file drift.

🤖 Generated with Claude Code

Copilot AI balanced review requested due to automatic review settings October 17, 2026 19:44

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@kylan11
kylan11 merged commit 89006f4 into mitis-cloud:main Oct 19, 2026
7 checks passed
@kylan11 kylan11 mentioned this pull request Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants