fix(gateway): Stop the edge XDP attach from taking a whole bond down - #587
Merged
Merged
Conversation
Attaching a native XDP program makes a driver reallocate its rings, and a driver that has to do that takes the link down while it does. The attach walked every resolved interface back to back, so both members of a bonded uplink went down inside the same second and the node lost its only path off the box. Two changes, both about blast radius. The attach now waits for each bond slave to rejoin its aggregate before touching the next one. Carrier alone is not enough to call a slave ready: on an LACP bond a port can hold carrier, sit in the aggregator and be synchronized while collecting and distributing are both clear, and in that state it moves no packets. The wait reads the slave's MII status and its LACP actor state together, and gives up rather than attaching to the surviving member when a slave does not come back. The kernel is now asked, through the netdev generic netlink family, which interfaces can take a native program at all, before any of them is touched. A driver with no ndo_bpf was previously discovered by attaching, which meant the healthy slaves attached ahead of it had already paid a link bounce by the time the failure surfaced, and the rollback cost them another. Support is only ever read as a refusal when the answer is definite; a kernel too old to serve the family, or a query that fails on its own terms, warns and proceeds as before. Neither guard makes a bond with link monitoring switched off safe. With miimon at 0 the bonding driver never polls carrier, so a slave that bounces is left in a failed state that only a monitor it does not run would clear, and no amount of care on this side brings it back. That is fixed in the deployment's bond configuration, not here. These guards bound the damage; they do not repair a bond that cannot notice a link returning. The sequencing is extracted behind an internal seam so the ordering can be driven in tests without a real program or a real NIC, and the existing root-gated test that builds a real kernel bond exercises the new wait. Fixes #583 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
scotwells
approved these changes
Sep 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Starting the edge datapath on a bonded uplink took every member of the bond down inside the same second, because attaching a native XDP program makes a driver reallocate its rings and drop the link while it does. The attach now waits for each bond slave to rejoin its aggregate before touching the next one, and asks the kernel which interfaces can take a native program before touching any of them, so an unsupported driver fails the process instead of being discovered by bouncing links. Carrier alone does not count as ready: a slave can hold carrier and be synchronized while it moves no packets, so the wait reads LACP actor state too.
Important
This bounds the damage, it does not repair a bond that cannot notice a link coming back. A bond running with link monitoring off leaves a bounced slave in a failed state permanently, which is fixed in the bond configuration — see datum-cloud/infra#5748.
Test plan
Fixes #583
🤖 Generated with Claude Code