Skip to content

fix(ebpf): Restore the uSID datapath on nodes that load it without full root - #536

Merged
scotwells merged 1 commit into
mainfrom
fix/usid-ingress-unprivileged-load
Sep 13, 2026
Merged

scotwells merged 1 commit into
mainfrom
fix/usid-ingress-unprivileged-load

Conversation

@scotwells

Copy link
Copy Markdown
Contributor

Every node that picks up the build with the GRO-merged return traffic fix (#535) rejects the eBPF uSID datapath at load time, so galactic-cni crash-loops and the node stops forwarding Instance traffic. This change makes the datapath load again with the capabilities galactic-cni actually runs with, and adds a CI check so this class of failure cannot reach a node again.

What went wrong

The kernel's eBPF verifier applies stricter rules to a loader that lacks CAP_PERFMON. One of those rules forbids moving a packet pointer by a variable amount. The new length check did exactly that to locate the inner TCP header past an IPv4 header of variable length. galactic-cni deliberately holds only BPF, NET_ADMIN, and NET_RAW, so the verifier rejected the datapath. CI and local tests loaded the datapath as full root, where the rule does not apply.

What changes

  • The datapath reads the inner transport header by offset instead of through a moved packet pointer. Behaviour from fix(ebpf): Keep downloads from general-purpose Instances through the edge from stalling #535 is unchanged: merged traffic is still checked against the route MTU one segment at a time, and a single oversized packet is still dropped as fragmentation-needed, for both IPv4 and IPv6. No capabilities are added to galactic-cni.
  • Each eBPF datapath (uSID, edge gateway, and NAT66) now has a CI test that loads it with only the capabilities its DaemonSet grants, read from the manifest itself. A control proves the test environment really enforces the reduced verifier rules, so the check cannot pass vacuously.

Verification

Fixes the rollout regression from #535. Related: #534, #532.

🤖 Generated with Claude Code

@scotwells
scotwells requested a review from a team as a code owner September 13, 2026 04:06
@scotwells
scotwells requested review from ecv and privateip September 13, 2026 04:06
The GRO-merged length check stepped a packet pointer past the inner
IPv4 header by its IHL-derived length. The verifier only allows a
variable packet pointer offset for a loader holding CAP_PERFMON.
galactic-cni holds BPF, NET_ADMIN, and NET_RAW, so every node rejected
the datapath and galactic-cni crash-looped. CI loaded as full root and
never saw the rule.

The transport header is now read by offset instead, which keeps the
per-segment MTU check unchanged for both families.

Each datapath now also has a load test that runs with only the
capabilities its DaemonSet grants, taken from the manifest, so a
program the verifier rejects for that loader fails CI.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@scotwells
scotwells force-pushed the fix/usid-ingress-unprivileged-load branch from 50cd477 to e4ef05c Compare September 13, 2026 04:06
@scotwells
scotwells merged commit ba14196 into main Sep 13, 2026
12 checks passed
@scotwells
scotwells deleted the fix/usid-ingress-unprivileged-load branch September 13, 2026 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants