fix(usid): Deliver inbound traffic to tap and veth Instances that share a VPC on one node

[scotwells]

Summary

A general-purpose Instance could become unreachable as soon as a unikernel Instance joined the same VPC on the same node. Inbound connections to it timed out, and no drop counter recorded the loss. With this change, both kinds of Instance receive inbound traffic, whatever order they attach in.

Cause

The datapath picks one of two kernel redirect methods to deliver decapsulated traffic. Veth attachments (unikernel Instances) use a method that crosses into the Instance's network namespace. Tap attachments (general-purpose Instances) need the plain method. That choice was stored once per VPC per node, so the last attachment to register decided it for every attachment in the VPC. When a veth registered after a tap, the kernel silently discarded every packet sent to the tap. The staging outage on eris-giune in VPC b followed exactly that order.

Fix

• The redirect choice now belongs to each attachment's host-side interface. It is recorded when the attachment is added, removed when it is deleted, and verified by the CNI CHECK operation.
• The choice lives in a new datapath map. No existing map changes layout, so rolling this out does not force the loader to recreate maps and wipe live routing state.
• An interface with no entry uses the plain redirect method. This covers attachments added before this change and any upgrade window before the new map is pinned. According to the kernel source, a plain redirect into a veth's host-side end is forwarded into the peer namespace, so it delivers to both kinds. The only cost is skipping the peer method's faster path. A test confirms this against a real kernel.
• During an upgrade, CNI ADD runs with the new CNI binary before the new datapath is loaded. In that window, ADD logs a missing map instead of failing, so Instances keep starting during the rollout.

Testing

A new datapath test sends real SRv6 frames through the program into tap and veth devices. Against the current main datapath, it reproduces the outage: the tap attachment is black-holed when the veth registers last. With this change, both attachments receive traffic in both registration orders, each through the expected redirect method. Other tests cover the no-entry fallback, deletion of one attachment's entry, and loading a pin set from the previous build without losing routing state.

Related

• Tap attachments for general-purpose Instances: Attach general-purpose workloads to a customer's VPC network #528, Let general-purpose instances start and route on a VPC network #531, fix(cni): Give general-purpose Instances their network's MTU #533
• Ingress sidecar route handling: fix: keep a deleted backend from tearing down a live one's route #526, fix(ingresssidecar): keep a route a second EndpointSlice still claims #518
• A separate fix for GRO-merged return packets on the decap path is in flight and edits the same datapath source file, outside the redirect step. The two changes don't overlap, but whichever merges second needs a rebase.

🤖 Generated with Claude Code