File each lost observation against the packet it belonged to

[copyleftdev]

Stacked on #15 — base is attribute-ring-buffer-loss, so the diff here is only this change. Merge #15 first and this retargets to main.

Evidence change

#15 narrowed a hole to a probe, but left every absence unusable. Knowing ip_rcv failed to emit somewhere says nothing about whether a particular packet's missing hop was never reached or merely never observed, so complete: false still downgraded every absence in the capture.

This files each hole against the packet it belonged to. Two packets from the same globally-incomplete capture (16,382 reserve failures), on a live kernel:

$ skbx explain light.jsonl event:...38e8
  "evidence": {"verdict": "lost", "observations_lost": 10}     6 hops observed
$ skbx explain light.jsonl event:...08e8
  "evidence": {"verdict": "complete"}                         10 hops observed

For the second packet, a function absent from those 10 hops was never reached. That is "this function was never hit" separated from "we lost that one" — per packet, inside a capture that is not complete overall.

Closes the question from the DEV thread that prompted #15.

Why a plain hash, not an LRU

This is the load-bearing decision. A full LRU evicts silently, and a silently evicted entry would make a packet look like it lost nothing — turning the feature into a lie under exactly the load that makes it matter. A plain hash refuses the insert and counts the refusal in kernel_skb_loss_unattributed.

That refusal is what makes the negative claim provable rather than merely likely: while the counter is zero the table is exhaustive, so a packet absent from it lost nothing. An LRU would trade that proof for a bounded amount of memory.

Reserve failures are the only loss a packet can own

loss_is_fully_attributed() requires the skb table to be exhaustive and kernel_recursion_misses, userspace_decode_failures, userspace_enrichment_failures and output_failures to all be zero. A recursion miss never reaches the emit path; a decode, enrichment or output failure discards a record after the kernel handed it over, when nothing knows which packet it described. Any of those leaves a hole no packet can be cleared of.

Read failures are deliberately excluded — they degrade fields on an event that was still emitted, and are already visible on it.

An earlier draft checked only the skb table and would have cleared packets in captures with output loss. Test loss_that_no_packet_owns_defeats_attribution pins each case.

Design note: this is not what the follow-up in #15 described

#15 proposed per-skb emit sequencing — stamp a counter into every record, infer loss from gaps. The identity turns out to be in scope at all 16 failure sites, which makes recording the loss directly strictly better:

	sequencing	this
hot-path cost	map op on every event	failure path only
record ABI	grows 8 bytes	unchanged
lost trailing event	invisible, chain just ends	recorded
wholly-lost packet	invisible, no records to inspect	recorded
opt-in required	yes	no

The trailing-event blind spot decided it: #15 already noted sequencing could not see a lost last hop. This has no such gap, and needs no flag because it costs nothing until something is already failing.

Packets that were previously invisible

The first draft of the summary line printed 363 of 320 packets affected. Not a formatting bug — 104 packets lost every observation, so they appear in the ledger but nowhere in the capture. A wholly-dropped chain previously left no trace at all. Replay now reports these as skbs_lost_entirely, separate from distinct_skbs, which counts only packets actually observed.

Change design

1. Kernel fact — which observation of which packet was dropped, keyed by the identity already stamped on every event, so it joins directly against a replayed chain.
2. Target validation — no new attachment surface; the identity is already resolved at every failure site.
3. Bounds — one BPF_MAP_TYPE_HASH, 4096 entries, preallocated. Written only on the failure path. Overflow is counted, never silent.
4. Failure visibility — kernel_skb_loss_unattributed is the single gate on whether any packet may be cleared; non-zero collapses the claim for all of them at once.
5. Checks — below.
6. Schema — append-only. New fields are #[serde(default)] and absent from required; the pre-existing fixture still parses and replays.

Checks run

• make check and make build, both exit 0.
• 14 new tests covering the exhaustiveness gate, each loss kind that defeats it, read failures that do not, the three explain verdicts, missing-footer, and entirely-lost packets.
• Live kernel 6.17.0-41-generic, 64 CPUs, across all three regimes: light loss (table under capacity, unattributed=0) → per-packet complete/lost; heavy loss (table full at 4096, 11,866 unfiled) → unknown for every packet; lossless → complete. Filed + unfiled reconciled against kernel_reserve_failures exactly.

Untested

Unchanged from #15: the TC/XDP program-site path never executes here. Now verified live. Both paths were exercised under forced loss in isolated network namespaces, and the per-packet ledger carries the program id the same way the per-probe table does:

"kernel_loss_by_skb": [{"skb": "0xffff8ac285589f00", "function": null, "program_id": 432, "reserve_failures": 1}]

explain rules correctly on TC-traced packets from a lossy capture — {"verdict": "lost", "observations_lost": 1} for an affected packet, {"verdict": "complete"} for one the ledger clears.

Full live suite: 13/14 pass. live-bpf-helper fails identically on main at b378c01, so it is pre-existing on this host, not caused by this change.

4096 entries is sized for the light-loss case this feature is for. Under heavy loss the table fills and every verdict degrades to unknown, which is correct but means the guarantee is only available when loss is bounded. That ceiling is a judgement call, not a measurement.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: eb2b48f2-bbf7-4ed2-9836-db8b43aa7cdd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[copyleftdev]

Closed automatically when its base branch was deleted during the #15 merge. Continued in #18, rebuilt on top of main with identical content plus the review fixes.