Repository navigation
fix(toolkit): dpkt, scapy and pyshark drop TCP and IPv4 behind some IPv6 extension-header chains #1626
Description
With engine='dpkt' or engine='scapy', TCP and IPv4 behind some IPv6 extension-header chains get no reassembly or trace input. The default engine reads past those headers and hands both over.
Repro on main (ee324e1dc), and unchanged on #1622 and #1624. Each capture is Ethernet, then IPv6 carrying the chain, then either TCP or two IPv4 fragments. The cells show the TCP reassembly/trace inputs and the IPv4 reassembly inputs.
| Chain | default | dpkt | scapy |
|---|---|---|---|
| Fragment (offset 0, M=0), then AH | 1/1, 2 | 0/0, 0 | 0/0, 0 |
| Fragment (offset 0, M=0), then Destination Options | 1/1, 2 | 0/0, 0 | 1/1, 2 |
| AH alone | 1/1, 2 | 1/1, 0 on #1622 | 0/0, 0 |
| Mobility (135) | 1/1, 2 | 0/0, 0 | 0/0, 0 |
| HIP (139), TCP only | 1/1 | 0/0 | 0/0 |
The #1622 cross-review traced dpkt's Fragment-first case to dpkt's own ip6.py:96 (if self.nxt == 44 and ext.frag_off). That line raises AttributeError when another extension header follows the Fragment header, so the frame degrades to RawPacket. dpkt does not decode 135, 139 or 140 at all.
Expected: dpkt and scapy step over the same chains as the default engine, within the #1604 bound, using the default engine's rules as #1624 does for IPv4.
Blocked until #1622 and #1624 merge, because they edit the dpkt and scapy carrier and IPv4 helpers.
Activity
JarryShaw commentedon Oct 11, 2026
Widening to pyshark. Over IPv6, pyshark also misses TCP behind Mobility (135) and HIP (139). Measured on
ee324e1dc(3.11, tshark 4.6.9), trace inputs: default 1, pyshark 0 for both; IPv6 plain TCP and Hop-by-Hop controls agree. The #1625 worker reports the cause: tshark's PDML carries no octets past those headers, so the adapter cannot read TCP from the PDML alone. Its fix for the IPv4 case adds a second tshark pass for those octets.Now blocked until #1622, #1624 and the #1625 PR merge.
[-]fix(toolkit): dpkt and scapy drop TCP and IPv4 behind some IPv6 extension-header chains[/-][+]fix(toolkit): dpkt, scapy and pyshark drop TCP and IPv4 behind some IPv6 extension-header chains[/+]JarryShaw commentedon Oct 12, 2026
#1622 and #1624 have merged. Still blocked until the #1625 PR merges (it replaces #1628, closed for conflicts).
JarryShaw commentedon Oct 13, 2026
Widening to dpkt's IPv6 reassembly input. When the Fragment header names another extension header, dpkt hands IPv6 reassembly nothing. So a datagram whose Fragmentable Part starts with Destination Options (RFC 8200 §4.5) gets no input from any of its fragments. Measured on
main(b375656e1), 3.14 and 3.11 (dpkt 1.9.8), IPv6 reassembly inputs:The first three raise
AttributeErrorat dpkt'sip6.py:96, as above, so the frame is kept raw. In the last, dpkt's walk (ip6.py:83-88) reads the later fragment's data as a Destination Options header and raisesNeedData(ip6.py:201), so Ethernet keeps the IPv6 packet as raw octets.JarryShaw commentedon Oct 13, 2026
Widening to Shim6 (140) over IPv6. Behind an 8-octet Shim6 header, dpkt and scapy give no TCP input and no IPv4 reassembly input. The default engine reads past the header with the generic
IPv6_Extparser registered for 140 (pcapkit/protocols/internet/ipv6_ext.py:670). Measured onmain(b375656e1), Python 3.14 with dpkt 1.9.8 and scapy 2.7.0; Python 3.11 with scapy 2.8.0 gives the same. TCP rows show TCP reassembly / trace inputs; the fragment row shows IPv4, IPv6 reassembly inputs:pyshark traces the TCP and pypcapfile gives the two IPv4 inputs, matching the default engine.
JarryShaw commentedon Oct 13, 2026
#1642 has merged, but this issue still spans dpkt and scapy, which #1655 and #1652 are editing. It stays blocked until both merge.
JarryShaw commentedon Oct 13, 2026
Unblocked: #1652 and #1655 have merged. Workers now have the remaining dpkt and scapy parts, one PR per engine file: dpkt together with #1651 and #1654, and scapy together with #1650. Each PR cites this issue as
Part of, so this closes only after both merge.fix(toolkit): dpkt walks IP layers off the octets as the default engi…
fix(toolkit): scapy walks link types, layer bounds and IPv6 extension…
7 remaining items
JarryShaw commentedon Oct 13, 2026
Reopened: I closed this early. Its pyshark part was not fixed. #1670's body lists it as left, and the #1671 worker re-measured it on
main(7aeeaf804): TCP behind IPv6 Mobility (135) or HIP (139), including 6in6 behind them, is traced by the default engine but not by pyshark, in 6 of this issue's cases. dpkt, scapy and pypcapfile agree with the default engine.Blocked until the #1671 PR merges, because it rewrites
pcapkit/toolkit/pyshark.py.JarryShaw commentedon Oct 14, 2026
Unblocked: #1686, the PR for #1671, merged at 01:17Z (
e86799b54). No open PR touchespcapkit/toolkit/pyshark.pyorpcapkit/foundation/engines/pyshark.py. A worker has the remaining pyshark part, TCP behind IPv6 Mobility (135) or HIP (139).JarryShaw commentedon Oct 14, 2026
Stopped under the ruling on #1687: third-party engines are best effort, and agreement-only work is being reverted rather than extended. This issue is agreement work, so I recommend closing it as won't-fix. That will be confirmed together with the revert list on #1687.
JarryShaw commentedon Oct 14, 2026
go ahead and close it
JarryShaw commentedon Oct 14, 2026
Closed as won't-fix, as ruled: under #1687, third-party engines are best effort, and this was agreement-only work.