fix(toolkit): pyshark traces TCP behind an IPv4 extension header as the default engine does (#1625)

[JarryShaw]

Please follow the guide below

• You will be asked some questions, please read them carefully and answer honestly

• Put an x into all the boxes [ ] relevant to your pull request (like that [x])

• Use Preview tab to see how your pull request will actually look like

• Searched for similar pull requests

• Followed the coding style (make pylint, make mypy, make isort)

• make test passes, and a test case covers the change

• Added a changelog entry under docs/source/changelog/ and regenerated CHANGELOG.md, if the change is user-visible — N/A — centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657

What is the purpose of your pull request?

Tick the commit type your subject line carries.

• fix — corrects a defect
• feat — adds a feature
• perf — changes performance, not behaviour
• refactor — changes neither behaviour nor performance
• test — tests only
• docs — documentation only
• ci — workflows or build tooling
• release — bumps the version or rolls up a distribution
• chore — anything else

---

Description of your pull request and other information

Closes #1625.

• Fix: behind an AH or IPv6 extension header in IPv4 (protocols 0, 43, 44, 51, 60, 135, 139), tcp_traceflow now reads the TCP from the frame's octets with pypcapfile's rules (_upper_layer, _past_extension_headers, _default_accepts). The approach matches open PR fix(toolkit): dpkt and scapy find TCP in IPv4 where the default engine does (#1606, #1617) #1624 for dpkt and scapy.
• Design change, a second tshark pass: the PDML pyshark reads carries no octets past a Mobility or HIP header. With tshark 4.6.9, nothing follows the 8-octet mipv6 layer, and HIP's fields stop at offset 74 of a 101-octet frame. -T pdml refuses -x. So on the first frame that needs octets, the engine starts its own tshark -T ek -x -J frame, filtered to those protocols. It reads it forward by frame number and kills and reaps it on close. A capture without such frames never starts it. It isn't a second pyshark FileCapture: pyshark's JSON parser raises KeyError on a Simple Packet Block, and each capture replaces the asyncio child watcher, which left the first pass's tshark a zombie.
• Failure handling: if the pass cannot start, exits with an error, writes unparsable output or lacks a frame, the engine warns once (PySharkWarning). That frame and later ones are traced as tshark dissects them, as on main. I chose this over raising so that no capture main extracts stops extracting.
• fix(toolkit): the dpkt, scapy and pyshark engines do not stop an extension-header chain at 32 #1609: this also fixes the IPv4 half of fix(toolkit): the dpkt, scapy and pyshark engines do not stop an extension-header chain at 32 #1609 for pyshark (33 Destination Options headers), so ipv4-33/pyshark/trace leaves its Gap row. IPv6 stays open, so fix(toolkit): the dpkt, scapy and pyshark engines do not stop an extension-header chain at 32 #1609 is not closed.
• Measured on 59 crafted frames, 3.11 / tshark 4.6.9, comparing trace inputs field by field: pyshark disagreed with default on 16 before, and on 1 after (ipv6-33, fix(toolkit): the dpkt, scapy and pyshark engines do not stop an extension-header chain at 32 #1609). TCP reassembly is still declined.

[github-actions]

Coverage: 89.73% (unit tier, Python 3.14, e9d316ad8, Unit Tests run success)

Package	Statements	Missed	Branches	Partial	Cover
pcapkit (top level)	104	3	20	3	95.16%
pcapkit/const	18858	954	2366	868	91.06%
pcapkit/corekit	2366	67	760	35	96.48%
pcapkit/dumpkit	275	3	100	3	98.40%
pcapkit/foundation	2961	153	1042	35	94.05%
pcapkit/interface	138	7	52	5	93.68%
pcapkit/protocols	17115	226	4570	191	98.06%
pcapkit/toolkit	992	134	364	6	85.40%
pcapkit/utilities	494	4	146	4	98.75%
pcapkit/vendor	4409	2342	1006	157	43.25%

Per-file detail: the coverage-html artifact of this run.

[JarryShaw]

Verdict: NEEDS CHANGES at 7d346ac4e.

Sonnet cross-review. The fix itself holds, and the review checked it against its own frames: 58 hand-built frames and 564 truncation cut points match the default engine field by field, and frame alignment holds on every pcapng shape tried. But the second tshark pass adds failure modes that main does not have:

1. Crash. A pcapng Simple Packet Block, which has no timestamp, that matches OCTETS_FILTER makes the JSON pass raise KeyError: 'frame.time_epoch' (pyshark tshark_json.py:111), which aborts extract(). I reproduced it: on main the same capture extracts and agrees with the default engine (Hop-by-Hop case); on the PR it raises.
2. Lifecycle. On early close with the second pass started, the first pass is left as an unreaped zombie (3 of 3 runs; main leaves none). A second pass that tshark rejects aborts the extraction, and one that dies mid-run silently drops the trace of every later matching frame.

Required: a second pass that fails or cannot be parsed must not crash the extraction or silently drop frames. Fall back to tshark's dissection with a warning, or raise from pcapkit.utilities.exceptions. Reap both processes on close. Add tests: an SPB frame (fails on this head), and a real-tshark lifecycle test, since test_close_closes_both_passes uses fakes. Nit: #1624 is not merged yet, so the body should say the approach matches it, not that it already applies to dpkt and scapy.

[JarryShaw]

Verdict: GOOD TO GO at e9d316ad8.

Sonnet cross-review, round 2. Both round-1 findings are fixed. Simple Packet Blocks trace as the default engine does, and early close leaves no tshark processes and no zombie.

• Re-run sets: the 58 frames, the 564 truncation points, every frame-alignment and pcapng shape, and 200 frames up to 65 KB show no new disagreement. The 4 remaining diffs, GRE and VXLAN carrying IPv4, also disagree on main.
• Fallback: killing the second pass mid-run gives exactly one PySharkWarning. Frames before that point match the default engine and frames after it match main, with 0 diffs over 2000 frames. A rejected filter, or a tshark without -J, also warns and finishes.
• Zombie cause: confirmed with raw pyshark. Each Capture replaces the asyncio child watcher, so two live captures leave a zombie and one capture plus a plain subprocess does not.

I reproduced the round-1 SPB probe agreeing on all 4 captures, the new tests passing (17) and failing against main's pyshark files (79), and no tshark left afterwards. Non-blocking: an engine dropped without close() leaves its subprocess to GC, as the first pass already does on main. tshark versions other than 4.6.9 are unverified, though an unexpected output shape takes the warned fallback. This conflicts with #1624 in tests/foundation/engines/test_ext_chain_limit_agreement_1604_unit.py, so whichever merges second needs a rebase.

[JarryShaw]

Resolve conflicts