feat(utilities): one config module for the tunable limits (PCAPKIT_* environment variables), and one home per duplicated protocol constant

Is your feature request related to a problem? Please describe.
The last four weeks added eight tunable limits and about 27 protocol constants, many duplicated across modules (asked in chat). Measured on main (6aeba80e8):

Tunables: FRAME_LAYER_LIMIT (64), EXTENSION_HEADER_LIMIT (32), IPV6_TCP_WARNED_LIMIT (32), dumpkit.common.FALLBACK_LIMIT, chardet.DETECT_CACHE_SIZE, corekit.io._READ_CHUNK, and the zero-pad guards in corekit/fields/field.py. Only PCAPKIT_DEVMODE, _VERBOSE and _SPHINX are environment variables today. PCAPKIT_VENDOR_RETRY (which crashes the import on a non-integer) and PCAPKIT_CI_MODE are undocumented.
Overrides don't propagate: ipv6.py:38-40, toolkit/dpkt.py, scapy.py and pypcapfile.py copy the two layer limits with from ... import. Changing the defining module therefore leaves those readers on the old value.
Duplicates: about 25 bare µs/ns resolution literals across four packages; the IPv4 (20), IPv6 (40) and TCP (20) header lengths in up to four places each, including a bare 20 in toolkit/pcap.py:178; PCAP and PCAP-NG magics; and two overlapping PCAP-NG minimum-block tables.

Describe the solution you'd like

New module: a stdlib-only pcapkit/utilities/config.py. It reads every PCAPKIT_* variable once at import, validates each one, and holds the tunables. utilities is the only cycle-free home: corekit.fields.strings imports utilities.chardet, so a corekit module would loop.
Readers: consumers read config.<NAME> at call time, with no from ... import copies, so one value reaches every engine.
Naming and docs: variables are named PCAPKIT_<NAME>, with one .. envvar:: list in the docs.
Protocol constants: they stay fixed. Each moves to its owning protocol module (IPv4/IPv6/TCP lengths, PCAP and PCAP-NG magics), or to a single resolution pair.
Sequencing: this comes after the refactor(toolkit): revert the cross-engine agreement tests and the adapter code that only mirrors the default engine #1708 dpkt and scapy reverts, which delete about ten of the duplicates.

Describe alternatives you've considered

A runtime config.configure(), like logging.configure, as a follow-up.
Per-Extractor arguments. context.py:20-29 avoided protocol-specific keyword arguments.

Additional context, the decisions I need from you:

No re-export: does the refactor(corekit)!: move the byte-order field helpers into corekit (#1711) #1715 no-re-export rule apply? Moving a constant would remove names such as pcapkit.protocols.protocol.FRAME_LAYER_LIMIT. That is a breaking change with a !. The alternative is to keep each definition where it is and centralise only the environment read.
Raising a limit: raising FRAME_LAYER_LIMIT past 64 reopens fix(protocols): deep layer nesting of any protocol exhausts the recursion limit and loses TCP silently #1610's RecursionError. Should a higher value be refused, capped, or allowed with a documented risk?
Bad values: should an invalid value warn and fall back to the default, or raise at import?
Vendor variables: should PCAPKIT_VENDOR_RETRY, _CI_MODE and _HTTP(S)_PROXY move into the module too?

no re-export
allow raising the limit and allow -1 as no limit. add document for the risks.
bad values warn and fallback to default.
moves as well.

Recorded:

No re-export. Each constant moves to its one home, and every old path goes. That makes it a breaking change, marked with a !.
Limits can be raised. -1 means no limit. The docs state the risk: past the default, a deep frame can hit Python's recursion limit, as in fix(protocols): deep layer nesting of any protocol exhausts the recursion limit and loses TCP silently #1610.
A bad value warns and falls back to the default. The warning has to go through the stdlib warnings module, because pcapkit's own warn imports logging and would cycle back into the config module.
The vendor variables move too. PCAPKIT_VENDOR_RETRY, _CI_MODE and _HTTP(S)_PROXY move, and each one gets documented.

This is blocked until the #1708 dpkt and scapy reverts merge: #1724, and the dpkt PR that follows it. Both rewrite the limit readers in toolkit/dpkt.py and scapy.py that this change has to switch to config.<NAME>.

maybe warn on -1 and values over 64 as well
is it possible we still use pcapkit's warning?

Recorded for 2: setting a limit to -1, or above its default (64 for the frame-layer limit), also warns, and the warning names the RecursionError risk.

For 3, yes, with one layout constraint. pcapkit.utilities.warnings imports only exceptions and logging (warnings.py:64-65), so config can import it as long as logging does not import config. So:

PCAPKIT_DEVMODE, _VERBOSE and _SPHINX stay in utilities/logging.py. warnings needs them while it is still loading.
- A bad value for one of them is recorded during import and warned through pcapkit's warn once warnings has finished loading.
- Today such a value falls back to False silently, and test_logging.py:60-69 pins that. That test would change.
Every other variable lives in utilities/config.py, which imports pcapkit.utilities.warnings directly: the limits, the caches and the vendor variables. Each bad or risky value warns through a new ConfigWarning(BaseWarning, RuntimeWarning), so devmode and the existing filters apply as usual.

Say if the three logging flags should move into config as well. That works too, but only if every warning is deferred until pcapkit/__init__.py finishes importing.

whats your recommendation?

Recommendation: keep PCAPKIT_DEVMODE, _VERBOSE and _SPHINX in utilities/logging.py, and put every other variable in utilities/config.py.

They are needed early: utilities.warnings reads these three while it is still loading, and logging loads before everything else in pcapkit. Moving them into config would mean logging imports config, and config imports warnings, which closes the cycle. The only way round it is to hold every config warning until pcapkit/__init__.py finishes, which is more machinery for three flags.
Bad values still warn: a bad value for one of the three is recorded while logging loads, and warnings reports it through pcapkit's warn once it has loaded. Today such a value falls back silently.
One documented list: the .. envvar:: list in the docs names all the variables, with each one's home module. Users see a single list even though the definitions live in two modules.

#1726 stays blocked until the #1708 dpkt revert (#1727) merges. The reverts rewrite the same limit readers this change switches to config.<NAME>.

take all defaults.

Recorded: all three defaults stand. Scapy's _READ_CHUNK stays unconfigurable. The proxy variables are still read on every call. Every limit raised above its default, or set to -1, warns about its own risk. Round 2, already running, implements exactly these.

docs(toolkit): drop agreement claims from adapter docstrings (#1708) (#…

feat(utilities)!: one config module for the tunable limits (#1726) (#…

actually, on second thoughts, vendor's envvars keep in vendor, they dont move to utilities.config

Recorded: the vendor variables stay in pcapkit.vendor. #1743 has already merged with them in config, so a follow-up PR moves them back. Defaults below, unless you say otherwise:

pcapkit.vendor.default.MAX_RETRY (PCAPKIT_VENDOR_RETRY) and CI_MODE (PCAPKIT_CI_MODE) return under their original names. This undoes feat(utilities)!: one config module for the tunable limits (#1726) #1743's removal of those two names, along with its versionremoved notes.
get_proxies() reads PCAPKIT_HTTP_PROXY and PCAPKIT_HTTPS_PROXY itself, on every call, as before.
feat(utilities)!: one config module for the tunable limits (#1726) #1743's handling stays: a bad value warns with ConfigWarning and falls back. The parsing helpers become public in config so vendor can reuse them without reaching into private names.
config.VENDOR_RETRY, config.CI_MODE, config.http_proxy and config.https_proxy are removed outright. They never shipped in a release, so they get no versionremoved.
The vendor variables are documented on the vendor page, and config.rst links there.

Ruling 7 (#1739) is unaffected and is in progress.

refactor(vendor): keep the vendor variables in pcapkit.vendor (#1726)

refactor(vendor): keep the vendor variables in pcapkit.vendor (#1726) (…

All of this has now landed:

Config module, rulings 1–3 and the 18:28Z defaults: feat(utilities)!: one config module for the tunable limits (#1726) #1743.
Vendor variables kept in pcapkit.vendor (the 22:28Z reversal): refactor(vendor): keep the vendor variables in pcapkit.vendor (#1726) #1745.
Ruling 7, one home per RFC-fixed constant: refactor(protocols): one home per duplicated RFC-fixed constant (#1726 ruling 7) #1739, through refactor(protocols): one home per duplicated RFC-fixed constant (#1739) #1752, on main as 3946bf527.

Closing.

docs(foundation): second pass over engines, toolkit and reassembly (#…

Navigation Menu

Repository navigation

feat(utilities): one config module for the tunable limits (PCAPKIT_* environment variables), and one home per duplicated protocol constant #1726

13 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

feat(utilities): one config module for the tunable limits (PCAPKIT_* environment variables), and one home per duplicated protocol constant #1726

Description

Activity

JarryShaw commented on Oct 14, 2026

JarryShaw commented on Oct 14, 2026

JarryShaw commented on Oct 14, 2026

JarryShaw commented on Oct 14, 2026

JarryShaw commented on Oct 14, 2026

JarryShaw commented on Oct 14, 2026

13 remaining items

JarryShaw commented on Oct 14, 2026

JarryShaw commented on Oct 14, 2026

JarryShaw commented on Oct 14, 2026

JarryShaw commented on Oct 14, 2026

JarryShaw commented on Oct 15, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions