Skip to content

ci(python): run round-trip sweeps on latest only, move experimental legs to their own workflow (#1732) - #1742

Merged
JarryShaw merged 1 commit into
mainfrom
ci/1732-sweeps-on-latest-only
Oct 14, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
ci/1732-sweeps-on-latest-only

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

Please follow the guide below

What is the purpose of your pull request?

Tick the commit type your subject line carries.

  • fix — corrects a defect
  • feat — adds a feature
  • perf — changes performance, not behaviour
  • refactor — changes neither behaviour nor performance
  • test — tests only
  • docs — documentation only
  • ci — workflows or build tooling
  • release — bumps the version or rolls up a distribution
  • chore — anything else

Closes #1732. This applies the owner's three rulings on that issue.

  • Sweeps on latest only: every other test leg skips the 41 *roundtrip* modules (303 tests). Every other integration leg drops its three *roundtrip*_runtime.py modules (251 tests become 239).
  • Option 2: 3.14t and GraalPy move to .github/workflows/experimental.yml ("Experimental Tests"). It reports separately and holds up nothing: no needs:, no workflow_call, and no workflow_run names it, so Unit Tests and its four consumers complete without it. Check names are unchanged. Caps are 140 for the job and 5/10/120 for the steps, with job-level continue-on-error. Ruleset 23497679 requires none of these checks.
  • Timing report, option A: util/ci_timing.py reports H1, H2 and H4 on non-blocking legs as warnings. Those are experimental.yml's legs, (KTLO) legs, and (experimental) legs from before the split. Blocking legs stay HARD.
  • Docs: workflows.rst now counts thirteen workflows, with a row and a graph node for the new one, and line citations are renumbered.
  • Tests: test_sweeps_on_latest_only_1732.py, test_workflows_at_a_glance_1732.py (pins the count, table and graph to the files) and test_ci_timing.py. Each new assertion fails against main.
  • Not fixed here: GraalPy's lost runners (ci(python): the GraalPy experimental leg loses its runner 18–25 minutes into its tests on every recent run #1740) and the KTLO half of ci(unit-tests): a KTLO or experimental leg that fails at job level, such as a lost runner, still fails Required checks passed #1738.

…egs to their own workflow (#1732)

- Owner's ruling: the round-trip sweeps run on `latest` alone. Every other
  `test` leg adds `--ignore-glob='*roundtrip*'` (ROUNDTRIP_SWEEP_GLOB); every
  other `integration` leg drops the three `*roundtrip*_runtime.py` modules
  from its selection and adds that glob (ROUNDTRIP_RUNTIME_GLOB), since
  pytest ignores no path it is handed. `latest` keeps both whole.
- Owner's option 2: free-threaded 3.14t and GraalPy move to a new workflow,
  experimental.yml ("Experimental Tests"), so neither `required-checks` nor
  any `workflow_run` consumer of "Unit Tests" waits on them. Check names
  unchanged; caps 140 (job) and 5/10/120 (steps); job-level
  `continue-on-error`; unit-tests.yml's `changes` script, byte for byte.
- `test` keeps main's caps: 45 for the job, no step caps.
- util/python_versions.py writes an `experimental` matrix output;
  util/update_python_versions.py counts its legs.
- util/ci_timing.py reads experimental.yml as one of its other workflows,
  and reports H1, H2 and H4 on a non-blocking job (any of experimental.yml's,
  a `(KTLO)` leg, or an `(experimental)` leg from before the split) as
  warnings, so they do not fail the run.
- workflows.rst, releasing.rst and the tier file describe it; line
  citations renumbered.

Tests: test_sweeps_on_latest_only_1732.py pins the structure;
test_workflows_at_a_glance_1732.py pins workflows.rst's workflow count, table
and graph to .github/workflows/*.yml; test_ci_timing.py the warning levels.

Closes #1732
@JarryShaw JarryShaw added the ci Pull requests that change CI or workflow configuration (ci: subject prefix) label Oct 14, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict: GOOD TO GO at 69d085079. A Sonnet cross-review checked each of five rounds (author: Opus). Rounds 1–4 each raised findings, all fixed in later rounds.

  • Nothing waits on Experimental Tests:
    • every .github/workflows/*.yml was parsed, and no needs:, workflow_run or workflow_call names it;
    • ruleset 23497679 requires only Required checks passed and Compat 3.11–3.15.
  • Sweep subset: the unit tier collects 4501 tests on latest and 4198 on the others. Integration collects 251 and 239, and the 12 dropped tests all belong to the three runtime modules.
  • Timing report, simulated through _report(): legacy and new experimental legs, and KTLO legs, past their share exit 0 with a ::warning. A blocking Python 3.14 leg still exits 1.
  • New assertions fail on main and pass here.
  • My own re-run in the PR's tree: the 1732, ci_timing, at-a-glance, 1660 and 1672 test files all pass.

@JarryShaw JarryShaw added the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage: 89.51% (unit tier, Python 3.15, 69d085079, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 3 20 3 95.16%
pcapkit/const 18860 954 2368 868 91.06%
pcapkit/corekit 2455 69 780 35 96.54%
pcapkit/dumpkit 275 3 100 3 98.40%
pcapkit/foundation 2906 164 1016 34 93.57%
pcapkit/interface 138 7 52 5 93.68%
pcapkit/protocols 17172 225 4588 190 98.07%
pcapkit/toolkit 865 198 298 6 74.38%
pcapkit/utilities 436 2 140 4 98.96%
pcapkit/vendor 4417 2342 1010 157 43.38%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw
JarryShaw merged commit 65122bf into main Oct 14, 2026
55 of 56 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 14, 2026
@JarryShaw
JarryShaw deleted the ci/1732-sweeps-on-latest-only branch October 14, 2026 22:27
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Pull requests that change CI or workflow configuration (ci: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

ci(python): the GraalPy experimental leg reaches 26% of the suite before its 25-minute cap

1 participant