Skip to content
This repository was archived by the owner on Oct 31, 2026. It is now read-only.

CI: a push to main runs what it changes; everything runs weekly - #249

Closed
bdbarnett wants to merge 1 commit into
mainfrom
ci/scope
Closed

bdbarnett wants to merge 1 commit into
mainfrom
ci/scope

Conversation

@bdbarnett

Copy link
Copy Markdown
Contributor

A push to main used to run everything: four Pythons on Linux plus Windows and macOS, the whole suite, and all 45 classes and 13 nodes of the lifecycle matrices. That is about 350 runner-minutes and over an hour of wall time for every merge, whatever the merge touched. Pull requests were already scoped by tools/scoped_tests.py --ci.

What changes

  • A push to main is checked for what it changes, the way a pull request is: tools/scoped_tests.py --ci diffs from the commit main moved from (github.event.before), so a merge is checked for the whole pull request it brings in. A shared change (the base classes, __init__, the kit, the probes, AUDIODSP_PIN, the workflow) still runs everything, including all 45 classes and every node. When that commit is missing from the history (a new branch, a force push), everything runs.
  • Everything still runs regularly: a new weekly run (Monday 06:23 UTC) and any manual run check the whole suite on every Python with the full matrices.
  • The mapping covers more files, where it used to fall back to the whole suite:
    • an instrument module runs the tests that name it and the ones that go through every instrument (audioinstruments.ALL), plus the instrument and roster tests;
    • a helper in tests/ or tests/support/ runs the tests that import it, directly or through other helpers (read with ast, imports inside functions included);
    • an effect module also runs the tests that name its module or class, and the ones that go through every effect (audioeffects.ALL, kit_options), beside its own tests and the rosters as before.
    • docs/reference/effect-controls.md runs test_metadata_contract, which compares it with the classes.
  • CONTRIBUTING.md and CLAUDE.md join the paths that skip the workflow; no test reads them.
  • The checkout fetches full history for every event, since a push now needs its base too. The Scope step prints what it chose and why.

Concurrency is unchanged: a new push to a pull request cancels the run it supersedes, and pushes to main are never cancelled, so each one keeps its own record.

How it was checked

Replay over the last 25 commits on main, each a merged pull request. "files" is test files run; "full" is the whole suite.

commit change files main push before PR before PR and main push after
392f32e Import kit_claims only where CPython runs the plants (#248) 1 full full 42 files
c2fdc92 A NaN constructor option is the option left out; CabinetSim 8 full full full
955ddb3 soundfont: a settable pitch-bend range and channel tuning (# 2 full full 23 files
6773c89 soundfont: play a region's filter, modulation envelope and L 20 full full full
3ca9f37 CI: install audiodsp at the pin by a bare git requirement (# 6 full full full
dde06b6 AGENTS.md: say how examples are written (#242) 1 skipped (docs) skipped (docs) skipped (docs)
5897a9c Release 0.4.2rc1 2 skipped (docs) skipped (docs) skipped (docs)
1650232 Point at PyDevices/gallery (#240) 1 skipped (docs) skipped (docs) skipped (docs)
54fb104 CONTRIBUTING: name the instrument gate's inputs by flag, not 1 full 11 files skipped (docs)
cd4c35b Shorten AGENTS.md to tips; the sound-change and seam-gate ru 3 full 11 files skipped (docs)
6c31188 Add simpletests for audioinstruments and audioeffects (#237) 5 full full full
47b7401 AirSpace: count the repeats twice over against Loop Bass's h 2 full 15 files 23 files
824c853 LadderFilter: bound the tail below Resonance 4 (#234) 4 full full 23 files
fe544c2 DigitalDelay, PingPongDelay: bound the tail with Repeat Cut 4 full 16 files 31 files
c5148b3 Player text and CLAIMS for seven effects; KIT_PLANTS across 36 full full 64 files
58964a8 HighPass, LowPass: CLAIMS tables for what the player text sa 5 full full 31 files
f6e6f59 Compressor: state the input level it stays clean at (#230) 2 full 15 files 29 files
1e381a2 Input ceilings: one stated level per effect, and the knobs t 38 full full 62 files
7976c00 Controls page: units for Compressor, Overdrive, Exciter and 5 full 18 files 35 files
45a16cb Overdrive: back off Mix 0, the output capacitor is charged o 3 full full 27 files
3f2f1f5 Player text: knob spans move to the Effect controls page (#2 58 full 44 files 67 files
3077ef0 Effect controls page, rendered from the classes (#225) 45 full full 71 files
3ce2c9a Distortion: a patch chosen before the first pull plays a fre 3 full full 30 files
653308f Player text and CLAIMS for eight more effects (#223) 17 full full 61 files
a83b345 Vibrato: Level spans -12 to 0 dB, so the whole knob is audib 2 full 15 files 26 files

In every row the new scope keeps each test file the change edited, and never runs less than the pull-request scope did before. The effect rows run more test files than before (the tests that name the class or go through every class), which a pull request now pays for too.

Against what Python really imports. Each of the 85 test modules was imported in a fresh interpreter and every repository file it loaded was listed. For each of those 541 (test, file) pairs outside lib/audioeffects/*.py, a change to the file runs that test, or the whole suite: 0 misses. Effect modules are left out of that check because importing audioeffects loads every effect; their mapping is by name, above.

Planted faults. Dropping the last macro label from tr808 and inverting before() in tests/support/fake_pump.py: the scope picks test_cpython_instruments, test_metadata_contract, test_sequencer and test_scheduling_seam, and running those three test modules fails them (8 errors in the instrument library test, 12 in the sequencer).

The Scope step itself, run locally for each event: a push with a valid before (scoped, or everything for a shared change), before all zeros, before not in the history, a scheduled run, and a pull request labelled lifecycle: all classes. A simulated push of the Vibrato merge runs 26 test files and the lifecycle matrix for Vibrato alone.

Runner time. Replayed over October's runs of this workflow since pull requests were scoped (27 pushes, 39 pull requests), with the measured time for runs that stay full and 8 minutes a job for a scoped run (25 for one that touches many classes): pushes go from about 8,400 to 3,700 runner-minutes. Pull requests change little, because they were already scoped. Over the whole of October, pushes would have taken about 20,200 runner-minutes instead of 32,600.

This pull request changes the workflow and the scope tool, so its own CI runs the whole suite.

A push to main now runs the tests its own change can move, from the
commit main moved from (github.event.before), the same way a pull request
does against its base. A shared change still runs everything, every class
and every node; so do a manual run and a new weekly run on Monday.

tools/scoped_tests.py --ci also maps an instrument module to the tests
that name it and the ones that go through every instrument, a helper in
tests/ or tests/support/ to the tests that import it, and an effect to the
tests that name it or go through every effect, where each used to run the
whole suite or only the class's own tests.
@bdbarnett

Copy link
Copy Markdown
Contributor Author

Closing: this repository is archived. Its packages now live in PyDevices/audioinstruments and PyDevices/audioeffects, each with its own scoped CI.

@bdbarnett bdbarnett closed this Oct 31, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant