Path-filter CI: only run the test suites that changed - #6
Merged
Conversation
Adds a `changes` job (dorny/paths-filter) that outputs a boolean per test category (api/accessibility/visual/performance/load/e2e), plus a workflow_dispatch run_all override to bypass it for a manual full run. Not yet consumed by any suite job.
First real consumer of the changes job: api-tests now skips when nothing under tests/api/ or its dependencies changed, with a fail-open guard against changes-job failure and an explicit lint success check (a skipped predecessor is otherwise ambiguous between "filtered out" and "lint failed upstream").
Extends the api-tests pattern down the existing chain. Reintroduces an explicit if: on these two jobs (they had none since 2026-08-12) using the proven-safe always()+!cancelled() pairing, not the bare always() that was removed back then.
Neither ran on PRs before (performance was nightly-only; load ran nowhere). Inserted into the existing serial chain after visual-tests, with load-tests last-active since it's the suite most likely to stress the shared dev backend (concurrent-mutation/degradation tests per its own pytest.ini marker description). security and data were deliberately not given their own jobs: tests/api/test_security.py already carries pytest.mark.api (so api-tests' -m api already runs it), and tests/data/test_data.py has zero test functions (shared constants only).
Extends needs: to include lint/load-tests/changes and adds the real gated if: condition as a ready-to-uncomment comment for whenever e2e is resumed. The active if: false pause (since 2026-08-11) is untouched.
test-summary now waits on both new jobs and reports their real test counts (via scripts/suite_summary.py, unchanged — it already handles a missing report gracefully) plus their raw job status in the collapsed details table.
Explains the changes job, the fail-open contract, the skip/failure- ambiguity gotcha that motivates checking lint's result directly in every gated job, and why always()+!cancelled() reappearing on accessibility-tests/visual-tests isn't a regression of the 2026-08-12 bare-always() bug.
saqibmanan
force-pushed
the
feat/ci-path-filtering
branch
from
August 14, 2026 17:41
a92fcb9 to
ad285f0
Compare
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
changesjob (dorny/paths-filter) that detects which test category changed (api/accessibility/visual/performance/load/e2e); each suite job now skips when its category didn't change, so a docs-only PR runs lint only instead of the full ~25min chain.performance-testsandload-testsas new PR-gated jobs — neither ran on PRs before (performance was nightly-only, load ran nowhere).security/datawere deliberately not given their own jobs (security is fully covered byapi-tests' existing marker;tests/data/test_data.pyhas zero actual tests) — seedocs/ci_workflow_notes.mdfor the reasoning.workflow_dispatchgained arun_allcheckbox to bypass filtering for a manual full run.docs/ci_workflow_notes.mdfor the 2026-08-12 incident this protects against).Test plan
actionlintclean (verified locally — only two pre-existing findings, both unrelated to this change)README.md→ confirm onlylint/changesruntests/api/**→ confirmapi-testsruns, others showskippedruffalongside an unrelatedtests/api/**change → confirm every downstream job showsskipped, not runworkflow_dispatchwithrun_all: true→ confirm every suite runs regardless of paths