Repository navigation
Add the non-regression suite on pinned repositories to CI - #66
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Second deliverable of the product track: a non-regression suite on real repositories, run in continuous integration.
test: define the non-regression suite on real repositories), the suite follows.tests/regression/repositories.tomlpins 12 public repositories by commit (the corpus of the audit campaign: Flask, Django, FastAPI, CLI tools, IoT and desktop code), plus the intentionally vulnerable Flask fixture kept in the tree undertests/regression/fixtures/vulnerable-flask/.tests/test_regression.pychecks out each repository at its commit (shallow fetch on first use, cached under.regression/orCORETRACE_REGRESSION_ROOT), analyses it with the bundled plugins and compares findings (file, line, rule, function) and coverage with the snapshot intests/regression/expected/<name>.json.CORETRACE_REGRESSION_UPDATE=1records an intended change.regressionmarker, deselected by default so the unit suite stays fast; a quick unmarked test keeps the manifest and the snapshots consistent.regressionjob (pytest -m regression, checkouts cached by manifest hash). The whole suite runs in about 25 seconds with cold clones, 6 seconds warm.