feat: stamp the tool version into measurement files and reports - #20
Open
repentsinner wants to merge 1 commit into
Open
feat: stamp the tool version into measurement files and reports#20repentsinner wants to merge 1 commit into
repentsinner wants to merge 1 commit into
Conversation
Reports are compared across vendors, and the tolerances, metrics, and primary-matrix estimator all change between versions. A report that cannot say which build produced it undercuts the reproducibility the project claims. Adds display_report.utilities.tool_identifier() and writes it to CSMF_Metadata.software on every path that builds measurement metadata, and onto the report header. Removes a false provenance record. The anonymizer hardcoded software="colour-workbench file stripper" — naming the project this code was extracted from, not this one — and analyze --strip-details wrote software=None. Both now record the real tool. The tool version identifies the code, not the measurement subject, so keeping it through anonymization leaks nothing. The version resolves through importlib.metadata, so it reads the static pyproject version today and tag-derived hatch-vcs versions once build/flywheel-and-main-rename lands. No coupling either way.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reports from this tool get shared and compared across vendors. The tolerance bands, the dE ITP math, and the primary-matrix estimator all change between versions — so a report that cannot say which build produced it undercuts the "open and independently verifiable" claim directly.
Removes a false provenance record
The anonymizer was writing the name of a different project into every file it touched:
colour-workbenchis the project this code was extracted from years ago — the same lineage as theole/ETCnaming already cleaned up. Any file passed through the anonymizer carries that string today. Meanwhileanalyze --strip-detailswrotesoftware=None, discarding provenance entirely.Both now record the real tool. The version identifies the code, not the measurement subject, so keeping it through anonymization leaks nothing.
What's added
display_report.utilities.tool_identifier()→"display-report 0.1.0", written toCSMF_Metadata.softwareon every path that builds measurement metadata:--device-namesuppliedDeviceInfo.to_metadata()analyze --strip-detailsNoneanonymize"colour-workbench file stripper"Plus a small right-aligned label on the report header, so the PDF itself carries the version rather than only the source file.
Version source
Resolved through
importlib.metadata, so it reads the staticpyprojectversion today (0.1.0) and switches to tag-derived hatch-vcs versions oncebuild/flywheel-and-main-renamelands. No ordering dependency between the two PRs — this works either way. APackageNotFoundErrorfallback covers running from an uninstalled source tree, and a test asserts the fallback is not what CI resolves.Verification
Two things worth knowing
A bug here was caught by ruff, not by the tests. My first patch put
tool_identifierinto the wrong import block, leavingF821 Undefined nameinside the--strip-detailsbranch. Tests stayed green because that branch has no coverage.Which is the real gap: this repo has zero
.csmffixtures.analyzeandanonymizecannot be exercised end to end at all, so the two paths this PR touches in those scripts are covered only by the linter. Worth adding a small anonymized fixture —anonymizeexists precisely to produce shareable ones.