fix(fixtures): bind synthetic provenance to the fixture byte determinants - #107
Merged
Merged
Conversation
…ants The synthetic fixture provenance recorded a sha256 of the whole uv.lock, and tests/test_synthetic_fixtures.py asserted it. That hash does not determine the fixture bytes. The generator reads the source spec, its own source, the repository's writer code, and SQLite plus SQLAlchemy. Every one of those is recorded already: the spec and generator by sha256, the two library versions by value, with require_exact_builder() refusing any other writer. Because the lock hash was written into the provenance, and the provenance is embedded in the recording config, the hash reached the sealed database bytes. That made it self-referential. Any lock rewrite forced a rewrite of two sealed binary fixtures, whether or not the rewrite touched anything the generator reads. Adding a platform-gated optional dependency did exactly that: it added two Linux-only wheels and migrated the lock file format, changed no resolved version of anything, and still broke the fixture suite. Drop lock_path and lock_sha256, and bump the provenance schema to v2 so a reader sees the narrower shape. The byte oracle is unchanged and remains the authority: check_generated() regenerates all five artifacts and compares them byte for byte, so a dependency change that really does move the bytes still fails. Two tests now prove that oracle bites: - a one-pixel edit to a source spec makes the committed fixtures stale - one flipped byte in any of the five generated artifacts is caught Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two new tamper tests assert that check_generated() raises 'generated bytes are stale'. On a builder that does not reproduce the committed bytes exactly, that message could report the builder instead of the tamper, and the assertion would hold for the wrong reason. Python 3.12 with the required SQLite and SQLAlchemy is the environment that test_public_fixtures_match_the_source_generator_byte_for_byte proves reproduces every committed byte, so it supplies the control. Restrict both tamper tests to it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
synthetic-provenance.jsonrecordedlock_pathandlock_sha256, a sha256 of the entireuv.lock, andtests/test_synthetic_fixtures.pyasserted it. This drops both fields and bumps the provenance schema tov2.Why
The lock hash does not determine the fixture bytes. The generator reads four things: the source spec, its own source, the repository's writer code, and the SQLite and SQLAlchemy versions. All four are already pinned precisely. The spec and the generator are recorded by sha256 and asserted against the files. The two library versions are recorded by value, and
require_exact_builder()raises if the running interpreter does not match them.Worse, the coupling was self-referential. The lock hash went into the provenance, the provenance is embedded in the recording config, and the config is inside the sealed database. So a lock rewrite changed the sealed binary fixtures by definition, whether or not it touched anything the generator reads.
That is not hypothetical. #96 adds a
linuxextra holding a single platform-gated PyGObject dependency.uv syncre-locks, and the result:pygobjectandpycairoversion = 1torevision = 3, about 2,195 lines changedI checked the causality directly rather than reasoning about it. With the environment resolved and installed from the new lock, and the recorded hash held at the old value, all five generated artifacts in both fixtures regenerate byte for byte identical. The lock's contents contributed nothing. Only the recorded hash did.
Every dependency PR from here on would have had to rewrite two sealed binary fixtures for no reason. Note also that
.github/workflows/test.ymlpinsastral-sh/setup-uv@v7atversion: "latest", so the lock format tracks whatever uv ships that week. The fixture seal should not.What still catches a real change
check_generated()is untouched and stays the authority. It regenerates all five artifacts into a temp directory and compares every byte. It runs on the 3.12 lane, where the SQLite and SQLAlchemy versions match, so it does the real comparison there rather than the guard-fires check.A dependency change that genuinely moves the bytes fails that comparison with
generated bytes are stale. A dependency change that does not move the bytes now passes, which is the correct outcome and the point of the change.Two tests were added so this is asserted rather than assumed:
test_a_changed_source_spec_makes_the_committed_fixtures_stalemoves one action coordinate by a single pixel in a copy of a source spec and requires the byte check to fail.test_every_committed_fixture_artifact_is_byte_checkedflips one byte in each of the five generated artifacts in turn and requires the byte check to fail each time.Both skip on a builder that cannot reproduce the committed bytes.
Verification
pytest tests/ --ignore=tests/test_browser_bridge.py: 611 passed, 13 skipped, locally on Python 3.12.13, SQLite 3.53.1, SQLAlchemy 2.0.52.ruff check openadapt_capture/: clean.v1provenance schema exists anywhere else in the workspace.Fixtures were regenerated with the committed generator, not hand-edited.