Keep the manifests an index is made of

[tamnd]

Every image in the package is unpullable. :debug, :release, :jit and the digest the lockfile pins all say manifest unknown, which is why gdb and tier1 fail on #125 and would fail on everything after it. It passes locally only because the images are already in the local Docker cache.

The index is still in the registry. What is gone is what it is made of:

amd64    sha256:49725194d7c1  ->  404  manifest unknown
unknown  sha256:9a203a6ff562  ->  200  the attestation
arm64    sha256:b0ce44129e22  ->  404  manifest unknown
unknown  sha256:13a9550cc691  ->  200  the attestation

We deleted them ourselves, in the tidy job of cpython-images. From this morning's run:

digests a release or the lockfile points at: 15
145 versions, deleting 18: 120 newest kept, 15 tagged, 15 pointed at by a release

Those eighteen are the halves the fifteen protected indexes are built out of.

Why the rule missed them

retention.doomed keeps the newest keep, anything tagged, and anything a release named. A per architecture manifest is none of the three. The tag goes on the index, not on the half. The lockfile records index digests, not halves. And because the build is fully cached against a fixed commit, a half keeps the same digest week after week, so its age never moves and it eventually drops past the floor of 120 and is deleted, breaking every index that lists it including the one published four minutes earlier in the same run.

What this does

retention.anchors says where the keeping starts, which is every tagged version plus everything a release named. retention.reachable walks from there through whatever each digest lists, breadth first, until nothing new turns up. Both are pure and take the membership lookup as an argument, so the interesting half is tested against a dictionary rather than a registry.

cpybuild expand is the command that joins them to the real thing. It reads the version list and the protected file the tidy job already produces, asks docker buildx imagetools inspect --raw what each digest lists, and prints the closure. --raw rather than the readable output, because the readable output resolves an index into a table of platforms and silently drops the attestation rows, and the attestations are versions too.

Two ways this could quietly turn back into the bug, and what stops each. A digest that will not resolve is reported on stderr and treated as holding nothing, because some of them genuinely do not resolve any more and refusing outright would mean the tidy up never runs again. And on top of that, expand exits non zero without printing anything if the whole walk found no parts at all, since with fifteen indexes in the package zero means Docker is not there or the login did not take, and the thing not to do with an answer that is probably wrong is hand it to something that deletes images.

Run against the real registry it turns the fifteen protected digests into thirty, and both halves that went missing this morning are in the thirty.

Getting the images back

Nothing about the content changed and the layer blobs are still in the registry, so a rebuild re pushes the same per architecture manifests under the same digests. That repairs the currently pinned de0d69b1 index in place, with no lockfile change and no new digest to write anywhere. This pull request touches tools/cpybuild/**, which is already a path trigger for cpython-images, so merging it rebuilds, republishes and then runs the fixed tidy up in one go.

Closes #126