Skip to content

[ci] Fix the empty Nightly release notes, and pin the build to the resolved core - #3236

Open
VietND96 wants to merge 3 commits into
SeleniumHQ:trunkfrom
NDViet:claude/nightly-release-log-empty-5hb6sx
Open

[ci] Fix the empty Nightly release notes, and pin the build to the resolved core#3236
VietND96 wants to merge 3 commits into
SeleniumHQ:trunkfrom
NDViet:claude/nightly-release-log-empty-5hb6sx

Conversation

@VietND96

Copy link
Copy Markdown
Member

Description

Two related fixes to the Nightly pipeline, both fallout from publishing by promotion (#3233).

1. The Nightly release notes were empty (https://github.com/SeleniumHQ/docker-selenium/releases/tag/nightly)

Since the nightly publishes by promotion, the deploy job builds nothing — make promote_nightly_images copies the tested manifests registry to registry. generate_release_notes.sh had not moved with it and still read everything from a build that no longer happens, so the notes came out blank in two independent ways:

  • The versions table was read out of <namespace>/<image>:<grid version>-<date>. A release publishes that tag; the nightly publishes :nightly and nothing else, so every docker run hit an image that exists nowhere and each cell was left empty. The tag to read is now IMAGE_TAG, which nightly.yml sets to nightly and a release leaves at the default.
  • The two "Published Docker images" lists came from docker images, i.e. the local store, which the promotion path never fills. They are now built by asking the registry for each published manifest — the same answer on either path, and it reports the digest users actually pull and the architectures the manifest covers instead of a local image id.

Nothing empty passes quietly any more: a version that comes back blank, a registry with none of the images in it, or a LATEST_TAG that would silently turn the changelog range into HEAD...trunk now fails the step by name. Images are promoted before this runs, so a failure leaves the previous notes up and turns the run red rather than replacing them with a blank page.

The reads also pull now, where before they were free, so each image is dropped as soon as its values are out — the browser images together are larger than the runner has free. Anything already in the store came from a local build and is left alone. The arm64 rows need emulators that the multi-architecture build used to register as a side effect, so the job sets up QEMU itself.

2. The build was not pinned to the core the reuse decision was made about

Keying the image hash on the jar's digest (#3232) decides correctly whether to build, but nothing then made the build use that jar. Base/Dockerfile downloads from

.../releases/download/${RELEASE}/selenium-server-${VERSION}.jar

and for the nightly that URL is a moving target: one asset name for a whole snapshot cycle, contents replaced every night. decide resolved the digest and did not pass it on — the build job inherited base-version and base-release only — so the core could change between the decision and the download.

The cost is not a slow cache. The tag names a core the image does not contain, and the next run keys on the new digest and rebuilds, leaving the mislabelled set published. The build is also a two-runner matrix downloading independently, minutes apart, so a re-upload between them puts two different cores into one manifest and merge_ci_images assembles it without complaint.

decide now outputs base-asset-digest, the build job passes it to make, and Base/Dockerfile verifies the downloaded jar against it and fails when it does not match. The nightly and release fallback build paths need no change — they run get-latest-upstream themselves, so the digest is already in the environment make reads.

Verification is skipped, out loud, for anything that is not a sha256: a local build, or an upstream asset old enough to predate the digest field, where identity falls back to the asset id and its upload time. Which of the two get-latest-upstream used is now in the log, because they fail differently — only a real sha256 can be checked against the jar, and a digest field that came back constant would silently return everything keyed on the core to treating every nightly as the same one.

Motivation and Context

The nightly release page has been publishing with no changelog, an empty version table and empty image lists, so there is no way to tell what a given nightly actually contains. Separately, a nightly could ship images built against a different Selenium core than the one the reuse decision was keyed on, and a multi-architecture manifest could carry two different cores.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • I have read the contributing document.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Verification

No Docker daemon was available while developing this, so the images were not built. What was checked:

  • generate_release_notes.sh was run end to end against a stubbed docker CLI. Both the nightly (IMAGE_TAG=nightly) and release (default tag) paths produce a full version table and 26+26 image rows; the release path resolves exactly the previous refs (base:4.48.1-20260910, video:ffmpeg-8.1-20260910), so release behaviour is unchanged.
  • Failure paths confirmed to exit non-zero with a named cause: empty and unresolvable LATEST_TAG, an unpublished image, and an empty registry.
  • Confirmed the pull → read → drop interleaving keeps peak disk at one image, and that images already present locally are not dropped.
  • The Base/Dockerfile change was parsed the way Docker parses it (comment lines stripped inside continuations, backslashes joined) and every RUN checked with sh -n. The digest branches were exercised against a real file: matching digest passes, a moved jar exits 1, and the non-sha256 and empty cases skip with a message.
  • make -n confirms BASE_ASSET_DIGEST reaches the build as --build-arg SERVER_JAR_DIGEST, and that it is empty (so verification is skipped) for a plain local build.

One thing left for the first CI run to answer: whether the upstream release assets carry GitHub's .digest field or fall through to the asset id and upload time. Both work, but they fail differently, which is why the branch taken is now logged.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HNaarX452mih58sA7QPTz2


Generated by Claude Code

…store

Since the nightly started publishing by promotion (SeleniumHQ#3233), the deploy job
builds nothing: `make promote_nightly_images` copies the tested manifests
registry to registry. generate_release_notes.sh had not moved with it and
still read everything from a build that no longer happens, so every section
of the nightly release came out blank.

Two separate ways, both silent:

- The versions were read out of `<namespace>/<image>:<grid version>-<date>`.
  A release publishes that tag; the nightly publishes :nightly and nothing
  else, so every `docker run` hit an image that exists nowhere and each cell
  of the table was left empty. The tag to read is now IMAGE_TAG, which
  nightly.yml sets to nightly and a release leaves at the default.

- The two "Published Docker images" lists came from `docker images`, i.e.
  the local store, which the promotion path never fills. They are now built
  by asking the registry for each published manifest, which is the same
  answer on either path - and reports the digest users actually pull and the
  architectures the manifest covers, instead of a local image id.

Nothing empty passes quietly any more. A version that comes back blank, a
registry with none of the images in it, or a LATEST_TAG that would silently
turn the changelog range into HEAD...trunk now fails the step by name. The
images are promoted before this runs, so a failure leaves the previous notes
up and turns the run red rather than replacing them with a blank page.

The reads also pull, where before they were free, so each image is dropped
again as soon as its values are out: the browser images together are larger
than the runner has free. Anything already in the store came from a local
build and is left alone. The arm64 rows need the emulators that the
multi-architecture build used to register as a side effect, so the job now
sets up QEMU itself.

The image list moves to `make print_ci_images` so the notes and the Makefile
cannot drift, and both workflows pass GHCR_NAMESPACE so the GHCR list follows
the repository owner rather than the script's SeleniumHQ default.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HNaarX452mih58sA7QPTz2
Keying the image hash on the jar's digest (SeleniumHQ#3232) decides correctly whether
to build, but nothing then makes the build use that jar. Base/Dockerfile
downloads from

  .../releases/download/${RELEASE}/selenium-server-${VERSION}.jar

and for the nightly that URL is a moving target: one asset name for a whole
snapshot cycle, contents replaced every night. decide resolved the digest and
did not pass it on - the build job inherited base-version and base-release
only - so between the decision and the download the core could change
underneath it.

What that costs is not a slow cache. The tag names a core the image does not
contain, and the next run keys on the new digest and rebuilds, leaving the
mislabelled set published. The build is also a two-runner matrix downloading
independently, minutes apart, so a re-upload between them puts two different
cores into one manifest and merge_ci_images assembles it without complaint.

A second revision in the tag would not have helped: the hash already carries
both identities, and the gap is between the decision and the build, not in
what the tag encodes. So pin it instead.

decide now outputs base-asset-digest, the build job passes it to make, and
Base/Dockerfile verifies the downloaded jar against it and fails when it does
not match. The nightly and release fallback build paths need no change - they
run get-latest-upstream themselves, so the digest is already in the
environment make reads.

Verification is skipped, out loud, for anything that is not a sha256: a local
build, or an upstream asset old enough to predate the digest field, where
identity falls back to the asset id and its upload time. Which of the two
get-latest-upstream used is now in the log, because they fail differently -
only a real sha256 can be checked against the jar, and a digest field that
came back constant would silently return everything keyed on the core to
treating every nightly as the same one.

The digest is also recorded as an image label, so a published image can be
traced back to its core without being pulled apart.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HNaarX452mih58sA7QPTz2
@qodo-code-review

Copy link
Copy Markdown
Contributor

ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

A pull request from a fork gets no secrets, so DOCKER_USERNAME and
DOCKER_PASSWORD arrive empty and

  docker login -u="" -p=""

exits 1. The step has no guard, so the job dies there - before the cluster is
built and before a single test runs.

The three sibling test workflows already handle this: helm-chart-test.yml and
k8s-dynamic-grid-test.yml end the same line with `|| true`, and docker-test.yml
does not log in to Docker Hub at all. k8s-scaling-test.yml is the only one that
was missed, which is exactly the pattern seen on SeleniumHQ#3236 - Docker, Dynamic Grid
and chart jobs green, scaling jobs red - and why it reads as an image problem:
what fails is the step that would have authenticated the pulls.

Nothing here needs the login. In build-in-job mode, which is what a fork run
uses, every grid image is built in the job itself; the rest are public. The
login only buys a higher anonymous pull limit, so losing it is a soft
degradation, never a failure - which is the judgement the other two workflows
already encode.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HNaarX452mih58sA7QPTz2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants