[FLINK-40248][ci] Add nightly snapshot publishing to GHA pipeline

[dalelane]

What is the purpose of the change

The nightly Azure pipeline publishes two things that the GitHub Actions nightly
workflow currently doesn't: release binaries uploaded to S3, and snapshot jars
deployed to repository.apache.org. That gap needs closing before Flink can move off
Azure Pipelines (FLINK-27075).

Which pipeline publishes a given branch will be configuration held in the GHA nightly
workflow itself.

Because that workflow is versioned per branch, each branch can be switched over
independently, and a branch that hasn't been switched over yet still builds its
snapshots on GHA (keeping them as build artifacts of the workflow run rather than
publishing them).

The aim of this is to make sure that the new GHA jobs get exercised on every branch
before they become the canonical source of anything we publish.

master is deliberately listed in both of the lists described below, which is the
first phase of the rollout proposed on FLINK-40248: GHA publishes master's binaries
under a gha-trial/ prefix while Azure continues to publish to the bucket root, so the
two can be compared over a series of nightly runs before anything depends on GHA's
output.

Removing master from the shadow list and removing Azure's cron_snapshot_deployment
job will be a follow-up, once the outputs have been confirmed to match.

Brief change log

• Adds snapshot_binary and snapshot_maven jobs to .github/workflows/nightly.yml,
  mirroring the cron_snapshot_deployment_binary and cron_snapshot_deployment_maven
  jobs in tools/azure-pipelines/build-nightly-dist.yml. Both run on JDK 11, the
  minimum supported version, matching the Azure jobs.
• PUBLISH_SNAPSHOT_BRANCHES lists branches whose snapshots this pipeline publishes.
  Because nightly.yml is versioned per branch, this is per-branch configuration,
  and a branch that has the change backported without being added to the list defaults
  closed: it builds, but publishes nothing.
• SHADOW_SNAPSHOT_BRANCHES lists branches that GHA and Azure are both publishing
  during comparison. Binaries for those branches are uploaded to a gha-trial/ prefix
  in the S3 bucket rather than the bucket root, so the two pipelines can't overwrite
  each other. Snapshot jars uploaded to Maven need no equivalent renaming, as they
  already have timestamped names.
• The binary release is uploaded as a GHA build artifact on every branch, whether or
  not it publishes, so its contents can be compared against what Azure produces.
• When a branch isn't publishing, the Maven job deploys into a local repository via
  -DaltDeploymentRepository and uploads that as a tarball artifact, instead of
  deploying to repository.apache.org.
• tools/ci/deploy_nightly_to_s3.sh: upload_to_s3 takes an optional destination
  prefix, defaulting to the existing behaviour of writing to the bucket root.
• tools/releasing/create_binary_release.sh: adds SKIP_PYTHON_WHEELS, defaulting to
  false so the manual release process is unaffected. Nightly builds set it to
  true, so a binary release can be produced without the 12 platform wheels that a
  release manager downloads by hand beforehand. Also enables nullglob around the
  wheel loop, so it doesn't iterate over a literal *.whl when no wheels are present.
• tools/azure-pipelines/build-nightly-dist.yml: sets SKIP_PYTHON_WHEELS=true, and
  now checks the exit code of create_binary_release.sh. Previously the nightly Azure
  build depended on the missing-wheels check failing, with that failure masked by the
  echo/find commands following it in the same inline script — which meant any
  other error in the script was silently ignored too.

Verifying this change

This change is a CI configuration change without test coverage. I'll verify by
running affected pipelines on my fork.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components),
  Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Sonnet 5

[flinkbot]

CI report:

• 5053bd5 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[dalelane]

re: dalelane#1 (comment)

I'm wondering whether it's feasible to have all the generated jars in a tar.gz build artifact or whether we hit some size limits here.

@XComp setting a retention time is a nod towards your comment on the storage requirements here. We could make this even shorter, I largely picked 5 days out of the air.

FWIW gzip'ping doesn't seem to make much difference.

I tried building a sample of modules - which suggests that the total size will be in the 1gb - 1.5gb ballpark. That is bigger than the snapshot binaries (which seem to be in the 600mb ballpark)

If this is too big, we could just tar the primary jars, and drop the sources/javadoc versions? Is that worth doing?