From 86d971a2839e6e8d97a0e01701011be699b0e3c6 Mon Sep 17 00:00:00 2001 From: Michael Ilyin Date: Tue, 11 Aug 2026 17:59:58 +0200 Subject: [PATCH 1/7] CI: publish the zenoh-flat-jni the snapshot was built against MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The nightly snapshot publication has never got past compiling: it resolves `zenoh-flat-jni:1.9.0`, and nothing of zenoh-flat-jni has ever been published under any version. Naming its own `1.9.0-SNAPSHOT` instead would fix the symptom and break two things — this repository's CI would then wait on that repository's CI, and the coordinate always holds the tip of *its* main while we compile against the commit `Cargo.lock` pins. JNI being a binary contract, that mismatch surfaces as `UnsatisfiedLinkError` at runtime, not as a build failure. So the publication publishes what it depends on. On `main` — every merge plus the weekday nightly, as before — it builds zenoh-flat-jni from the pinned commit and uploads it as `1.9.0-java-SNAPSHOT`, then builds and uploads the SDK against that. Self-sufficient, because it uses that repository's *source at a commit we choose*, never an artifact its CI produced; coherent, because the dependency our POM names is what we compiled against. - `flat_jni_pin` reads the pin from `Cargo.lock` and the commit stamp from the three published coordinates. Rebuilding means ten cross-compiled targets and half an hour, and the pin moves about once a day, so it happens only when the published copy is not already that commit. Anything missing or unreadable reads as "not ours" and rebuilds. - `publish_flat_jni_copy` calls zenoh-flat-jni's own publication workflow rather than duplicating its build matrix. `uses:` cannot hold an expression, so the workflow file comes from its `main` and the pin goes in `branch:` — the coupling is to a file in that repository, never to a run of its CI. - `-java` keeps our copy from overwriting the one zenoh-flat-jni publishes or zenoh-kotlin's; the three can legitimately pin different commits at once. The name is fixed, so it is overwritten rather than accumulated. - `--refresh-dependencies` on both SDK invocations: Gradle caches changing modules for 24 hours and setup-gradle restores that cache, so without it the SDK could compile against yesterday's copy while publishing a POM naming the coordinate that now holds today's. - `ci/consumer-smoke-test` resolves the published snapshot from a clean build with no connection to this one and takes a key expression through JNI. That is the only check that the POM, the transitive dependency and the native library work for someone who is not us. - `bump-and-tag.bash` now checks the value `gradle.properties` ends up with: main inherits a snapshot, so *omitting* `zenoh-flat-jni-version` is how a release would reach one, which the input-only check did not cover. - CI runs on `main` are serialized rather than cancelled. Two uploads cannot be made atomic, and cancelling mid-publication is what splits them; PUBLISHING.md says so rather than claiming coherence by construction. The test job is untouched: it still builds zenoh-flat-jni from source through the composite build, which keeps that path from rotting. Verified end to end locally through Maven Local: the qualified copy publishes with its commit stamp, the SDK compiles and publishes against it, and an outside consumer resolves both and prints `zenoh-java smoke test OK`. Needs eclipse-zenoh/zenoh-flat-jni's `source-repository`/`version-qualifier` inputs on `main` first. --- .github/workflows/ci.yml | 71 +++++++++- .github/workflows/publish.yml | 49 ++++++- CI.md | 45 ++++-- PUBLISHING.md | 76 +++++++++- README.md | 10 +- build.gradle.kts | 13 +- ci/consumer-smoke-test/build.gradle.kts | 49 +++++++ ci/consumer-smoke-test/settings.gradle.kts | 1 + .../src/main/java/smoke/SmokeTest.java | 48 +++++++ ci/scripts/bump-and-tag.bash | 31 ++-- ci/scripts/flat-jni-copy.bash | 134 ++++++++++++++++++ gradle.properties | 13 +- 12 files changed, 493 insertions(+), 47 deletions(-) create mode 100644 ci/consumer-smoke-test/build.gradle.kts create mode 100644 ci/consumer-smoke-test/settings.gradle.kts create mode 100644 ci/consumer-smoke-test/src/main/java/smoke/SmokeTest.java create mode 100755 ci/scripts/flat-jni-copy.bash diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 024d8cd3..84d6d294 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -26,6 +26,15 @@ on: env: CARGO_TERM_COLOR: always +# A run on main publishes two coordinates from two separate uploads — our copy +# of zenoh-flat-jni, then this SDK naming it. Nothing makes that pair atomic, so +# runs are serialized rather than cancelled: cancelling a run mid-publication is +# exactly what leaves the two naming different commits. Per ref, so branches do +# not queue behind each other. +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: false + # Third-party actions are pinned to a commit, with the version in a trailing # comment: a tag is mutable, and a moved tag runs code nobody reviewed. Actions # under eclipse-zenoh/ are ours and stay on a branch. @@ -105,13 +114,65 @@ jobs: - name: Check whether all jobs pass run: echo '${{ toJson(needs) }}' | jq -e 'all(.result == "success")' - # Publish snapshot packages. Only triggered when CI runs on main. - # One job, because both publications now come from a single Gradle - # invocation — see .github/workflows/publish.yml. - publish_snapshot_package: - name: Publish snapshot package + # Everything below publishes, and only from main: every merge there plus the + # weekday nightly above. Branches and pull requests publish nothing. + # + # The snapshot must be usable by someone who is not us, which means its + # zenoh-flat-jni dependency has to exist — and be the commit this SDK compiled + # against. It must also not wait on zenoh-flat-jni's CI. Both follow from one + # rule: this job publishes what it depends on. See CI.md. + + # Which commit that is, and whether the copy already published is it. + flat_jni_pin: + name: Resolve the zenoh-flat-jni pin if: contains(fromJSON('["refs/heads/main"]'), github.ref) needs: ci + runs-on: ubuntu-latest + outputs: + commit: ${{ steps.pin.outputs.commit }} + qualifier: ${{ steps.pin.outputs.qualifier }} + rebuild: ${{ steps.pin.outputs.rebuild }} + steps: + - uses: actions/checkout@v4 + - id: pin + run: bash ci/scripts/flat-jni-copy.bash + + # Our own copy, built from that commit by zenoh-flat-jni's own publication + # workflow — its cross-compilation matrix is six desktop targets and four + # Android ABIs, and duplicating it here is how the two would drift. + # + # The workflow file is taken from its main, reviewed like any dependency; + # `uses:` cannot hold an expression, so the pin cannot go there. What is built + # is `branch:`, and that is the whole of the coupling: this needs a *file* in + # that repository, never a run of its CI. + # + # Half an hour when it runs, so it runs only when the pin has moved. + publish_flat_jni_copy: + name: Publish our zenoh-flat-jni copy + needs: flat_jni_pin + if: needs.flat_jni_pin.outputs.rebuild == 'true' + uses: eclipse-zenoh/zenoh-flat-jni/.github/workflows/publish.yml@main + permissions: + contents: read + packages: write + with: + snapshot: true + # A called workflow runs with the caller's context, so it has to be told + # whose sources to check out. + source-repository: eclipse-zenoh/zenoh-flat-jni + branch: ${{ needs.flat_jni_pin.outputs.commit }} + version-qualifier: ${{ needs.flat_jni_pin.outputs.qualifier }} + secrets: inherit + + # Then the SDK snapshot, naming the copy above. Reached both ways: the copy + # was rebuilt, or it was already current and skipped. + publish_snapshot_package: + name: Publish snapshot package + needs: [flat_jni_pin, publish_flat_jni_copy] + if: >- + ${{ !cancelled() + && needs.flat_jni_pin.result == 'success' + && needs.publish_flat_jni_copy.result != 'failure' }} uses: ./.github/workflows/publish.yml permissions: contents: read diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 56ca32f8..8be6f689 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -57,8 +57,14 @@ jobs: # Assembles the artifact and generates its POM without uploading, so a # `maven_publish: false` rehearsal actually proves something. Only the # remote upload below is gated. + # + # `--refresh-dependencies` because zenohFlatJniVersion is a snapshot that + # the job before this one may have just replaced. Gradle caches changing + # modules for 24 hours and setup-gradle restores that cache, so without it + # this could compile against yesterday's copy while publishing a POM that + # names the coordinate now holding today's. - name: Assemble and verify the publication - run: ./gradlew publishJvmPublicationToMavenLocal publishAndroidReleasePublicationToMavenLocal --info -Pandroid=true + run: ./gradlew publishJvmPublicationToMavenLocal publishAndroidReleasePublicationToMavenLocal --info --refresh-dependencies -Pandroid=true - name: Set pub mode env var # Note: This step is intended to allow publishing snapshot packages. @@ -77,7 +83,8 @@ jobs: run: | ./gradlew publishJvmPublicationToSonatypeRepository \ publishAndroidReleasePublicationToSonatypeRepository \ - ${{ env.RELEASE }} --info -PremotePublication=true -Pandroid=true ${{ env.PUB_MODE }} + ${{ env.RELEASE }} --info --refresh-dependencies \ + -PremotePublication=true -Pandroid=true ${{ env.PUB_MODE }} env: CENTRAL_SONATYPE_TOKEN_USERNAME: ${{ secrets.CENTRAL_SONATYPE_TOKEN_USERNAME}} CENTRAL_SONATYPE_TOKEN_PASSWORD: ${{ secrets.CENTRAL_SONATYPE_TOKEN_PASSWORD }} @@ -92,3 +99,41 @@ jobs: with: name: problem-reports-${{ github.job }}.zip path: ${{ github.workspace }}/build/reports/problems/ + + # Everything above runs inside this repository's own build. What a user gets + # is the published coordinate and whatever it transitively pulls in — which is + # where a snapshot has failed before, by naming a zenoh-flat-jni that did not + # exist. So resolve it as an outsider would and run it. + # + # Snapshots only: a release goes to a staging repository and is not public at + # this point. + consumer_test: + name: Consume the published snapshot + needs: publish_package + if: ${{ inputs.snapshot == true && inputs.maven_publish == true }} + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + with: + ref: ${{ inputs.branch }} + + - uses: actions/setup-java@v4 + with: + distribution: temurin + java-version: 11 + + - name: Setup Gradle + uses: gradle/actions/setup-gradle@v4 + + # A separate Gradle build with no path or composite dependency on this + # one, resolving from the snapshot repository with `--refresh-dependencies` + # so a cached copy of the same coordinates cannot stand in for what was + # just uploaded. + - name: Resolve and run it + working-directory: ci/consumer-smoke-test + run: | + set -euo pipefail + version="$(tr -d '[:space:]' < "$GITHUB_WORKSPACE/version.txt")-SNAPSHOT" + echo "Consuming org.eclipse.zenoh:zenoh-java:$version" >> "$GITHUB_STEP_SUMMARY" + ../../gradlew run --no-daemon --refresh-dependencies -PcandidateVersion="$version" diff --git a/CI.md b/CI.md index b8f2f6b4..b47f6c03 100644 --- a/CI.md +++ b/CI.md @@ -11,7 +11,7 @@ source yourself, [README.md](README.md#where-the-native-library-comes-from). - [The pin](#the-pin) - [Lockfile synchronization](#lockfile-synchronization) - [Moving the pin by hand](#moving-the-pin-by-hand) -- [Publishing does not use any of this](#publishing-does-not-use-any-of-this) +- [What publishing uses](#what-publishing-uses) ## What CI runs @@ -151,18 +151,37 @@ freezes resolution at a commit, so the sync can no longer move the pin and the bot goes silent. (The other way to defeat it — committing the `path = "…"` form — is covered above.) -## Publishing does not use any of this +## What publishing uses -A release resolves `org.eclipse.zenoh:zenoh-flat-jni:$zenohFlatJniVersion` from -Maven Central, like any other consumer. The pin, the lockfile and the composite -build play no part in it — they exist so that *testing* against unreleased -bindings is reproducible. +A **release** does not use any of this. It resolves +`org.eclipse.zenoh:zenoh-flat-jni:$zenohFlatJniVersion` from Maven Central like +any other consumer; the pin, the lockfile and the composite build play no part. +There, `zenohFlatJniVersion` says which **release** the SDK is published against +and `Cargo.lock` says which **commit** it is tested against, and moving one does +not move the other. -The two are deliberately independent: `zenohFlatJniVersion` in -`gradle.properties` says which **release** this SDK is built and published -against, `Cargo.lock` says which **commit** it is tested against, and moving one -does not move the other. A release must in fact avoid the composite build -entirely — the artifact would be built from source on the builder's disk while -the POM still claimed the released version — so `build.gradle.kts` fails any -`publish*` task while an included build is present. See +A **snapshot** makes them the same commit, by construction. It has to satisfy +two things at once — it must publish even if zenoh-flat-jni's CI has never run, +and the dependency it names must exist and be the code it compiled against — and +the only construction that does both is to publish what it depends on: + +```text +Cargo.lock pin ──> build zenoh-flat-jni from that commit + publish it as 1.9.0-java-SNAPSHOT + | + v + build the SDK against that coordinate + publish zenoh-java:-SNAPSHOT +``` + +So on `main`, the pin drives the publication as well as the tests, and +`gradle.properties` names our own copy rather than a zenoh-flat-jni release. +The mechanics — the qualifier, the commit stamp that decides whether the copy +needs rebuilding, what the pairing does and does not guarantee — are in +[PUBLISHING.md](PUBLISHING.md#the-snapshot-publication). + +Either way a publication must avoid the composite build: the artifact would be +built from source on the builder's disk while the POM still claimed a resolved +version, so `build.gradle.kts` fails any `publish*` task while an included build +is present. See [PUBLISHING.md](PUBLISHING.md#building-against-zenoh-flat-jni-source). diff --git a/PUBLISHING.md b/PUBLISHING.md index acf9546c..19c9df7e 100644 --- a/PUBLISHING.md +++ b/PUBLISHING.md @@ -16,6 +16,8 @@ which covers them once for both repositories. - [What this repository publishes](#what-this-repository-publishes) - [Relationship to zenoh-flat-jni](#relationship-to-zenoh-flat-jni) +- [The snapshot publication](#the-snapshot-publication) + - [What it does not guarantee](#what-it-does-not-guarantee) - [Running a release](#running-a-release) - [Before the first run](#before-the-first-run) - [Rehearsal (dry run)](#rehearsal-dry-run) @@ -89,12 +91,82 @@ Central, because a release must not depend on a snapshot: - snapshots are mutable and are eventually removed, so even where it resolved it would not stay reproducible. -`ci/scripts/bump-and-tag.bash` refuses a `-SNAPSHOT` value outright rather than -letting that reach a published POM. +`ci/scripts/bump-and-tag.bash` refuses to let that reach a published POM. It +checks the value `gradle.properties` ends up with, not the workflow input: +between releases that file names a snapshot, so *omitting* the input is the way +a release would reach one. Rehearsals are not constrained this way — see [Rehearsing before zenoh-flat-jni is released](#rehearsing-before-zenoh-flat-jni-is-released). +## The snapshot publication + +Between releases, every merge to `main` and the weekday nightly upload a +mutable pre-release build to the [Central snapshot +repository](https://central.sonatype.com/repository/maven-snapshots/). Its +purpose is to keep the upload machinery exercised — signing keys, credentials, +what Central accepts — and to give people a way to try the current `main`. + +It publishes **five** coordinates, not two: + +```text +org.eclipse.zenoh:zenoh-java:-SNAPSHOT +org.eclipse.zenoh:zenoh-java-android:-SNAPSHOT +org.eclipse.zenoh:zenoh-flat-jni:1.9.0-java-SNAPSHOT (+ -jvm, -android) +``` + +The last three are **our own copy** of zenoh-flat-jni, built from the commit +`Cargo.lock` pins. Publishing what we depend on is what makes the snapshot both +self-sufficient and coherent: + +- **self-sufficient** — if zenoh-flat-jni's CI were switched off entirely, this + publication still works. It uses that repository's *source at a commit we + choose*, never an artifact its CI produced. +- **coherent** — the dependency our POM names is the code we compiled against. + Pointing instead at zenoh-flat-jni's own `1.9.0-SNAPSHOT` would name the tip + of *its* `main` while we compiled against our pin; JNI being a binary + contract, that mismatch surfaces as `UnsatisfiedLinkError` at runtime rather + than as a build failure. + +The `-java` qualifier keeps our copy from overwriting the one zenoh-flat-jni +publishes itself, or zenoh-kotlin's — the three can legitimately pin different +commits at the same moment. The names are fixed rather than derived from a +commit, so each is overwritten in place and storage does not grow with the +number of builds. (Central still stores snapshots as timestamped builds and +cleans them after 90 days, so it is the consumer-facing *name* that is constant, +not the bytes behind it.) + +Rebuilding that copy means cross-compiling ten targets, on the order of half an +hour, and the pin moves roughly once a day — so it is rebuilt only when it has +to be. Every POM zenoh-flat-jni publishes carries the commit it was built from: + +```console +$ curl -s .../1.9.0-java-SNAPSHOT/maven-metadata.xml # ~2.9 kB +$ curl -s .../zenoh-flat-jni-1.9.0-java--.pom # ~1.8 kB +e75529ce… +``` + +`ci/scripts/flat-jni-copy.bash` reads that stamp from all three coordinates and +compares it with the pin; anything missing or different means rebuild. Run it +locally to see the decision, or `--self-test` to check its parsers. + +### What it does not guarantee + +The two uploads are separate Gradle invocations and a snapshot repository has no +staging-and-flip, so nothing makes the pair atomic. `main`'s CI runs are +serialized rather than cancelled — cancelling mid-publication is what splits +them — but a failure during the second upload still leaves a split state until +the next successful run. That is accepted for a mutable pre-release artifact; +strict coherence would need the SDK to name an immutable, timestamped snapshot, +which conflicts with the fixed names above. + +Every publication is followed by `ci/consumer-smoke-test`, a separate Gradle +build with no connection to this one, which resolves the published +`zenoh-java:-SNAPSHOT` from the snapshot repository with +`--refresh-dependencies` and runs a key-expression round trip through JNI. That +is the check that the whole chain — POM, transitive zenoh-flat-jni, native +library — works for someone who is not us. + ## Running a release Everything is driven from **Actions → Release → Run workflow** on the default diff --git a/README.md b/README.md index c4b3a32a..1bd0f794 100644 --- a/README.md +++ b/README.md @@ -139,9 +139,13 @@ Three ways to build. Pick by what you are doing: | build the bindings from source too | `./gradlew build -PuseLocalJni=true` | yes | | build against my own checkout | `./gradlew build -PlocalJniDir=../zenoh-flat-jni` | yes | -**The default** downloads `org.eclipse.zenoh:zenoh-flat-jni` from Maven Central -with the native library already inside it. Nothing is compiled from Rust and no -toolchain is needed. +**The default** downloads `org.eclipse.zenoh:zenoh-flat-jni` with the native +library already inside it. Nothing is compiled from Rust and no toolchain is +needed. On `main` that is `1.9.0-java-SNAPSHOT`, published from this repository +alongside the SDK snapshot ([CI.md](CI.md#what-publishing-uses)); a release +names a zenoh-flat-jni release on Maven Central. Either way it is one coordinate +in `gradle.properties`, and if it has not been published yet the other two rows +build without it. **`-PuseLocalJni=true`** builds the bindings from source, as `Cargo.toml` says — the usual Rust arrangement, and the one CI uses. A `git` dependency there diff --git a/build.gradle.kts b/build.gradle.kts index b9234aee..5b70787e 100644 --- a/build.gradle.kts +++ b/build.gradle.kts @@ -76,12 +76,13 @@ subprojects { repositories { google() mavenCentral() - // A rehearsal has to build against a zenoh-flat-jni that is not released - // yet; its own rehearsal publishes -SNAPSHOT here. This - // repository enters the resolution path *only* when a snapshot version - // was explicitly asked for, and even then only for that one module — so - // a release, whose version never ends in -SNAPSHOT, cannot resolve a - // mutable artifact by accident. + // Between releases this SDK builds against a snapshot: our own copy of + // zenoh-flat-jni, `1.9.0-java-SNAPSHOT`, published by the same job that + // publishes this SDK's snapshot (see CI.md). A rehearsal can name + // another one. This repository enters the resolution path *only* when a + // snapshot version was asked for, and even then only for that one + // module — so a release, whose version never ends in -SNAPSHOT, cannot + // resolve a mutable artifact by accident. if (zenohFlatJniVersion.endsWith("-SNAPSHOT")) { maven { name = "centralSnapshots" diff --git a/ci/consumer-smoke-test/build.gradle.kts b/ci/consumer-smoke-test/build.gradle.kts new file mode 100644 index 00000000..1234cbc8 --- /dev/null +++ b/ci/consumer-smoke-test/build.gradle.kts @@ -0,0 +1,49 @@ +// +// An external consumer of the published zenoh-java artifact — deliberately not +// part of the main Gradle build, with no path, project or composite dependency +// on it. What it proves is what this repository cannot prove from inside its own +// build: that the published coordinate resolves, that the POM's transitive +// zenoh-flat-jni dependency exists and is resolvable too, and that the native +// library inside it loads. +// +// Run against a candidate: +// gradle run -PcandidateVersion= [-PcandidateRepository=] +// +plugins { + java + application +} + +val candidateVersion: String by project + +// Defaults to the Maven Central snapshot repository, which is where this +// repository's snapshot publication puts both coordinates. +val candidateRepository: String = + project.findProperty("candidateRepository")?.toString() + ?: "https://central.sonatype.com/repository/maven-snapshots" + +repositories { + // The content filters make the resolution source unambiguous: anything + // org.eclipse.zenoh can only come from the candidate repository, never from + // a released copy of the same coordinates on Central. + maven { + name = "candidate" + url = uri(candidateRepository) + content { includeGroup("org.eclipse.zenoh") } + } + mavenCentral { + content { excludeGroup("org.eclipse.zenoh") } + } +} + +dependencies { + implementation("org.eclipse.zenoh:zenoh-java:$candidateVersion") +} + +java { + toolchain { languageVersion.set(JavaLanguageVersion.of(11)) } +} + +application { + mainClass.set("smoke.SmokeTest") +} diff --git a/ci/consumer-smoke-test/settings.gradle.kts b/ci/consumer-smoke-test/settings.gradle.kts new file mode 100644 index 00000000..14c4a2cb --- /dev/null +++ b/ci/consumer-smoke-test/settings.gradle.kts @@ -0,0 +1 @@ +rootProject.name = "zenoh-java-smoke-test" diff --git a/ci/consumer-smoke-test/src/main/java/smoke/SmokeTest.java b/ci/consumer-smoke-test/src/main/java/smoke/SmokeTest.java new file mode 100644 index 00000000..81912520 --- /dev/null +++ b/ci/consumer-smoke-test/src/main/java/smoke/SmokeTest.java @@ -0,0 +1,48 @@ +// +// Copyright (c) 2026 ZettaScale Technology +// +// This program and the accompanying materials are made available under the +// terms of the Eclipse Public License 2.0 which is available at +// http://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0 +// which is available at https://www.apache.org/licenses/LICENSE-2.0. +// +// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0 +// +// Contributors: +// ZettaScale Zenoh Team, +// + +package smoke; + +import io.zenoh.exceptions.ZError; +import io.zenoh.keyexpr.KeyExpr; + +/** + * The minimum that proves a published zenoh-java artifact is usable: loading the + * native library out of the transitive zenoh-flat-jni dependency, crossing the + * JNI boundary in both directions, and doing it through the Java-facing API a + * user of this SDK actually calls. + * + * Key expressions rather than a session, because they need no network, no ports + * and no discovery — a CI runner cannot make them flaky. + */ +public final class SmokeTest { + + public static void main(String[] args) throws ZError { + String expr = "demo/example/**"; + KeyExpr ke = KeyExpr.tryFrom(expr); + + if (!expr.equals(ke.toString())) { + throw new IllegalStateException("key expression round-tripped as `" + ke + "`, expected `" + expr + "`"); + } + if (!ke.intersects(KeyExpr.tryFrom("demo/example/smoke"))) { + throw new IllegalStateException("`" + expr + "` should intersect `demo/example/smoke`"); + } + if (ke.intersects(KeyExpr.tryFrom("other/key"))) { + throw new IllegalStateException("`" + expr + "` should not intersect `other/key`"); + } + + System.out.println("zenoh-java smoke test OK on " + + System.getProperty("os.name") + " " + System.getProperty("os.arch")); + } +} diff --git a/ci/scripts/bump-and-tag.bash b/ci/scripts/bump-and-tag.bash index e40ecfc4..f20c0085 100644 --- a/ci/scripts/bump-and-tag.bash +++ b/ci/scripts/bump-and-tag.bash @@ -27,19 +27,6 @@ git commit version.txt -m "chore: Bump version to \`$version\`" # real release, never a snapshot: consumers do not have the snapshot repository # configured, and snapshots are mutable and eventually removed. if [[ -n "$flat_jni_version" ]]; then - # A *release* may not depend on a snapshot: consumers do not configure the - # snapshot repository, and snapshots mutate and expire. A rehearsal may — that - # is how the SDK is exercised before the binding is released at all. - case "$flat_jni_version" in - *-SNAPSHOT) - if [[ "$live_run" == "true" ]]; then - echo "error: refusing to release against a snapshot dependency ($flat_jni_version)" >&2 - exit 1 - fi - echo "note: rehearsing against snapshot $flat_jni_version" - ;; - esac - sed -i.bak -E "s|^zenohFlatJniVersion=.*|zenohFlatJniVersion=$flat_jni_version|" gradle.properties rm -f gradle.properties.bak @@ -53,6 +40,24 @@ if [[ -n "$flat_jni_version" ]]; then fi fi +# Checked on the value the release will actually build against, not on the +# workflow input: main now inherits `1.9.0-java-SNAPSHOT`, our own mutable copy +# of zenoh-flat-jni, so omitting the input is exactly how a release would reach +# a snapshot dependency — which the earlier input-only check did not cover. +# A rehearsal may depend on one; that is how the SDK is exercised before the +# binding is released at all. +effective_flat_jni_version=$(sed -n 's/^zenohFlatJniVersion=//p' gradle.properties | tr -d '[:space:]') +case "$effective_flat_jni_version" in + *-SNAPSHOT) + if [[ "$live_run" == "true" ]]; then + echo "error: refusing to release against a snapshot dependency ($effective_flat_jni_version)." >&2 + echo " Pass zenoh-flat-jni-version naming a release that is on Maven Central." >&2 + exit 1 + fi + echo "note: rehearsing against snapshot $effective_flat_jni_version" + ;; +esac + if [[ ${live_run} ]]; then git tag --force "$version" -m "v$version" fi diff --git a/ci/scripts/flat-jni-copy.bash b/ci/scripts/flat-jni-copy.bash new file mode 100755 index 00000000..7ce72471 --- /dev/null +++ b/ci/scripts/flat-jni-copy.bash @@ -0,0 +1,134 @@ +#!/usr/bin/env bash +# +# Is the zenoh-flat-jni copy this repository publishes already the commit we pin? +# +# The SDK snapshot names org.eclipse.zenoh:zenoh-flat-jni:, +# and this repository is what puts that coordinate there — built from the commit +# Cargo.lock pins, so the dependency exists and is the code the SDK compiled +# against, whether or not zenoh-flat-jni's CI has ever run. Producing it means +# cross-compiling ten targets, on the order of half an hour, so it is rebuilt +# only when the published copy is not already that commit. +# +# The published copy says which commit it was built from as a POM property, so +# the check costs two small requests per coordinate rather than a 39 MB +# download. Anything missing — no metadata, no POM, no stamp — reads as "not +# ours" and rebuilds, which is the safe direction. +# +# Writes `commit`, `version`, `qualifier` and `rebuild` to $GITHUB_OUTPUT when +# running under Actions, and prints them either way. `--self-test` runs the +# parsers against fixtures and exits. +# +set -euo pipefail + +cd "$(dirname "${BASH_SOURCE[0]}")/../.." + +readonly repository=${SNAPSHOT_REPOSITORY:-https://central.sonatype.com/repository/maven-snapshots} +readonly group_path=org/eclipse/zenoh +# The qualifier that keeps our copy from overwriting zenoh-flat-jni's own +# snapshot or zenoh-kotlin's. Checked against gradle.properties below, and +# handed to zenoh-flat-jni's publication workflow, so the two cannot drift. +readonly qualifier=${FLAT_JNI_QUALIFIER:-java} +# All three coordinates, not just the root: one Gradle invocation uploads them, +# but a partial failure leaves them at different builds, and it is the platform +# ones that carry the native libraries. +readonly artifacts=(zenoh-flat-jni zenoh-flat-jni-jvm zenoh-flat-jni-android) + +# The commit Cargo.lock pins for the git dependency on zenoh-flat-jni. Reads +# stdin so it can be tested without a lockfile. +pinned_commit() { + grep -Eom1 'zenoh-flat-jni\.git[^#"]*#[0-9a-f]{40}' | grep -Eo '[0-9a-f]{40}$' +} + +# The file name the newest timestamped build of a snapshot has, from that +# version's maven-metadata.xml on stdin: +# ---. +timestamped_name() { # + local metadata timestamp build + metadata=$(cat) + timestamp=$(sed -n 's:.*\(.*\).*:\1:p' <<<"$metadata" | head -1) + build=$(sed -n 's:.*\(.*\).*:\1:p' <<<"$metadata" | head -1) + [[ -n $timestamp && -n $build ]] || return 1 + printf '%s-%s-%s-%s.%s' "$1" "${2%-SNAPSHOT}" "$timestamp" "$build" "$3" +} + +# The commit a published POM on stdin was built from; empty when it has no stamp. +pom_commit() { + sed -n 's:.*\(.*\).*:\1:p' | head -1 +} + +# The stamp of the published copy of one coordinate; empty if it is not there. +published_commit() { # + local base_url="$repository/$group_path/$1/$2" metadata name + metadata=$(curl -sf "$base_url/maven-metadata.xml") || return 0 + name=$(timestamped_name "$1" "$2" pom <<<"$metadata") || return 0 + { curl -sf "$base_url/$name" || true; } | pom_commit +} + +self_test() { + local got + got=$(pinned_commit <<<'source = "git+https://github.com/eclipse-zenoh/zenoh-flat-jni.git?branch=main#e75529ce3758401ce213456e7b8e4e5667635cf8"') + [[ $got == e75529ce3758401ce213456e7b8e4e5667635cf8 ]] || { echo "pinned_commit: $got" >&2; exit 1; } + + # The real lockfile too, so a change to how Cargo writes it fails here + # rather than by silently rebuilding on every run. + got=$(pinned_commit &2; exit 1; } + + got=$(timestamped_name zenoh-flat-jni 1.9.0-java-SNAPSHOT pom <<'EOF' + + + 20260810.012355 + 1 + + +EOF + ) + [[ $got == zenoh-flat-jni-1.9.0-java-20260810.012355-1.pom ]] || { echo "timestamped_name: $got" >&2; exit 1; } + + # A release-style metadata carries no block: no name to build. + if timestamped_name zenoh-flat-jni 1.9.0 pom <<<'1.9.0' >/dev/null; then + echo "timestamped_name accepted metadata with no snapshot block" >&2 + exit 1 + fi + + got=$(pom_commit <<<' e75529ce3758401ce213456e7b8e4e5667635cf8') + [[ $got == e75529ce3758401ce213456e7b8e4e5667635cf8 ]] || { echo "pom_commit: $got" >&2; exit 1; } + + got=$(pom_commit <<<'1.9.0-java-SNAPSHOT') + [[ -z $got ]] || { echo "pom_commit on an unstamped POM: $got" >&2; exit 1; } + + echo "flat-jni-copy.bash self-test OK" +} + +main() { + local version commit rebuild=false stamp + version=$(sed -n 's/^zenohFlatJniVersion=//p' gradle.properties | tr -d '[:space:]') + [[ $version == *-$qualifier-SNAPSHOT ]] || { + echo "::error::zenohFlatJniVersion=$version is not a -$qualifier-SNAPSHOT copy;" \ + "only that coordinate is ours to publish" >&2 + exit 1 + } + commit=$(pinned_commit >"$GITHUB_OUTPUT" + fi +} + +if [[ ${1:-} == --self-test ]]; then self_test; else main; fi diff --git a/gradle.properties b/gradle.properties index b14ed9ce..24bcd088 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,10 +1,17 @@ org.jetbrains.dokka.experimental.gradle.pluginMode=V2Enabled org.jetbrains.dokka.experimental.gradle.pluginMode.noWarn=true -# The zenoh-flat-jni release this SDK is built against. Rewritten by -# ci/scripts/bump-and-tag.bash at release time; override for a rehearsal with +# The zenoh-flat-jni this SDK is built against. Rewritten by +# ci/scripts/bump-and-tag.bash at release time, where it names a real Maven +# Central release; override for a rehearsal with # -PzenohFlatJniVersion=1.9.0-rc3-SNAPSHOT. -zenohFlatJniVersion=1.9.0 +# +# Between releases it names *our own copy* of zenoh-flat-jni — the `-java` +# qualifier — built from the commit Cargo.lock pins and published by the same +# job that publishes this SDK's snapshot. That is what makes a snapshot +# self-sufficient: its dependency exists, and is the code it compiled against, +# whether or not zenoh-flat-jni's CI has ever run. See CI.md. +zenohFlatJniVersion=1.9.0-java-SNAPSHOT # Build zenoh-flat-jni from source, as Cargo.toml says, instead of resolving the # published artifact. See README.md. Never enable this for a release. From a96e01305f489d8e707bb028e3ad12aa64f222cd Mon Sep 17 00:00:00 2001 From: Michael Ilyin Date: Tue, 11 Aug 2026 19:46:08 +0200 Subject: [PATCH 2/7] CI: tell zenoh-flat-jni which base version we expect MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Its coordinate comes from its own version.txt, ours from gradle.properties. A pin that moves past a version bump there would publish 1.10.0-java-SNAPSHOT while this SDK still resolves 1.9.0-java-SNAPSHOT — and only fail afterwards. flat-jni-copy.bash now emits the base version and ci.yml passes it as expected-base-version, so eclipse-zenoh/zenoh-flat-jni#37's preflight catches the mismatch before anything is built. --- .github/workflows/ci.yml | 5 +++++ ci/scripts/flat-jni-copy.bash | 13 +++++++++---- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 84d6d294..e10be422 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -130,6 +130,7 @@ jobs: runs-on: ubuntu-latest outputs: commit: ${{ steps.pin.outputs.commit }} + base: ${{ steps.pin.outputs.base }} qualifier: ${{ steps.pin.outputs.qualifier }} rebuild: ${{ steps.pin.outputs.rebuild }} steps: @@ -162,6 +163,10 @@ jobs: source-repository: eclipse-zenoh/zenoh-flat-jni branch: ${{ needs.flat_jni_pin.outputs.commit }} version-qualifier: ${{ needs.flat_jni_pin.outputs.qualifier }} + # It derives the coordinate from its own version.txt; this is what we + # expect that to be, so a pin that moved past a version bump there fails + # before publishing something we cannot resolve. + expected-base-version: ${{ needs.flat_jni_pin.outputs.base }} secrets: inherit # Then the SDK snapshot, naming the copy above. Reached both ways: the copy diff --git a/ci/scripts/flat-jni-copy.bash b/ci/scripts/flat-jni-copy.bash index 7ce72471..b8b8c8c7 100755 --- a/ci/scripts/flat-jni-copy.bash +++ b/ci/scripts/flat-jni-copy.bash @@ -14,8 +14,8 @@ # download. Anything missing — no metadata, no POM, no stamp — reads as "not # ours" and rebuilds, which is the safe direction. # -# Writes `commit`, `version`, `qualifier` and `rebuild` to $GITHUB_OUTPUT when -# running under Actions, and prints them either way. `--self-test` runs the +# Writes `commit`, `version`, `base`, `qualifier` and `rebuild` to $GITHUB_OUTPUT +# when running under Actions, and prints them either way. `--self-test` runs the # parsers against fixtures and exits. # set -euo pipefail @@ -101,13 +101,17 @@ EOF } main() { - local version commit rebuild=false stamp + local version base commit rebuild=false stamp version=$(sed -n 's/^zenohFlatJniVersion=//p' gradle.properties | tr -d '[:space:]') [[ $version == *-$qualifier-SNAPSHOT ]] || { echo "::error::zenohFlatJniVersion=$version is not a -$qualifier-SNAPSHOT copy;" \ "only that coordinate is ours to publish" >&2 exit 1 } + # zenoh-flat-jni derives the coordinate from its own version.txt, so it needs + # to be told what we expect: a pin that moved past a version bump there would + # otherwise publish 1.10.0-java-SNAPSHOT while we still resolve this. + base=${version%-$qualifier-SNAPSHOT} commit=$(pinned_commit >"$GITHUB_OUTPUT" From 5514d99370807717543dbb9395583b1fc7d94929 Mon Sep 17 00:00:00 2001 From: Michael Ilyin Date: Tue, 11 Aug 2026 21:30:45 +0200 Subject: [PATCH 3/7] CI: publish the commit that triggered the run, and check the copy is whole MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review of #525. - **The SDK publication was resolving `main`, not the run's commit.** Concurrency queues a newer run; it does not hold the branch still. So run A could read Cargo.lock at A, spend half an hour building the JNI copy from A's pin, and then publish SDK source B against it — recreating exactly the mismatch this job exists to prevent, and persisting if B's queued run later fails. It now passes `github.sha`. - **Three matching stamps were not proof of a finished publication.** The POM is uploaded before the Gradle module metadata and before the jar or aar, so a run that died in between would leave three readable stamps, no module metadata for a consumer to resolve a variant against, and a decision to skip rebuilding — permanently, since every later run reads the same three stamps. Nothing downstream would catch it: the consumer smoke test runs on Linux and the broken variant could be the Android one. The decision now also requires each coordinate's metadata to advertise `pom`, `module` and its binary (`jar`, or `aar` for Android), which is what a complete upload lists. Verified against the real 1.9.0-rc8-SNAPSHOT, and the two ways to be unfinished are in the self-test. - **PUBLISHING.md contradicted itself.** The rehearsal sections still said an empty `zenoh-flat-jni-version` falls back to an unreleased `1.9.0` and dies while compiling, and sent operators to zenoh-flat-jni's own rehearsal snapshot. The fallback is now our own `1.9.0-java-SNAPSHOT`, which resolves — what it cannot do is reach a live release, and it is `bump-and-tag.bash` that stops that. "No consumer test" was also listed as a known gap; the gap now is only that a *release* candidate is not consumable, being in staging. --- .github/workflows/ci.yml | 8 ++++- .github/workflows/publish.yml | 2 +- PUBLISHING.md | 46 +++++++++++++-------------- ci/scripts/flat-jni-copy.bash | 59 +++++++++++++++++++++++++++++++---- 4 files changed, 84 insertions(+), 31 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e10be422..ab02f678 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -184,5 +184,11 @@ jobs: packages: write with: snapshot: true - branch: ${{ github.ref_name }} + # The commit that triggered this run, not `main`. Concurrency queues a + # newer run; it does not hold the branch still. Passing the branch name, + # this job would check out whatever `main` had become while the JNI copy + # was being built for half an hour — publishing SDK source B against the + # copy built from A's pin, which is the mismatch the whole job exists to + # prevent. + branch: ${{ github.sha }} secrets: inherit diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 8be6f689..e5dd8c38 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -15,7 +15,7 @@ on: description: "If the publication is for a snapshot version." default: false branch: - description: Target branch + description: Branch, tag or commit to publish from type: string required: false maven_publish: diff --git a/PUBLISHING.md b/PUBLISHING.md index 19c9df7e..37d39235 100644 --- a/PUBLISHING.md +++ b/PUBLISHING.md @@ -190,7 +190,7 @@ builds and publishes. | --- | --- | | `live-run` | **unchecked** | | `version` | a fresh provisional number, not one already used | -| `zenoh-flat-jni-version` | a version that exists — today a snapshot, see [below](#rehearsing-the-release-workflow-with-a-snapshot). Empty falls back to `gradle.properties`, which names an unreleased version and fails | +| `zenoh-flat-jni-version` | a version that exists — today a snapshot, see [below](#rehearsing-the-release-workflow-with-a-snapshot). Empty falls back to `gradle.properties`, which names our own `1.9.0-java-SNAPSHOT` copy: fine for a rehearsal, refused for a live run | | `maven_publish` | checked — or uncheck for the very first run | `live-run` and `maven_publish` behave exactly as in zenoh-flat-jni: unchecking @@ -238,12 +238,15 @@ release is blocked. | CI, snapshot publication | `zenoh-flat-jni:-SNAPSHOT` | signing, credentials, a real upload | | live release | `zenoh-flat-jni:` on Central | **blocked until that exists** | -A snapshot may depend on a snapshot, because nothing published is permanent. So -the answer is to consume the snapshot that zenoh-flat-jni's *own* rehearsal -published — a rehearsal there with `maven_publish` enabled uploads -`zenoh-flat-jni:-SNAPSHOT` to the Central snapshot repository. +A snapshot may depend on a snapshot, because nothing published is permanent — +which is what [The snapshot publication](#the-snapshot-publication) above rests +on. `gradle.properties` names `1.9.0-java-SNAPSHOT`, our own copy, and every +merge to `main` republishes it from the pinned commit. So a rehearsal needs +nothing set: the default already resolves. -Nothing needs editing. Name the version on the command line: +Name another one on the command line to build against a different +zenoh-flat-jni — the snapshot it publishes itself, or one from a rehearsal +there: ```bash ./gradlew build -PzenohFlatJniVersion=1.9.0-rc8-SNAPSHOT @@ -270,21 +273,16 @@ Then run **Release** from the Actions tab with: | `maven_publish` | checked to rehearse the upload too, unchecked to stop at assembly | | `version`, `branch` | leave empty unless you are rehearsing a specific one | -**`zenoh-flat-jni-version` is the field that matters.** Left empty, the build -falls back to `zenohFlatJniVersion` in `gradle.properties` — currently `1.9.0`, -which is not on Maven Central, and the run dies while compiling: +**`zenoh-flat-jni-version` decides what the rehearsal builds against.** Left +empty it falls back to `zenohFlatJniVersion` in `gradle.properties` — +`1.9.0-java-SNAPSHOT`, our own copy, which `main` republishes on every merge. A +rehearsal against that is a real rehearsal, and it is also what the nightly +scheduled run of `release.yml` does, passing no inputs at all. -```text -Could not find org.eclipse.zenoh:zenoh-flat-jni:1.9.0 -``` - -That is the conditional repository below doing its job, not a broken build: a -non-snapshot version never gets the snapshot repository on its resolution path. - -`release.yml` used to run on a weekday schedule as well, and a scheduled run -passes no inputs — so it failed exactly this way every night. The schedule is -gone: the workflow is `workflow_dispatch` only, and a rehearsal is something you -start on purpose, with the version filled in. +What that fallback cannot do is reach a **live** release: +`ci/scripts/bump-and-tag.bash` refuses a `-SNAPSHOT` binding, and it checks the +value `gradle.properties` ends up with rather than the input, precisely because +an omitted input now inherits one. The Central snapshot repository is declared **conditionally** in `build.gradle.kts`, and this is the part worth understanding: @@ -373,9 +371,11 @@ repository. - **The rewritten release path has never run.** The workflows were repaired for a repository that no longer contains Rust; no rehearsal has yet exercised them. -- **No consumer test.** Unlike zenoh-flat-jni, nothing resolves the published - `zenoh-java` artifact from a repository and runs it before release. The tests - here run against the build's own output. +- **No consumer test before a *release*.** Every snapshot publication is followed + by `ci/consumer-smoke-test`, which resolves the published artifact from the + snapshot repository and runs it — but a release goes to a staging repository + and is not resolvable at that point, so nothing consumes a release candidate + the way zenoh-flat-jni's own dry-run repository lets it consume one. - **The Android artifact has no runtime test**, and its `ndkVersion` and NDK setup step are retained although no native code is built here — unverified whether the Android Gradle Plugin still needs them. diff --git a/ci/scripts/flat-jni-copy.bash b/ci/scripts/flat-jni-copy.bash index b8b8c8c7..94edc5aa 100755 --- a/ci/scripts/flat-jni-copy.bash +++ b/ci/scripts/flat-jni-copy.bash @@ -31,7 +31,19 @@ readonly qualifier=${FLAT_JNI_QUALIFIER:-java} # All three coordinates, not just the root: one Gradle invocation uploads them, # but a partial failure leaves them at different builds, and it is the platform # ones that carry the native libraries. -readonly artifacts=(zenoh-flat-jni zenoh-flat-jni-jvm zenoh-flat-jni-android) +# +# Each with the extension of its binary, because a matching stamp is not on its +# own proof of a finished publication. The POM goes up before the Gradle module +# metadata and before the jar or aar, so a run that died in between would leave +# three readable stamps, no module metadata for Gradle to resolve a variant +# against, and a decision to skip rebuilding — forever, since the next run reads +# the same three stamps. Nothing downstream would catch it either: the consumer +# smoke test runs on Linux, and the broken variant could be the Android one. +readonly artifacts=( + zenoh-flat-jni:jar + zenoh-flat-jni-jvm:jar + zenoh-flat-jni-android:aar +) # The commit Cargo.lock pins for the git dependency on zenoh-flat-jni. Reads # stdin so it can be tested without a lockfile. @@ -56,10 +68,25 @@ pom_commit() { sed -n 's:.*\(.*\).*:\1:p' | head -1 } -# The stamp of the published copy of one coordinate; empty if it is not there. -published_commit() { # +# Whether the metadata on stdin advertises an unclassified artifact of each given +# extension. Maven appends an entry as each file lands, so a publication that +# stopped part-way lists fewer than a finished one. The anchor is what excludes +# the sources and javadoc jars: the schema puts before , +# so only a main artifact starts its entry with the extension. +advertises() { # … + local blocks ext + blocks=$(tr -d '[:space:]' | sed 's::\n:g') + for ext in "$@"; do + grep -q "^$ext" <<<"$blocks" || return 1 + done +} + +# The stamp of the published copy of one coordinate; empty if it is not there, or +# not all of it is. +published_commit() { # local base_url="$repository/$group_path/$1/$2" metadata name metadata=$(curl -sf "$base_url/maven-metadata.xml") || return 0 + advertises pom module "$3" <<<"$metadata" || return 0 name=$(timestamped_name "$1" "$2" pom <<<"$metadata") || return 0 { curl -sf "$base_url/$name" || true; } | pom_commit } @@ -97,11 +124,30 @@ EOF got=$(pom_commit <<<'1.9.0-java-SNAPSHOT') [[ -z $got ]] || { echo "pom_commit on an unstamped POM: $got" >&2; exit 1; } + # A finished publication, then the two ways one can be unfinished: stopped + # after the POM, and carrying a sources jar but no main jar. The layout is + # what zenoh-flat-jni really publishes — checked against 1.9.0-rc8-SNAPSHOT. + local finished=" + pom + module + sourcesjar + jar" + advertises pom module jar <<<"$finished" || { echo "advertises: finished rejected" >&2; exit 1; } + if advertises pom module aar <<<"$finished"; then + echo "advertises: accepted a jar publication as an aar one" >&2; exit 1 + fi + if advertises pom module jar <<<'pom'; then + echo "advertises: accepted a publication that stopped after the POM" >&2; exit 1 + fi + if advertises jar <<<'sourcesjar'; then + echo "advertises: took the sources jar for the main one" >&2; exit 1 + fi + echo "flat-jni-copy.bash self-test OK" } main() { - local version base commit rebuild=false stamp + local version base commit rebuild=false entry artifact stamp version=$(sed -n 's/^zenohFlatJniVersion=//p' gradle.properties | tr -d '[:space:]') [[ $version == *-$qualifier-SNAPSHOT ]] || { echo "::error::zenohFlatJniVersion=$version is not a -$qualifier-SNAPSHOT copy;" \ @@ -114,8 +160,9 @@ main() { base=${version%-$qualifier-SNAPSHOT} commit=$(pinned_commit Date: Tue, 11 Aug 2026 21:44:41 +0200 Subject: [PATCH 4/7] CI: require the whole coordinate to be at one build, not merely present MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Review of #525. The presence check missed the case that matters more than a first publication: an overwrite that failed part-way. Every in a snapshot's maven-metadata.xml carries its own , updated as that file lands. After the first complete publication all three extensions are listed and stay listed, so an overwrite that replaced the POM and then failed leaves the POM at build N+1 with the module metadata and the binary still at N. The old check saw three extensions, read the N+1 POM, found the pin in its stamp, and skipped rebuilding — permanently, and with a coordinate split across two JNI builds. So the check is now agreement rather than presence: the unclassified pom, module and binary entries must all name the same -, and the POM is fetched by that name, so a metadata entry pointing at a file that never landed reads as no stamp and rebuilds. `timestamped_name` became `snapshot_value`, since the build identifier rather than one file name is what is being compared. The self-test gains the split case, which the previous fixtures could not express. Checked against the published 1.9.0-rc8-SNAPSHOT: all three coordinates whole, at 1.9.0-rc8-20260810.012355-1. --- ci/scripts/flat-jni-copy.bash | 107 +++++++++++++++++++++------------- 1 file changed, 65 insertions(+), 42 deletions(-) diff --git a/ci/scripts/flat-jni-copy.bash b/ci/scripts/flat-jni-copy.bash index 94edc5aa..a8609469 100755 --- a/ci/scripts/flat-jni-copy.bash +++ b/ci/scripts/flat-jni-copy.bash @@ -34,11 +34,12 @@ readonly qualifier=${FLAT_JNI_QUALIFIER:-java} # # Each with the extension of its binary, because a matching stamp is not on its # own proof of a finished publication. The POM goes up before the Gradle module -# metadata and before the jar or aar, so a run that died in between would leave -# three readable stamps, no module metadata for Gradle to resolve a variant -# against, and a decision to skip rebuilding — forever, since the next run reads -# the same three stamps. Nothing downstream would catch it either: the consumer -# smoke test runs on Linux, and the broken variant could be the Android one. +# metadata and before the jar or aar, so a publication that died in between — +# whether it was the first or an overwrite — leaves a readable stamp with no +# module metadata for Gradle to resolve a variant against, and a decision to skip +# rebuilding. Forever, since the next run reads the same stamp. Nothing +# downstream would catch it either: the consumer smoke test runs on Linux, and +# the broken variant could be the Android one. readonly artifacts=( zenoh-flat-jni:jar zenoh-flat-jni-jvm:jar @@ -51,16 +52,18 @@ pinned_commit() { grep -Eom1 'zenoh-flat-jni\.git[^#"]*#[0-9a-f]{40}' | grep -Eo '[0-9a-f]{40}$' } -# The file name the newest timestamped build of a snapshot has, from that -# version's maven-metadata.xml on stdin: -# ---. -timestamped_name() { # +# Which timestamped build a snapshot currently resolves to, from that version's +# maven-metadata.xml on stdin: +# -- +# It is also the middle of every file name in that build: +# -. +snapshot_value() { # local metadata timestamp build metadata=$(cat) timestamp=$(sed -n 's:.*\(.*\).*:\1:p' <<<"$metadata" | head -1) build=$(sed -n 's:.*\(.*\).*:\1:p' <<<"$metadata" | head -1) [[ -n $timestamp && -n $build ]] || return 1 - printf '%s-%s-%s-%s.%s' "$1" "${2%-SNAPSHOT}" "$timestamp" "$build" "$3" + printf '%s-%s-%s' "${1%-SNAPSHOT}" "$timestamp" "$build" } # The commit a published POM on stdin was built from; empty when it has no stamp. @@ -68,27 +71,36 @@ pom_commit() { sed -n 's:.*\(.*\).*:\1:p' | head -1 } -# Whether the metadata on stdin advertises an unclassified artifact of each given -# extension. Maven appends an entry as each file lands, so a publication that -# stopped part-way lists fewer than a finished one. The anchor is what excludes -# the sources and javadoc jars: the schema puts before , -# so only a main artifact starts its entry with the extension. -advertises() { # … - local blocks ext +# Whether the metadata on stdin says every one of the given extensions is at the +# given build. Each carries its own , updated as that +# file lands, so an overwrite that failed part-way leaves the POM at build N+1 +# while the module metadata and the binary are still at N — which is the case a +# presence check cannot see, since all three entries exist either way and have +# since the first publication. +# +# The anchor is what excludes the sources and javadoc jars: the schema puts +# before , so only a main artifact starts its entry with +# the extension. +all_at() { # … + local blocks value ext blocks=$(tr -d '[:space:]' | sed 's::\n:g') + value=${1//./\\.} + shift for ext in "$@"; do - grep -q "^$ext" <<<"$blocks" || return 1 + grep -q "^$ext$value" <<<"$blocks" || return 1 done } # The stamp of the published copy of one coordinate; empty if it is not there, or -# not all of it is. +# not all of it is at the same build. published_commit() { # - local base_url="$repository/$group_path/$1/$2" metadata name + local base_url="$repository/$group_path/$1/$2" metadata value metadata=$(curl -sf "$base_url/maven-metadata.xml") || return 0 - advertises pom module "$3" <<<"$metadata" || return 0 - name=$(timestamped_name "$1" "$2" pom <<<"$metadata") || return 0 - { curl -sf "$base_url/$name" || true; } | pom_commit + value=$(snapshot_value "$2" <<<"$metadata") || return 0 + all_at "$value" pom module "$3" <<<"$metadata" || return 0 + # The POM is fetched by that name, so a metadata entry naming a file that + # never landed reads as no stamp — and rebuilds. + { curl -sf "$base_url/$1-$value.pom" || true; } | pom_commit } self_test() { @@ -101,7 +113,7 @@ self_test() { got=$(pinned_commit &2; exit 1; } - got=$(timestamped_name zenoh-flat-jni 1.9.0-java-SNAPSHOT pom <<'EOF' + got=$(snapshot_value 1.9.0-java-SNAPSHOT <<'EOF' 20260810.012355 @@ -110,11 +122,11 @@ self_test() { EOF ) - [[ $got == zenoh-flat-jni-1.9.0-java-20260810.012355-1.pom ]] || { echo "timestamped_name: $got" >&2; exit 1; } + [[ $got == 1.9.0-java-20260810.012355-1 ]] || { echo "snapshot_value: $got" >&2; exit 1; } - # A release-style metadata carries no block: no name to build. - if timestamped_name zenoh-flat-jni 1.9.0 pom <<<'1.9.0' >/dev/null; then - echo "timestamped_name accepted metadata with no snapshot block" >&2 + # A release-style metadata carries no block: no build to name. + if snapshot_value 1.9.0 <<<'1.9.0' >/dev/null; then + echo "snapshot_value accepted metadata with no snapshot block" >&2 exit 1 fi @@ -124,23 +136,34 @@ EOF got=$(pom_commit <<<'1.9.0-java-SNAPSHOT') [[ -z $got ]] || { echo "pom_commit on an unstamped POM: $got" >&2; exit 1; } - # A finished publication, then the two ways one can be unfinished: stopped - # after the POM, and carrying a sources jar but no main jar. The layout is - # what zenoh-flat-jni really publishes — checked against 1.9.0-rc8-SNAPSHOT. + # A finished publication of build -1, in the layout zenoh-flat-jni really + # produces — checked against the published 1.9.0-rc8-SNAPSHOT. + local n=1.9.0-java-20260810.012355-1 local finished=" - pom - module - sourcesjar - jar" - advertises pom module jar <<<"$finished" || { echo "advertises: finished rejected" >&2; exit 1; } - if advertises pom module aar <<<"$finished"; then - echo "advertises: accepted a jar publication as an aar one" >&2; exit 1 + pom$n + module$n + sourcesjar$n + jar$n" + all_at "$n" pom module jar <<<"$finished" || { echo "all_at: finished rejected" >&2; exit 1; } + if all_at "$n" pom module aar <<<"$finished"; then + echo "all_at: accepted a jar publication as an aar one" >&2; exit 1 + fi + if all_at "$n" pom module jar <<<"pom$n"; then + echo "all_at: accepted a publication that stopped after the POM" >&2; exit 1 fi - if advertises pom module jar <<<'pom'; then - echo "advertises: accepted a publication that stopped after the POM" >&2; exit 1 + if all_at "$n" jar <<<"sourcesjar$n"; then + echo "all_at: took the sources jar for the main one" >&2; exit 1 fi - if advertises jar <<<'sourcesjar'; then - echo "advertises: took the sources jar for the main one" >&2; exit 1 + + # The case a presence check cannot see: an overwrite that replaced the POM + # and then failed, leaving the module metadata and the binary at the build + # before it. Every extension is still listed; only the values disagree. + local m=1.9.0-java-20260811.030000-2 + if all_at "$m" pom module jar <<<" + pom$m + module$n + jar$n"; then + echo "all_at: accepted a coordinate split across two builds" >&2; exit 1 fi echo "flat-jni-copy.bash self-test OK" From 8503c612cfd79a31e1b413dc195d6a3243a83414 Mon Sep 17 00:00:00 2001 From: Michael Ilyin Date: Wed, 12 Aug 2026 01:06:16 +0200 Subject: [PATCH 5/7] docs: stop describing a nightly this repository no longer has #526 removes both weekday schedules - CI's 06:00 snapshot run and release.yml's 00:00 dry run - so three sentences added here describe a trigger that will not exist: - the ci.yml comment above the publishing jobs, "every merge there plus the weekday nightly above"; - "every merge to `main` and the weekday nightly upload a mutable pre-release build"; - the rehearsal section, which offered the nightly release run as an example of leaving `zenoh-flat-jni-version` empty. The last one was the weakest of the three anyway: that run failed while compiling every night, because an empty input fell back to a version not on Maven Central. What this branch changes about it is the part worth saying - the fallback is now our own published copy, so an empty field is a sound default rather than a guaranteed failure. The crons themselves are left to #526; touching them here would only conflict. --- .github/workflows/ci.yml | 4 ++-- PUBLISHING.md | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ab02f678..14c95281 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -114,8 +114,8 @@ jobs: - name: Check whether all jobs pass run: echo '${{ toJson(needs) }}' | jq -e 'all(.result == "success")' - # Everything below publishes, and only from main: every merge there plus the - # weekday nightly above. Branches and pull requests publish nothing. + # Everything below publishes, and only from main, on every merge there. + # Branches and pull requests publish nothing. # # The snapshot must be usable by someone who is not us, which means its # zenoh-flat-jni dependency has to exist — and be the commit this SDK compiled diff --git a/PUBLISHING.md b/PUBLISHING.md index 37d39235..b3bb61c0 100644 --- a/PUBLISHING.md +++ b/PUBLISHING.md @@ -101,8 +101,8 @@ Rehearsals are not constrained this way — see ## The snapshot publication -Between releases, every merge to `main` and the weekday nightly upload a -mutable pre-release build to the [Central snapshot +Between releases, every merge to `main` uploads a mutable pre-release build to +the [Central snapshot repository](https://central.sonatype.com/repository/maven-snapshots/). Its purpose is to keep the upload machinery exercised — signing keys, credentials, what Central accepts — and to give people a way to try the current `main`. @@ -276,8 +276,8 @@ Then run **Release** from the Actions tab with: **`zenoh-flat-jni-version` decides what the rehearsal builds against.** Left empty it falls back to `zenohFlatJniVersion` in `gradle.properties` — `1.9.0-java-SNAPSHOT`, our own copy, which `main` republishes on every merge. A -rehearsal against that is a real rehearsal, and it is also what the nightly -scheduled run of `release.yml` does, passing no inputs at all. +rehearsal against that is a real rehearsal — leaving the field empty is now a +sound default rather than the guaranteed compile failure it used to be. What that fallback cannot do is reach a **live** release: `ci/scripts/bump-and-tag.bash` refuses a `-SNAPSHOT` binding, and it checks the From 37c57574fd2e3a79a3098d7d94fd170fee5ed430 Mon Sep 17 00:00:00 2001 From: Michael Ilyin Date: Wed, 12 Aug 2026 01:23:21 +0200 Subject: [PATCH 6/7] CI: pin the actions this branch adds #526 pins every third-party action in these workflows; the four steps added here - the checkout in flat_jni_pin, and the checkout, setup-java and setup-gradle in consumer_test - were written against the unpinned form and would land on main as the only tags left. Same commits, same trailing comments. --- .github/workflows/ci.yml | 2 +- .github/workflows/publish.yml | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 14c95281..3b865f43 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -134,7 +134,7 @@ jobs: qualifier: ${{ steps.pin.outputs.qualifier }} rebuild: ${{ steps.pin.outputs.rebuild }} steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - id: pin run: bash ci/scripts/flat-jni-copy.bash diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index e5dd8c38..839602c6 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -114,17 +114,17 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v4 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: ref: ${{ inputs.branch }} - - uses: actions/setup-java@v4 + - uses: actions/setup-java@b6effb05e454b25005698d916606bdc6ffcbf961 # v5.7.0 with: distribution: temurin java-version: 11 - name: Setup Gradle - uses: gradle/actions/setup-gradle@v4 + uses: gradle/actions/setup-gradle@0723195856401067f7a2779048b490ace7a47d7c # v5.0.2 # A separate Gradle build with no path or composite dependency on this # one, resolving from the snapshot repository with `--refresh-dependencies` From b7a2984fb944753865ff892d8ae5b4219df3f6be Mon Sep 17 00:00:00 2001 From: Michael Ilyin Date: Wed, 12 Aug 2026 01:52:58 +0200 Subject: [PATCH 7/7] Repin zenoh-flat-jni at the commit that carries the publication inputs eclipse-zenoh/zenoh-flat-jni#37 is merged, so `publish.yml@main` now accepts `source-repository`, `version-qualifier` and `expected-base-version`, and every POM it publishes carries `zenoh.flatJniCommit`. The pin here was e75529c, which predates all of that: the preflight added by that PR rejects it by design, and the workflow this branch adds could not start at all while the inputs it names were not on main. Only the zenoh-flat-jni line moves. `cargo update -p zenoh-flat-jni --precise` also re-resolved the zenoh git dependencies to the tip of their branch, which this branch has no business carrying: zenoh-flat-jni@6b5c04c was tested against 773126fd, and the lockfile-sync bot is what moves that rev in step with Zenoh's own. They are pinned back, leaving a one-line diff. --- Cargo.lock | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Cargo.lock b/Cargo.lock index c19ea899..8f01bcf5 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3912,7 +3912,7 @@ dependencies = [ [[package]] name = "zenoh-flat-jni" version = "1.9.0" -source = "git+https://github.com/eclipse-zenoh/zenoh-flat-jni.git?branch=main#e75529ce3758401ce213456e7b8e4e5667635cf8" +source = "git+https://github.com/eclipse-zenoh/zenoh-flat-jni.git?branch=main#6b5c04c8ee89422d6631f457f27ac79d2d0f3f5f" dependencies = [ "jni 0.21.1", "konst 0.3.17",