diff --git a/AGENTS.md b/AGENTS.md index 187417a..311f5f1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -738,10 +738,12 @@ artifact's tag (`cliff.toml`), and pushes a signed `chore(release)` commit and t no release PR. The repo squash-merges with the PR title as the subject, so the PR title is the changelog entry (`pr-title.yml` checks it is conventional); release.sh refuses a commit that touched the artifact and that git-cliff cannot parse as conventional, unless -`--allow-unconventional`. Until an artifact's first release, its manifest stays at -the `0.0.0` placeholder and its changelog changes are logged by hand under -`[Unreleased]`, which becomes the first release's section; after it, do not edit the -changelogs by hand. +`--allow-unconventional`. Every artifact is released and publishing is on +(`PUBLISH_ENABLED` is `true`): `decdn.node` and `decdn.publisher` are on Galaxy, the +chart on `oci://ghcr.io/decdn/charts` and Artifact Hub. So every changelog section is +generated: never edit the changelogs by hand or add an `[Unreleased]` section (release.sh +refuses one). release.sh's `0.0.0`-placeholder path is only for a new artifact's first +release. **CI.** `ci.yml` is the blocking gate: `pre-commit`, `scripts` and `actionlint` on every PR, the Ansible, chart, compose and cloud-init jobs path-filtered, KICS on the first three (KICS has diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5d25344..ffa8d69 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -12,9 +12,9 @@ the subjects into the collections' and the chart's changelogs at release (see [RELEASING.md](RELEASING.md)). Write a title an operator can read as a changelog entry; the `pr-title.yml` workflow refuses a title that is not conventional, with a lowercase type, one of `feat` `fix` `perf` `refactor` `docs` `chore` `revert` `security` `ci` -`test` `style` `build`. Until an -artifact's first release, also add an entry under `[Unreleased]` in its changelog by -hand: that section becomes the first release's notes. +`test` `style` `build`. Never edit a released artifact's changelog by hand: every +section after its first is generated. Only a new artifact's first section is written by +hand, under `[Unreleased]` (see [RELEASING.md](RELEASING.md)). ## One-time setup diff --git a/README.md b/README.md index 8422598..1b1ae05 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,8 @@ [![CI](https://github.com/decdn/devops/actions/workflows/ci.yml/badge.svg)](https://github.com/decdn/devops/actions/workflows/ci.yml) [![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/decdn-node)](https://artifacthub.io/packages/search?repo=decdn-node) +[![decdn.node on Galaxy](https://img.shields.io/ansible/collection/v/decdn/node?label=decdn.node)](https://galaxy.ansible.com/ui/repo/published/decdn/node/) +[![decdn.publisher on Galaxy](https://img.shields.io/ansible/collection/v/decdn/publisher?label=decdn.publisher)](https://galaxy.ansible.com/ui/repo/published/decdn/publisher/) [![Ansible](https://img.shields.io/badge/Ansible-%E2%89%A5%202.15-1A1918?logo=ansible&logoColor=white)](https://docs.ansible.com/) [![ansible-lint: production](https://img.shields.io/badge/ansible--lint-production-blue)](https://ansible.readthedocs.io/projects/lint/) [![IaC scan: KICS](https://img.shields.io/badge/IaC%20scan-KICS-7B61FF)](https://kics.io/) @@ -22,9 +24,9 @@ Four ways to deploy the node, cache or origin: | Path | For | Start here | |------|-----|------------| | **Docker Compose** (`compose/`), recommended | One host that already runs Docker. The quickest start, [below](#quickstart-docker-compose). Profiles for a cache node or an origin, the onboarding sponsor (`sponsord` + its onramp), an iroh relay, an iroh DNS server and Grafana Alloy, driven by the `decdn-compose` wrapper (set-up, preflight, guarded restarts). | [`compose/README.md`](compose/README.md) | -| **Ansible** (`ansible/`) | VMs and bare metal, one node or a fleet. Hardens the host too (firewall, SSH, patching). Also published as the `decdn.node` and `decdn.publisher` Galaxy collections. | [`ansible/README.md`](ansible/README.md) | +| **Ansible** (`ansible/`) | VMs and bare metal, one node or a fleet. Hardens the host too (firewall, SSH, patching). Also published on Ansible Galaxy as the [`decdn.node`](https://galaxy.ansible.com/ui/repo/published/decdn/node/) and [`decdn.publisher`](https://galaxy.ansible.com/ui/repo/published/decdn/publisher/) collections. | [`ansible/README.md`](ansible/README.md) | | **cloud-init** (`cloud-init/`) | One VM, no control machine: paste the user-data into your provider's "create server" form. It runs the Ansible playbook on the host itself, hardening included. One template per kind of operator: a cache node, or a publisher's origin with the onboarding sponsor (`sponsord` + its onramp). | [`cloud-init/README.md`](cloud-init/README.md) | -| **Helm** (`charts/decdn-node/`) | Kubernetes, one release per node, cache or origin. | [`charts/decdn-node/README.md`](charts/decdn-node/README.md) | +| **Helm** (`charts/decdn-node/`) | Kubernetes, one release per node, cache or origin. Published to `oci://ghcr.io/decdn/charts/decdn-node`, signed, and listed on [Artifact Hub](https://artifacthub.io/packages/helm/decdn-node/decdn-node). | [`charts/decdn-node/README.md`](charts/decdn-node/README.md) | The iroh relay and DNS server a publisher may run deploy with Ansible or Compose. diff --git a/RELEASING.md b/RELEASING.md index 13cb575..81578b0 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -10,8 +10,8 @@ The repo publishes three artifacts, each on its own version and released by its Release an artifact when it has changes; the others are not touched. Their versions are independent. The one dependency is Galaxy's: `decdn.publisher` requires `decdn.node` -(its playbooks run `decdn.node`'s baseline, Alloy and origin nodes), so the publisher -collection's first release comes after the node collection's. `scripts/release.sh` +(its playbooks run `decdn.node`'s baseline, Alloy and origin nodes), so a publisher +release that raises its `decdn.node` floor comes after that node release. `scripts/release.sh` refuses any publisher release whose `decdn.node: ">=X.Y.Z"` no `node-collection-v*` tag on origin satisfies yet. A tag is not a Galaxy upload, so the publish job also asks Galaxy, before uploading a `decdn.publisher`, for a `decdn.node` that satisfies @@ -26,8 +26,8 @@ Each workflow's `build` job runs on every matching tag push. It gates the tag (`scripts/check-release-version.sh`), re-runs that artifact's checks (`make galaxy-check-node`, `make galaxy-check-publisher` or `make lint-helm`), packages it, and uploads it with a `SHA256SUMS` as a workflow artifact. The `publish` job pushes it to Galaxy or ghcr.io and creates the GitHub -Release, but **only** when the repository variable `PUBLISH_ENABLED` is `true`. Until -then, a tag push is a dry run. So is a manual *Run workflow* (`workflow_dispatch`) with an +Release, but **only** when the repository variable `PUBLISH_ENABLED` is `true`, as it +is now; set it to anything else and a tag push is a dry run. So is a manual *Run workflow* (`workflow_dispatch`) with an existing tag. Before a tag exists, the rehearsal is `scripts/release.sh`'s dry run and the artifact's make target (below). The repo's *Latest* release is the highest `decdn.node` version, the node operator's collection: a node collection release is marked Latest only @@ -114,7 +114,7 @@ change lands in several, run the script once for each. **Prerequisites.** - [git-cliff](https://git-cliff.org) 2.9 or later, for `require_conventional` - (`cargo install git-cliff`; CI tests with 2.14.1). The first release does not use it. + (`cargo install git-cliff`; CI tests with 2.14.1). - Git configured to sign, with `user.signingkey` set explicitly. The script signs a throwaway commit object first, so a key that cannot sign fails before the checks. - A clean `main` equal to `origin/main`, and no local tag of the artifact above @@ -126,13 +126,13 @@ change lands in several, run the script once for each. as the squash commit's subject (*Settings → General → Pull Requests*, and the `main` ruleset's allowed merge methods), and `pr-title` a required status check. -**The first release** of each artifact starts from the `0.0.0` placeholder in its -manifest and names its level: `scripts/release.sh chart minor --execute` cuts `0.1.0`. -Without a tag, git-cliff would render the whole history, so the first release does not -use it. It renames the changelog's hand-written `## [Unreleased]` section to the dated -version instead, so until then, changes are logged there by hand. After that, no -`[Unreleased]` section is kept: the script refuses one, because every later section is -generated. +**A first release.** All three artifacts are past theirs. A new artifact starts from +the `0.0.0` placeholder in its manifest and names its level: +`scripts/release.sh minor --execute` cuts `0.1.0`. Without a tag, git-cliff +would render the whole history, so the first release does not use it. It renames the +changelog's hand-written `## [Unreleased]` section to the dated version instead, so +until then, changes are logged there by hand. After that, no `[Unreleased]` section is +kept: the script refuses one, because every later section is generated. The chart's workflow file name (`release-chart.yml`) and tag shape (`decdn-node-X.Y.Z`) are both part of its cosign certificate identity and of its attestations' signer @@ -141,6 +141,10 @@ either once a chart is published. ## First release only +Done for all three artifacts: the Galaxy namespace and key, the `release` environment, +`PUBLISH_ENABLED`, the public GHCR package and the Artifact Hub listing with its +`repositoryID`. Kept for a new artifact, or to rebuild the setup. + Before the first publish: - **Galaxy namespace.** The `decdn` namespace must exist on galaxy.ansible.com and the diff --git a/ansible/README.md b/ansible/README.md index f5368f5..68b897a 100644 --- a/ansible/README.md +++ b/ansible/README.md @@ -1,8 +1,9 @@ # ansible — deCDN deployment Declarative Ansible project for deploying deCDN nodes, one or a fleet, over a -hardened host baseline. The roles also ship as two Galaxy collections: `decdn.node` -(what a node operator runs) and `decdn.publisher` (what a publisher runs beside its +hardened host baseline. The roles also ship as two collections on Ansible Galaxy: +[`decdn.node`](https://galaxy.ansible.com/ui/repo/published/decdn/node/) (what a node operator runs) and +[`decdn.publisher`](https://galaxy.ansible.com/ui/repo/published/decdn/publisher/) (what a publisher runs beside its origin nodes). | Playbook | Purpose | Make target | @@ -513,8 +514,9 @@ the RPC URL when it is not provisioned on the host instead). Highlights: ## Packaging as Galaxy collections (`decdn.node`, `decdn.publisher`) -The seven roles are also packaged as two distributable collections, for operators who -bring their own playbooks: +The seven roles are also packaged as two collections, published on Ansible Galaxy, for +operators who bring their own playbooks (`ansible-galaxy collection install +decdn.node` or `decdn.publisher`): | Collection | Roles | For | |---|---|---| @@ -542,5 +544,4 @@ each collection on its own version (the Helm chart is released separately), cut `main` by `scripts/release.sh`: see [RELEASING.md](../RELEASING.md). Each `galaxy//CHANGELOG.md` section is generated at release from the conventional commit subjects that touched that collection's roles, its overlay or -`galaxy/build.sh`, so give the PR a title that reads as a changelog entry. Until a -collection's first release, also log the change under its `[Unreleased]` by hand. +`galaxy/build.sh`, so give the PR a title that reads as a changelog entry. diff --git a/ansible/galaxy/node/README.md b/ansible/galaxy/node/README.md index 7a39b67..208c6ad 100644 --- a/ansible/galaxy/node/README.md +++ b/ansible/galaxy/node/README.md @@ -25,17 +25,7 @@ collection, which depends on this one. ## Install -> **Not on Galaxy yet.** The first release (`node-collection-v0.1.0`) has not been -> cut; see [RELEASING.md](https://github.com/decdn/devops/blob/main/RELEASING.md). -> Until then, build and install it from a checkout: -> -> ```bash -> git clone https://github.com/decdn/devops && cd devops/ansible -> make build-node -> ansible-galaxy collection install build/decdn-node-*.tar.gz -> ``` - -Once published: +From [Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/decdn/node/): ```bash ansible-galaxy collection install decdn.node @@ -49,6 +39,20 @@ collections: version: ">=0.1.0" ``` +Each release is listed in the +[CHANGELOG](https://github.com/decdn/devops/blob/main/ansible/galaxy/node/CHANGELOG.md) +and as a `node-collection-vX.Y.Z` +[GitHub Release](https://github.com/decdn/devops/releases) with the tarball and its +`SHA256SUMS`. To run changes not released yet, build it from a checkout. The build +carries the last release's version until the next one, so `--force` replaces an +installed release: + +```bash +git clone https://github.com/decdn/devops && cd devops/ansible +make build-node +ansible-galaxy collection install --force build/decdn-node-*.tar.gz +``` + ## Usage A minimal node playbook — baseline first (so the admin key lands before SSH diff --git a/ansible/galaxy/publisher/README.md b/ansible/galaxy/publisher/README.md index 24424af..3bbbd76 100644 --- a/ansible/galaxy/publisher/README.md +++ b/ansible/galaxy/publisher/README.md @@ -28,21 +28,8 @@ repository — four roles and nothing else: ## Install -> **Not on Galaxy yet.** The first release (`publisher-collection-v0.1.0`) has not -> been cut, and it follows `decdn.node`'s; see -> [RELEASING.md](https://github.com/decdn/devops/blob/main/RELEASING.md). Until then, -> build and install both from a checkout. A checkout's builds are both `0.0.0`, below -> the `decdn.node >=0.1.0` this collection declares, so install the node collection -> first (which pulls its own dependencies) and this one without dependency resolution: -> -> ```bash -> git clone https://github.com/decdn/devops && cd devops/ansible -> make build -> ansible-galaxy collection install build/decdn-node-*.tar.gz -> ansible-galaxy collection install --no-deps build/decdn-publisher-*.tar.gz -> ``` - -Once published: +From [Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/decdn/publisher/), +which also installs `decdn.node`: ```bash ansible-galaxy collection install decdn.publisher @@ -56,6 +43,21 @@ collections: version: ">=0.1.0" ``` +Each release is listed in the +[CHANGELOG](https://github.com/decdn/devops/blob/main/ansible/galaxy/publisher/CHANGELOG.md) +and as a `publisher-collection-vX.Y.Z` +[GitHub Release](https://github.com/decdn/devops/releases) with the tarball and its +`SHA256SUMS`. To run changes not released yet, build both collections from a checkout +and install the node collection first. The builds carry the last releases' versions +until the next ones, so `--force` replaces installed releases: + +```bash +git clone https://github.com/decdn/devops && cd devops/ansible +make build +ansible-galaxy collection install --force build/decdn-node-*.tar.gz +ansible-galaxy collection install --force build/decdn-publisher-*.tar.gz +``` + ## Usage A minimal sponsor playbook — `decdn.node`'s baseline first (so the admin key lands diff --git a/charts/decdn-node/README.md b/charts/decdn-node/README.md index b4a02e3..2ef026f 100644 --- a/charts/decdn-node/README.md +++ b/charts/decdn-node/README.md @@ -158,14 +158,19 @@ config: ``` ```bash -# From a checkout of this repo: -helm install decdn-node-1 charts/decdn-node -n decdn -f values-node-1.yaml - -# From the OCI registry, at a released version (CHANGELOG.md): +# From the OCI registry, at a released version: helm install decdn-node-1 oci://ghcr.io/decdn/charts/decdn-node --version X.Y.Z \ -n decdn -f values-node-1.yaml + +# Changes not released yet, from a checkout of this repo: +helm install decdn-node-1 charts/decdn-node -n decdn -f values-node-1.yaml ``` +The chart is listed on +[Artifact Hub](https://artifacthub.io/packages/helm/decdn-node/decdn-node), and each +version's changes are in its +[CHANGELOG](https://github.com/decdn/devops/blob/main/charts/decdn-node/CHANGELOG.md). + Published charts are signed with cosign, and every release after 0.1.0 also carries build-provenance attestations; [RELEASING.md](https://github.com/decdn/devops/blob/main/RELEASING.md#verifying-a-release) diff --git a/docs/node-operators.md b/docs/node-operators.md index f15859e..cf2c1c0 100644 --- a/docs/node-operators.md +++ b/docs/node-operators.md @@ -21,11 +21,12 @@ loopback, and the chain's contract addresses taken from upstream's deployment ma | **Docker Compose** (one Docker host), recommended | [`compose/README.md`](../compose/README.md) | `decdn-compose init node` | | **Ansible** (VMs, bare metal, a fleet) | [`ansible/README.md`](../ansible/README.md) | `inventory/hosts-node.yml.example`, `make deploy-node` (`playbooks/node.yml`) | | **cloud-init** (one VM, no control machine) | [`cloud-init/README.md`](../cloud-init/README.md) | `cloud-init/user-data-node.yaml` | -| **Helm** (Kubernetes, one release per node) | [`charts/decdn-node/README.md`](../charts/decdn-node/README.md) | the chart, with no `config.cache.origin` | +| **Helm** (Kubernetes, one release per node) | [`charts/decdn-node/README.md`](../charts/decdn-node/README.md) | `oci://ghcr.io/decdn/charts/decdn-node`, with no `config.cache.origin` | [`requirements.md`](requirements.md) has the hardware, network and platform requirements, -and compares the paths. The Galaxy collection for your own playbooks is -[`decdn.node`](../ansible/galaxy/node/README.md). +and compares the paths. For your own playbooks, the roles are the +[`decdn.node`](../ansible/galaxy/node/README.md) collection on +[Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/decdn/node/) (`ansible-galaxy collection install decdn.node`). A node without an origin backend fills a miss by pulling from other nodes (`node_to_node_pull_through_enabled`). The Ansible role and the chart turn that on for diff --git a/docs/publishers.md b/docs/publishers.md index b61f0c6..1460589 100644 --- a/docs/publishers.md +++ b/docs/publishers.md @@ -30,9 +30,10 @@ Only the origin is required. With Ansible, start from `ansible/inventory/hosts-publisher.yml.example`; `make deploy-publisher` (`playbooks/publisher.yml`) deploys all four in that order, and each has its own target (`deploy-origin`, `deploy-sponsord`, `deploy-relay`, `deploy-dns`). The roles also ship as -the [`decdn.publisher`](../ansible/galaxy/publisher/README.md) Galaxy collection, which -depends on [`decdn.node`](../ansible/galaxy/node/README.md) for the baseline, observability -and the origin node itself. +the [`decdn.publisher`](../ansible/galaxy/publisher/README.md) collection on +[Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/decdn/publisher/) (`ansible-galaxy collection install decdn.publisher`), +which depends on [`decdn.node`](../ansible/galaxy/node/README.md) for the baseline, +observability and the origin node itself. ## Origin nodes