diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9bcfeb5..5a1a948 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -283,6 +283,11 @@ jobs: exit 1 fi + # Skips while the published template has no bitbucket-pipelines.yml. + - name: go test -run TestTemplatePipelines + if: steps.template.outputs.public == 'true' + run: TOUCHMARK_E2E_TEMPLATE="$PWD/.ci/template" go test -count=1 -v -run '^TestTemplatePipelines$' ./internal/e2e/bitbucket/ + release-snapshot: # The release without publishing anything (docs/project/release.md): # the scripts of the release and of the Action by their tests; diff --git a/docs/agents.md b/docs/agents.md index 203b7c8..7ef9a4b 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -47,8 +47,8 @@ no lock file, no header, no marker in their files. It never checks a target out and never runs a script, hook, filter or LFS from the hub or a target. It delivers nothing to a repository without the opt-in file, unless `targets.yml` subscribes it (`opt_in: assumed`). There is no pull mode (targets do not -fetch from the hub), no remote pack sources, no Azure DevOps driver, and no hub on -Bitbucket Pipelines: Bitbucket Cloud is a provider a hub elsewhere delivers to. +fetch from the hub), no remote pack sources, no Azure DevOps driver, and no `setup` for +Bitbucket: a hub on Bitbucket Cloud runs in Bitbucket Pipelines, set up by hand. ## Mental model diff --git a/docs/concepts/security.md b/docs/concepts/security.md index 0dbc42c..c9f47e3 100644 --- a/docs/concepts/security.md +++ b/docs/concepts/security.md @@ -34,7 +34,7 @@ reviewed merges reach. `security.write_isolation` in `hub.yml` says how: | Mode | The write key | touchmark checks | |---|---|---| -| `platform` (default) | in the hub's CI: GitHub's environment `touchmark-distribute` limited to the default branch; GitLab's protected, masked, hidden variable scoped to that environment | a probe in the jobs any branch can start; on GitHub also the environment's branch policy, through the hub channel | +| `platform` (default) | in the hub's CI: GitHub's environment `touchmark-distribute` limited to the default branch; GitLab's protected, masked, hidden variable scoped to that environment; on Bitbucket Premium a secured variable of the deployment `touchmark-distribute`, which only the default branch may deploy to | a probe in the jobs any branch can start; on GitHub also the environment's branch policy, through the hub channel; on Bitbucket the hub's statement in `security.reason`, since the API does not show the deployment permissions | | `external` | released by an external secret store through the CI's OIDC token, bound to the default branch (GitHub `sub` = `repo:ORG/HUB:ref:refs/heads/main` or `…:environment:touchmark-distribute`; GitLab `project_path`, `ref`, `ref_type`, `ref_protected`) | the job's own context | | `none` | anywhere; the risk is accepted | nothing; `security.reason` is required, and every report and `doctor` show it as a warning | @@ -52,10 +52,20 @@ their own workflow instead of running yours. So the probe runs where an attacker - **Gitea and Forgejo Actions:** every branch sees every secret, so `platform` cannot hold and `distribute` refuses it there. Run the hub's CI on GitHub or GitLab, or use `external`, or `none` with every branch protected. +- **Bitbucket Pipelines:** `touchmark probe` runs in every pipeline, in a step without + the deployment, and fails if it sees a write key: a repository or workspace variable + reaches every branch. A deployment variable reaches any step that names the + deployment, in any branch's `bitbucket-pipelines.yml`, unless the environment's + deployment permissions (Premium) admit the default branch alone; Bitbucket's API does + not show them, so under `platform` `distribute` and `doctor` refuse to run unless + `security.reason` states them. Without Premium use `none` with a reason, or + `external`. See [A hub on Bitbucket Cloud](../getting-started/bitbucket.md). **The guards** of `distribute` (and of `doctor` in CI), each exit 2 with nothing written: the run is not on the default branch, or is a pull request event; on GitLab the ref is -not protected or the job is not in the environment `touchmark-distribute`; the probe saw +not protected or the job is not in the environment `touchmark-distribute`; on Bitbucket +the step has no `deployment: touchmark-distribute`, the default branch could not be read, +or `platform` comes without the statement in `security.reason`; the probe saw a write key or did not run; an operation flag was given in CI; git is older than 2.45; the write key belongs to another account than `hub.yml`'s `writer`. `plan` in CI exits 2 when it sees any write key. A run on a commit that is no longer the tip of the default diff --git a/docs/getting-started/bitbucket.md b/docs/getting-started/bitbucket.md new file mode 100644 index 0000000..0aef5e1 --- /dev/null +++ b/docs/getting-started/bitbucket.md @@ -0,0 +1,182 @@ +# A hub on Bitbucket Cloud + +A hub hosted on Bitbucket Cloud, whose CI is Bitbucket Pipelines: the template's +`bitbucket-pipelines.yml` checks every pull request to the hub and delivers from the +default branch. Bitbucket Data Center is not supported. + +!!! warning "Not verified on a live Bitbucket yet" + The Bitbucket support is built from Atlassian's documentation and tested against a + stand-in of its API, not on a live workspace. The points a first live run must + confirm are listed under [Assumed, not yet seen live](#assumed-not-yet-seen-live). + +`touchmark setup` has nothing for Bitbucket yet: the steps below are by hand. + +## Where the write key can live + +Bitbucket Pipelines gives a step the variables of a deployment environment only when the +step names it (`deployment: touchmark-distribute`). But any branch's copy of +`bitbucket-pipelines.yml` can name it too, so the write key stays on the default branch +only when the environment's **deployment permissions** stop the steps of other branches +([they pause](https://support.atlassian.com/bitbucket-cloud/docs/set-up-and-monitor-deployments/)). +Those permissions are a **Premium** feature, and Bitbucket's API does not show them, so +touchmark cannot check them. Choose by plan: + +| Plan | `hub.yml` | What keeps the write key | +|---|---|---| +| Premium | `write_isolation: platform` and a `reason` that states the restriction | the deployment `touchmark-distribute` admits the default branch only, and only admins or selected users deploy to it; you state it, touchmark cannot read it | +| Free, Standard | `write_isolation: none` and a `reason` | nothing but who may push: every branch of the hub can read the write key; protect the branches and keep write access to the hub tight | +| any | `write_isolation: external` | an external secret store that releases the key to the default branch only | + +```yaml +# hub.yml of a hub on Bitbucket Premium +version: 1 +id: acme-eng +writer: "{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d}" # the writer account's UUID +security: + write_isolation: platform + reason: "Bitbucket Premium: only admins may deploy to touchmark-distribute, from main only" +``` + +Under `platform` without a `reason`, which is how the template ships, `distribute` and +`doctor` refuse to run on Bitbucket Pipelines (exit 2): a hub on Free or Standard is +never taken for isolated. On every plan the `probe` step fails a pipeline that sees a +write key in a repository or workspace variable. + +## Steps + +1. **Create the hub.** In your workspace choose *Create → Repository → Import + repository*, and give the URL of + [the template](https://github.com/bedrock-python/engineering-assets-template), + `https://github.com/bedrock-python/engineering-assets-template.git`. Keep it private + unless the hub may be public. Turn Pipelines on under *Repository settings → + Pipelines → Settings*. +2. **Create two bot accounts**, Atlassian accounts that only touchmark uses, each with an + API token (Atlassian account settings → *Security → API tokens*, with scopes): + - the *reader*, with read access to your target repositories and the scopes + `read:user`, `read:workspace`, `read:repository`, `read:pullrequest` (all + `…:bitbucket`); + - the *writer*, with write access to the targets only (not admin, and no access to + the hub), and the reader's scopes plus `write:repository` and `write:pullrequest`. + A token's scopes do not narrow the repositories it reaches, the account's permissions + do. See [Bitbucket Cloud](../guide/providers.md#bitbucket-cloud) for the details. +3. **Create the hub's access token.** Under *Repository settings → Security → Access + tokens* of the hub, create one with *Repositories: Read* and *Pull requests: Write*. + Pipelines gives a step no API token of its own: touchmark reads the hub's default + branch, its tip and visibility with this token, and keeps `plan`'s comment in the hub + pull request with it. It reaches the hub alone, and every pipeline of the hub sees + it, so give it nothing more; `Pull requests: Write` also lets it merge, so restrict + who may merge into the default branch (step 5). +4. **Store the keys** under *Repository settings → Pipelines → Repository variables* + and *→ Deployments*, all **Secured**: + - repository variables: `TOUCHMARK_READ_TOKEN` (the reader's API token) and + `TOUCHMARK_PIPELINES_TOKEN` (the hub's access token); + - create the deployment environment `touchmark-distribute` and store the writer's + API token as its variable `TOUCHMARK_WRITE_TOKEN`, and nowhere else: not as a + repository or workspace variable; + - on Premium, open the environment's settings: allow deployments from the default + branch only, **and** let only admins deploy (*Only allow admins to deploy to this + environment*) or only named users or groups (*Allow selected users or groups to + deploy to this environment*). The branch alone is not enough: anyone with write + access to the hub can run the custom pipelines on the default branch and pass + them variables ([below](#what-the-pipelines-do)). + The hub delivers to Bitbucket Cloud with these one-provider names; with `providers` + in `hub.yml` they carry the provider's id (`TOUCHMARK_BB_WRITE_TOKEN`). +5. **Protect the hub.** Under *Repository settings → Branch restrictions*, let no one + push to the default branch directly and only your maintainers merge into it; keep the + bot accounts and the hub's access token out of those lists. Bitbucket reads none of + the template's CODEOWNERS files: replace `@acme/hub-maintainers` in them with your + team, or delete them with the other platforms' CI files (`touchmark check` warns + about the placeholder). +6. **Schedule the runs.** Under *Pipelines → Schedules*, create two schedules on the + default branch: a daily one for the custom pipeline `distribute`, and a weekly one + for `doctor`. +7. **Describe your hub.** In `hub.yml` set `id`, `writer` to the writer account's UUID + in quotes (`GET https://api.bitbucket.org/2.0/user` with its token shows it), and + `security` as above. List your repositories in `targets.yml` (`org: ` with + `match`, or `repo: /`; Bitbucket has no topics). +8. **Open a pull request** and check the `probe`, `check` and `plan` steps. `plan` + keeps its report in one comment of the pull request, and the report files are the + step's artifacts. +9. **Merge it.** The push pipeline of the default branch runs `probe`, then + `distribute`, which opens a pull request in every repository that has + [opted in](opt-in.md). *Run pipeline* runs the custom pipelines `distribute` and + `doctor` by hand. + +## What the pipelines do + +| Pipeline | Steps | +|---|---| +| pull request to the hub | `probe`, `check`, `plan --strict --comment` | +| push to `main` or `master`, the custom pipeline `distribute` | `probe`, then `distribute` with `deployment: touchmark-distribute` | +| the custom pipeline `doctor` | `probe`, then `doctor` with `deployment: touchmark-distribute` | + +- **The default branch.** Pipelines names no default branch, so touchmark reads it with + `TOUCHMARK_PIPELINES_TOKEN` (a public hub answers without it) and refuses + `distribute` and `doctor` on any other branch, or when it cannot read it. The push + pipeline starts on `main` and `master`: add your default branch to its pattern if it + has another name. +- **The deployment.** `distribute` and `doctor` run only in a step with + `deployment: touchmark-distribute` (`BITBUCKET_DEPLOYMENT_ENVIRONMENT`). Bitbucket + runs one deployment to an environment at a time, so two runs never overlap. +- **Time.** The `distribute` step may run 6 hours (`max-time: 360`); touchmark starts no + target after 5 hours 30 minutes (`--deadline 5h30m`), and the next run picks up what is + left. +- **Reports.** Bitbucket has no step summary or annotations: the report is in the step's + log, and `touchmark-report.{md,json,jsonl}` and `touchmark-doctor.{md,json}` are the + steps' artifacts (kept 14 days). +- **The plan comment** is written with the hub's access token. Bitbucket shows HTML as + text, so the comment has no folded sections, and its marker is a Markdown link + reference definition, `[touchmark-plan]: # "touchmark plan: "`, which renders + as nothing. An access token cannot tell its own account, so touchmark edits the newest + comment with the marker that Bitbucket lets it edit, and writes a new one otherwise. +- **The fingerprint** of the hub is `bitbucket.org/`, lowercase and + without braces: `bitbucket.org/3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d`. `--hub-fp` and + `previous_fingerprints` also take the UUID as `BITBUCKET_REPO_UUID` gives it, in + braces. +- **Custom pipelines** accept variables when they run: anyone with write access to the + hub can run them, through the website or the API, and a pipeline variable overrides a + deployment variable. The template's scripts reference no variable, so the website asks + for none, but the API takes any. touchmark's git ignores the variables that would + change its config or the programs it runs (`GIT_CONFIG_*`, `GIT_SSH_COMMAND`, + `LD_PRELOAD`, …; see the [threat model](../project/threat-model.md#tokens-and-git)), + but whoever runs the step chooses its whole environment. So restrict the deployment + `touchmark-distribute` to admins or named users, not only to the default branch: on + Premium, deployment permissions pause a run by anyone else. + +## Check it + +`touchmark doctor --hub-token`, run from a clone of the hub, reads where the keys live +with the token in `TOUCHMARK_HUB_TOKEN`: a one-off access token of the hub with +*Repositories: Read* and *Pipelines: Read*, or a maintainer's API token with +`read:repository:bitbucket` and `read:pipeline:bitbucket` (not the pipelines' own +token, which needs no Pipelines scope). It finds that a write key in a repository or workspace variable fails, one in the +deployment `touchmark-distribute` is unknown (the API does not show who may deploy; it +is `ok` when the environment admits only admins), and a variable that is not secured +warns. Workspace variables need a workspace administrator's token; without one they are +reported as not read. + +## Assumed, not yet seen live + +- Pipelines runs the step scripts with the image's shell, not its `touchmark` + entrypoint, and the image's user (uid 65532) may write the report files into the + clone. If not, set `run-as-user` on the steps. +- The `plan` step's `git fetch` writes into the clone's `.git`, and Bitbucket clones as + root ([BCLOUD-23217](https://jira.atlassian.com/browse/BCLOUD-23217)): if Pipelines + runs the step as the image's user (uid 65532), the fetch may fail with a permission + error. Then set `run-as-user: 0` on the `plan` step. +- `git fetch origin` works in a pull request's step with Pipelines' own credentials. +- `BITBUCKET_DEPLOYMENT_ENVIRONMENT` holds the environment's name (`touchmark-distribute`). +- An access token with *Pull requests: Write* may comment on the hub's pull requests, + and a `PUT` of another account's comment answers 403. +- Bitbucket renders a link reference definition in a comment as nothing. +- `restrictions.admin_only` of a deployment environment, which the API returns but does + not document. +- Custom pipeline variables via the API: a run started through the API with variables + the YAML does not declare gets them in its environment, overriding deployment + variables, and a deployment restricted to admins or selected users pauses it when + someone else starts it. +- A scheduled run, and the push pipeline after a merge, pass a deployment restricted to + admins or selected users; Atlassian says deployment permissions hold scheduled and + automated deployments too, until an allowed user resumes them. + +Next: [opt a repository in](opt-in.md). diff --git a/docs/guide/ci.md b/docs/guide/ci.md index 1946f70..53a44b6 100644 --- a/docs/guide/ci.md +++ b/docs/guide/ci.md @@ -1,7 +1,8 @@ # Run it in CI The [hub template](https://github.com/bedrock-python/engineering-assets-template) ships -the CI files for GitHub Actions, GitLab CI, Gitea and Forgejo Actions. Copy them rather +the CI files for GitHub Actions, GitLab CI, Gitea and Forgejo Actions, and Bitbucket +Pipelines; each platform reads only its own. Copy them rather than writing your own: the jobs, the probe and the environment fit together. This page explains what they rely on. @@ -132,6 +133,57 @@ runner's 3-hour limit, and `doctor` weekly in a workflow of its own. Remember th and Forgejo Actions give every branch every secret: see [A hub on Gitea or Forgejo](../getting-started/gitea-forgejo.md). +## Bitbucket Pipelines + +Every step runs in the image, pinned by digest, with the hub's whole history; the +template's `bitbucket-pipelines.yml`: + +```yaml +image: ghcr.io/bedrock-python/touchmark:X.Y.Z@sha256: +clone: + depth: full # touchmark reads the hub's whole history + +pipelines: + pull-requests: + "**": + - step: { name: probe, clone: { enabled: false }, script: [touchmark probe] } + - step: { name: check, script: [touchmark check --hub .] } + - step: + name: plan + script: + - git fetch --no-tags origin "+refs/heads/$BITBUCKET_PR_DESTINATION_BRANCH:refs/remotes/origin/$BITBUCKET_PR_DESTINATION_BRANCH" + - touchmark plan --hub . --strict --comment + artifacts: [touchmark-report.md, touchmark-report.json] + branches: + "{main,master}": + - step: { name: probe, clone: { enabled: false }, script: [touchmark probe] } + - step: + name: distribute + deployment: touchmark-distribute + max-time: 360 + script: [touchmark distribute --hub . --deadline 5h30m] + artifacts: [touchmark-report.md, touchmark-report.json, touchmark-report.jsonl] + custom: + distribute: [...] # probe, then distribute: Run pipeline and a daily schedule + doctor: [...] # probe, then doctor in the deployment: a weekly schedule +``` + +- **The write key** is a secured variable of the deployment `touchmark-distribute`, which + only a step with `deployment: touchmark-distribute` gets. touchmark checks + `BITBUCKET_DEPLOYMENT_ENVIRONMENT` in `distribute` and `doctor`. Which branches may + deploy is a Premium setting the API does not show: see + [A hub on Bitbucket Cloud](../getting-started/bitbucket.md#where-the-write-key-can-live). +- **The hub channel** uses `TOUCHMARK_PIPELINES_TOKEN`, an access token of the hub: + Pipelines names no default branch and gives a step no API token, so touchmark reads + the default branch, its tip and the hub's visibility through the API, and keeps + `plan`'s comment with it. +- **`probe`** runs without the deployment at the start of every pipeline, so a write key + in a repository or workspace variable stops it before anything else. +- **Schedules** are set up in the website (*Pipelines → Schedules*) for the custom + pipelines `distribute` (daily) and `doctor` (weekly), on the default branch. +- **The hub's fingerprint** is `bitbucket.org/` and `BITBUCKET_REPO_UUID` without its + braces, lowercase. + ## Reports, summaries and annotations In CI, `plan` and `distribute` leave their report as `touchmark-report.json` and @@ -149,6 +201,10 @@ job's artifacts. the job's log and its artifacts (`expose_as` shows it on the merge request). - **Gitea 1.27 with runner 2.0:** the step summary. **Forgejo:** the log and the artifacts. +- **Bitbucket Pipelines:** no step summary or annotations: the log and the step's + artifacts. `plan --comment` keeps the report in one comment of the hub pull request + with `TOUCHMARK_PIPELINES_TOKEN`, without HTML (Bitbucket shows it as text), its marker + a Markdown link reference definition. In a public hub, targets that are not public are only counted in all of these outputs, never named. diff --git a/docs/guide/local.md b/docs/guide/local.md index 5c0ec72..18a4752 100644 --- a/docs/guide/local.md +++ b/docs/guide/local.md @@ -104,7 +104,7 @@ last commit (`check`, `plan`, `status`, `apply`). `plan` needs the read key, the hub's fingerprint, and providers it can name: a `providers` list or `platform` (and `base_url`) in `hub.yml`, or an origin on github.com, -a `*.ghe.com` host or gitlab.com. +a `*.ghe.com` host, gitlab.com or bitbucket.org. ```sh export TOUCHMARK_READ_TOKEN=… # or TOUCHMARK_READ_APP_ID and TOUCHMARK_READ_APP_KEY @@ -113,7 +113,9 @@ touchmark plan --hub . --hub-fp github.com/712345678 --only acme/billing --forma ``` The fingerprint is the hub's host and repository id: on GitHub -`gh api repos/OWNER/HUB --jq .id`, on GitLab the project id on the project's page. A +`gh api repos/OWNER/HUB --jq .id`, on GitLab the project id on the project's page, on +Bitbucket the repository's UUID (`bitbucket.org/{…}`, its `uuid` in +`GET https://api.bitbucket.org/2.0/repositories/WORKSPACE/HUB`). A local run cannot read the tip of the hub's default branch, and says so in a warning. `plan` needs git 2.45 or newer, as `distribute` does: both read every target with it. diff --git a/docs/guide/providers.md b/docs/guide/providers.md index 1afc874..c70a988 100644 --- a/docs/guide/providers.md +++ b/docs/guide/providers.md @@ -3,7 +3,7 @@ One hub can deliver to GitHub, GitLab, Gitea and Forgejo at once, including self-managed instances. Each platform is a **provider** in `hub.yml`, with its own reader and writer; `targets.yml` names repositories by provider. Bitbucket Cloud is a provider -too, delivered to from a hub on another platform: see [Bitbucket Cloud](#bitbucket-cloud). +too, and can host the hub: see [Bitbucket Cloud](#bitbucket-cloud). ## One provider: the shorthand @@ -129,9 +129,10 @@ covers it. ## Bitbucket Cloud A provider of type `bitbucket` is Bitbucket Cloud at `https://bitbucket.org` (leave `url` -out). `plan`, `distribute` and `doctor` work on it from a hub on GitHub or GitLab, or -wherever you run touchmark; a hub whose own CI is Bitbucket Pipelines is not supported -yet, and `setup` has nothing for it. Bitbucket Data Center is not supported. +out). `plan`, `distribute` and `doctor` work on it from a hub on GitHub or GitLab, from +a hub on Bitbucket itself, whose CI is Bitbucket Pipelines (see +[A hub on Bitbucket Cloud](../getting-started/bitbucket.md)), or wherever you run +touchmark; `setup` has nothing for it yet. Bitbucket Data Center is not supported. ```yaml # hub.yml @@ -143,7 +144,8 @@ providers: **Accounts.** Make two bot accounts (Atlassian accounts that only touchmark uses), each with an API token; app passwords are gone, and repository, project and workspace access -tokens are not supported, since they cannot tell who they are. A token's scopes do not +tokens are not supported as reader or writer, since they cannot tell who they are (a +hub on Bitbucket uses one for its own repository only, `TOUCHMARK_PIPELINES_TOKEN`). A token's scopes do not narrow the repositories it reaches, the account's permissions do: that is why the writer is an account of its own. diff --git a/docs/project/e2e.md b/docs/project/e2e.md index 0decf3f..5216d92 100644 --- a/docs/project/e2e.md +++ b/docs/project/e2e.md @@ -261,7 +261,7 @@ The tests read GitLab from the environment. Without `TOUCHMARK_E2E_GITLAB_URL`, ## The hub template -The hub template ([engineering-assets-template](https://github.com/bedrock-python/engineering-assets-template), a repository of its own) ships CI files for GitHub, GitLab and Gitea that run touchmark from its release image. Its jobs are run for real, from the template's working tree, every job in the touchmark image under test, as that image's user: +The hub template ([engineering-assets-template](https://github.com/bedrock-python/engineering-assets-template), a repository of its own) ships CI files for GitHub, GitLab, Gitea and Bitbucket that run touchmark from its release image. Its jobs are run for real, from the template's working tree, every job in the touchmark image under test, as that image's user: ```sh # the image built from this working tree's Dockerfile @@ -273,9 +273,11 @@ bash scripts/e2e/gitlab.sh --template ../engineering-assets-template --touchmark docker run --rm --name touchmark-e2e-template-github -v "$PWD:/src:ro" -v "$PWD/../engineering-assets-template:/template:ro" -w /src \ -e TOUCHMARK_E2E_TEMPLATE=/template golang:1.27 \ sh -c 'git config --global --add safe.directory /src && go test -count=1 -run TestTemplateWorkflow ./internal/e2e/github/' +# the Bitbucket Pipelines file: its pipelines and steps, played without Bitbucket, on any OS +TOUCHMARK_E2E_TEMPLATE=../engineering-assets-template go test -count=1 -run TestTemplatePipelines ./internal/e2e/bitbucket/ ``` -Without `--run`, `--template` adds `TestTemplate` to the whole suite. Without `--template`, `TestTemplate` skips, and so does `TestTemplateWorkflow` without `TOUCHMARK_E2E_TEMPLATE`. `gitea.sh` runs Gitea's runner, so `--template` applies to the Gitea images; on a Forgejo image the script says so and `TestTemplate` skips. +Without `--run`, `--template` adds `TestTemplate` to the whole suite. Without `--template`, `TestTemplate` skips, and so do `TestTemplateWorkflow` and `TestTemplatePipelines` without `TOUCHMARK_E2E_TEMPLATE` (`TestTemplatePipelines` also when the template has no `bitbucket-pipelines.yml`). `gitea.sh` runs Gitea's runner, so `--template` applies to the Gitea images; on a Forgejo image the script says so and `TestTemplate` skips. ### What `--template` adds (`scripts/e2e/template-lib.sh`) @@ -300,6 +302,8 @@ The cleanup removes the runner, the registry, the forwarder and the image tags, - **`TestTemplate` on Gitea** (`internal/e2e/template_test.go`), the template README's "On Gitea or Forgejo": the hub (the template in one `[skip ci]` commit, the job images pinned to the image under test), the reader's and writer's tokens as Actions secrets; the pull request that sets `id`, `writer` and `security.write_isolation: none` with the template's reason: `check` and `plan` pass (`plan` with `--strict`, as a pull request from the hub's own branch gets the secrets), `distribute` is skipped, and `plan --comment` leaves its report in a comment through the job token; the merge: `distribute` alone, pull requests by the writer in both targets; Run workflow (`POST …/actions/workflows/{file}/dispatches`) of the doctor workflow and of the main one. No job log holds a token. - **`TestTemplateWorkflow`** (`internal/e2e/github/template_test.go`): a dry run of `.github/workflows/engineering-assets.yml` against `ghfake` as GHES, without GitHub and without act. The test plays the runner: each job's `if` (a small evaluator for the expressions the workflow uses: contexts, `==`, `!=`, `&&`, `||`, `!`, `always()`; strings compare without case, as GitHub's do), `needs`, the secrets and variables a job sees (the repository's, and the environment's when the job names one), the environment's deployment branch policy, each job's `GITHUB_TOKEN` (an App installed on the hub alone, minted with the job's `permissions`), `actions/checkout` (a pull request's merge commit), `run:` steps in bash with `GITHUB_OUTPUT`, and the touchmark Action: this tree's `scripts/action/run.sh` with the step's inputs (the defaults from `action.yml`), the version of a release and env, a `gh` on `PATH` that accepts the attestation of that release's digest, and a `docker` that resolves the release's tag to that digest and runs touchmark, built from this tree, by that digest, with only the environment and the user `run.sh` gives the container. The runs: the pull request that describes the hub (`check`, `plan` and its comment), its merge (`probe`, `distribute`), the doctor schedule, the daily schedule (nothing to change), Run workflow; Dependabot's pull request that bumps the Action and a pull request from a fork, which get no Actions secrets (a fork's `GITHUB_TOKEN` read-only too): `plan` runs offline, without `--strict`, and passes with its warning; then a write key leaked into a repository secret: the plan job's probe step fails, and `distribute` and `doctor` refuse (`TOUCHMARK_KEY_EXPOSED=true`). +- **`TestTemplatePipelines`** (`internal/e2e/bitbucket/template_test.go`): no live Bitbucket and no runner. It reads `bitbucket-pipelines.yml` and checks what keeps the write key on the default branch: the image pinned by digest and a full clone; the pipelines (a pull request: `probe`, `check`, `plan --strict --comment`; the push of `main` and `master` and the custom `distribute`: `probe`, then `distribute`; the custom `doctor`: `probe`, then `doctor`; no default or tag pipeline); only `distribute` and `doctor` with `deployment: touchmark-distribute`; no script that names a `TOUCHMARK_` variable, and none in a custom pipeline that names any variable (Run pipeline would ask for it). Then it plays each pipeline with the variables Bitbucket gives a step (the repository's to every step, the deployment's to its steps only) through touchmark's probe and guards: everything runs on the default branch of a hub that states its Premium restriction; a custom pipeline on another branch, and `platform` without the statement, are refused; a write key leaked into a repository variable stops every pipeline at its probe. + ### What the runs found (2026-10-02) | Finding | Fixed | diff --git a/docs/project/threat-model.md b/docs/project/threat-model.md index 4975738..d11d484 100644 --- a/docs/project/threat-model.md +++ b/docs/project/threat-model.md @@ -59,6 +59,7 @@ How to read the tests column: | The redaction registry: every token and JWT in its forms, secrets drivers mint registered, `::add-mask::` on GitHub Actions, merged automata so that masking stays linear in the targets | `redact`, the drivers, `cli.maskNewSecrets` | `TestAddForms`, `TestOnAdd`, `TestManySecrets`, `TestGitHubMask`, `TestMaskNewSecrets`, `TestJWT`, `TestInstallationTokens`, `FuzzWriter` | — | | A secret in a body, title, label, comment, commit message or branch name stops the target before its first write: `failed:secret-exposure` | `distribute` (`targetExec.leak`); config files refuse token-like strings, and a URL with credentials (`user:password@`), which `targets.yml` refuses, has them cut from every message | `TestLeakEveryText`, `TestExecuteSecretExposure`, `TestExecuteRefusesBeforeWriting` *(git 2.45)*, `TestExecuteSecretInPath` *(git 2.45)*, `TestParseRejectsSecrets`, `TestParseOperationsSecrets`, `TestDisplayURL` | — | | No touchmark token, `GITHUB_TOKEN` or `CI_JOB_TOKEN` in the environment of a target's git; the environment keeps `PATH`, system and temporary directories, proxies and `SSL_CERT_*`; `GIT_CONFIG_NOSYSTEM`, an empty global config, a temporary `HOME`, `GIT_ATTR_NOSYSTEM`, `GIT_TERMINAL_PROMPT=0`, empty `credential.helper` and askpass | `gitx.Isolation`, `gitx.Git` | `TestTargetIsolation`, `TestInitTargetDirAndHome`, `TestRunIgnoresInheritedEnv`, `TestRunInherit`, `TestCanaries` | — | +| No git of touchmark's, in the hub or a target, inherits a variable that adds config or names a program git runs: `GIT_CONFIG_COUNT` with every `GIT_CONFIG_KEY_*` and `GIT_CONFIG_VALUE_*`, `GIT_CONFIG_PARAMETERS`, `GIT_CONFIG_GLOBAL`, `GIT_CONFIG_SYSTEM`; `GIT_EXEC_PATH`, `GIT_SSH`, `GIT_SSH_COMMAND`, `GIT_ASKPASS`, `SSH_ASKPASS`, `GIT_PROXY_COMMAND`, `GIT_EXTERNAL_DIFF`, pagers and editors; `GIT_TRACE*` and `GIT_CURL_VERBOSE`; `GIT_TEMPLATE_DIR`; the variables that point git at another repository, index, object store or namespace, or change pathspecs and attributes; the loader's `LD_*` and `DYLD_*`. A CI variable or a custom pipeline run with variables cannot make git run `core.fsmonitor` or a hook. touchmark's own isolation sets its `GIT_CONFIG_*` explicitly; variables that only take config away (`GIT_CONFIG_NOSYSTEM`, `GIT_ATTR_NOSYSTEM`, `GIT_CEILING_DIRECTORIES`) stay | `gitx.Git` (`droppedEnv`, `droppedPrefixes`) | `TestRunDropsInheritedConfig`, `TestDroppedEnv`, `TestHostileGitConfigEnv`, `TestRunIgnoresInheritedEnv` | Whoever sets touchmark's environment also sets `PATH` and `HOME`: they choose the git that runs and, for the local commands, the user's global config. Keep variables of the job that runs `distribute` to the maintainers: on Bitbucket, where anyone who can run a custom pipeline can pass variables, restrict the deployment `touchmark-distribute` to admins or named users, not only to the branch. The hub's and a target's own `.git/config` apply to the local commands: they are the user's checkouts. | | A timeout or cancel kills git's whole process tree (on Windows the launcher's children too); stale `shallow.lock` files go before the next fetch | `gitx` (`configureCancel`, `removeStaleLocks`) | `TestCancelledFetch`, `TestNetworkTimeout`, `TestStaleLocksRemoved`, `TestRunCancelled` | — | | git's settings in a target: hooks off, `core.fsmonitor=false`, `protocol.allow=never` but https (and http to a loopback host), `http.followRedirects=false`, `transfer.fsckObjects=true`, `submodule.recurse=false`, `GIT_NO_LAZY_FETCH=1` | `gitx.newIsolated` | `TestTargetIsolation`, `TestFetchRefusesMalformedObjects`, `TestReadBlobNeverFetches`, `TestInitTargetRemotes`, `TestCheckRemoteScope` | `core.hooksPath` points at an empty directory of the isolation home, not at `/dev/null`: the same effect on every OS. | | Commands are argv without a shell, paths through `-z` or after `--`; only git runs | `gitx` | `TestOnlyGitRuns`, `TestQuoteStdinPath`, `TestHashPathsEdgeCases`, `FuzzParseLsTree`, `FuzzParsePush` | — | diff --git a/docs/reference/environment.md b/docs/reference/environment.md index b88ac8c..254c028 100644 --- a/docs/reference/environment.md +++ b/docs/reference/environment.md @@ -32,13 +32,17 @@ done. A Gitea or Forgejo admin token is refused. | `GITHUB_TOKEN` | the hub channel on GitHub Actions (and on Gitea and Forgejo when `GITEA_TOKEN` is unset): the tip of the default branch, the hub's visibility, the environment's branch policy, `plan --comment` | | `CI_JOB_TOKEN` | the hub channel on GitLab CI | | `GITEA_TOKEN` | the hub channel on Gitea and Forgejo Actions | +| `TOUCHMARK_PIPELINES_TOKEN` | the hub channel on Bitbucket Pipelines, which gives a step no API token: an access token of the hub repository with *Repositories: Read* (the default branch, its tip, the visibility) and *Pull requests: Write* (`plan --comment`) | | `CI` | any value but `""`, `false` and `0` makes a run a CI run: operation flags exit 2, and an unknown hub visibility counts as public | From the CI, touchmark also reads the platform's own variables: the server and API URLs, the repository and its id (the hub's fingerprint), the ref and the event, the step summary file, the job's timeout and start (the default `--deadline`), the merge request -and its base (the scope of `plan`), the project's visibility, and on GitLab -`CI_COMMIT_REF_PROTECTED`, `CI_ENVIRONMENT_NAME` and `CI_SERVER_TLS_CA_FILE`. +and its base (the scope of `plan`), the project's visibility, on GitLab +`CI_COMMIT_REF_PROTECTED`, `CI_ENVIRONMENT_NAME` and `CI_SERVER_TLS_CA_FILE`, and on +Bitbucket Pipelines `BITBUCKET_REPO_UUID`, `BITBUCKET_REPO_FULL_NAME`, `BITBUCKET_BRANCH`, +`BITBUCKET_TAG`, `BITBUCKET_PR_ID`, `BITBUCKET_DEPLOYMENT_ENVIRONMENT` and +`BITBUCKET_REPO_IS_PRIVATE`. ## In the GitHub Action diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md index 19c559e..7089ae7 100644 --- a/docs/reference/glossary.md +++ b/docs/reference/glossary.md @@ -18,8 +18,9 @@ propose the same content to that repository again. with the writer, from the hub's default branch. **fingerprint** — the hub's identity: the host and immutable repository id of the hub, -like `github.com/712345678`. In the commit trailers and the marker. CI provides it; -`--hub-fp` outside CI. +like `github.com/712345678`; on Bitbucket the repository's UUID, lowercase and without +braces, like `bitbucket.org/3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d`. In the commit +trailers and the marker. CI provides it; `--hub-fp` outside CI. **hub** — the repository, created from the template, that holds the packs, `hub.yml`, `targets.yml` and `.touchmark/operations.yml`. diff --git a/docs/reference/hub.md b/docs/reference/hub.md index c6f7a02..93184d3 100644 --- a/docs/reference/hub.md +++ b/docs/reference/hub.md @@ -25,7 +25,7 @@ memory: auto_close_cooldown: 30d # a bot's close: proposed again after 30d, then 60d; the third counts security: write_isolation: platform # platform | external | none - # reason: "..." # required with none + # reason: "..." # required with none, and with platform on Bitbucket Pipelines private_targets_in_public_hub: skip # skip | deliver providers: - id: gh # TOUCHMARK_GH_* variables; gh:owner/name in targets.yml @@ -63,7 +63,9 @@ The schema cannot say everything. `touchmark check` also requires: neither a current pack nor claimed by two packs; - `commit.message` and `pr.title` are not blank and do not start with `Draft:` or `WIP:` (set `pr.draft` instead); `commit.message` holds no git scissors line; -- `security.reason` is set when `write_isolation` is `none`; +- `security.reason` is set when `write_isolation` is `none` (and `distribute` and + `doctor` on Bitbucket Pipelines need it under `platform` too: see + [A hub on Bitbucket Cloud](../getting-started/bitbucket.md)); - no line of the file looks like a token (`ghp_`, `github_pat_`, `ghs_`, `glpat-`, `-----BEGIN`). @@ -80,7 +82,7 @@ In a hub pull request, `providers` and `ca_file` are read from the default branc | `id` | string, 3 to 40 characters, required | Short slug naming this hub, used in the sync branch name and in pull request markers. The template's placeholder "change-me" fails `touchmark check`. | | `branch` | string | Sync branch in every target. Default: touchmark/<id>. | | `branch_aliases` | list of strings | Former sync branch names. Open pull requests on them are still recognised as this hub's own. | -| `previous_fingerprints` | list of strings | Former fingerprints of this hub (host/numeric-repository-id), after the hub moved. | +| `previous_fingerprints` | list of strings | Former fingerprints of this hub (host/repository-id: a numeric id, or a Bitbucket repository UUID with or without braces), after the hub moved. | | `opt_in_file` | string | Path of the opt-in file in targets. Default: .engineering-assets.yml. | | `commit` | object | Commit settings. The commit author is always the provider's writer account. | | `commit.message` | string | Commit message. Default: "chore: sync engineering assets". Must not start with Draft: or WIP:, which turns a GitLab merge request into a draft. | @@ -99,8 +101,8 @@ In a hub pull request, `providers` and `ca_file` are read from the default branc | `memory` | object | Memory of pull requests closed without merging. | | `memory.auto_close_cooldown` | string | How long content closed by a bot waits before it is proposed again, in days or hours. Default: 30d. | | `security` | object | — | -| `security.write_isolation` | one of `platform`, `external`, `none` | Where the write key lives. platform: in the hub's CI platform, checked by a probe; external: issued by an external vault over OIDC; none: risk accepted, needs a reason. Default: platform. | -| `security.reason` | string | Why the write key cannot be isolated. Required with write_isolation: none. | +| `security.write_isolation` | one of `platform`, `external`, `none` | Where the write key lives. platform: in the hub's CI platform, checked by a probe (on Bitbucket Pipelines only with a reason); external: issued by an external vault over OIDC; none: risk accepted, needs a reason. Default: platform. | +| `security.reason` | string | Why the write key cannot be isolated. Required with write_isolation: none, and on Bitbucket Pipelines with platform, where it states the deployment permissions (Premium) that keep the write key on the default branch, which touchmark cannot read. | | `security.private_targets_in_public_hub` | one of `skip`, `deliver` | What a public hub does with non-public targets: skip them, or deliver and let their names reach public CI logs. Default: skip. | | `providers` | list of objects | Platforms this hub delivers to. Targets refer to them by id. | | `providers[].id` | string, required | Name targets.yml uses for this provider, and the <ID> of its TOUCHMARK_<ID>_\* variables. | diff --git a/docs/reference/output.md b/docs/reference/output.md index 0920bb5..cd1a2c5 100644 --- a/docs/reference/output.md +++ b/docs/reference/output.md @@ -72,7 +72,7 @@ and the branch head before and after. | `engine` | string, required | touchmark's version. | | `hub` | object, required | The hub the run read. | | `hub.id` | string, required | hub.yml's id. | -| `hub.fingerprint` | string, required | The hub's host and immutable repository id, e.g. github.com/712345678. | +| `hub.fingerprint` | string, required | The hub's host and immutable repository id, e.g. github.com/712345678; on Bitbucket the repository UUID, lowercase without braces. | | `hub.commit` | string, required | Full commit id: 40 or 64 lowercase hex digits. | | `hub.pr` | integer ≥ 1 | The hub pull request plan ran on. | | `outcome` | one of `completed`, `superseded`, required | completed, or superseded when the hub's default branch had moved on and nothing was inspected or written. | @@ -189,7 +189,7 @@ hub, for each provider's write identity, and for each target. See | `engine` | string, required | touchmark's version. | | `hub` | object, required | The hub the run read. | | `hub.id` | string, required | hub.yml's id. | -| `hub.fingerprint` | string, required | The hub's host and immutable repository id, e.g. github.com/712345678. | +| `hub.fingerprint` | string, required | The hub's host and immutable repository id, e.g. github.com/712345678; on Bitbucket the repository UUID, lowercase without braces. | | `hub.commit` | string, required | Full commit id: 40 or 64 lowercase hex digits. | | `strict` | boolean, required | The run had --strict: a check that warns or is unknown, or an incomplete resolve, exits with code 3. | | `hub_token` | boolean, required | doctor --hub-token read, with a maintainer's token, where the hub keeps its secrets. | diff --git a/internal/cli/bitbucket_hub_test.go b/internal/cli/bitbucket_hub_test.go new file mode 100644 index 0000000..9e98382 --- /dev/null +++ b/internal/cli/bitbucket_hub_test.go @@ -0,0 +1,185 @@ +package cli + +import ( + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "strings" + "sync" + "testing" + + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/hubch" + "github.com/bedrock-python/touchmark/internal/redact" + "github.com/bedrock-python/touchmark/internal/report" +) + +// bbHubToken is the hub's access token on Bitbucket Pipelines. +const bbHubToken = "bb-hub-access-token-0123456789" + +// bbPipelines is a step of the hub's pipelines on Bitbucket. +func bbPipelines(extra map[string]string) map[string]string { + vars := map[string]string{ + "CI": "true", + "BITBUCKET_BUILD_NUMBER": "12", + "BITBUCKET_REPO_UUID": "{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d}", + "BITBUCKET_REPO_FULL_NAME": "acme/engineering-assets", + "BITBUCKET_REPO_IS_PRIVATE": "true", + hubch.BitbucketTokenVar: bbHubToken, + } + for k, v := range extra { + vars[k] = v + } + return vars +} + +// bbHubAPI serves the hub repository's answers to the hub channel and +// records the comments written on pull request 41. +type bbHubAPI struct { + *httptest.Server + mu sync.Mutex + writes []string // "METHOD path: raw" + auth map[string]bool +} + +func newBBHubAPI(t *testing.T) *bbHubAPI { + t.Helper() + s := &bbHubAPI{auth: map[string]bool{}} + const repo = "/2.0/repositories/acme/engineering-assets" + s.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + s.mu.Lock() + defer s.mu.Unlock() + s.auth[r.Header.Get("Authorization")] = true + w.Header().Set("Content-Type", "application/json") + if r.Header.Get("Authorization") != "Bearer "+bbHubToken { + http.Error(w, `{"type": "error"}`, http.StatusNotFound) + return + } + switch { + case r.Method == http.MethodGet && r.URL.Path == repo: + fmt.Fprint(w, `{"uuid": "{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d}", "full_name": "acme/engineering-assets", "is_private": true, "mainbranch": {"name": "master"}}`) + case r.Method == http.MethodGet && r.URL.Path == repo+"/pullrequests/41/comments": + fmt.Fprint(w, `{"pagelen": 100, "values": []}`) + case r.Method == http.MethodPost && r.URL.Path == repo+"/pullrequests/41/comments": + var in struct { + Content struct { + Raw string `json:"raw"` + } `json:"content"` + } + _ = json.NewDecoder(r.Body).Decode(&in) + s.writes = append(s.writes, r.Method+" "+r.URL.Path+": "+in.Content.Raw) + w.WriteHeader(http.StatusCreated) + fmt.Fprint(w, `{"id": 9}`) + default: + http.Error(w, `{"type": "error"}`, http.StatusNotFound) + } + })) + t.Cleanup(s.Close) + return s +} + +// detectAt is hubch.Detect of vars with the API of srv. +func detectAt(vars map[string]string, srv *httptest.Server) hubch.Context { + c := hubch.Detect(func(k string) string { return vars[k] }, nil) + c.APIURL = srv.URL + "/2.0" + return c +} + +// On Bitbucket, Pipelines names no default branch: the hub channel reads it +// with the hub's access token before any guard. +func TestHubRepositoryBitbucket(t *testing.T) { + srv := newBBHubAPI(t) + vars := bbPipelines(map[string]string{"BITBUCKET_BRANCH": "master"}) + hctx, warnings := hubRepository(t.Context(), detectAt(vars, srv.Server), func(k string) string { return vars[k] }) + if len(warnings) != 0 || hctx.DefaultBranch != "master" || hctx.Visibility != "private" { + t.Errorf("context %+v, warnings %q", hctx, warnings) + } + // Without the token a private hub is not read: a warning, no default + // branch, and so distribute's guard refuses. + delete(vars, hubch.BitbucketTokenVar) + hctx, warnings = hubRepository(t.Context(), detectAt(vars, srv.Server), func(k string) string { return vars[k] }) + if hctx.DefaultBranch != "" || len(warnings) != 1 || !strings.Contains(warnings[0], "a private hub needs TOUCHMARK_PIPELINES_TOKEN") { + t.Errorf("without a token: %+v, %q", hctx, warnings) + } + // Other CIs are left as they are. + gh := hubch.Context{CI: hubch.GitHubActions, DefaultBranch: ""} + if got, w := hubRepository(t.Context(), gh, func(string) string { return "" }); got != gh || w != nil { + t.Errorf("GitHub: %+v, %q", got, w) + } +} + +// plan --comment on a Bitbucket hub pull request: one comment through the +// hub channel with the hub's access token, its marker a Markdown reference +// definition and its body without HTML, which Bitbucket would show as text. +func TestPlanCommentBitbucket(t *testing.T) { + srv := newBBHubAPI(t) + vars := bbPipelines(map[string]string{"BITBUCKET_BRANCH": "feature", "BITBUCKET_PR_ID": "41", "BITBUCKET_PR_DESTINATION_BRANCH": "master"}) + getenv := func(k string) string { return vars[k] } + hctx := detectAt(vars, srv.Server) + if pr := hubPR(hctx, getenv); pr != 41 { + t.Fatalf("hubPR = %d", pr) + } + reg := redact.New() + reg.Add(bbHubToken) + ch, err := hubch.New(hctx, httpx.New(httpx.Options{Redact: reg}), hubch.Token(hctx, getenv)) + if err != nil { + t.Fatal(err) + } + rep := report.NewDelivery("plan", "dev") + rep.Hub.ID = "acme-eng" + rep.Paths = []report.PathChange{{Action: "add", Path: "AGENTS.md", Targets: 2}} + if w := postPlanComment(t.Context(), hctx, ch, 41, rep, reg); w != "" { + t.Fatalf("warning %q", w) + } + srv.mu.Lock() + defer srv.mu.Unlock() + if len(srv.writes) != 1 { + t.Fatalf("writes %q", srv.writes) + } + body := srv.writes[0] + if !strings.HasSuffix(body, "\n\n[touchmark-plan]: # \"touchmark plan: acme-eng\"\n") || strings.Contains(body, "<") { + t.Errorf("comment:\n%s", body) + } + if !srv.auth["Bearer "+bbHubToken] || len(srv.auth) != 1 { + t.Errorf("Authorization headers %v", srv.auth) + } + // Every other CI keeps its HTML comment, byte for byte. + for _, ci := range []hubch.CI{hubch.GitHubActions, hubch.GiteaActions, hubch.ForgejoActions} { + if got := planCommentMarkerOf(ci, "acme-eng"); got != "" { + t.Errorf("%s: marker %q", ci, got) + } + } +} + +func TestHubAPIHostBitbucket(t *testing.T) { + for _, tc := range []struct{ typ, host, want string }{ + {"bitbucket", "bitbucket.org", "api.bitbucket.org"}, + {"bitbucket", "Bitbucket.org:443", "api.bitbucket.org"}, + {"github", "bitbucket.org", "bitbucket.org"}, + } { + if got := hubAPIHost(tc.typ, tc.host); got != tc.want { + t.Errorf("hubAPIHost(%s, %s) = %s, want %s", tc.typ, tc.host, got, tc.want) + } + } +} + +// --hub-fp takes a Bitbucket repository UUID as BITBUCKET_REPO_UUID gives +// it, and keeps it in the form the CI reports. +func TestHubFPBitbucket(t *testing.T) { + var f fingerprintFlag + if err := f.Set("bitbucket.org/{3F2A8D4E-1B6C-4F0A-9E7D-5C2B1A0F9E8D}"); err != nil || string(f) != "bitbucket.org/3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d" { + t.Errorf("--hub-fp: %q, %v", f, err) + } + for _, bad := range []string{"bitbucket.org/{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d", "bitbucket.org/3f2a8d4e", "bitbucket.org/{}"} { + if err := f.Set(bad); err == nil { + t.Errorf("--hub-fp %s: no error", bad) + } + } + vars := bbPipelines(map[string]string{"BITBUCKET_BRANCH": "master"}) + hctx := hubch.Detect(func(k string) string { return vars[k] }, nil) + fp, warnings, err := planFingerprint(hctx, "bitbucket.org/3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d") + if err != nil || len(warnings) != 0 || fp != "bitbucket.org/3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d" { + t.Errorf("planFingerprint: %q, %q, %v", fp, warnings, err) + } +} diff --git a/internal/cli/delivery.go b/internal/cli/delivery.go index d0785df..e5aa22a 100644 --- a/internal/cli/delivery.go +++ b/internal/cli/delivery.go @@ -231,6 +231,48 @@ type failedChannel struct{ err error } func (c failedChannel) Head(context.Context) (string, error) { return "", c.err } +// hubRepository fills in what the CI does not say about the hub: Bitbucket +// Pipelines names no default branch, so on Bitbucket the hub channel reads +// it, with the hub's access token (TOUCHMARK_PIPELINES_TOKEN; a public hub +// answers without one), and the visibility when BITBUCKET_REPO_IS_PRIVATE +// was missing. It reads the hub repository only, before any guard: the +// guards then compare the run's branch with it. When the channel cannot +// read it the context stays without a default branch, with a warning: +// distribute and doctor refuse to run then (distribute.DistributeGuard), +// and plan reads its providers without credentials. Every other CI is +// returned as it is. +func hubRepository(ctx context.Context, hctx hubch.Context, getenv func(string) string) (hubch.Context, []string) { + if hctx.CI != hubch.BitbucketPipelines || hctx.DefaultBranch != "" { + return hctx, nil + } + reg := redact.New() + token := hubch.Token(hctx, getenv) + reg.Add(token, basicUsers...) + info, err := func() (hubch.RepoInfo, error) { + ch, err := hubch.New(hctx, httpx.New(httpx.Options{Redact: reg}), token) + if err != nil { + return hubch.RepoInfo{}, err + } + r, ok := ch.(hubch.RepositoryReader) + if !ok { + return hubch.RepoInfo{}, errors.New("hub channel: it reads no repository") + } + return r.Repository(ctx) + }() + if err != nil { + hint := "" + if token == "" { + hint = "; a private hub needs " + hubch.BitbucketTokenVar + ", an access token of the hub repository with Repositories: Read" + } + return hctx, []string{reg.Replace(fmt.Sprintf("Bitbucket Pipelines names no default branch of the hub, and the hub channel could not read it: %v%s", err, hint))} + } + hctx.DefaultBranch = info.DefaultBranch + if hctx.Visibility == "" { + hctx.Visibility = info.Visibility + } + return hctx, nil +} + // channelVisibility fills in the hub's visibility from the channel when the // CI's event payload names none (GitHub's and Gitea's schedule event). It // returns hctx as it was, with a warning, when the channel cannot tell: the diff --git a/internal/cli/distribute.go b/internal/cli/distribute.go index 2a5f247..4bf86a4 100644 --- a/internal/cli/distribute.go +++ b/internal/cli/distribute.go @@ -216,10 +216,12 @@ func runDistribute(ctx context.Context, e *env, o *options, d *distOptions) (err return err } hctx := hubch.Detect(e.getenv, os.ReadFile) + hctx, repoWarnings := hubRepository(ctx, hctx, e.getenv) fp, warnings, err := planFingerprint(hctx, string(o.hubFP)) if err != nil { return err } + warnings = append(warnings, repoWarnings...) gitV, err := gitVersion(ctx) if err != nil { return configError(err) diff --git a/internal/cli/doctor.go b/internal/cli/doctor.go index de3e1a8..530d647 100644 --- a/internal/cli/doctor.go +++ b/internal/cli/doctor.go @@ -87,10 +87,12 @@ func runDoctor(ctx context.Context, e *env, o *options, d *doctorOptions) (err e return err } hctx := hubch.Detect(e.getenv, os.ReadFile) + hctx, repoWarnings := hubRepository(ctx, hctx, e.getenv) fp, warnings, err := planFingerprint(hctx, string(o.hubFP)) if err != nil { return err } + warnings = append(warnings, repoWarnings...) ci := inCI(hctx, e.getenv) if d.hubToken && ci { return configErrorf("--hub-token is for a maintainer's local run: a CI job must not hold a maintainer's token of the hub") @@ -289,10 +291,11 @@ func isolationChecks(ctx context.Context, hctx hubch.Context, cfg *config.Hub, g // The token goes to the hub's own API only, whatever the hub.yml of this // checkout says (a branch under review may say anything): the provider's // API must be on the fingerprint's host (api.github.com for github.com, -// api. for GHE.com), and a provider with a ca_file must have the -// same endpoints and CA bundle on the default branch (origin/HEAD), whose -// bundle is used (trustedTokenProvider). The host is printed on stderr -// before the first request. +// api. for GHE.com, api.bitbucket.org for bitbucket.org, whose +// fingerprint's id is the repository's UUID), and a provider with a +// ca_file must have the same endpoints and CA bundle on the default branch +// (origin/HEAD), whose bundle is used (trustedTokenProvider). The host is +// printed on stderr before the first request. func (h *hub) keyLocation(ctx context.Context, e *env, rps []config.ResolvedProvider, fp, token string, reg *redact.Registry) ([]report.DoctorCheck, error) { i := strings.LastIndexByte(fp, '/') host, id := strings.ToLower(fp[:i]), fp[i+1:] @@ -323,6 +326,8 @@ func (h *hub) keyLocation(ctx context.Context, e *env, rps []config.ResolvedProv in.Platform, in.APIURL = "github", "https://api."+host case host == "gitlab.com": in.Platform, in.APIURL = "gitlab", "https://gitlab.com/api/v4" + case host == "bitbucket.org": + in.Platform, in.APIURL = "bitbucket", "https://api.bitbucket.org/2.0" default: return nil, configErrorf("--hub-token: no provider of %s is on the hub's host %s, so touchmark cannot tell its platform", config.HubFile, host) } @@ -351,9 +356,9 @@ func (h *hub) keyLocation(ctx context.Context, e *env, rps []config.ResolvedProv // hubAPIHost is the host name of the API of a hub on host (a fingerprint's // host, perhaps with a port) on platform typ: api.github.com for -// github.com, api. for GHE.com, the hub's host itself elsewhere -// (GitHub Enterprise Server, GitLab, Gitea and Forgejo serve their API -// under their own URL). +// github.com, api. for GHE.com, api.bitbucket.org for bitbucket.org, +// the hub's host itself elsewhere (GitHub Enterprise Server, GitLab, Gitea +// and Forgejo serve their API under their own URL). func hubAPIHost(typ, host string) string { name := strings.ToLower(host) if hn, _, err := net.SplitHostPort(name); err == nil { @@ -364,6 +369,8 @@ func hubAPIHost(typ, host string) string { return "api.github.com" case typ == "github" && strings.HasSuffix(name, ".ghe.com"): return "api." + name + case typ == "bitbucket" && name == "bitbucket.org": + return "api.bitbucket.org" } return name } diff --git a/internal/cli/hostile_env_test.go b/internal/cli/hostile_env_test.go new file mode 100644 index 0000000..93063d0 --- /dev/null +++ b/internal/cli/hostile_env_test.go @@ -0,0 +1,58 @@ +package cli + +import ( + "os" + "os/exec" + "path/filepath" + "testing" +) + +// A hostile environment around touchmark (a CI variable, a custom pipeline +// run with variables) that gives git a core.fsmonitor command through +// GIT_CONFIG_COUNT does not run it: neither check and status reading the +// hub, nor status and apply in a target. The control first shows that +// plain git in the same environment runs the command. +func TestHostileGitConfigEnv(t *testing.T) { + h := baseHub(t) + tg := newRepo(t, false) + tg.write(optInFile, optInBase) + tg.commit("opt in") + marker := filepath.Join(t.TempDir(), "fsmonitor-ran") + hostile := map[string]string{ + "GIT_CONFIG_COUNT": "1", + "GIT_CONFIG_KEY_0": "core.fsmonitor", + "GIT_CONFIG_VALUE_0": "echo ran >>'" + filepath.ToSlash(marker) + "'", + } + + control := exec.Command("git", "-C", tg.dir, "status", "--porcelain") + control.Env = os.Environ() + for k, v := range hostile { + control.Env = append(control.Env, k+"="+v) + } + if out, err := control.CombinedOutput(); err != nil { + t.Fatalf("git status: %v\n%s", err, out) + } + if _, err := os.Stat(marker); err != nil { + t.Fatalf("plain git did not run the hostile core.fsmonitor (%v): the test proves nothing", err) + } + if err := os.Remove(marker); err != nil { + t.Fatal(err) + } + + for k, v := range hostile { + t.Setenv(k, v) + } + s := newScenario(t, "", h, tg, "--repo", "acme/svc") + s.run(exitOK, "check") + s.report("status") + s.report("apply") + if got := changes(s.report("status")); len(got) != 0 { + t.Errorf("status after apply plans %q", got) + } + s.wantContent("AGENTS.md", text("base AGENTS.md v1")) + if _, err := os.Stat(marker); err == nil { + t.Fatal("touchmark's git ran the core.fsmonitor command of the inherited GIT_CONFIG_COUNT") + } else if !os.IsNotExist(err) { + t.Fatal(err) + } +} diff --git a/internal/cli/plan.go b/internal/cli/plan.go index 42db517..2c32d3b 100644 --- a/internal/cli/plan.go +++ b/internal/cli/plan.go @@ -118,10 +118,12 @@ func runPlan(ctx context.Context, e *env, o *options, p *planOptions) (err error return err } hctx := hubch.Detect(e.getenv, os.ReadFile) + hctx, repoWarnings := hubRepository(ctx, hctx, e.getenv) fp, warnings, err := planFingerprint(hctx, string(o.hubFP)) if err != nil { return err } + warnings = append(warnings, repoWarnings...) rps, err := h.resolveProviders(ctx, hctx, e.getenv) if err != nil { return configError(err) @@ -302,8 +304,9 @@ func planFingerprint(hctx hubch.Context, flag string) (string, []string, error) warnings = append(warnings, fmt.Sprintf("--hub-fp %s differs from %s, the fingerprint CI reports", flag, ci)) } if flag == "" { - return "", nil, configErrorf("the hub's fingerprint is unknown: CI provides it (GITHUB_REPOSITORY_ID, CI_PROJECT_ID); " + - "outside CI pass --hub-fp HOST/ID, the hub's host and repository id, e.g. github.com/712345678") + return "", nil, configErrorf("the hub's fingerprint is unknown: CI provides it (GITHUB_REPOSITORY_ID, CI_PROJECT_ID, BITBUCKET_REPO_UUID); " + + "outside CI pass --hub-fp HOST/ID, the hub's host and repository id, e.g. github.com/712345678 " + + "(on Bitbucket the repository's UUID, BITBUCKET_REPO_UUID: bitbucket.org/{…})") } return flag, warnings, nil } @@ -679,7 +682,8 @@ func (h *hub) caPool(ctx context.Context, name, rev string) (*x509.CertPool, err // hubPR returns the number of the hub pull request a CI run builds: from // GITHUB_REF (refs/pull//merge) on Actions, CI_MERGE_REQUEST_IID on -// GitLab; 0 when the run builds none. +// GitLab, BITBUCKET_PR_ID on Bitbucket Pipelines; 0 when the run builds +// none. func hubPR(hctx hubch.Context, getenv func(string) string) int64 { var n string switch hctx.CI { @@ -691,6 +695,8 @@ func hubPR(hctx hubch.Context, getenv func(string) string) int64 { n, _, _ = strings.Cut(rest, "/") case hubch.GitLabCI: n = strings.TrimSpace(getenv("CI_MERGE_REQUEST_IID")) + case hubch.BitbucketPipelines: + n = strings.TrimSpace(getenv("BITBUCKET_PR_ID")) } v, err := strconv.ParseInt(n, 10, 64) if err != nil || v <= 0 { @@ -775,12 +781,13 @@ func (l *refList) Set(s string) error { } // fingerprintRe is a hub fingerprint: a host with an optional port, and a -// positive repository id. -var fingerprintRe = regexp.MustCompile(`^[A-Za-z0-9.-]+(:[0-9]{1,5})?/[1-9][0-9]{0,19}$`) +// positive repository id or a Bitbucket repository UUID, with or without +// its braces (BITBUCKET_REPO_UUID has them). +var fingerprintRe = regexp.MustCompile(`^[A-Za-z0-9.-]+(:[0-9]{1,5})?/([1-9][0-9]{0,19}|[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}|\{[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}\})$`) // fingerprintFlag is the value of --hub-fp: HOST[:PORT]/ID in the form CI // reports it (config.CanonicalFingerprint: the host lowercased, without the -// default ports 443 and 80). +// default ports 443 and 80; a Bitbucket UUID lowercased, without braces). type fingerprintFlag string func (f *fingerprintFlag) String() string { return string(*f) } @@ -788,7 +795,7 @@ func (f *fingerprintFlag) String() string { return string(*f) } func (f *fingerprintFlag) Set(s string) error { s = strings.TrimSpace(s) if !fingerprintRe.MatchString(s) { - return fmt.Errorf("%q is not HOST/ID, like github.com/712345678", s) + return fmt.Errorf("%q is not HOST/ID, like github.com/712345678 or bitbucket.org/{repository-uuid}", s) } *f = fingerprintFlag(config.CanonicalFingerprint(s)) return nil diff --git a/internal/cli/plan_comment.go b/internal/cli/plan_comment.go index 94c1180..a9d1c35 100644 --- a/internal/cli/plan_comment.go +++ b/internal/cli/plan_comment.go @@ -14,8 +14,9 @@ import ( // holds plan's Markdown report and is edited by every later run, through // the hub channel with the CI's own token (GITHUB_TOKEN with // pull-requests: write in the plan job; the Actions token of Gitea and -// Forgejo). GitLab has no comment: CI_JOB_TOKEN cannot write merge request -// notes. +// Forgejo; on Bitbucket the hub's access token in TOUCHMARK_PIPELINES_TOKEN +// with Pull requests: Write). GitLab has no comment: CI_JOB_TOKEN cannot +// write merge request notes. // planCommentMarker is the hidden line that tells plan's comment from the // others: one per hub, so that two hubs planning in one repository (a @@ -24,6 +25,22 @@ func planCommentMarker(hubID string) string { return "" } +// planCommentMarkerOf is the marker of plan's comment in CI ci. Bitbucket +// shows an HTML comment as text, so there the marker is a Markdown link +// reference definition, which Markdown does not show either (as the marker +// of a sync pull request there, marker.FrameRefDef): +// +// [touchmark-plan]: # "touchmark plan: " +// +// It needs a blank line before it, which the report's last line gives. +// Every other CI keeps the HTML comment, byte for byte. +func planCommentMarkerOf(ci hubch.CI, hubID string) string { + if ci == hubch.BitbucketPipelines { + return `[touchmark-plan]: # "touchmark plan: ` + hubID + `"` + } + return planCommentMarker(hubID) +} + // postPlanComment keeps plan's comment in the hub pull request pr through // ch, with rep rendered as its body (masked with reg), and returns a warning // when it could not: the run builds no hub pull request, the CI has no @@ -45,13 +62,21 @@ func postPlanComment(ctx context.Context, hctx hubch.Context, ch hubch.Channel, if !ok { return what + "not posted: no channel to the hub for its CI token" } - marker := planCommentMarker(rep.Hub.ID) + marker := planCommentMarkerOf(hctx.CI, rep.Hub.ID) + write := rep.WriteComment + if hctx.CI == hubch.BitbucketPipelines { + // Bitbucket shows HTML as text: no
sections. + write = rep.WritePlainComment + } var b strings.Builder - if err := rep.WriteComment(&b, report.MaxComment, "\n"+marker+"\n"); err != nil { + if err := write(&b, report.MaxComment, "\n"+marker+"\n"); err != nil { return what + err.Error() } body := reg.Replace(b.String()) if _, err := c.UpsertComment(ctx, pr, marker, body); err != nil { + if hctx.CI == hubch.BitbucketPipelines { + return what + fmt.Sprintf("not posted: %v (the hub's access token in %s needs Pull requests: Write)", err, hubch.BitbucketTokenVar) + } return what + fmt.Sprintf("not posted: %v", err) } return "" diff --git a/internal/cli/setup.go b/internal/cli/setup.go index da75938..9465478 100644 --- a/internal/cli/setup.go +++ b/internal/cli/setup.go @@ -98,8 +98,9 @@ func runSetup(ctx context.Context, e *env, o *options, s *setupOptions) (err err case "gitea", "forgejo": return giteaRefusal(platform) case "bitbucket": - return configErrorf("setup does not set up bitbucket yet: a hub on Bitbucket Pipelines comes in a later release; " + - "a hub on GitHub or GitLab delivers to Bitbucket Cloud as one of its providers") + return configErrorf("setup does not set up bitbucket yet: set up a hub on Bitbucket Pipelines by hand (the bot accounts and their API "+ + "tokens, the hub's access token, the repository and deployment variables), as %s shows; a hub on GitHub or GitLab "+ + "delivers to Bitbucket Cloud as one of its providers", docsurl.GettingStarted("bitbucket")) case "github", "gitlab": default: return usageErrorf("unknown platform %q: setup sets up github or gitlab", platform) diff --git a/internal/config/hub.go b/internal/config/hub.go index 5933f90..f42bd65 100644 --- a/internal/config/hub.go +++ b/internal/config/hub.go @@ -65,7 +65,8 @@ func validateHub(h *Hub, doc document, p *problems) { checkBranches(h, doc, p) for i, fp := range h.PreviousFingerprints { if !fingerprintRe.MatchString(fp) { - p.errorf(fmt.Sprintf("previous_fingerprints[%d]", i), "%q must be host/numeric-id, like github.com/712345678", fp) + p.errorf(fmt.Sprintf("previous_fingerprints[%d]", i), "%q must be host/repository-id, like github.com/712345678 "+ + "or bitbucket.org/{repository-uuid}", fp) } } if doc.has("opt_in_file") { @@ -369,7 +370,9 @@ func setDefault(v *string, def string) { // CanonicalFingerprint returns a hub fingerprint (host[:port]/id) in the // form CI reports it, so that spellings of one hub compare // equal and hash to the same marker fp: the host lowercased, the default -// ports 443 and 80 dropped, and the repository id without leading zeros. +// ports 443 and 80 dropped, and the repository id without leading zeros. A +// Bitbucket repository UUID, as BITBUCKET_REPO_UUID gives it +// ({0f1e…-…}), loses its braces and is lowercased: bitbucket.org/0f1e…-…. // Anything that is not host[:port]/id is returned unchanged. func CanonicalFingerprint(fp string) string { if !fingerprintRe.MatchString(fp) { @@ -380,6 +383,9 @@ func CanonicalFingerprint(fp string) string { if name, port, ok := strings.Cut(host, ":"); ok && (port == "443" || port == "80") { host = name } + if strings.Contains(id, "-") { + return host + "/" + strings.ToLower(strings.Trim(id, "{}")) + } if id = strings.TrimLeft(id, "0"); id == "" { id = "0" } diff --git a/internal/config/providers.go b/internal/config/providers.go index 41151dc..d60f07d 100644 --- a/internal/config/providers.go +++ b/internal/config/providers.go @@ -106,7 +106,7 @@ func (h *Hub) ResolveProviders(getenv func(string) string) ([]ResolvedProvider, // remote is on originHost ("" when unknown). Outside CI, a hub without // providers takes its implicit provider from that host when it is a public // instance whose type the host tells: github.com and *.ghe.com (github at -// https://), gitlab.com (gitlab). Any other host is a self-managed +// https://), gitlab.com (gitlab), bitbucket.org (bitbucket). Any other host is a self-managed // instance, whose type only hub.yml can tell: the error asks for providers // there. In CI the environment decides, whatever the origin. func (h *Hub) ResolveProvidersWithOrigin(getenv func(string) string, originHost string) ([]ResolvedProvider, error) { @@ -154,7 +154,8 @@ func (h *Hub) ResolveProvidersWithOrigin(getenv func(string) string, originHost // ciEnv is what the CI environment says about the hub's own platform. type ciEnv struct { // kind is the provider type of the CI: "github", "gitlab", "gitea", - // "forgejo", or "" outside CI. + // "forgejo", "bitbucket" (Bitbucket Pipelines, always on Bitbucket + // Cloud), or "" outside CI. kind string // serverVar names the variable with the platform's URL, and serverURL // is its value, trimmed. @@ -177,6 +178,11 @@ func readCIEnv(getenv func(string) string) ciEnv { env.kind, env.serverVar = "github", "GITHUB_SERVER_URL" case isTrue("GITLAB_CI"): env.kind, env.serverVar = "gitlab", "CI_SERVER_URL" + case strings.TrimSpace(getenv("BITBUCKET_BUILD_NUMBER")) != "": + // Bitbucket Pipelines runs on Bitbucket Cloud only, and names no + // server URL: the hub is on https://bitbucket.org. + env.kind, env.serverURL = "bitbucket", bitbucketURL + return env default: return env } @@ -207,8 +213,8 @@ func implicitProvider(h *Hub, env ciEnv, originHost string) (Provider, error) { default: var ok bool if p, ok = originProvider(originHost); !ok { - return Provider{}, fmt.Errorf("%s declares no providers, no CI environment names one (GitHub Actions, GitLab CI, Gitea or Forgejo Actions), "+ - "and the hub's origin remote is not on github.com, a *.ghe.com host or gitlab.com; add providers to %s, or set platform and base_url there", HubFile, HubFile) + return Provider{}, fmt.Errorf("%s declares no providers, no CI environment names one (GitHub Actions, GitLab CI, Gitea or Forgejo Actions, Bitbucket Pipelines), "+ + "and the hub's origin remote is not on github.com, a *.ghe.com host, gitlab.com or bitbucket.org; add providers to %s, or set platform and base_url there", HubFile, HubFile) } } if h != nil { @@ -219,14 +225,16 @@ func implicitProvider(h *Hub, env ciEnv, originHost string) (Provider, error) { // OriginTellsProvider reports whether a hub without providers can take its // implicit provider outside CI from an origin remote on host: only a public -// instance (github.com, a *.ghe.com host, gitlab.com) tells its type. +// instance (github.com, a *.ghe.com host, gitlab.com, bitbucket.org) tells +// its type. func OriginTellsProvider(host string) bool { _, ok := originProvider(host) return ok } // originProvider returns the provider of a public instance at host, the -// host of a hub's origin remote: only there does the host tell the type. +// host of a hub's origin remote: only there does the host tell the type +// (Bitbucket Cloud's bitbucket.org included). func originProvider(host string) (Provider, bool) { host = strings.ToLower(strings.TrimSpace(host)) switch { @@ -236,6 +244,8 @@ func originProvider(host string) (Provider, bool) { return Provider{ID: "github", Type: "github", URL: "https://" + host}, true case host == "gitlab.com": return Provider{ID: "gitlab", Type: "gitlab", URL: "https://gitlab.com"}, true + case host == "bitbucket.org": + return Provider{ID: "bitbucket", Type: "bitbucket", URL: bitbucketURL}, true } return Provider{}, false } diff --git a/internal/config/providers_test.go b/internal/config/providers_test.go index 1d5298b..36c0a99 100644 --- a/internal/config/providers_test.go +++ b/internal/config/providers_test.go @@ -216,6 +216,12 @@ func TestResolveProvidersImplicit(t *testing.T) { env: envOf("FORGEJO_ACTIONS", "true", "GITEA_ACTIONS", "true", "GITHUB_ACTIONS", "true", "GITHUB_SERVER_URL", "https://codeberg.org"), want: implicit("forgejo", "https://codeberg.org", "codeberg.org", "https://codeberg.org/api/v1", ""), }, + { + // Pipelines names no server: it runs on Bitbucket Cloud only. + name: "Bitbucket Pipelines", + env: envOf("BITBUCKET_BUILD_NUMBER", "17", "CI", "true", "BITBUCKET_REPO_UUID", "{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d}"), + want: implicit("bitbucket", "https://bitbucket.org", "bitbucket.org", "https://api.bitbucket.org/2.0", ""), + }, } legacy, _, err := ParseHub(nil) if err != nil { @@ -270,7 +276,7 @@ func TestResolveProvidersErrors(t *testing.T) { name: "no providers outside CI", hub: &Hub{ID: "acme-eng"}, env: envOf("CI", "true", "GITHUB_ACTIONS", "1"), - want: "hub.yml declares no providers, no CI environment names one (GitHub Actions, GitLab CI, Gitea or Forgejo Actions), and the hub's origin remote is not on github.com", + want: "hub.yml declares no providers, no CI environment names one (GitHub Actions, GitLab CI, Gitea or Forgejo Actions, Bitbucket Pipelines), and the hub's origin remote is not on github.com", }, { name: "no server URL", @@ -411,6 +417,7 @@ func TestResolveProvidersOrigin(t *testing.T) { {"GitHub.com", implicit("github", "https://github.com", "github.com", "https://api.github.com", "https://api.github.com/graphql")}, {"acme.ghe.com", implicit("github", "https://acme.ghe.com", "acme.ghe.com", "https://api.acme.ghe.com", "https://api.acme.ghe.com/graphql")}, {"gitlab.com", implicit("gitlab", "https://gitlab.com", "gitlab.com", "https://gitlab.com/api/v4", "https://gitlab.com/api/graphql")}, + {"bitbucket.org", implicit("bitbucket", "https://bitbucket.org", "bitbucket.org", "https://api.bitbucket.org/2.0", "")}, } { got, err := h.ResolveProvidersWithOrigin(envOf(), tc.origin) if err != nil || !reflect.DeepEqual(got, []ResolvedProvider{tc.want}) { @@ -447,6 +454,10 @@ func TestCanonicalFingerprint(t *testing.T) { "host/0": "host/0", "github.com": "github.com", "": "", + // Bitbucket: BITBUCKET_REPO_UUID's braces go, and the case. + "bitbucket.org/{3F2A8D4E-1B6C-4F0A-9E7D-5C2B1A0F9E8D}": "bitbucket.org/3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d", + "Bitbucket.org/3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d": "bitbucket.org/3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d", + "bitbucket.org/{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d": "bitbucket.org/{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d", } { if got := CanonicalFingerprint(in); got != want { t.Errorf("CanonicalFingerprint(%q) = %q, want %q", in, got, want) @@ -456,6 +467,10 @@ func TestCanonicalFingerprint(t *testing.T) { if want := []string{"gitlab.example.com/1234", "github.com/1"}; !reflect.DeepEqual(h.PreviousFingerprints, want) { t.Errorf("previous_fingerprints = %q, want %q", h.PreviousFingerprints, want) } + h = mustParseHub(t, []byte("version: 1\nid: acme-eng\nprevious_fingerprints: [\"bitbucket.org/{3F2A8D4E-1B6C-4F0A-9E7D-5C2B1A0F9E8D}\"]\n")) + if want := []string{"bitbucket.org/3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d"}; !reflect.DeepEqual(h.PreviousFingerprints, want) { + t.Errorf("previous_fingerprints = %q, want %q", h.PreviousFingerprints, want) + } } // TestResolveProvidersProcessEnv: a nil getenv reads the process diff --git a/internal/config/testdata/hub/invalid/fingerprint-invalid.yml b/internal/config/testdata/hub/invalid/fingerprint-invalid.yml index 6c995c4..3d15ba7 100644 --- a/internal/config/testdata/hub/invalid/fingerprint-invalid.yml +++ b/internal/config/testdata/hub/invalid/fingerprint-invalid.yml @@ -1,4 +1,4 @@ -# error: previous_fingerprints[0]: "github.com" must be host/numeric-id +# error: previous_fingerprints[0]: "github.com" must be host/repository-id version: 1 id: acme-eng previous_fingerprints: [github.com] diff --git a/internal/config/testdata/hub/invalid/fingerprint-uuid-brace.yml b/internal/config/testdata/hub/invalid/fingerprint-uuid-brace.yml new file mode 100644 index 0000000..cc898b7 --- /dev/null +++ b/internal/config/testdata/hub/invalid/fingerprint-uuid-brace.yml @@ -0,0 +1,4 @@ +# error: previous_fingerprints[0]: "bitbucket.org/{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d" must be host/repository-id +version: 1 +id: acme-eng +previous_fingerprints: ["bitbucket.org/{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d"] diff --git a/internal/config/testdata/hub/valid/fingerprint-uuid.yml b/internal/config/testdata/hub/valid/fingerprint-uuid.yml new file mode 100644 index 0000000..8d75b20 --- /dev/null +++ b/internal/config/testdata/hub/valid/fingerprint-uuid.yml @@ -0,0 +1,6 @@ +version: 1 +id: acme-eng +previous_fingerprints: + - "bitbucket.org/{3F2A8D4E-1B6C-4F0A-9E7D-5C2B1A0F9E8D}" + - bitbucket.org/0b7e5a2c-9d4f-4e1b-8a3c-6f5d2e1c0b9a + - github.com/712345678 diff --git a/internal/config/validate.go b/internal/config/validate.go index f507277..34a7376 100644 --- a/internal/config/validate.go +++ b/internal/config/validate.go @@ -57,7 +57,7 @@ const ( repoURLPattern = `^` + urlBase + `(/` + refSegment + `){2,}/?$` namespaceURLPattern = `^` + urlBase + `(/` + refSegment + `)+/?$` excludeURLPattern = `^` + urlBase + `(/` + urlGlobSegment + `){2,}/?$` - fingerprintPattern = `^[A-Za-z0-9.-]+(:[0-9]{1,5})?/[0-9]+$` + fingerprintPattern = `^[A-Za-z0-9.-]+(:[0-9]{1,5})?/([0-9]+|[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}|\{[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}\})$` cooldownPattern = `^[1-9][0-9]{0,4}[dh]$` accountPattern = `^[^` + space + `]+$` labelPattern = `^[^,]*[^,` + space + `][^,]*$` diff --git a/internal/decide/branch.go b/internal/decide/branch.go index b4ddd7a..2d29810 100644 --- a/internal/decide/branch.go +++ b/internal/decide/branch.go @@ -113,8 +113,9 @@ var ( // trailerHubIDRe is the hub id syntax (hub.yml id, marker hub). trailerHubIDRe = regexp.MustCompile(`^[a-z0-9]+(-[a-z0-9]+)*$`) // trailerFingerprintRe is the fingerprint syntax (hub.yml - // previous_fingerprints, marker fp): host, optional port, numeric id. - trailerFingerprintRe = regexp.MustCompile(`^[A-Za-z0-9.-]+(:[0-9]{1,5})?/[0-9]+$`) + // previous_fingerprints, marker fp) in canonical form: host, optional port, + // and a numeric id or a lowercase Bitbucket repository UUID without braces. + trailerFingerprintRe = regexp.MustCompile(`^[A-Za-z0-9.-]+(:[0-9]{1,5})?/([0-9]+|[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})$`) ) // trailerGitPrefixes are the prefixes git itself writes; one of them lets diff --git a/internal/distribute/bitbucket_hub_test.go b/internal/distribute/bitbucket_hub_test.go new file mode 100644 index 0000000..e7a64b8 --- /dev/null +++ b/internal/distribute/bitbucket_hub_test.go @@ -0,0 +1,172 @@ +package distribute + +import ( + "strings" + "testing" + + "github.com/bedrock-python/touchmark/internal/config" + "github.com/bedrock-python/touchmark/internal/gitx" + "github.com/bedrock-python/touchmark/internal/hubch" + "github.com/bedrock-python/touchmark/internal/report" +) + +// bbDeploy is the deployment step of a hub on Bitbucket Pipelines: its +// default branch (read by the hub channel), in the deployment +// touchmark-distribute. +var bbDeploy = hubch.Context{CI: hubch.BitbucketPipelines, Host: "bitbucket.org", RepoID: "3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d", + DefaultBranch: "main", RefName: "main", RefIsBranch: true, Event: "push", Environment: "touchmark-distribute"} + +// premium is a hub that states the Premium deployment restriction. +func premium() *config.Hub { + return hubWithIsolation("platform", "Bitbucket Premium: only main may deploy to touchmark-distribute") +} + +// The guards of distribute and doctor on Bitbucket Pipelines: the default +// branch, no pull request, the deployment touchmark-distribute, and under +// platform the hub's statement of the restriction. +func TestDistributeGuardBitbucket(t *testing.T) { + with := func(edit func(*hubch.Context)) hubch.Context { + c := bbDeploy + edit(&c) + return c + } + for _, tc := range []struct { + name string + c hubch.Context + hub *config.Hub + want []string + }{ + {"premium", bbDeploy, premium(), nil}, + {"none", bbDeploy, hubWithIsolation("none", "Bitbucket Standard"), nil}, + {"external", bbDeploy, hubWithIsolation("external", ""), nil}, + // The template's hub.yml says platform and nothing else: refused, so a + // hub on Free or Standard is not taken for isolated. + {"platform without a statement", bbDeploy, hubWithIsolation("platform", ""), []string{"only Bitbucket Premium offers", "security.reason"}}, + {"the default isolation", bbDeploy, &config.Hub{ID: "acme-eng"}, []string{"only Bitbucket Premium offers"}}, + {"no deployment", with(func(c *hubch.Context) { c.Environment = "" }), premium(), []string{"deployment: touchmark-distribute"}}, + {"another deployment", with(func(c *hubch.Context) { c.Environment = "production" }), premium(), []string{"BITBUCKET_DEPLOYMENT_ENVIRONMENT"}}, + {"the deployment's name in another case", with(func(c *hubch.Context) { c.Environment = "Touchmark-Distribute" }), premium(), nil}, + {"pull request", with(func(c *hubch.Context) { c.Event, c.RefName, c.RefIsBranch = "pull_request", "feature", false }), premium(), + []string{"does not run for the pull_request event", "feature, which is not a branch"}}, + {"other branch", with(func(c *hubch.Context) { c.RefName = "feature" }), premium(), []string{"default branch main, not on feature"}}, + {"tag", with(func(c *hubch.Context) { c.RefName, c.RefIsBranch = "v1", false }), premium(), []string{"v1, which is not a branch"}}, + {"default branch unread", with(func(c *hubch.Context) { c.DefaultBranch = "" }), premium(), + []string{"Bitbucket Pipelines names none", "TOUCHMARK_PIPELINES_TOKEN"}}, + } { + t.Run(tc.name, func(t *testing.T) { + err := DistributeGuard(t.Context(), GuardInput{Context: tc.c, Hub: tc.hub, Getenv: guardEnv{}.get, GitVersion: gitx.DeliveryMinVersion}) + if len(tc.want) == 0 { + if err != nil { + t.Errorf("error %v", err) + } + return + } + if err == nil { + t.Fatalf("no error, want %q", tc.want) + } + for _, w := range tc.want { + if !strings.Contains(err.Error(), w) { + t.Errorf("error %q lacks %q", err, w) + } + } + }) + } + // doctor holds the write key too: the same context guards. + if err := DoctorGuard(with(func(c *hubch.Context) { c.Environment = "" }), guardEnv{}.get); err == nil || !strings.Contains(err.Error(), "doctor needs a step with deployment") { + t.Errorf("doctor without the deployment: %v", err) + } + if err := DoctorGuard(bbDeploy, guardEnv{}.get); err != nil { + t.Errorf("doctor in the deployment: %v", err) + } +} + +// The probe of a Bitbucket step sees what the step's variables give it: the +// repository variables (the read key, the hub's access token) pass, a write +// key among them fails, as on the other platforms. +func TestProbeBitbucket(t *testing.T) { + step := guardEnv{"BITBUCKET_BUILD_NUMBER": "7", "TOUCHMARK_READ_TOKEN": "r-0123456789", "TOUCHMARK_PIPELINES_TOKEN": "h-0123456789"} + environ := func(e guardEnv) func() []string { + return func() []string { + var out []string + for k, v := range e { + out = append(out, k+"="+v) + } + return out + } + } + if res := Probe(step.get, environ(step)); len(res.Exposed) != 0 { + t.Errorf("a step without the deployment: %q", res.Exposed) + } + step["TOUCHMARK_WRITE_TOKEN"] = "w-0123456789" + if res := Probe(step.get, environ(step)); len(res.Exposed) != 1 || res.Exposed[0] != "TOUCHMARK_WRITE_TOKEN" { + t.Errorf("a write key in a repository variable: %q", res.Exposed) + } +} + +func TestIsolationChecksBitbucket(t *testing.T) { + status := func(h *config.Hub) (report.CheckStatus, string) { + cs := IsolationChecks(IsolationInput{Context: bbDeploy, Hub: h}) + if len(cs) != 1 { + t.Fatalf("checks %+v", cs) + } + return cs[0].Status, cs[0].Detail + } + if s, d := status(hubWithIsolation("platform", "")); s != report.StatusFail || !strings.Contains(d, "only Bitbucket Premium offers") { + t.Errorf("platform without a statement: %s %s", s, d) + } + if s, d := status(premium()); s != report.StatusUnknown || !strings.Contains(d, "the hub states: Bitbucket Premium") { + t.Errorf("premium: %s %s", s, d) + } + if s, _ := status(hubWithIsolation("none", "Standard plan")); s != report.StatusWarn { + t.Errorf("none: %s", s) + } +} + +func TestKeyLocationBitbucket(t *testing.T) { + envs := []hubch.NamedEnvironment{{Name: "touchmark-distribute"}, {Name: "pages"}} + ks := hubch.KeyStore{Platform: "bitbucket", DefaultBranch: "main", Environments: envs, Secrets: []hubch.Secret{ + {Name: "TOUCHMARK_READ_TOKEN", Where: "repository", Masked: true}, + {Name: "TOUCHMARK_WRITE_TOKEN", Where: "repository", Masked: true}, + {Name: "TOUCHMARK_SIGNING_KEY", Where: "workspace", Masked: true}, + {Name: "TOUCHMARK_WRITE_TOKEN", Where: "environment", Environment: "touchmark-distribute", Masked: true}, + {Name: "TOUCHMARK_BB_WRITE_TOKEN", Where: "environment", Environment: "pages"}, + }} + lines := keyChecks(KeyLocationChecks(ks, nil, premium()), "key-location") + for _, want := range []struct { + status report.CheckStatus + sub string + }{ + {report.StatusFail, "repository variable TOUCHMARK_WRITE_TOKEN: the pipelines of every branch"}, + {report.StatusFail, "workspace variable TOUCHMARK_SIGNING_KEY"}, + {report.StatusUnknown, "deployment variable TOUCHMARK_WRITE_TOKEN of environment touchmark-distribute: Bitbucket's API does not show"}, + {report.StatusWarn, "deployment variable TOUCHMARK_BB_WRITE_TOKEN of environment pages: distribute's step deploys to touchmark-distribute"}, + {report.StatusWarn, "TOUCHMARK_BB_WRITE_TOKEN of environment pages is not secured"}, + } { + if !hasCheck(lines, want.status, want.sub) { + t.Errorf("no %s check with %q in:\n%s", want.status, want.sub, strings.Join(lines, "\n")) + } + } + if len(lines) != 5 { + t.Errorf("%d checks:\n%s", len(lines), strings.Join(lines, "\n")) + } + // Admins only: the step of anyone else pauses. + ks.Environments[0].AdminOnly = true + if lines := keyChecks(KeyLocationChecks(ks, nil, premium()), "key-location"); !hasCheck(lines, report.StatusOK, "only admins may deploy there") { + t.Errorf("admin only:\n%s", strings.Join(lines, "\n")) + } + // none: the risk is the hub's. + ks.Environments[0].AdminOnly = false + if lines := keyChecks(KeyLocationChecks(ks, nil, hubWithIsolation("none", "Standard")), "key-location"); !hasCheck(lines, report.StatusWarn, "accepted the risk") { + t.Errorf("none:\n%s", strings.Join(lines, "\n")) + } + // No environment touchmark-distribute fails; one that could not be + // listed is unknown. + bare := hubch.KeyStore{Platform: "bitbucket", DefaultBranch: "main"} + if lines := keyChecks(KeyLocationChecks(bare, nil, premium()), "key-location"); !hasCheck(lines, report.StatusFail, "the hub has no environment touchmark-distribute") { + t.Errorf("no environment:\n%s", strings.Join(lines, "\n")) + } + bare.EnvironmentsUnread = "the repository's deployment environments: HTTP 403" + if lines := keyChecks(KeyLocationChecks(bare, nil, premium()), "key-location"); !hasCheck(lines, report.StatusUnknown, "is not known") { + t.Errorf("unread environments:\n%s", strings.Join(lines, "\n")) + } +} diff --git a/internal/distribute/distribute.go b/internal/distribute/distribute.go index e35be00..6053dca 100644 --- a/internal/distribute/distribute.go +++ b/internal/distribute/distribute.go @@ -182,8 +182,10 @@ const ( const sweepOffOnly = "off: --only restricts the run to some targets" // fingerprintRe is the form of a hub fingerprint: host, optional port and -// the numeric repository id. -var fingerprintRe = regexp.MustCompile(`^[A-Za-z0-9.-]+(:[0-9]{1,5})?/[0-9]+$`) +// the repository id: numeric, or a Bitbucket repository UUID (with or +// without braces, any case; config.CanonicalFingerprint makes it lowercase +// without braces). +var fingerprintRe = regexp.MustCompile(`^[A-Za-z0-9.-]+(:[0-9]{1,5})?/([0-9]+|[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}|\{[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}\})$`) // HeadGuard reports whether guard I8 (a run from an old hub commit does // nothing) applies to a run in context c: a local run, or a CI run that diff --git a/internal/distribute/doctor_hub.go b/internal/distribute/doctor_hub.go index 6690a2b..280b317 100644 --- a/internal/distribute/doctor_hub.go +++ b/internal/distribute/doctor_hub.go @@ -20,9 +20,9 @@ import ( // DoctorGuard runs the guards of doctor before any platform call: in CI // (a CI hubch recognizes, or CI set), the job builds the hub's default -// branch for no pull request, and on GitLab in the protected environment -// touchmark-distribute, as distribute's (the writer is used on the default -// branch only). A local run passes. +// branch for no pull request, and on GitLab and Bitbucket in the +// environment touchmark-distribute, as distribute's (the writer is used on +// the default branch only). A local run passes. func DoctorGuard(c hubch.Context, getenv func(string) string) error { if !guardInCI(c, getenv) { return nil @@ -57,6 +57,9 @@ type IsolationInput struct { // pipelines and doctor --hub-token tell; // - platform, on Gitea and Forgejo Actions: fail, their secrets reach // every branch's jobs; +// - platform, on Bitbucket Pipelines: fail without security.reason +// (BitbucketPlatformRefusal); with it unknown: the API does not show +// who may deploy, and the reason states it; // - platform, locally: unknown, doctor --hub-token tells. func IsolationChecks(in IsolationInput) []report.DoctorCheck { mode := "platform" @@ -114,6 +117,18 @@ func IsolationChecks(in IsolationInput) []report.DoctorCheck { case hubch.GiteaActions, hubch.ForgejoActions: c.Status, c.Detail = report.StatusFail, "security.write_isolation: platform cannot hold on Gitea and Forgejo Actions, whose secrets every branch's jobs see: "+ "run distribute on GitHub Actions or GitLab CI, or set write_isolation to external, or to none with a reason; see "+docsurl.WriteIsolation + case hubch.BitbucketPipelines: + reason := "" + if in.Hub != nil { + reason = strings.TrimSpace(in.Hub.Security.Reason) + } + if reason == "" { + c.Status, c.Detail = report.StatusFail, BitbucketPlatformRefusal + break + } + c.Status, c.Detail = report.StatusUnknown, "Bitbucket's API does not show which branches may deploy to "+distributeEnvironment+ + " (Premium deployment permissions); the hub states: "+reason+". `touchmark probe` in the hub's pipelines fails when a repository "+ + "or workspace variable holds a write key, and doctor --hub-token reads the variables" default: c.Status, c.Detail = report.StatusUnknown, "a local run cannot see where the hub's CI keeps the write key: doctor --hub-token reads it with a maintainer's token" } @@ -156,6 +171,12 @@ func IsWriteKeyName(name string, known []string) bool { // no_one_allowed warns. // - Gitea and Forgejo: a write key in an Actions secret fails, unless // the hub accepted the risk (none: warn). +// - Bitbucket: a write key in a repository or workspace variable fails +// (every branch's pipelines, pull requests included, read it); one in +// a deployment variable of touchmark-distribute is unknown (the API +// does not show which branches may deploy), ok when only admins may +// deploy there, a warning under none, and a warning in another +// environment; one not secured warns. // // Under security.write_isolation external no write key is expected in the // hub, and one found there is graded the same. A key found nowhere is @@ -184,6 +205,8 @@ func KeyLocationChecks(ks hubch.KeyStore, known []string, cfg *config.Hub) []rep out = append(out, githubKeyCheck(s, envs, ks.DefaultBranch)) case "gitlab": out = append(out, gitlabKeyChecks(s)...) + case "bitbucket": + out = append(out, bitbucketKeyChecks(s, envs, mode)...) default: if mode == "none" { add(report.StatusWarn, "Actions secret %s (%s): every branch's jobs read it; the hub accepted the risk (security.write_isolation: none)", s.Name, s.Where) @@ -195,13 +218,13 @@ func KeyLocationChecks(ks hubch.KeyStore, known []string, cfg *config.Hub) []rep if found == 0 { add(report.StatusUnknown, "no secret or variable named like a write key was found where this token can see; the key may live elsewhere (an organization secret not shared with the hub, a parent group, an external store)") } - if ks.Platform == "github" { + if ks.Platform == "github" || ks.Platform == "bitbucket" { switch _, ok := envs[distributeEnvironment]; { case ok || mode != "platform": case ks.EnvironmentsUnread != "": add(report.StatusUnknown, "whether the hub has the environment %s is not known: %s", distributeEnvironment, ks.EnvironmentsUnread) default: - add(report.StatusFail, "the hub has no environment %s: keep the write key in its secrets, with Deployment branches set to the default branch only", distributeEnvironment) + add(report.StatusFail, "the hub has no environment %s: keep the write key in its secrets (its deployment variables on Bitbucket), with deployments limited to the default branch", distributeEnvironment) } } if ks.Platform == "gitlab" { @@ -244,6 +267,39 @@ func githubKeyCheck(s hubch.Secret, envs map[string]hubch.NamedEnvironment, defa return c } +// bitbucketKeyChecks grade one write-key variable on Bitbucket. +func bitbucketKeyChecks(s hubch.Secret, envs map[string]hubch.NamedEnvironment, mode string) []report.DoctorCheck { + c := report.DoctorCheck{Name: "key-location"} + where := s.Where + " variable " + s.Name + switch s.Where { + case "repository", "workspace": + c.Status, c.Detail = report.StatusFail, fmt.Sprintf("%s: the pipelines of every branch, pull requests included, read it; "+ + "move it into the deployment environment %s", where, distributeEnvironment) + case "environment": + where = fmt.Sprintf("deployment variable %s of environment %s", s.Name, s.Environment) + switch { + case s.Environment != distributeEnvironment: + c.Status, c.Detail = report.StatusWarn, fmt.Sprintf("%s: distribute's step deploys to %s", where, distributeEnvironment) + case mode == "none": + c.Status, c.Detail = report.StatusWarn, where+": any branch's pipeline may deploy there and read it; the hub accepted the risk (security.write_isolation: none)" + case envs[s.Environment].AdminOnly: + c.Status, c.Detail = report.StatusOK, where+": only admins may deploy there (restrictions.admin_only), so the step of anyone else pauses" + default: + c.Status, c.Detail = report.StatusUnknown, where+": Bitbucket's API does not show which branches may deploy there; with Premium "+ + "restrict it to the default branch (Repository settings > Deployments) and state that in security.reason; without Premium "+ + "any branch's pipeline reads it" + } + default: + c.Status, c.Detail = report.StatusUnknown, fmt.Sprintf("variable %s (%s)", s.Name, s.Where) + } + out := []report.DoctorCheck{c} + if !s.Masked { + out = append(out, report.DoctorCheck{Name: "key-location", Status: report.StatusWarn, + Detail: where + " is not secured: its value shows in the settings and the API, and the logs do not mask it; make it Secured"}) + } + return out +} + // gitlabKeyChecks grade one write-key variable on GitLab. func gitlabKeyChecks(s hubch.Secret) []report.DoctorCheck { where := "project variable " + s.Name diff --git a/internal/distribute/guards.go b/internal/distribute/guards.go index fd93b2e..4c92000 100644 --- a/internal/distribute/guards.go +++ b/internal/distribute/guards.go @@ -48,12 +48,14 @@ const ( // - in CI: the run builds the hub's default branch (RefIsBranch and // RefName == DefaultBranch) and the event is not pull_request* or // merge_request_event; on GitLab also CI_COMMIT_REF_PROTECTED=true and -// CI_ENVIRONMENT_NAME=touchmark-distribute; +// CI_ENVIRONMENT_NAME=touchmark-distribute; on Bitbucket Pipelines the +// step's deployment touchmark-distribute (BITBUCKET_DEPLOYMENT_ENVIRONMENT); // - write isolation (security.write_isolation): "platform" on GitHub // Actions needs TOUCHMARK_KEY_EXPOSED=false (true: the key is visible // outside the environment; unset: the probe job is missing); "none" // needs security.reason (config enforces it) and adds a red report -// warning; "external" skips the probe. +// warning; "external" skips the probe. "platform" on Bitbucket +// Pipelines needs security.reason too (guardIsolation). // // A dry run gets every check: it holds the write credential as distribute // does, and builds its drivers from the hub.yml of the ref it runs on (no @@ -72,7 +74,10 @@ const ( // every branch's jobs see: refused there, with the ways out (external, // or none with a reason). On GitLab the probe runs in the hub's merge // request pipelines instead (`touchmark probe`), and a local run has no -// probe. The red warning for "none" is the caller's to add to the +// probe. On Bitbucket Pipelines it holds only on Premium, whose +// deployment permissions keep other branches (or people) out of the +// environment; the API does not show them, so the hub states them in +// security.reason, and without that statement "platform" is refused. The red warning for "none" is the caller's to add to the // report: the guards return errors only. // // Every failed check is reported, joined, never a variable's value. @@ -116,7 +121,8 @@ func guardInCI(c hubch.Context, getenv func(string) string) bool { } // guardContext checks that a CI job of distribute builds the hub's default -// branch, for no pull request, and on GitLab in the protected environment. +// branch, for no pull request, and on GitLab and Bitbucket in the +// environment that holds the write key. func guardContext(c hubch.Context) []error { return guardContextOf("distribute", c) } // guardContextOf is guardContext for command cmd (distribute, doctor): the @@ -128,12 +134,17 @@ func guardContextOf(cmd string, c hubch.Context) []error { errs = append(errs, fmt.Errorf("%s does not run for the %s event: only a run of the hub's default branch may hold the write key", cmd, ev)) } switch { - case c.CI != hubch.GitHubActions && c.CI != hubch.GitLabCI && c.CI != hubch.GiteaActions && c.CI != hubch.ForgejoActions: - errs = append(errs, fmt.Errorf("%s cannot tell which ref this CI job builds: it runs in CI only on GitHub Actions, GitLab CI, Gitea Actions and Forgejo Actions", cmd)) + case c.CI != hubch.GitHubActions && c.CI != hubch.GitLabCI && c.CI != hubch.GiteaActions && c.CI != hubch.ForgejoActions && + c.CI != hubch.BitbucketPipelines: + errs = append(errs, fmt.Errorf("%s cannot tell which ref this CI job builds: it runs in CI only on GitHub Actions, GitLab CI, Gitea Actions, "+ + "Forgejo Actions and Bitbucket Pipelines", cmd)) case !c.RefIsBranch: errs = append(errs, fmt.Errorf("%s runs only on the hub's default branch; this job builds %s, which is not a branch", cmd, guardRef(c.RefName))) case c.DefaultBranch != "" && c.RefName != c.DefaultBranch: errs = append(errs, fmt.Errorf("%s runs only on the hub's default branch %s, not on %s", cmd, c.DefaultBranch, c.RefName)) + case c.DefaultBranch == "" && c.CI == hubch.BitbucketPipelines: + errs = append(errs, fmt.Errorf("%s cannot tell whether %s is the hub's default branch: Bitbucket Pipelines names none, and the hub "+ + "channel could not read it; give the hub a repository access token with Repositories: Read in %s", cmd, c.RefName, hubch.BitbucketTokenVar)) case c.DefaultBranch == "" && c.Event != "schedule": errs = append(errs, fmt.Errorf("the CI names no default branch of the hub, so %s cannot tell whether %s is it", cmd, c.RefName)) } @@ -145,6 +156,10 @@ func guardContextOf(cmd string, c hubch.Context) []error { errs = append(errs, fmt.Errorf("%s needs the job's environment %s on GitLab (CI_ENVIRONMENT_NAME), which holds the write key", cmd, distributeEnvironment)) } } + if c.CI == hubch.BitbucketPipelines && !strings.EqualFold(c.Environment, distributeEnvironment) { + errs = append(errs, fmt.Errorf("%s needs a step with deployment: %s on Bitbucket Pipelines (BITBUCKET_DEPLOYMENT_ENVIRONMENT), "+ + "whose deployment variables hold the write key", cmd, distributeEnvironment)) + } return errs } @@ -174,10 +189,33 @@ func guardIsolation(in GuardInput, getenv func(string) string) error { case hubch.GiteaActions, hubch.ForgejoActions: return errors.New("security.write_isolation: platform cannot hold on Gitea and Forgejo Actions, whose secrets every branch's jobs see: " + "run distribute on GitHub Actions or GitLab CI, or set write_isolation to external, or to none with a reason") + case hubch.BitbucketPipelines: + if in.Hub == nil || strings.TrimSpace(in.Hub.Security.Reason) == "" { + return errors.New(BitbucketPlatformRefusal) + } } return nil } +// BitbucketPlatformRefusal is why security.write_isolation platform without +// security.reason is refused on Bitbucket Pipelines. +// +// Any branch's bitbucket-pipelines.yml may name a step with deployment: +// touchmark-distribute and so read its deployment variables, the write key +// among them, unless the environment's deployment permissions (Premium: +// only some branches, or only admins, may deploy; a step of another +// branch pauses; +// https://support.atlassian.com/bitbucket-cloud/docs/set-up-and-monitor-deployments/) +// stop it. Bitbucket's API does not show the deployment branch +// restriction, so touchmark cannot check it: the hub states it in +// security.reason, reviewed like the rest of hub.yml. The template's +// hub.yml does not, so a hub on Bitbucket is safe by default: it must +// either restrict the environment and say so, or accept the risk (none). +const BitbucketPlatformRefusal = "security.write_isolation: platform on Bitbucket Pipelines needs the deployment environment " + distributeEnvironment + + " restricted to the default branch (or to admins), which only Bitbucket Premium offers and its API does not show: restrict it " + + "(Repository settings > Deployments) and state that in security.reason; on Free and Standard set write_isolation to none with a reason, " + + "or to external; see " + docsurl.WriteIsolation + // DistributeEnvironment is the environment of the hub's CI that holds the // write key and runs distribute. const DistributeEnvironment = distributeEnvironment diff --git a/internal/docsurl/docsurl_test.go b/internal/docsurl/docsurl_test.go index 16d3e78..75583cf 100644 --- a/internal/docsurl/docsurl_test.go +++ b/internal/docsurl/docsurl_test.go @@ -30,7 +30,7 @@ func slug(heading string) string { // and its anchor a heading of that page. func TestPagesExist(t *testing.T) { docs := filepath.Join("..", "..", "docs") - for _, url := range []string{WriteIsolation, Migrate, GiteaForgejo, GettingStarted("github"), GettingStarted("gitlab")} { + for _, url := range []string{WriteIsolation, Migrate, GiteaForgejo, GettingStarted("github"), GettingStarted("gitlab"), GettingStarted("bitbucket")} { rest, ok := strings.CutPrefix(url, Base) if !ok { t.Errorf("%s is not under %s", url, Base) diff --git a/internal/e2e/bitbucket/doc.go b/internal/e2e/bitbucket/doc.go new file mode 100644 index 0000000..5fcafe7 --- /dev/null +++ b/internal/e2e/bitbucket/doc.go @@ -0,0 +1,13 @@ +// Package bitbuckete2e checks the hub template's Bitbucket Pipelines file, +// bitbucket-pipelines.yml of the template's working tree in +// TOUCHMARK_E2E_TEMPLATE, without Bitbucket: TestTemplatePipelines reads +// its pipelines and plays what Bitbucket Pipelines gives each step (the +// default variables, the repository variables, and a deployment's +// variables only to a step that names it), then runs touchmark's own +// guards on those steps. Like the GitHub package it has no build tag, and +// skips without TOUCHMARK_E2E_TEMPLATE or when the template has no +// bitbucket-pipelines.yml. +// +// What only a live Bitbucket can confirm is listed in +// docs/getting-started/bitbucket.md ("Assumed, not yet seen live"). +package bitbuckete2e diff --git a/internal/e2e/bitbucket/template_test.go b/internal/e2e/bitbucket/template_test.go new file mode 100644 index 0000000..53b7ef0 --- /dev/null +++ b/internal/e2e/bitbucket/template_test.go @@ -0,0 +1,376 @@ +package bitbuckete2e + +import ( + "errors" + "os" + "path/filepath" + "regexp" + "slices" + "strings" + "testing" + + "go.yaml.in/yaml/v3" + + "github.com/bedrock-python/touchmark/internal/config" + "github.com/bedrock-python/touchmark/internal/distribute" + "github.com/bedrock-python/touchmark/internal/gitx" + "github.com/bedrock-python/touchmark/internal/hubch" +) + +// The file's shape, as far as the test reads it +// (support.atlassian.com/bitbucket-cloud/docs/bitbucket-pipelines-configuration-reference/). +type pipelinesFile struct { + Image any `yaml:"image"` + Clone cloneOpts `yaml:"clone"` + Pipelines struct { + Default []item `yaml:"default"` + Branches map[string][]item `yaml:"branches"` + Tags map[string][]item `yaml:"tags"` + PullRequests map[string][]item `yaml:"pull-requests"` + Custom map[string][]item `yaml:"custom"` + } `yaml:"pipelines"` +} + +type cloneOpts struct { + Depth any `yaml:"depth"` + Enabled *bool `yaml:"enabled"` +} + +type item struct { + Step *step `yaml:"step"` + Parallel any `yaml:"parallel"` + Stage any `yaml:"stage"` +} + +type step struct { + Name string `yaml:"name"` + Image any `yaml:"image"` + Deployment string `yaml:"deployment"` + Trigger string `yaml:"trigger"` + Clone *cloneOpts `yaml:"clone"` + MaxTime int `yaml:"max-time"` + Script []string `yaml:"script"` + Artifacts []string `yaml:"artifacts"` +} + +// pipeline is one pipeline of the file: its steps, and what starts it. +type pipeline struct { + name string + steps []step + // env is what Bitbucket sets in every step of a run of it. + env map[string]string +} + +const ( + hubUUID = "{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d}" + readTok = "reader-api-token-0123456789" + hubTok = "hub-access-token-0123456789" + writeTok = "writer-api-token-0123456789" +) + +var ( + imageRe = regexp.MustCompile(`^ghcr\.io/bedrock-python/touchmark:[^@\s]+@sha256:[0-9a-f]{64}$`) + commandRe = regexp.MustCompile(`^touchmark (probe|check|plan|distribute|doctor)\b`) +) + +// TestTemplatePipelines checks the security properties of the template's +// bitbucket-pipelines.yml: +// +// - every step runs the touchmark image, pinned by digest, with the hub's +// whole history; no step sets an image of its own; +// - the pipelines and their touchmark commands: a pull request runs +// probe, check, plan --strict --comment; the push pipeline of main and +// master, and the custom pipeline distribute, run probe then +// distribute; the custom pipeline doctor runs probe then doctor; there +// is no default or tag pipeline, which would run on any branch or tag; +// - only distribute and doctor run with deployment: touchmark-distribute, +// and every pipeline starts with the probe, without it; +// - no script names a TOUCHMARK_ variable, and the custom pipelines' +// scripts name none at all: the website asks for variables a custom +// pipeline's script references, and a pipeline variable overrides a +// deployment variable; +// - played with the variables Bitbucket gives each step (the repository's +// to every step, the deployment's to its steps only): the probe passes +// and touchmark's guards admit distribute and doctor on the default +// branch of a hub that states its Premium restriction, and refuse them +// on another branch, in a pull request, and under platform without the +// statement; with the write key leaked into a repository variable, +// every pipeline stops at its probe, before any step that holds it. +func TestTemplatePipelines(t *testing.T) { + dir := os.Getenv("TOUCHMARK_E2E_TEMPLATE") + if dir == "" { + t.Skip("TOUCHMARK_E2E_TEMPLATE is not set: it names the hub template's working tree") + } + data, err := os.ReadFile(filepath.Join(dir, "bitbucket-pipelines.yml")) + if errors.Is(err, os.ErrNotExist) { + t.Skip("the template has no bitbucket-pipelines.yml") + } + if err != nil { + t.Fatal(err) + } + var f pipelinesFile + if err := yaml.Unmarshal(data, &f); err != nil { + t.Fatalf("bitbucket-pipelines.yml: %v", err) + } + if s, _ := f.Image.(string); !imageRe.MatchString(s) { + t.Errorf("image %v: want touchmark pinned by digest", f.Image) + } + if d, _ := f.Clone.Depth.(string); d != "full" { + t.Errorf("clone depth %v: touchmark reads the hub's whole history", f.Clone.Depth) + } + if len(f.Pipelines.Default) > 0 || len(f.Pipelines.Tags) > 0 { + t.Error("a default or tag pipeline runs on any branch or tag") + } + + pr := map[string]string{"BITBUCKET_BRANCH": "feature", "BITBUCKET_PR_ID": "41", "BITBUCKET_PR_DESTINATION_BRANCH": "main"} + main := map[string]string{"BITBUCKET_BRANCH": "main"} + var all []pipeline + add := func(name string, items []item, env map[string]string) { + var steps []step + for i, it := range items { + if it.Step == nil || it.Parallel != nil || it.Stage != nil { + t.Errorf("%s: item %d is not a plain step", name, i) + continue + } + steps = append(steps, *it.Step) + } + all = append(all, pipeline{name: name, steps: steps, env: env}) + } + if len(f.Pipelines.PullRequests) != 1 || f.Pipelines.PullRequests["**"] == nil { + t.Fatalf("pull-requests %v: want one pipeline for every branch, \"**\"", keys(f.Pipelines.PullRequests)) + } + add("pull-requests **", f.Pipelines.PullRequests["**"], pr) + var branchPattern string + for pattern, items := range f.Pipelines.Branches { + branchPattern = pattern + add("branches "+pattern, items, main) + } + if len(f.Pipelines.Branches) != 1 || !matchesBoth(branchPattern) { + t.Errorf("branches %v: want one pipeline for main and master", keys(f.Pipelines.Branches)) + } + for name, items := range f.Pipelines.Custom { + add("custom "+name, items, main) + } + if got := keys(f.Pipelines.Custom); !slices.Equal(got, []string{"distribute", "doctor"}) { + t.Errorf("custom pipelines %q, want distribute and doctor", got) + } + + want := map[string][]string{ + "pull-requests **": {"probe", "check", "plan"}, + "branches " + branchPattern: {"probe", "distribute"}, + "custom distribute": {"probe", "distribute"}, + "custom doctor": {"probe", "doctor"}, + } + for _, p := range all { + var cmds []string + for _, s := range p.steps { + cmd := checkStep(t, p, s) + cmds = append(cmds, cmd) + } + if !slices.Equal(cmds, want[p.name]) { + t.Errorf("%s runs %q, want %q", p.name, cmds, want[p.name]) + } + } + if t.Failed() { + return + } + + premium := &config.Hub{ID: "acme-eng", Security: config.Security{WriteIsolation: "platform", + Reason: "Bitbucket Premium: only main may deploy to touchmark-distribute"}} + repoVars := map[string]string{"TOUCHMARK_READ_TOKEN": readTok, "TOUCHMARK_PIPELINES_TOKEN": hubTok} + deployVars := map[string]string{"TOUCHMARK_WRITE_TOKEN": writeTok} + + // As configured: every step runs; the probe passes; the deployment + // steps pass touchmark's guards on the default branch. + for _, p := range all { + ran := play(t, p, repoVars, deployVars, premium, "") + if len(ran) != len(p.steps) { + t.Errorf("%s: ran %q of %d steps", p.name, ran, len(p.steps)) + } + } + // The custom pipelines run by hand on another branch: refused. + for _, p := range all { + if strings.HasPrefix(p.name, "custom ") { + q := p + q.env = map[string]string{"BITBUCKET_BRANCH": "feature"} + if ran := play(t, q, repoVars, deployVars, premium, "default branch main, not on feature"); len(ran) == len(p.steps) { + t.Errorf("%s on another branch ran every step", p.name) + } + } + } + // Free or Standard, the template's hub.yml: platform without a + // statement is refused. + plain := &config.Hub{ID: "acme-eng", Security: config.Security{WriteIsolation: "platform"}} + for _, p := range all { + if p.name == "custom doctor" || strings.HasPrefix(p.name, "branches ") { + play(t, p, repoVars, deployVars, plain, "only Bitbucket Premium offers") + } + } + // The write key leaked into a repository variable: every pipeline stops + // at its probe. + leaked := map[string]string{"TOUCHMARK_WRITE_TOKEN": writeTok} + for k, v := range repoVars { + leaked[k] = v + } + for _, p := range all { + if ran := play(t, p, leaked, deployVars, premium, ""); len(ran) != 0 { + t.Errorf("%s with a leaked write key ran %q", p.name, ran) + } + } +} + +// checkStep checks one step and returns its touchmark command. +func checkStep(t *testing.T, p pipeline, s step) string { + t.Helper() + if s.Image != nil { + t.Errorf("%s: step %s sets its own image", p.name, s.Name) + } + if s.Trigger != "" && s.Trigger != "automatic" { + t.Errorf("%s: step %s has trigger %s", p.name, s.Name, s.Trigger) + } + var cmd string + for _, line := range s.Script { + if strings.Contains(line, "TOUCHMARK_") { + t.Errorf("%s: step %s names a touchmark variable: %s", p.name, s.Name, line) + } + if strings.HasPrefix(p.name, "custom ") && strings.Contains(line, "$") { + t.Errorf("%s: step %s references a variable, which Run pipeline would ask for: %s", p.name, s.Name, line) + } + if m := commandRe.FindStringSubmatch(line); m != nil { + if cmd != "" { + t.Errorf("%s: step %s runs two touchmark commands", p.name, s.Name) + } + cmd = m[1] + } + } + switch { + case cmd == "": + t.Errorf("%s: step %s runs no touchmark command", p.name, s.Name) + case cmd == "distribute" || cmd == "doctor": + if s.Deployment != "touchmark-distribute" { + t.Errorf("%s: %s runs with deployment %q, want touchmark-distribute", p.name, cmd, s.Deployment) + } + case s.Deployment != "": + t.Errorf("%s: %s runs with deployment %s, which holds the write key", p.name, cmd, s.Deployment) + } + script := strings.Join(s.Script, "\n") + if cmd == "plan" && (!strings.Contains(script, "--strict") || !strings.Contains(script, "--comment")) { + t.Errorf("%s: plan without --strict --comment", p.name) + } + if cmd == "distribute" && (s.MaxTime < 330 || !strings.Contains(script, "--deadline")) { + t.Errorf("%s: distribute needs a deadline under its max-time (%d minutes)", p.name, s.MaxTime) + } + if s.Name != cmd { + t.Errorf("%s: step %q runs %s", p.name, s.Name, cmd) + } + if s.Clone != nil && s.Clone.Depth != nil { + t.Errorf("%s: step %s clones a shallow history", p.name, s.Name) + } + return cmd +} + +// play runs pipeline p as Bitbucket would with these variables, step by +// step, until one fails: the probe through distribute.Probe, distribute +// and doctor through touchmark's guards (with the default branch main, as +// the hub channel reads it), and the other steps only through the guard of +// plan (no write key in the step). It returns the steps that ran to the +// end. wantErr, when set, is part of the error that must stop it. +func play(t *testing.T, p pipeline, repoVars, deployVars map[string]string, hub *config.Hub, wantErr string) []string { + t.Helper() + var ran []string + for _, s := range p.steps { + env := map[string]string{ + "CI": "true", "BITBUCKET_BUILD_NUMBER": "17", "BITBUCKET_REPO_UUID": hubUUID, + "BITBUCKET_REPO_FULL_NAME": "acme/engineering-assets", "BITBUCKET_REPO_IS_PRIVATE": "true", + } + for k, v := range p.env { + env[k] = v + } + for k, v := range repoVars { + env[k] = v + } + if s.Deployment != "" { + env["BITBUCKET_DEPLOYMENT_ENVIRONMENT"] = s.Deployment + for k, v := range deployVars { + env[k] = v + } + } + getenv := func(k string) string { return env[k] } + environ := func() []string { + var out []string + for k, v := range env { + out = append(out, k+"="+v) + } + return out + } + var err error + switch s.Name { + case "probe": + if res := distribute.Probe(getenv, environ); len(res.Exposed) > 0 { + err = errors.New("probe: " + strings.Join(res.Exposed, ", ") + " visible") + } + case "distribute", "doctor": + hctx := hubch.Detect(getenv, nil) + if hctx.CI != hubch.BitbucketPipelines || hctx.Fingerprint() != "bitbucket.org/3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d" { + t.Fatalf("%s: the step's context %+v", p.name, hctx) + } + hctx.DefaultBranch = "main" + if s.Name == "doctor" { + err = distribute.DoctorGuard(hctx, getenv) + if err == nil { + err = isolationFails(hctx, hub) + } + } else { + err = distribute.DistributeGuard(t.Context(), distribute.GuardInput{Context: hctx, Hub: hub, Getenv: getenv, GitVersion: gitx.DeliveryMinVersion}) + } + default: + if res := distribute.Probe(getenv, environ); len(res.Exposed) > 0 { + err = errors.New(s.Name + " sees " + strings.Join(res.Exposed, ", ")) + } + } + if err != nil { + if wantErr != "" && !strings.Contains(err.Error(), wantErr) { + t.Errorf("%s: step %s: %v, want %q", p.name, s.Name, err, wantErr) + } + if wantErr == "" && !strings.HasPrefix(err.Error(), "probe: ") { + t.Errorf("%s: step %s: %v", p.name, s.Name, err) + } + return ran + } + ran = append(ran, s.Name) + } + if wantErr != "" { + t.Errorf("%s: every step ran, want %q", p.name, wantErr) + } + return ran +} + +// isolationFails is doctor's write-isolation check as an error when it +// fails: doctor reports it, and exits 1. +func isolationFails(hctx hubch.Context, hub *config.Hub) error { + for _, c := range distribute.IsolationChecks(distribute.IsolationInput{Context: hctx, Hub: hub}) { + if c.Status == "fail" { + return errors.New(c.Detail) + } + } + return nil +} + +// matchesBoth reports whether a branches pattern of the file takes main +// and master: a name, or a {a,b} alternation of names. +func matchesBoth(pattern string) bool { + alts := []string{pattern} + if strings.HasPrefix(pattern, "{") && strings.HasSuffix(pattern, "}") { + alts = strings.Split(pattern[1:len(pattern)-1], ",") + } + return slices.Contains(alts, "main") && slices.Contains(alts, "master") +} + +func keys[V any](m map[string]V) []string { + var out []string + for k := range m { + out = append(out, k) + } + slices.Sort(out) + return out +} diff --git a/internal/gitx/gitx.go b/internal/gitx/gitx.go index e6b9b29..5120590 100644 --- a/internal/gitx/gitx.go +++ b/internal/gitx/gitx.go @@ -4,9 +4,11 @@ // Every command gets: // - "-c core.quotePath=false -c color.ui=false -c log.showSignature=false"; // - GIT_TERMINAL_PROMPT=0, GIT_OPTIONAL_LOCKS=0, LC_ALL=C, LANGUAGE=C; -// - no inherited variables that select another repository or change -// pathspec matching (GIT_DIR, GIT_LITERAL_PATHSPECS, …; see Run), nor -// touchmark's own (TOUCHMARK_*: credentials among them); +// - no inherited variables that select another repository, change +// pathspec matching, add config or name a program git runs +// (GIT_DIR, GIT_LITERAL_PATHSPECS, GIT_CONFIG_*, GIT_SSH_COMMAND, +// LD_PRELOAD, …; see droppedEnv), nor touchmark's own (TOUCHMARK_*: +// credentials among them); // - NUL-separated output (-z) wherever git offers it; // - on a cancelled context, a stop of the whole process tree it started // (SIGTERM to its process group on Unix, so git cleans up its lock @@ -105,18 +107,22 @@ var defaultEnv = []string{ } // droppedEnv lists inherited variables that are not passed to git; Git.Env -// may still set them explicitly. Neither is any variable whose name starts -// with droppedPrefix: touchmark's own, among them the write credentials and -// signing keys a CI job has in its environment until distribute reads them. -// git needs none of them, and whatever git starts (a hook, a filter, a -// credential helper of a repository's config) must not see them. +// may still set them explicitly (the isolation of a target's repository and +// the hub channel set GIT_CONFIG_GLOBAL and GIT_CONFIG_COUNT this way). +// Neither is any variable whose name starts with one of droppedPrefixes. // -// The first group would point git at another repository, index or object -// store than Dir (git sets them for hooks, for example): the non-config -// entries of `git rev-parse --local-env-vars`. The second group changes how -// pathspecs match; GIT_LITERAL_PATHSPECS, for one, would silently turn the -// ":(top)" pathspec of History into a path that matches nothing. +// Whoever sets the environment touchmark starts in (a CI variable of the +// hub, a custom pipeline run with variables, a step before touchmark's) must +// not be able to change what git does in the hub or a target through it: +// run a program, read another config or repository, or write a file. The +// variables that only take something away from git (GIT_CONFIG_NOSYSTEM, +// GIT_ATTR_NOSYSTEM, GIT_CEILING_DIRECTORIES) stay: they cannot make git run +// or read anything, and tests rely on them to keep the machine's config out. var droppedEnv = []string{ + // Another repository, index or object store than Dir (git sets them + // for hooks, for example): the non-config entries of + // `git rev-parse --local-env-vars`, and GIT_NAMESPACE, which moves every + // ref git reads. "GIT_ALTERNATE_OBJECT_DIRECTORIES", "GIT_OBJECT_DIRECTORY", "GIT_DIR", @@ -130,20 +136,91 @@ var droppedEnv = []string{ "GIT_INTERNAL_SUPER_PREFIX", "GIT_SHALLOW_FILE", "GIT_COMMON_DIR", + "GIT_NAMESPACE", + "GIT_DISCOVERY_ACROSS_FILESYSTEM", + // How pathspecs match; GIT_LITERAL_PATHSPECS, for one, would silently + // turn the ":(top)" pathspec of History into a path that matches + // nothing. "GIT_LITERAL_PATHSPECS", "GIT_GLOB_PATHSPECS", "GIT_NOGLOB_PATHSPECS", "GIT_ICASE_PATHSPECS", + + // Config from outside the repository: any setting, core.fsmonitor or + // core.hooksPath among them, names a program git runs on an ordinary + // read of the index. GIT_CONFIG_COUNT and its GIT_CONFIG_KEY_n and + // GIT_CONFIG_VALUE_n (droppedPrefixes) and GIT_CONFIG_PARAMETERS carry + // settings themselves; GIT_CONFIG_GLOBAL and GIT_CONFIG_SYSTEM name a + // file of them, and GIT_CONFIG the file git config writes. + "GIT_CONFIG_COUNT", + "GIT_CONFIG_PARAMETERS", + "GIT_CONFIG_GLOBAL", + "GIT_CONFIG_SYSTEM", + "GIT_CONFIG", + + // Where attributes come from (git 2.41): they decide how + // HashPaths hashes a target's file. + "GIT_ATTR_SOURCE", + + // Programs git runs: its own helpers (GIT_EXEC_PATH), ssh and its + // password prompts, a proxy, a diff tool, a pager and an editor. + "GIT_EXEC_PATH", + "GIT_SSH", + "GIT_SSH_COMMAND", + "GIT_SSH_VARIANT", + "GIT_ASKPASS", + "SSH_ASKPASS", + "GIT_PROXY_COMMAND", + "GIT_EXTERNAL_DIFF", + "GIT_PAGER", + "PAGER", + "GIT_EDITOR", + "GIT_SEQUENCE_EDITOR", + "EDITOR", + "VISUAL", + + // Files git copies into a new repository, hooks among them. + "GIT_TEMPLATE_DIR", + + // Prints request headers, credentials included (GIT_TRACE* is a + // prefix). + "GIT_CURL_VERBOSE", +} + +// droppedPrefixes start the names of inherited variables no git process +// gets (see droppedEnv): +// - TOUCHMARK_: touchmark's own, among them the write credentials and +// signing keys a CI job has in its environment until distribute reads +// them; git needs none of them, and whatever git starts (a hook, a +// filter, a credential helper of a repository's config) must not see +// them; +// - GIT_CONFIG_KEY_ and GIT_CONFIG_VALUE_: the settings of +// GIT_CONFIG_COUNT, dropped even when the count is, so a git that reads +// them some other way finds none; +// - GIT_TRACE: GIT_TRACE, GIT_TRACE2_EVENT and the rest take a file name +// to write to, and print headers and packets; +// - LD_ and DYLD_: the dynamic loader's, which load a library of the +// caller's choice into git (LD_PRELOAD, LD_LIBRARY_PATH, LD_AUDIT, +// DYLD_INSERT_LIBRARIES, …) or write its debug output to a file. +var droppedPrefixes = []string{ + "TOUCHMARK_", + "GIT_CONFIG_KEY_", + "GIT_CONFIG_VALUE_", + "GIT_TRACE", + "LD_", + "DYLD_", } // Run runs git with args and returns stdout. stdin may be nil. // // A command that exits non-zero returns an *Error. Variables that locate -// another repository (GIT_DIR, GIT_WORK_TREE, GIT_INDEX_FILE, …) or change -// pathspec matching (GIT_LITERAL_PATHSPECS, …) are not inherited from the -// environment; set them through Env when needed. Neither are touchmark's -// own variables (TOUCHMARK_*). +// another repository (GIT_DIR, GIT_WORK_TREE, GIT_INDEX_FILE, …), change +// pathspec matching (GIT_LITERAL_PATHSPECS, …), add config +// (GIT_CONFIG_COUNT, GIT_CONFIG_PARAMETERS, GIT_CONFIG_GLOBAL, …) or name a +// program git runs (GIT_SSH_COMMAND, GIT_EXEC_PATH, LD_PRELOAD, …) are not +// inherited from the environment (droppedEnv); set them through Env when +// needed. Neither are touchmark's own variables (TOUCHMARK_*). func (g *Git) Run(ctx context.Context, stdin io.Reader, args ...string) ([]byte, error) { out, err := g.runPartial(ctx, stdin, args...) if err != nil { @@ -206,17 +283,15 @@ func (g *Git) environ() []string { return append(env, g.Env...) } -// droppedPrefix starts the names of touchmark's variables, which no git -// process inherits (see droppedEnv). -const droppedPrefix = "TOUCHMARK_" - // isDroppedEnv reports whether the "NAME=value" entry kv sets one of -// droppedEnv, or a variable of touchmark's. Names compare -// case-insensitively, as on Windows. +// droppedEnv, or a variable whose name starts with one of droppedPrefixes. +// Names compare case-insensitively, as on Windows. func isDroppedEnv(kv string) bool { name := envName(kv) - if len(name) >= len(droppedPrefix) && strings.EqualFold(name[:len(droppedPrefix)], droppedPrefix) { - return true + for _, p := range droppedPrefixes { + if len(name) >= len(p) && strings.EqualFold(name[:len(p)], p) { + return true + } } return slices.ContainsFunc(droppedEnv, func(v string) bool { return strings.EqualFold(v, name) }) } diff --git a/internal/gitx/gitx_test.go b/internal/gitx/gitx_test.go index d43546f..46f77c9 100644 --- a/internal/gitx/gitx_test.go +++ b/internal/gitx/gitx_test.go @@ -4,6 +4,7 @@ import ( "context" "errors" "os" + "os/exec" "path/filepath" "slices" "strings" @@ -145,6 +146,100 @@ func TestRunInherit(t *testing.T) { } } +// TestRunDropsInheritedConfig: config the environment carries +// (GIT_CONFIG_COUNT with its KEY_n/VALUE_n, GIT_CONFIG_PARAMETERS, +// GIT_CONFIG_GLOBAL, GIT_CONFIG_SYSTEM) never reaches git, while the same +// variables set through Env do. Each variable is first shown to work on this +// git, so a pass is not a git that ignores it. +func TestRunDropsInheritedConfig(t *testing.T) { + r := newTestRepo(t, false) + home := t.TempDir() + evil := filepath.Join(t.TempDir(), "evil.gitconfig") + if err := os.WriteFile(evil, []byte("[core]\n\tfsmonitor = evil-global\n"), 0o644); err != nil { + t.Fatal(err) + } + cases := []struct { + name string + env []string + want string + }{ + {"GIT_CONFIG_COUNT", []string{"GIT_CONFIG_COUNT=1", "GIT_CONFIG_KEY_0=core.fsmonitor", "GIT_CONFIG_VALUE_0=evil-count"}, "evil-count"}, + {"GIT_CONFIG_PARAMETERS", []string{"GIT_CONFIG_PARAMETERS='core.fsmonitor'='evil-parameters'"}, "evil-parameters"}, + {"GIT_CONFIG_GLOBAL", []string{"GIT_CONFIG_GLOBAL=" + evil}, "evil-global"}, + {"GIT_CONFIG_SYSTEM", []string{"GIT_CONFIG_SYSTEM=" + evil}, "evil-global"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + // The control: git itself reads the variable. + cmd := exec.Command("git", "-C", r.dir, "config", "--get", "core.fsmonitor") + cmd.Env = append(os.Environ(), append([]string{"HOME=" + home, "XDG_CONFIG_HOME=" + home}, tc.env...)...) + if out, err := cmd.Output(); err != nil || strings.TrimSpace(string(out)) != tc.want { + t.Fatalf("plain git reads core.fsmonitor = %q, %v; want %q", out, err, tc.want) + } + // Through Git, inherited: dropped. + for _, kv := range tc.env { + name, value, _ := strings.Cut(kv, "=") + t.Setenv(name, value) + } + g := &Git{Dir: r.dir, Env: []string{"HOME=" + home, "XDG_CONFIG_HOME=" + home}} + out, err := g.Run(t.Context(), nil, "config", "--get", "core.fsmonitor") + var gitErr *Error + if !errors.As(err, &gitErr) || gitErr.Code != 1 { + t.Errorf("Git inherits %s: core.fsmonitor = %q, %v", tc.name, out, err) + } + // Through Env: kept, as touchmark's own isolation needs. + g.Env = append(g.Env, tc.env...) + out, err = g.Run(t.Context(), nil, "config", "--get", "core.fsmonitor") + if err != nil || strings.TrimSpace(string(out)) != tc.want { + t.Errorf("Env %s: core.fsmonitor = %q, %v; want %q", tc.name, out, err, tc.want) + } + }) + } +} + +// TestDroppedEnv: which inherited variables git gets. Names compare +// case-insensitively; prefixes drop whole families. +func TestDroppedEnv(t *testing.T) { + t.Parallel() + dropped := []string{ + "GIT_CONFIG_COUNT=1", "GIT_CONFIG_KEY_0=core.fsmonitor", "GIT_CONFIG_VALUE_0=x", + "git_config_key_12=x", "GIT_CONFIG_PARAMETERS='a.b'='c'", "GIT_CONFIG_GLOBAL=/x", + "GIT_CONFIG_SYSTEM=/x", "GIT_CONFIG=/x", "GIT_EXEC_PATH=/x", "GIT_SSH=/x", + "GIT_SSH_COMMAND=x", "GIT_ASKPASS=x", "SSH_ASKPASS=x", "GIT_PROXY_COMMAND=x", + "GIT_EXTERNAL_DIFF=x", "GIT_PAGER=x", "PAGER=x", "EDITOR=x", "GIT_EDITOR=x", + "VISUAL=x", "GIT_TRACE=/x", "GIT_TRACE2_EVENT=/x", "GIT_TRACE_PACKET=1", + "GIT_CURL_VERBOSE=1", "GIT_TEMPLATE_DIR=/x", "GIT_ALTERNATE_OBJECT_DIRECTORIES=/x", + "GIT_OBJECT_DIRECTORY=/x", "GIT_DIR=/x", "GIT_WORK_TREE=/x", "GIT_INDEX_FILE=/x", + "GIT_NAMESPACE=x", "GIT_ATTR_SOURCE=HEAD", "LD_PRELOAD=/x.so", "LD_LIBRARY_PATH=/x", + "LD_AUDIT=/x.so", "DYLD_INSERT_LIBRARIES=/x.dylib", "DYLD_LIBRARY_PATH=/x", + "TOUCHMARK_GITHUB_TOKEN=x", + } + kept := []string{ + "GIT_CONFIG_NOSYSTEM=1", "GIT_ATTR_NOSYSTEM=1", "GIT_CEILING_DIRECTORIES=/tmp", + "GIT_AUTHOR_NAME=a", "HOME=/home/a", "PATH=/bin", "HTTPS_PROXY=http://proxy", + "LDFLAGS=-s", "GIT_SSL_CAINFO=/ca.pem", + } + for _, kv := range dropped { + if !isDroppedEnv(kv) { + t.Errorf("%q is inherited", kv) + } + } + for _, kv := range kept { + if isDroppedEnv(kv) { + t.Errorf("%q is dropped", kv) + } + } + g := &Git{ + Inherit: func() []string { return slices.Concat(dropped, kept) }, + Env: []string{"GIT_CONFIG_GLOBAL=/own", "GIT_CONFIG_COUNT=1", "GIT_CONFIG_KEY_0=a.b", "GIT_CONFIG_VALUE_0=c"}, + } + env := g.environ() + want := slices.Concat(kept, defaultEnv, g.Env) + if !slices.Equal(env, want) { + t.Errorf("environ() = %q\nwant %q", env, want) + } +} + func TestRunStartFailure(t *testing.T) { t.Parallel() g := &Git{Bin: "touchmark-no-such-git-binary"} @@ -198,7 +293,7 @@ func TestEnvName(t *testing.T) { t.Errorf("envName(%q) = %q, want %q", kv, got, want) } } - if !isDroppedEnv("git_dir=/x") || !isDroppedEnv("GIT_LITERAL_PATHSPECS=1") || isDroppedEnv("GIT_DIRX=/x") || isDroppedEnv("GIT_CONFIG_COUNT=1") { + if !isDroppedEnv("git_dir=/x") || !isDroppedEnv("GIT_LITERAL_PATHSPECS=1") || isDroppedEnv("GIT_DIRX=/x") || !isDroppedEnv("GIT_CONFIG_COUNT=1") || isDroppedEnv("GIT_CONFIG_NOSYSTEM=1") { t.Error("isDroppedEnv misclassifies") } // touchmark's own variables, credentials first, reach no git process. diff --git a/internal/hubch/bitbucket.go b/internal/hubch/bitbucket.go new file mode 100644 index 0000000..866130b --- /dev/null +++ b/internal/hubch/bitbucket.go @@ -0,0 +1,308 @@ +package hubch + +import ( + "context" + "errors" + "fmt" + "net/http" + "net/url" + "regexp" + "strconv" + "strings" + + "github.com/bedrock-python/touchmark/internal/httpx" +) + +// A hub on Bitbucket Cloud, run by Bitbucket Pipelines. Sources: +// - the default variables: +// https://support.atlassian.com/bitbucket-cloud/docs/variables-and-secrets/ +// - pull request pipelines (not run for pull requests from forks): +// https://support.atlassian.com/bitbucket-cloud/docs/pipeline-start-conditions/ +// - the REST API: https://developer.atlassian.com/cloud/bitbucket/rest/intro/ +// (a repository by UUID: /2.0/repositories/{}/{uuid}). + +// BitbucketTokenVar names the hub's own API token on Bitbucket Pipelines. +// Pipelines gives a step no token for the API, so the hub keeps one in a +// secured repository variable: an access token of the hub repository with +// Repositories: Read (the default branch and its tip) and, for plan +// --comment, Pull requests: Write. It is the counterpart of GITHUB_TOKEN and +// CI_JOB_TOKEN, and every pipeline of the hub sees it; it holds no write key +// of any target. +const BitbucketTokenVar = "TOUCHMARK_PIPELINES_TOKEN" + +// The places of Bitbucket Cloud, the only home of Bitbucket Pipelines. +const ( + bitbucketServerURL = "https://bitbucket.org" + bitbucketHost = "bitbucket.org" + bitbucketAPIURL = "https://api.bitbucket.org/2.0" +) + +// bitbucketUUIDRe is a UUID in canonical form: lowercase, without braces. +var bitbucketUUIDRe = regexp.MustCompile(`^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$`) + +// BitbucketRepoID returns a Bitbucket repository UUID in the form the hub +// fingerprint carries it: lowercase, without the braces Bitbucket writes +// around it (BITBUCKET_REPO_UUID is "{0f1e…}"), so that the fingerprint is +// bitbucket.org/0f1e…. Anything that is not a UUID, with or without one +// pair of braces, gives "". +func BitbucketRepoID(s string) string { + s = strings.TrimSpace(s) + if strings.HasPrefix(s, "{") && strings.HasSuffix(s, "}") { + s = s[1 : len(s)-1] + } + s = strings.ToLower(s) + if !bitbucketUUIDRe.MatchString(s) { + return "" + } + return s +} + +// detectBitbucket reads Bitbucket Pipelines' default variables: +// - the hub is on Bitbucket Cloud (ServerURL https://bitbucket.org, API +// https://api.bitbucket.org/2.0); RepoID is BITBUCKET_REPO_UUID in +// canonical form (BitbucketRepoID), RepoPath BITBUCKET_REPO_FULL_NAME; +// - a pull request pipeline (BITBUCKET_PR_ID set, whatever its value) is +// Event pull_request, its RefName the source branch BITBUCKET_BRANCH, +// and no branch run; a branch pipeline (BITBUCKET_BRANCH) is Event push +// and a branch run; a tag pipeline (BITBUCKET_TAG) is Event push, not a +// branch run. Pipelines does not tell a push from a custom pipeline run +// by hand or on a schedule: both are push runs of their branch; +// - Environment is BITBUCKET_DEPLOYMENT_ENVIRONMENT, set in a step with a +// deployment; +// - Visibility is private or public from BITBUCKET_REPO_IS_PRIVATE; +// - DefaultBranch stays empty: no variable names it, and the hub channel +// reads it (RepositoryReader). +func detectBitbucket(env func(string) string) Context { + c := Context{CI: BitbucketPipelines, ServerURL: bitbucketServerURL, Host: bitbucketHost, APIURL: bitbucketAPIURL} + c.RepoID = BitbucketRepoID(env("BITBUCKET_REPO_UUID")) + c.RepoPath = env("BITBUCKET_REPO_FULL_NAME") + branch, tag := env("BITBUCKET_BRANCH"), env("BITBUCKET_TAG") + switch { + case env("BITBUCKET_PR_ID") != "": + c.Event, c.RefName = "pull_request", branch + case branch != "": + c.Event, c.RefName, c.RefIsBranch = "push", branch, true + case tag != "": + c.Event, c.RefName = "push", tag + } + c.Environment = env("BITBUCKET_DEPLOYMENT_ENVIRONMENT") + switch strings.ToLower(env("BITBUCKET_REPO_IS_PRIVATE")) { + case "true": + c.Visibility = "private" + case "false": + c.Visibility = "public" + } + c.RepositoryURL = cleanURL(env("BITBUCKET_GIT_HTTP_ORIGIN")) + return c +} + +// RepoInfo is what the hub channel reads of the hub repository itself. +type RepoInfo struct { + // DefaultBranch is the repository's main branch. + DefaultBranch string + // Visibility is "public" or "private". + Visibility string +} + +// RepositoryReader is a Channel that can read the hub repository's default +// branch and visibility: the channel of Bitbucket, whose CI variables name +// no default branch. +type RepositoryReader interface { + Repository(ctx context.Context) (RepoInfo, error) +} + +// bitbucketChannel reads the hub repository through Bitbucket Cloud's REST +// API, and keeps plan's comment in a hub pull request. +type bitbucketChannel struct { + client *httpx.Client + auth *httpx.Auth // nil without a token + repo string // {api}/repositories/{workspace}/{slug} + uuid string // canonical, "" when the CI named none + branch string // the default branch, "" until read + info *RepoInfo // the repository, once read +} + +// newBitbucket returns the channel of a hub on Bitbucket: the repository +// c.RepoPath (workspace/slug) under c.APIURL, with token, when set, sent as +// a Bearer token to the API's host only. When c names its default branch, +// Head reads that branch; else the repository's main branch. +func newBitbucket(c Context, client *httpx.Client, token string) (Channel, error) { + if c.DefaultBranch != "" { + if err := checkBranch(c.DefaultBranch); err != nil { + return nil, fmt.Errorf("hub channel: default branch: %w", err) + } + } + ws, slug, ok := strings.Cut(c.RepoPath, "/") + if !ok || !validName(ws) || !validName(slug) { + return nil, fmt.Errorf("hub channel: invalid repository path %q", c.RepoPath) + } + api, err := url.Parse(c.APIURL) + if err != nil || api.Host == "" || api.User != nil || (api.Scheme != "https" && api.Scheme != "http") { + return nil, errors.New("hub channel: the API URL is not an absolute http(s) URL without credentials") + } + api.RawQuery, api.ForceQuery, api.Fragment, api.RawFragment = "", false, "", "" + if client == nil { + client = httpx.New(httpx.Options{}) + } + ch := &bitbucketChannel{ + client: client, + repo: strings.TrimRight(api.String(), "/") + "/repositories/" + url.PathEscape(ws) + "/" + url.PathEscape(slug), + uuid: c.RepoID, + branch: c.DefaultBranch, + } + if token != "" { + if err := checkToken(token); err != nil { + return nil, err + } + value := "Bearer " + token + ch.auth = &httpx.Auth{Hosts: []string{api.Host}, Header: func(context.Context) (string, error) { return value, nil }} + } + return ch, nil +} + +// Repository reads the hub repository (GET /repositories/{workspace}/{slug}): +// its main branch and is_private. The answer must name the repository the +// CI named (its uuid), and a main branch. +func (ch *bitbucketChannel) Repository(ctx context.Context) (RepoInfo, error) { + if ch.info != nil { + return *ch.info, nil + } + var repo struct { + UUID string `json:"uuid"` + IsPrivate *bool `json:"is_private"` + MainBranch *struct { + Name string `json:"name"` + } `json:"mainbranch"` + } + if _, err := ch.client.JSON(ctx, http.MethodGet, ch.repo, ch.auth, nil, &repo); err != nil { + return RepoInfo{}, fmt.Errorf("hub channel: read the hub repository: %w", err) + } + if ch.uuid != "" && BitbucketRepoID(repo.UUID) != ch.uuid { + return RepoInfo{}, fmt.Errorf("hub channel: the answer names repository %q, not {%s}", repo.UUID, ch.uuid) + } + if repo.MainBranch == nil || checkBranch(repo.MainBranch.Name) != nil { + return RepoInfo{}, errors.New("hub channel: the hub repository's answer names no main branch") + } + if repo.IsPrivate == nil { + return RepoInfo{}, errors.New("hub channel: the hub repository's answer does not say whether it is private") + } + info := RepoInfo{DefaultBranch: repo.MainBranch.Name, Visibility: "public"} + if *repo.IsPrivate { + info.Visibility = "private" + } + ch.info = &info + return info, nil +} + +// Visibility reads the hub repository's visibility (VisibilityReader). +func (ch *bitbucketChannel) Visibility(ctx context.Context) (string, error) { + info, err := ch.Repository(ctx) + return info.Visibility, err +} + +// Head reads the tip of the default branch: GET +// /repositories/{workspace}/{slug}/refs/branches/{branch}, whose target.hash +// is the full commit id. +func (ch *bitbucketChannel) Head(ctx context.Context) (string, error) { + branch := ch.branch + if branch == "" { + info, err := ch.Repository(ctx) + if err != nil { + return "", err + } + branch = info.DefaultBranch + } + var ref struct { + Name string `json:"name"` + Target struct { + Hash string `json:"hash"` + } `json:"target"` + } + if _, err := ch.client.JSON(ctx, http.MethodGet, ch.repo+"/refs/branches/"+escapeSegments(branch), ch.auth, nil, &ref); err != nil { + return "", fmt.Errorf("hub channel: read the tip of branch %q: %w", branch, err) + } + sha := strings.ToLower(ref.Target.Hash) + if ref.Name != branch || !isOID(sha) { + return "", fmt.Errorf("hub channel: unexpected answer for branch %q: name %q, commit %q", branch, ref.Name, ref.Target.Hash) + } + return sha, nil +} + +// bitbucketCommentPage is the page size of the comment listing, Bitbucket's +// largest pagelen. +const bitbucketCommentPage = 100 + +// UpsertComment keeps plan's comment in hub pull request pr (Commenter). +// It lists the pull request's comments (GET +// …/pullrequests/{pr}/comments, oldest first, 100 a page) and takes those +// that are neither deleted nor inline and whose raw content holds marker. +// An access token cannot learn its own account (GET /2.0/user refuses it), +// so the comment is not told by its author: each candidate, newest first, is +// replaced (PUT …/comments/{id}), and a refusal (HTTP 403 or 404: another +// account's comment, which only its author may edit) moves on to the next. +// Only an accepted edit proves the comment touchmark's, so a candidate whose +// content is already body is replaced all the same (the edit changes +// nothing) and counts as unchanged once Bitbucket accepts it: a comment of +// someone else's that copies the body exactly does not stand in for +// touchmark's. Without one that takes the edit, it creates the comment +// (POST …/comments). A comment of anyone else that holds the marker is thus +// never changed. +func (ch *bitbucketChannel) UpsertComment(ctx context.Context, pr int64, marker, body string) (CommentResult, error) { + if pr <= 0 { + return "", fmt.Errorf("hub channel: no pull request to comment on (#%d)", pr) + } + if marker == "" || !strings.Contains(body, marker) { + return "", errors.New("hub channel: the comment's body lacks its marker") + } + comments := ch.repo + "/pullrequests/" + strconv.FormatInt(pr, 10) + "/comments" + type candidate struct { + id int64 + raw string + } + var found []candidate + for page := 1; page <= maxCommentPages; page++ { + var list struct { + Next string `json:"next"` + Values []struct { + ID int64 `json:"id"` + Deleted bool `json:"deleted"` + Content struct { + Raw string `json:"raw"` + } `json:"content"` + Inline *struct{} `json:"inline"` + } `json:"values"` + } + u := fmt.Sprintf("%s?pagelen=%d&page=%d", comments, bitbucketCommentPage, page) + if _, err := ch.client.JSON(ctx, http.MethodGet, u, ch.auth, nil, &list); err != nil { + return "", fmt.Errorf("hub channel: list the comments of #%d: %w", pr, err) + } + for _, c := range list.Values { + if c.ID > 0 && !c.Deleted && c.Inline == nil && strings.Contains(c.Content.Raw, marker) { + found = append(found, candidate{c.ID, c.Content.Raw}) + } + } + if list.Next == "" || len(list.Values) == 0 { + break + } + } + payload := map[string]any{"content": map[string]string{"raw": body}} + for i := len(found) - 1; i >= 0; i-- { + c := found[i] + edit := comments + "/" + strconv.FormatInt(c.id, 10) + _, err := ch.client.JSON(ctx, http.MethodPut, edit, ch.auth, payload, nil) + if err == nil { + if c.raw == body { + return CommentUnchanged, nil + } + return CommentUpdated, nil + } + if s := status(err); s == http.StatusForbidden || s == http.StatusNotFound { + continue + } + return "", fmt.Errorf("hub channel: update comment %d of #%d: %w", c.id, pr, err) + } + if _, err := ch.client.JSON(ctx, http.MethodPost, comments, ch.auth, payload, nil); err != nil { + return "", fmt.Errorf("hub channel: comment on #%d: %w", pr, err) + } + return CommentCreated, nil +} diff --git a/internal/hubch/bitbucket_test.go b/internal/hubch/bitbucket_test.go new file mode 100644 index 0000000..48220f3 --- /dev/null +++ b/internal/hubch/bitbucket_test.go @@ -0,0 +1,489 @@ +package hubch + +import ( + "encoding/json" + "fmt" + "net/http" + "net/http/httptest" + "slices" + "strconv" + "strings" + "sync" + "testing" + + "github.com/bedrock-python/touchmark/internal/httpx" +) + +// bbUUID is the hub repository's UUID as Bitbucket writes it, and bbID as +// the fingerprint carries it. +const ( + bbUUID = "{3F2A8D4E-1B6C-4F0A-9E7D-5C2B1A0F9E8D}" + bbID = "3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d" + bbTok = "bb-hub-access-token-0123456789" +) + +// pipelinesEnv is a step of Bitbucket Pipelines (the default variables of +// https://support.atlassian.com/bitbucket-cloud/docs/variables-and-secrets/), +// with extra variables over it. +func pipelinesEnv(extra map[string]string) func(string) string { + vars := map[string]string{ + "CI": "true", + "BITBUCKET_BUILD_NUMBER": "17", + "BITBUCKET_REPO_UUID": bbUUID, + "BITBUCKET_REPO_FULL_NAME": "acme/engineering-assets", + "BITBUCKET_WORKSPACE": "acme", + "BITBUCKET_REPO_SLUG": "engineering-assets", + "BITBUCKET_COMMIT": "3f2c0000000000000000000000000000000000aa", + "BITBUCKET_GIT_HTTP_ORIGIN": "http://bitbucket.org/acme/engineering-assets", + "BITBUCKET_REPO_IS_PRIVATE": "true", + } + for k, v := range extra { + if v == "" { + delete(vars, k) + } else { + vars[k] = v + } + } + return envOf(vars) +} + +func TestDetectBitbucket(t *testing.T) { + base := Context{CI: BitbucketPipelines, ServerURL: "https://bitbucket.org", Host: "bitbucket.org", APIURL: "https://api.bitbucket.org/2.0", + RepoID: bbID, RepoPath: "acme/engineering-assets", Visibility: "private", RepositoryURL: "http://bitbucket.org/acme/engineering-assets"} + with := func(f func(*Context)) Context { + c := base + f(&c) + return c + } + for _, tc := range []struct { + name string + extra map[string]string + want Context + }{ + {"push to a branch", map[string]string{"BITBUCKET_BRANCH": "main"}, + with(func(c *Context) { c.Event, c.RefName, c.RefIsBranch = "push", "main", true })}, + {"the deployment step", map[string]string{"BITBUCKET_BRANCH": "main", "BITBUCKET_DEPLOYMENT_ENVIRONMENT": "touchmark-distribute"}, + with(func(c *Context) { + c.Event, c.RefName, c.RefIsBranch, c.Environment = "push", "main", true, "touchmark-distribute" + })}, + {"a pull request", map[string]string{"BITBUCKET_BRANCH": "feature", "BITBUCKET_PR_ID": "41", "BITBUCKET_PR_DESTINATION_BRANCH": "main"}, + with(func(c *Context) { c.Event, c.RefName = "pull_request", "feature" })}, + // A pull request's pipeline is never a branch run, even of the + // default branch, whatever its id says. + {"a pull request from main", map[string]string{"BITBUCKET_BRANCH": "main", "BITBUCKET_PR_ID": "x"}, + with(func(c *Context) { c.Event, c.RefName = "pull_request", "main" })}, + {"a tag", map[string]string{"BITBUCKET_TAG": "v1"}, + with(func(c *Context) { c.Event, c.RefName = "push", "v1" })}, + {"a public hub", map[string]string{"BITBUCKET_BRANCH": "main", "BITBUCKET_REPO_IS_PRIVATE": "false"}, + with(func(c *Context) { c.Event, c.RefName, c.RefIsBranch, c.Visibility = "push", "main", true, "public" })}, + {"no visibility, a garbled UUID", map[string]string{"BITBUCKET_BRANCH": "main", "BITBUCKET_REPO_IS_PRIVATE": "", "BITBUCKET_REPO_UUID": "{3f2a}"}, + with(func(c *Context) { + c.Event, c.RefName, c.RefIsBranch, c.Visibility, c.RepoID = "push", "main", true, "", "" + })}, + } { + got := Detect(pipelinesEnv(tc.extra), nil) + if got != tc.want { + t.Errorf("%s:\n got %+v\n want %+v", tc.name, got, tc.want) + } + } + c := Detect(pipelinesEnv(map[string]string{"BITBUCKET_BRANCH": "main"}), nil) + if fp := c.Fingerprint(); fp != "bitbucket.org/"+bbID { + t.Errorf("fingerprint %q", fp) + } + // GitLab's and Actions' flags win: a Bitbucket variable in their jobs + // does not make them Bitbucket. + if c := Detect(envOf(map[string]string{"GITLAB_CI": "true", "BITBUCKET_BUILD_NUMBER": "1"}), nil); c.CI != GitLabCI { + t.Errorf("GitLab with a stray variable: %s", c.CI) + } + if tok := Token(c, pipelinesEnv(map[string]string{BitbucketTokenVar: " " + bbTok + " ", "GITHUB_TOKEN": "x"})); tok != bbTok { + t.Errorf("token %q", tok) + } +} + +func TestBitbucketRepoID(t *testing.T) { + for in, want := range map[string]string{ + bbUUID: bbID, + bbID: bbID, + " " + bbUUID + " ": bbID, + "{" + bbID: "", + "{{" + bbID + "}}": "", + "712345678": "", + "": "", + "3f2a8d4e1b6c4f0a9e": "", + } { + if got := BitbucketRepoID(in); got != want { + t.Errorf("BitbucketRepoID(%q) = %q, want %q", in, got, want) + } + } +} + +// bbServer is Bitbucket Cloud's API for one hub, acme/engineering-assets, +// with its pull request 41 and the comments on it, as +// https://api.bitbucket.org/swagger.json describes them. The hub's access +// token comments as account "{bot}"; a comment of another account refuses +// its edit with 403. +type bbServer struct { + *httptest.Server + mu sync.Mutex + private bool + comments []bbComment + next int64 + log []string + auth []string +} + +type bbComment struct { + ID int64 `json:"id"` + Deleted bool `json:"deleted"` + Content struct { + Raw string `json:"raw"` + } `json:"content"` + Inline *struct { + Path string `json:"path"` + } `json:"inline,omitempty"` + User struct { + UUID string `json:"uuid"` + } `json:"user"` +} + +func newBBServer(t *testing.T) *bbServer { + t.Helper() + s := &bbServer{next: 500, private: true} + const repo = "/2.0/repositories/acme/engineering-assets" + s.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + s.mu.Lock() + defer s.mu.Unlock() + s.log = append(s.log, r.Method+" "+r.URL.Path+"?"+r.URL.RawQuery) + s.auth = append(s.auth, r.Header.Get("Authorization")) + w.Header().Set("Content-Type", "application/json") + authed := r.Header.Get("Authorization") == "Bearer "+bbTok + if !authed && (s.private || r.Method != http.MethodGet) { + // Bitbucket answers a private repository's anonymous reader as + // if it did not exist. + http.Error(w, `{"type": "error", "error": {"message": "Repository not found"}}`, http.StatusNotFound) + return + } + var in struct { + Content struct { + Raw string `json:"raw"` + } `json:"content"` + } + if r.Method != http.MethodGet { + if err := json.NewDecoder(r.Body).Decode(&in); err != nil || in.Content.Raw == "" { + http.Error(w, `{"type": "error"}`, http.StatusBadRequest) + return + } + } + comments := repo + "/pullrequests/41/comments" + switch { + case r.Method == http.MethodGet && r.URL.Path == repo: + fmt.Fprintf(w, `{"type": "repository", "uuid": %q, "full_name": "acme/engineering-assets", "is_private": %v, "mainbranch": {"type": "branch", "name": "trunk"}}`, bbUUID, s.private) + case r.Method == http.MethodGet && r.URL.Path == repo+"/refs/branches/trunk": + fmt.Fprint(w, `{"type": "branch", "name": "trunk", "target": {"type": "commit", "hash": "0123456789ABCDEF0123456789abcdef01234567"}}`) + case r.Method == http.MethodGet && r.URL.Path == comments: + size, _ := strconv.Atoi(r.URL.Query().Get("pagelen")) + n, _ := strconv.Atoi(r.URL.Query().Get("page")) + if size <= 0 || size > 100 || n <= 0 { + http.Error(w, `{"type": "error"}`, http.StatusBadRequest) + return + } + lo, hi := min((n-1)*size, len(s.comments)), min(n*size, len(s.comments)) + page := map[string]any{"pagelen": size, "page": n, "values": s.comments[lo:hi]} + if hi < len(s.comments) { + page["next"] = fmt.Sprintf("https://api.bitbucket.org%s?pagelen=%d&page=%d", comments, size, n+1) + } + _ = json.NewEncoder(w).Encode(page) + case r.Method == http.MethodPost && r.URL.Path == comments: + c := bbComment{ID: s.next} + c.Content.Raw, c.User.UUID = in.Content.Raw, "{bot}" + s.next++ + s.comments = append(s.comments, c) + w.WriteHeader(http.StatusCreated) + _ = json.NewEncoder(w).Encode(c) + case r.Method == http.MethodPut && strings.HasPrefix(r.URL.Path, comments+"/"): + id, _ := strconv.ParseInt(strings.TrimPrefix(r.URL.Path, comments+"/"), 10, 64) + for i := range s.comments { + if s.comments[i].ID != id { + continue + } + if s.comments[i].User.UUID != "{bot}" { + http.Error(w, `{"type": "error", "error": {"message": "You are not allowed to edit this comment"}}`, http.StatusForbidden) + return + } + s.comments[i].Content.Raw = in.Content.Raw + _ = json.NewEncoder(w).Encode(s.comments[i]) + return + } + http.Error(w, `{"type": "error"}`, http.StatusNotFound) + default: + http.Error(w, `{"type": "error"}`, http.StatusNotFound) + } + })) + t.Cleanup(s.Close) + return s +} + +func (s *bbServer) add(author, raw string) { + s.mu.Lock() + defer s.mu.Unlock() + c := bbComment{ID: s.next} + c.Content.Raw, c.User.UUID = raw, author + s.next++ + s.comments = append(s.comments, c) +} + +func (s *bbServer) raws() []string { + s.mu.Lock() + defer s.mu.Unlock() + var out []string + for _, c := range s.comments { + out = append(out, c.User.UUID+": "+c.Content.Raw) + } + return out +} + +func (s *bbServer) takeLog() []string { + s.mu.Lock() + defer s.mu.Unlock() + log := s.log + s.log = nil + return log +} + +func (s *bbServer) channel(t *testing.T, token string) Channel { + t.Helper() + c := Detect(pipelinesEnv(map[string]string{"BITBUCKET_PR_ID": "41", "BITBUCKET_BRANCH": "feature"}), nil) + c.APIURL = s.URL + "/2.0" + ch, err := New(c, httpx.New(httpx.Options{}), token) + if err != nil { + t.Fatal(err) + } + return ch +} + +func TestBitbucketChannel(t *testing.T) { + s := newBBServer(t) + ch := s.channel(t, bbTok) + info, err := ch.(RepositoryReader).Repository(t.Context()) + if err != nil || info != (RepoInfo{DefaultBranch: "trunk", Visibility: "private"}) { + t.Fatalf("repository %+v, %v", info, err) + } + head, err := ch.Head(t.Context()) + if err != nil || head != "0123456789abcdef0123456789abcdef01234567" { + t.Errorf("head %q, %v", head, err) + } + if v, err := ch.(VisibilityReader).Visibility(t.Context()); err != nil || v != "private" { + t.Errorf("visibility %q, %v", v, err) + } + // The repository is read once; the token went with every request. + if log := s.takeLog(); !slices.Equal(log, []string{ + "GET /2.0/repositories/acme/engineering-assets?", + "GET /2.0/repositories/acme/engineering-assets/refs/branches/trunk?", + }) { + t.Errorf("requests %q", log) + } + for _, a := range s.auth { + if a != "Bearer "+bbTok { + t.Errorf("authorization %q", a) + } + } + // Without a token a private hub cannot be read; a public one can. + if _, err := s.channel(t, "").(RepositoryReader).Repository(t.Context()); err == nil || !strings.Contains(err.Error(), "404") { + t.Errorf("anonymous read of a private hub: %v", err) + } + s.private = false + if info, err := s.channel(t, "").(RepositoryReader).Repository(t.Context()); err != nil || info.Visibility != "public" { + t.Errorf("anonymous read of a public hub: %+v, %v", info, err) + } + // An answer for another repository is refused. + c := Detect(pipelinesEnv(map[string]string{"BITBUCKET_BRANCH": "main", "BITBUCKET_REPO_UUID": "{0b7e5a2c-9d4f-4e1b-8a3c-6f5d2e1c0b9a}"}), nil) + c.APIURL = s.URL + "/2.0" + other, err := New(c, nil, bbTok) + if err != nil { + t.Fatal(err) + } + if _, err := other.Head(t.Context()); err == nil || !strings.Contains(err.Error(), "names repository") { + t.Errorf("another repository's answer: %v", err) + } +} + +func TestBitbucketChannelErrors(t *testing.T) { + c := Detect(pipelinesEnv(map[string]string{"BITBUCKET_BRANCH": "main"}), nil) + for name, mutate := range map[string]func(*Context){ + "no path": func(c *Context) { c.RepoPath = "" }, + "bad path": func(c *Context) { c.RepoPath = "acme/../x/y" }, + "bad API": func(c *Context) { c.APIURL = "ftp://api.bitbucket.org" }, + "bad branch": func(c *Context) { c.DefaultBranch = "-x" }, + "API with key": func(c *Context) { c.APIURL = "https://u:p@api.bitbucket.org/2.0" }, + } { + cc := c + mutate(&cc) + if _, err := New(cc, nil, bbTok); err == nil { + t.Errorf("%s: no error", name) + } + } + if _, err := New(c, nil, "a token"); err == nil || strings.Contains(err.Error(), "a token") { + t.Errorf("a token with a space: %v", err) + } +} + +func TestBitbucketUpsertComment(t *testing.T) { + const marker = `[touchmark-plan]: # "touchmark plan: acme-eng"` + body := func(s string) string { return s + "\n\n" + marker + "\n" } + s := newBBServer(t) + ch := s.channel(t, bbTok).(Commenter) + + // Someone else's comments, one quoting the marker, an inline one and a + // deleted one with it, come first. + s.add("{alice}", "LGTM") + s.add("{mallory}", body("planted")) + s.add("{bot}", body("inline")) + s.comments[len(s.comments)-1].Inline = &struct { + Path string `json:"path"` + }{"hub.yml"} + s.add("{bot}", body("deleted")) + s.comments[len(s.comments)-1].Deleted = true + s.takeLog() + + // None of them is plan's own: mallory's refuses the edit, so a new one + // is made. + res, err := ch.UpsertComment(t.Context(), 41, marker, body("first")) + if err != nil || res != CommentCreated { + t.Fatalf("first: %s, %v", res, err) + } + if log := s.takeLog(); !slices.Equal(log, []string{ + "GET /2.0/repositories/acme/engineering-assets/pullrequests/41/comments?pagelen=100&page=1", + "PUT /2.0/repositories/acme/engineering-assets/pullrequests/41/comments/501?", + "POST /2.0/repositories/acme/engineering-assets/pullrequests/41/comments?", + }) { + t.Errorf("requests %q", log) + } + // The next run edits its own comment, newest first. + if res, err := ch.UpsertComment(t.Context(), 41, marker, body("second")); err != nil || res != CommentUpdated { + t.Errorf("second: %s, %v", res, err) + } + s.takeLog() + if res, err := ch.UpsertComment(t.Context(), 41, marker, body("second")); err != nil || res != CommentUnchanged { + t.Errorf("same: %s, %v", res, err) + } + // Unchanged only once the edit proves the comment its own. + if log := s.takeLog(); len(log) != 2 || !strings.HasPrefix(log[1], "PUT ") || !strings.HasSuffix(log[1], "/comments/504?") { + t.Errorf("requests %q", log) + } + want := []string{"{alice}: LGTM", "{mallory}: " + body("planted"), "{bot}: " + body("inline"), "{bot}: " + body("deleted"), "{bot}: " + body("second")} + if got := s.raws(); !slices.Equal(got, want) { + t.Errorf("comments\n got %q\n want %q", got, want) + } + // Past a page of 100 comments it still finds its own. + for i := range 150 { + s.add("{alice}", fmt.Sprintf("note %d", i)) + } + s.takeLog() + if res, err := ch.UpsertComment(t.Context(), 41, marker, body("third")); err != nil || res != CommentUpdated { + t.Errorf("third: %s, %v", res, err) + } + if log := s.takeLog(); len(log) != 3 || !strings.HasSuffix(log[1], "page=2") || !strings.HasPrefix(log[2], "PUT ") { + t.Errorf("requests %q", log) + } + // Errors: no marker, no pull request, a refused token. + if _, err := ch.UpsertComment(t.Context(), 41, marker, "no marker"); err == nil { + t.Error("a body without its marker") + } + if _, err := ch.UpsertComment(t.Context(), 0, marker, body("x")); err == nil { + t.Error("no pull request") + } + if _, err := s.channel(t, "").(Commenter).UpsertComment(t.Context(), 41, marker, body("x")); err == nil { + t.Error("an anonymous comment") + } +} + +// Someone else's comment that holds exactly the body plan would write is +// not taken for plan's own: equal content proves nothing until an edit is +// accepted, so plan tries it, is refused, and comments itself. +func TestBitbucketUpsertCommentCopiedBody(t *testing.T) { + const marker = `[touchmark-plan]: # "touchmark plan: acme-eng"` + body := "the plan\n\n" + marker + "\n" + s := newBBServer(t) + ch := s.channel(t, bbTok).(Commenter) + s.add("{mallory}", body) + s.takeLog() + + res, err := ch.UpsertComment(t.Context(), 41, marker, body) + if err != nil || res != CommentCreated { + t.Fatalf("first: %s, %v", res, err) + } + if log := s.takeLog(); !slices.Equal(log, []string{ + "GET /2.0/repositories/acme/engineering-assets/pullrequests/41/comments?pagelen=100&page=1", + "PUT /2.0/repositories/acme/engineering-assets/pullrequests/41/comments/500?", + "POST /2.0/repositories/acme/engineering-assets/pullrequests/41/comments?", + }) { + t.Errorf("requests %q", log) + } + if got, want := s.raws(), []string{"{mallory}: " + body, "{bot}: " + body}; !slices.Equal(got, want) { + t.Errorf("comments\n got %q\n want %q", got, want) + } + // The next run with the same body finds its own: unchanged. + if res, err := ch.UpsertComment(t.Context(), 41, marker, body); err != nil || res != CommentUnchanged { + t.Errorf("same: %s, %v", res, err) + } + if got := s.raws(); len(got) != 2 { + t.Errorf("comments %q", got) + } +} + +func TestReadKeyStoreBitbucket(t *testing.T) { + const repo = "/2.0/repositories/acme/engineering-assets" + const env = "{1c2d3e4f-0000-4000-8000-000000000001}" + const pages = "{1c2d3e4f-0000-4000-8000-000000000002}" + s := newKeyServer(t, "Authorization", "Bearer "+maintainerToken, map[string]string{ + "/2.0/repositories/{}/{" + bbID + "}": `{"uuid": "` + bbUUID + `", "full_name": "acme/engineering-assets", "is_private": true, "mainbranch": {"name": "main"}}`, + repo + "/pipelines_config/variables": `{"pagelen": 100, "values": [{"key": "TOUCHMARK_READ_TOKEN", "secured": true, "value": ""}, + {"key": "TOUCHMARK_WRITE_TOKEN", "secured": false, "value": "leaked-value"}]}`, + "/2.0/workspaces/acme/pipelines-config/variables": "status 403", + repo + "/environments": `{"pagelen": 100, "values": [{"uuid": "` + env + `", "name": "touchmark-distribute", "restrictions": {"admin_only": true}}, + {"uuid": "` + pages + `", "name": "pages"}]}`, + repo + "/deployments_config/environments/" + env + "/variables": `{"values": [{"key": "TOUCHMARK_WRITE_TOKEN", "secured": true}]}`, + repo + "/deployments_config/environments/" + pages + "/variables": `{"values": []}`, + }) + ks, err := ReadKeyStore(t.Context(), KeyStoreInput{Platform: "bitbucket", APIURL: s.URL + "/2.0", RepoID: bbUUID, Token: maintainerToken}) + if err != nil { + t.Fatal(err) + } + if ks.RepoPath != "acme/engineering-assets" || ks.DefaultBranch != "main" || ks.Visibility != "private" { + t.Errorf("store %+v", ks) + } + want := []string{"TOUCHMARK_READ_TOKEN repository", "TOUCHMARK_WRITE_TOKEN environment:touchmark-distribute", "TOUCHMARK_WRITE_TOKEN repository"} + if got := secretNames(ks); !slices.Equal(got, want) { + t.Errorf("secrets %q, want %q", got, want) + } + for _, sec := range ks.Secrets { + if sec.Masked != (sec.Name == "TOUCHMARK_READ_TOKEN" || sec.Where == "environment") { + t.Errorf("secret %+v: secured is %v", sec, sec.Masked) + } + } + if strings.Contains(fmt.Sprintf("%+v", ks), "leaked-value") { + t.Error("the store keeps a variable's value") + } + if len(ks.Environments) != 2 || !ks.Environments[0].AdminOnly || ks.Environments[1].AdminOnly { + t.Errorf("environments %+v", ks.Environments) + } + if len(ks.Unread) != 1 || !strings.Contains(ks.Unread[0], "workspace's Pipelines variables: HTTP 403") { + t.Errorf("unread %q", ks.Unread) + } + // Every request carried the token, to the API's host only. + for _, a := range s.auth { + if a != "Bearer "+maintainerToken { + t.Errorf("authorization %q", a) + } + } + // The fingerprint's form, without braces, reads the same hub. + if _, err := ReadKeyStore(t.Context(), KeyStoreInput{Platform: "bitbucket", APIURL: s.URL + "/2.0", RepoID: bbID, Token: maintainerToken}); err != nil { + t.Errorf("the canonical id: %v", err) + } + if _, err := ReadKeyStore(t.Context(), KeyStoreInput{Platform: "bitbucket", APIURL: s.URL + "/2.0", RepoID: "712345678", Token: maintainerToken}); err == nil { + t.Error("a numeric id on Bitbucket") + } + if _, err := ReadKeyStore(t.Context(), KeyStoreInput{Platform: "bitbucket", APIURL: s.URL + "/2.0", RepoID: bbID, Token: "wrong-token-0123456789"}); err == nil { + t.Error("a refused token") + } +} diff --git a/internal/hubch/comment.go b/internal/hubch/comment.go index 1cda7a6..d32988a 100644 --- a/internal/hubch/comment.go +++ b/internal/hubch/comment.go @@ -13,7 +13,9 @@ import ( // Commenter is a Channel that keeps one comment of touchmark in a hub pull // request up to date (plan --comment): the REST channel of // GitHub, Gitea and Forgejo, with the CI's own token (GitHub's GITHUB_TOKEN -// with pull-requests: write, the Actions token of Gitea and Forgejo). +// with pull-requests: write, the Actions token of Gitea and Forgejo), and +// the channel of Bitbucket with the hub's access token +// (TOUCHMARK_PIPELINES_TOKEN with Pull requests: Write; bitbucket.go). // GitLab's channel is none: CI_JOB_TOKEN cannot write merge request notes. type Commenter interface { // UpsertComment finds the comment on pull request pr that the CI's diff --git a/internal/hubch/hubch.go b/internal/hubch/hubch.go index 5aabbf0..46dab9d 100644 --- a/internal/hubch/hubch.go +++ b/internal/hubch/hubch.go @@ -35,7 +35,10 @@ const ( GitLabCI CI = "gitlab-ci" GiteaActions CI = "gitea-actions" ForgejoActions CI = "forgejo-actions" - Local CI = "local" + // BitbucketPipelines is Bitbucket Pipelines, which runs on Bitbucket + // Cloud only (bitbucket.org). + BitbucketPipelines CI = "bitbucket-pipelines" + Local CI = "local" ) // GitLabUser is the user of the job token in git's Basic header; register @@ -66,11 +69,14 @@ type Context struct { RefName string RefIsBranch bool // Event is the triggering event: push, pull_request, schedule, - // workflow_dispatch, merge_request_event, web, … + // workflow_dispatch, merge_request_event, web, … Bitbucket Pipelines + // tells only pull requests (pull_request) from the rest (push: a branch + // or tag pipeline, a custom pipeline run by hand or on a schedule). Event string // RefProtected is GitLab's CI_COMMIT_REF_PROTECTED. RefProtected bool - // Environment is GitLab's CI_ENVIRONMENT_NAME. + // Environment is GitLab's CI_ENVIRONMENT_NAME, or Bitbucket's + // BITBUCKET_DEPLOYMENT_ENVIRONMENT (the deployment of the step). Environment string // Visibility is "public", "internal" or "private"; "" when unknown. Visibility string @@ -98,6 +104,9 @@ type Context struct { // CI_COMMIT_REF_NAME, CI_COMMIT_BRANCH (set only for branches), // CI_PIPELINE_SOURCE, CI_COMMIT_REF_PROTECTED, CI_ENVIRONMENT_NAME, // CI_PROJECT_VISIBILITY, CI_REPOSITORY_URL, CI_SERVER_TLS_CA_FILE. +// - Bitbucket Pipelines (BITBUCKET_BUILD_NUMBER set): see +// detectBitbucket. Pipelines names no default branch: the hub channel +// reads it (RepositoryReader). // - Otherwise Local with everything empty. // // readFile reads the event payload; errors reading or parsing it leave the @@ -139,6 +148,8 @@ func Detect(getenv func(string) string, readFile func(string) ([]byte, error)) C return detectActions(GitHubActions, env, readFile) case isTrue(env("GITLAB_CI")): return detectGitLab(env) + case env("BITBUCKET_BUILD_NUMBER") != "": + return detectBitbucket(env) } return Context{CI: Local} } @@ -391,7 +402,9 @@ type Channel interface { // Token returns the hub CI's own token from the environment: GITHUB_TOKEN // on GitHub Actions, CI_JOB_TOKEN on GitLab CI, GITEA_TOKEN or GITHUB_TOKEN -// on Gitea and Forgejo Actions; "" when absent or Local. +// on Gitea and Forgejo Actions, TOUCHMARK_PIPELINES_TOKEN on Bitbucket +// Pipelines (BitbucketTokenVar: Pipelines gives a step no API token of its +// own); "" when absent or Local. // // The caller registers the token with its redact.Registry before use, on // GitLab with the basic user GitLabUser. @@ -410,6 +423,8 @@ func Token(c Context, getenv func(string) string) string { return t } return env("GITHUB_TOKEN") + case BitbucketPipelines: + return env(BitbucketTokenVar) } return "" } @@ -419,6 +434,8 @@ func Token(c Context, getenv func(string) string) string { // GET {api}/repos/{path}/git/ref/heads/{branch}; Gitea and Forgejo // GET {api}/repos/{path}/branches/{branch} — with the token sent only to // the host of c.APIURL; +// - Bitbucket: REST over client (newBitbucket), with the hub's access +// token as a Bearer token sent only to the host of c.APIURL; // - GitLab: `git ls-remote` of c.RepositoryURL with the job token passed // as an http.extraHeader through GIT_CONFIG_COUNT/KEY/VALUE environment // variables (never in the URL or argv), because CI_JOB_TOKEN cannot @@ -448,6 +465,8 @@ func New(c Context, client *httpx.Client, token string) (Channel, error) { return newREST(c, client, token) case GitLabCI: return newGit(c, token) + case BitbucketPipelines: + return newBitbucket(c, client, token) case Local, "": return nil, errors.New("hub channel: not running in a known CI") } @@ -588,6 +607,11 @@ type Environment struct { // Policies are its custom deployment branch and tag policies, when it // has them: name patterns (fnmatch) with type "branch" or "tag". Policies []EnvironmentPolicy + // AdminOnly is a Bitbucket deployment environment's + // restrictions.admin_only (Premium: deployments by others pause). The + // field is not in Bitbucket's API reference (assumed from the API's + // answers); Bitbucket shows no deployment branch restriction at all. + AdminOnly bool } // EnvironmentPolicy is one custom deployment branch or tag policy. diff --git a/internal/hubch/hubch_test.go b/internal/hubch/hubch_test.go index 0d05a75..dac07b2 100644 --- a/internal/hubch/hubch_test.go +++ b/internal/hubch/hubch_test.go @@ -857,8 +857,9 @@ func TestGitLabChannelIsolation(t *testing.T) { t.Setenv("GIT_SSL_NO_VERIFY", "1") t.Setenv("GIT_TRACE", "1") - // The fixture works: git that inherits the environment reads the decoy. - out, err := (&gitx.Git{}).Run(t.Context(), nil, "ls-remote", hubURL, "refs/heads/main") + // The fixture works: git that inherits the environment reads the decoy + // (plain git: gitx.Git drops these variables too). + out, err := exec.CommandContext(t.Context(), "git", "ls-remote", hubURL, "refs/heads/main").Output() if err != nil || !strings.HasPrefix(string(out), other) { t.Fatalf("plain git read %q, %v; want the decoy %s", out, err, other) } diff --git a/internal/hubch/keys.go b/internal/hubch/keys.go index de3484c..b57f1f3 100644 --- a/internal/hubch/keys.go +++ b/internal/hubch/keys.go @@ -15,18 +15,20 @@ import ( // Where the hub keeps its secrets, as a maintainer's token of the hub reads // it: `touchmark doctor --hub-token`. The token goes to the // hub's API host only, in a header; the values of secrets and variables are -// never kept (the APIs of GitHub and Gitea never return them; GitLab's -// variables carry them, and the decoding drops them). +// never kept (the APIs of GitHub and Gitea never return them; GitLab's and +// Bitbucket's variables carry those not masked or secured, and the decoding +// drops them). // KeyStoreInput locates the hub for ReadKeyStore. type KeyStoreInput struct { - // Platform is "github", "gitlab", "gitea" or "forgejo"; "bitbucket" is - // refused for now. + // Platform is "github", "gitlab", "gitea", "forgejo" or "bitbucket". Platform string // APIURL is the REST base of the hub's platform (…/api/v3, /api/v4, // /api/v1, https://api.github.com). APIURL string - // RepoID is the hub repository's immutable id (the fingerprint's). + // RepoID is the hub repository's immutable id (the fingerprint's): a + // number, or on Bitbucket the repository's UUID (with or without + // braces). RepoID string // Token is the maintainer's token. Token string @@ -44,8 +46,9 @@ type KeyStore struct { // name, where they live and how they are kept. Values are never read. Secrets []Secret // Environments are the hub's deployment environments on GitHub, with - // who may use them; EnvironmentsUnread says why they could not be listed - // ("" when they were). + // who may use them, and on Bitbucket (whose API does not show which + // branches may deploy); EnvironmentsUnread says why they could not be + // listed ("" when they were). Environments []NamedEnvironment EnvironmentsUnread string // ProtectedBranches and ProtectedTags are GitLab's: the refs whose @@ -65,18 +68,21 @@ type KeyStore struct { Unread []string } -// Secret is one secret (GitHub, Gitea, Forgejo) or CI/CD variable (GitLab). +// Secret is one secret (GitHub, Gitea, Forgejo), CI/CD variable (GitLab) +// or Pipelines variable (Bitbucket). type Secret struct { Name string // Where is "repository", "organization" (shared with the hub), - // "dependabot", "environment" (Environment names it), "project" or - // "group" (Group names it, GitLab). + // "dependabot", "environment" (Environment names it; a deployment + // variable on Bitbucket), "project" or "group" (Group names it, GitLab), + // "workspace" (Bitbucket). Where string Environment string Group string // Protected, Masked and Scope are GitLab's: a protected variable // reaches pipelines of protected refs only; Scope is its environment - // scope ("*" for every job). + // scope ("*" for every job). On Bitbucket Masked is a secured + // variable: hidden in the logs and the API. Protected bool Masked bool Scope string @@ -115,7 +121,9 @@ const ( // its CI/CD variables and those of each group above it (protected, // masked, environment scope); its protected branches and tags; // - Gitea and Forgejo: the repository by id, the names of its Actions -// secrets and of its organization's. +// secrets and of its organization's; +// - Bitbucket: the repository by UUID; its Pipelines variables, its +// workspace's, and each deployment environment's (secured or not). // // A listing the token may not read is noted in Unread, or in the field of // its own (EnvironmentsUnread, ProtectedBranchesUnread, @@ -125,7 +133,12 @@ func ReadKeyStore(ctx context.Context, in KeyStoreInput) (KeyStore, error) { if in.Client == nil { in.Client = httpx.New(httpx.Options{}) } - if repoID(in.RepoID) == "" { + switch { + case in.Platform == "bitbucket": + if in.RepoID = BitbucketRepoID(in.RepoID); in.RepoID == "" { + return KeyStore{}, errors.New("hub keys: the hub's repository UUID is unknown") + } + case repoID(in.RepoID) == "": return KeyStore{}, errors.New("hub keys: the hub's repository id is unknown") } if err := checkToken(in.Token); err != nil { @@ -150,7 +163,8 @@ func ReadKeyStore(ctx context.Context, in KeyStoreInput) (KeyStore, error) { k.auth = &httpx.Auth{Hosts: []string{api.Host}, Header: k.header("token ")} return k.gitea(ctx) case "bitbucket": - return KeyStore{}, errors.New("hub keys: a hub on Bitbucket is not supported yet") + k.auth = &httpx.Auth{Hosts: []string{api.Host}, Header: k.header("Bearer ")} + return k.bitbucket(ctx) } return KeyStore{}, fmt.Errorf("hub keys: unknown platform %q", in.Platform) } diff --git a/internal/hubch/keys_bitbucket.go b/internal/hubch/keys_bitbucket.go new file mode 100644 index 0000000..0ea95f9 --- /dev/null +++ b/internal/hubch/keys_bitbucket.go @@ -0,0 +1,129 @@ +package hubch + +import ( + "context" + "errors" + "fmt" + "strings" +) + +// bitbucketVariable is a Pipelines variable of Bitbucket (repository, +// workspace or deployment); its value is not decoded. +type bitbucketVariable struct { + Key string `json:"key"` + Secured bool `json:"secured"` +} + +// bitbucket reads the key store of a hub on Bitbucket Cloud: +// - the repository by UUID, GET /repositories/{}/{uuid} (the empty +// workspace field is documented: "Repository object and UUID" in +// https://developer.atlassian.com/cloud/bitbucket/rest/intro/); +// - its repository variables, GET …/pipelines_config/variables; +// - its workspace's variables, GET /workspaces/{ws}/pipelines-config/variables +// (workspace administrators only: otherwise unread); +// - its deployment environments, GET …/environments, and each one's +// variables, GET …/deployments_config/environments/{uuid}/variables. +// +// Bitbucket's API does not show which branches may deploy to an +// environment (a Premium setting), so the key store holds none; an +// environment's restrictions.admin_only, not in the API reference, is read +// when present. +func (k *keyReader) bitbucket(ctx context.Context) (KeyStore, error) { + var repo struct { + UUID string `json:"uuid"` + FullName string `json:"full_name"` + IsPrivate bool `json:"is_private"` + MainBranch *struct { + Name string `json:"name"` + } `json:"mainbranch"` + } + u := k.base + "/repositories/%7B%7D/%7B" + k.in.RepoID + "%7D" + if err := k.get(ctx, u, &repo); err != nil { + return KeyStore{}, fmt.Errorf("hub keys: read the hub repository {%s}: %w", k.in.RepoID, err) + } + if BitbucketRepoID(repo.UUID) != k.in.RepoID { + return KeyStore{}, fmt.Errorf("hub keys: the answer names repository %q, not {%s}", repo.UUID, k.in.RepoID) + } + ws, slug, ok := strings.Cut(repo.FullName, "/") + if !ok || !validName(ws) || !validName(slug) { + return KeyStore{}, fmt.Errorf("hub keys: the hub repository's name %q is not workspace/slug", repo.FullName) + } + k.ks = KeyStore{Platform: "bitbucket", RepoPath: repo.FullName, Visibility: "public"} + if repo.IsPrivate { + k.ks.Visibility = "private" + } + if repo.MainBranch != nil { + k.ks.DefaultBranch = repo.MainBranch.Name + } + for _, l := range []struct{ what, where, u string }{ + {"the repository's Pipelines variables", "repository", k.segs("repositories", ws, slug, "pipelines_config", "variables")}, + {"the workspace's Pipelines variables", "workspace", k.segs("workspaces", ws, "pipelines-config", "variables")}, + } { + vars, err := bitbucketList[bitbucketVariable](ctx, k, l.u) + if err != nil { + if err := k.unread(l.what, err); err != nil { + return KeyStore{}, err + } + continue + } + for _, v := range vars { + k.ks.Secrets = append(k.ks.Secrets, Secret{Name: v.Key, Where: l.where, Masked: v.Secured}) + } + } + type environment struct { + UUID string `json:"uuid"` + Name string `json:"name"` + Restrictions *struct { + AdminOnly bool `json:"admin_only"` + } `json:"restrictions"` + } + envs, err := bitbucketList[environment](ctx, k, k.segs("repositories", ws, slug, "environments")) + if err != nil { + if err := k.unread("the repository's deployment environments", err); err != nil { + return KeyStore{}, err + } + k.ks.EnvironmentsUnread = k.ks.Unread[len(k.ks.Unread)-1] + } + for _, e := range envs { + ne := NamedEnvironment{Name: e.Name} + if e.Restrictions != nil { + ne.AdminOnly = e.Restrictions.AdminOnly + } + k.ks.Environments = append(k.ks.Environments, ne) + if BitbucketRepoID(e.UUID) == "" { + k.ks.Unread = append(k.ks.Unread, fmt.Sprintf("the variables of environment %s: no UUID in the answer", e.Name)) + continue + } + vars, err := bitbucketList[bitbucketVariable](ctx, k, k.segs("repositories", ws, slug, "deployments_config", "environments", e.UUID, "variables")) + if err != nil { + if err := k.unread("the variables of environment "+e.Name, err); err != nil { + return KeyStore{}, err + } + continue + } + for _, v := range vars { + k.ks.Secrets = append(k.ks.Secrets, Secret{Name: v.Key, Where: "environment", Environment: e.Name, Masked: v.Secured}) + } + } + return k.ks, nil +} + +// bitbucketList reads a paginated Bitbucket listing of T (values), 100 a +// page, until a page without next. +func bitbucketList[T any](ctx context.Context, k *keyReader, u string) ([]T, error) { + var out []T + for page := 1; page <= keyPages; page++ { + var list struct { + Next string `json:"next"` + Values []T `json:"values"` + } + if err := k.get(ctx, fmt.Sprintf("%s?pagelen=%d&page=%d", u, keyPageSize, page), &list); err != nil { + return nil, err + } + out = append(out, list.Values...) + if list.Next == "" || len(list.Values) == 0 { + return out, nil + } + } + return out, errors.New("more items than touchmark reads") +} diff --git a/internal/marker/marker.go b/internal/marker/marker.go index 13beee3..702b757 100644 --- a/internal/marker/marker.go +++ b/internal/marker/marker.go @@ -136,8 +136,9 @@ var ( // hyphens, so an id never holds "--", which HTML comments forbid. hubRe = regexp.MustCompile(`^[a-z0-9]+(-[a-z0-9]+)*$`) // fingerprintRe is the fingerprint syntax of hub.yml - // (previous_fingerprints): host, optional port, numeric repository id. - fingerprintRe = regexp.MustCompile(`^[A-Za-z0-9.-]+(:[0-9]{1,5})?/[0-9]+$`) + // (previous_fingerprints) in canonical form: host, optional port, and a + // numeric repository id or a lowercase Bitbucket repository UUID. + fingerprintRe = regexp.MustCompile(`^[A-Za-z0-9.-]+(:[0-9]{1,5})?/([0-9]+|[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})$`) ) // errTooLarge marks an Encode failure caused by size alone: the marker does @@ -424,7 +425,7 @@ func checkData(m Marker) error { case d.Hub != m.Hub: return fmt.Errorf("data hub %.40q differs from the header's %.40q", d.Hub, m.Hub) case !fingerprintRe.MatchString(d.FP): - return fmt.Errorf("data fp %.40q is not host/numeric-id", d.FP) + return fmt.Errorf("data fp %.40q is not host/repository-id", d.FP) case FP16(d.FP) != m.FP16: return fmt.Errorf("data fp %.40q does not hash to the header's fp %s", d.FP, m.FP16) } diff --git a/internal/report/delivery.go b/internal/report/delivery.go index a0f6cd8..b9fc6e5 100644 --- a/internal/report/delivery.go +++ b/internal/report/delivery.go @@ -755,7 +755,7 @@ func (d *Delivery) mdGroups(m *mdWriter) { } for _, g := range groups { m.line("") - m.open(fmt.Sprintf("
%s (%d)", mdText(g.label), len(g.targets))) + m.details(fmt.Sprintf("%s (%d)", mdText(g.label), len(g.targets))) m.line("") hidden := 0 for _, t := range g.targets { @@ -769,7 +769,7 @@ func (d *Delivery) mdGroups(m *mdWriter) { m.line(fmt.Sprintf("- %d private in public hub, not named", hidden)) } m.line("") - m.close("
") + m.endDetails() } } @@ -815,6 +815,10 @@ type mdWriter struct { // openTag is set while a
section is open. openTag bool cut bool + // plain writes Markdown without HTML, for a platform that shows HTML as + // text (Bitbucket escapes it): a section is a bold title, not + //
. + plain bool } // cutNote ends a cut report. @@ -859,6 +863,23 @@ func (m *mdWriter) close(s string) { } } +// details opens a collapsible section titled summary (Markdown text): a +//
block, or with plain a bold line. +func (m *mdWriter) details(summary string) { + if m.plain { + m.line("**" + summary + "**") + return + } + m.open("
" + summary + "") +} + +// endDetails closes the section details opened. +func (m *mdWriter) endDetails() { + if !m.plain { + m.close("
") + } +} + // finish returns the Markdown, cut with a note when it did not fit. func (m *mdWriter) finish() string { if !m.cut { diff --git a/internal/report/plan.go b/internal/report/plan.go index fe4990a..f48ba29 100644 --- a/internal/report/plan.go +++ b/internal/report/plan.go @@ -264,10 +264,21 @@ const ( // line appended after it. closing is Markdown the caller made safe, such as // the comment's hidden marker; it counts against the limit. func (d *Delivery) WriteComment(w io.Writer, limit int, closing string) error { + return d.writeComment(w, limit, closing, false) +} + +// WritePlainComment is WriteComment without HTML, for a platform whose +// comments show HTML as text (Bitbucket): the collapsible sections become +// bold titles over their lists. closing must hold no HTML either. +func (d *Delivery) WritePlainComment(w io.Writer, limit int, closing string) error { + return d.writeComment(w, limit, closing, true) +} + +func (d *Delivery) writeComment(w io.Writer, limit int, closing string, plain bool) error { if limit <= 0 { limit = MaxComment } - m := &mdWriter{limit: limit - len(closing), note: commentCut} + m := &mdWriter{limit: limit - len(closing), note: commentCut, plain: plain} d.markdown(m) _, err := io.WriteString(w, m.finish()+closing) if err != nil { @@ -338,7 +349,7 @@ func (d *Delivery) mdPaths(m *mdWriter) { return } m.line("") - m.open(fmt.Sprintf("
%s (%d)", mdText(d.pathsTitle()), len(d.Paths))) + m.details(fmt.Sprintf("%s (%d)", mdText(d.pathsTitle()), len(d.Paths))) m.line("") m.line("| Change | Path | Targets |") m.line("|---|---|---:|") @@ -355,5 +366,5 @@ func (d *Delivery) mdPaths(m *mdWriter) { m.line(fmt.Sprintf("and %s more; the JSON report lists them all", plural(n, "path"))) } m.line("") - m.close("
") + m.endDetails() } diff --git a/internal/report/plan_test.go b/internal/report/plan_test.go index c13370a..5b76e08 100644 --- a/internal/report/plan_test.go +++ b/internal/report/plan_test.go @@ -1,6 +1,7 @@ package report import ( + "regexp" "strings" "testing" ) @@ -131,6 +132,44 @@ func TestWriteComment(t *testing.T) { } } +// WritePlainComment renders the same report without HTML (Bitbucket shows +// HTML as text): every section a bold title, and the cut note and closing +// line as WriteComment has them. +func TestWritePlainComment(t *testing.T) { + d := planned() + const marker = "\n[touchmark-plan]: # \"touchmark plan: acme-eng\"\n" + var html, plain strings.Builder + if err := d.WriteComment(&html, 0, marker); err != nil { + t.Fatal(err) + } + if err := d.WritePlainComment(&plain, 0, marker); err != nil { + t.Fatal(err) + } + s := plain.String() + if !strings.Contains(html.String(), "
") { + t.Fatalf("the sample report has no section to compare:\n%s", html.String()) + } + if strings.Contains(s, "<") || !strings.HasSuffix(s, marker) { + t.Errorf("plain comment:\n%s", s) + } + for _, m := range regexp.MustCompile(`
(.*)`).FindAllStringSubmatch(html.String(), -1) { + if !strings.Contains(s, "\n**"+m[1]+"**\n") { + t.Errorf("plain comment lacks the section title **%s**:\n%s", m[1], s) + } + } + for i := range 3000 { + d.Targets = append(d.Targets, DeliveryTarget{Provider: "bb", Host: "bitbucket.org", RepoID: "9", Path: "acme/many-" + strings.Repeat("x", i%40), + Outcome: OutcomeOpened}) + } + var cut strings.Builder + if err := d.WritePlainComment(&cut, 20000, marker); err != nil { + t.Fatal(err) + } + if s := cut.String(); len(s) > 20000 || !strings.HasSuffix(s, commentCut+marker) || strings.Contains(s, "<") { + t.Errorf("cut plain comment of %d bytes:\n%s", len(s), s[max(0, len(s)-400):]) + } +} + // The schema accepts a plan's scope, estimate and --assume-opt-in, and // refuses them malformed. func TestPlanSchema(t *testing.T) { diff --git a/schemas/doctor.schema.json b/schemas/doctor.schema.json index 973d6e4..d375c15 100644 --- a/schemas/doctor.schema.json +++ b/schemas/doctor.schema.json @@ -79,9 +79,9 @@ "pattern": "^[a-z0-9]+(-[a-z0-9]+)*$" }, "fingerprint": { - "description": "The hub's host and immutable repository id, e.g. github.com/712345678.", + "description": "The hub's host and immutable repository id, e.g. github.com/712345678; on Bitbucket the repository UUID, lowercase without braces.", "type": "string", - "pattern": "^[A-Za-z0-9.-]+(:[0-9]{1,5})?/[0-9]+$" + "pattern": "^[A-Za-z0-9.-]+(:[0-9]{1,5})?/([0-9]+|[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})$" }, "commit": { "description": "Full commit id: 40 or 64 lowercase hex digits.", diff --git a/schemas/hub.schema.json b/schemas/hub.schema.json index 1d3549d..54d2779 100644 --- a/schemas/hub.schema.json +++ b/schemas/hub.schema.json @@ -29,11 +29,11 @@ "items": { "$ref": "#/$defs/branch" } }, "previous_fingerprints": { - "description": "Former fingerprints of this hub (host/numeric-repository-id), after the hub moved.", + "description": "Former fingerprints of this hub (host/repository-id: a numeric id, or a Bitbucket repository UUID with or without braces), after the hub moved.", "type": ["array", "null"], "items": { "type": "string", - "pattern": "^[A-Za-z0-9.-]+(:[0-9]{1,5})?/[0-9]+$" + "pattern": "^[A-Za-z0-9.-]+(:[0-9]{1,5})?/([0-9]+|[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}|\\{[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}\\})$" } }, "opt_in_file": { @@ -136,11 +136,11 @@ "additionalProperties": false, "properties": { "write_isolation": { - "description": "Where the write key lives. platform: in the hub's CI platform, checked by a probe; external: issued by an external vault over OIDC; none: risk accepted, needs a reason. Default: platform.", + "description": "Where the write key lives. platform: in the hub's CI platform, checked by a probe (on Bitbucket Pipelines only with a reason); external: issued by an external vault over OIDC; none: risk accepted, needs a reason. Default: platform.", "enum": ["platform", "external", "none"] }, "reason": { - "description": "Why the write key cannot be isolated. Required with write_isolation: none.", + "description": "Why the write key cannot be isolated. Required with write_isolation: none, and on Bitbucket Pipelines with platform, where it states the deployment permissions (Premium) that keep the write key on the default branch, which touchmark cannot read.", "type": "string" }, "private_targets_in_public_hub": { diff --git a/schemas/report.schema.json b/schemas/report.schema.json index 8272d96..d48721f 100644 --- a/schemas/report.schema.json +++ b/schemas/report.schema.json @@ -124,9 +124,9 @@ "pattern": "^[a-z0-9]+(-[a-z0-9]+)*$" }, "fingerprint": { - "description": "The hub's host and immutable repository id, e.g. github.com/712345678.", + "description": "The hub's host and immutable repository id, e.g. github.com/712345678; on Bitbucket the repository UUID, lowercase without braces.", "type": "string", - "pattern": "^[A-Za-z0-9.-]+(:[0-9]{1,5})?/[0-9]+$" + "pattern": "^[A-Za-z0-9.-]+(:[0-9]{1,5})?/([0-9]+|[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})$" }, "commit": { "$ref": "#/$defs/commit" }, "pr": { diff --git a/zensical.toml b/zensical.toml index fd54d9a..a526de3 100644 --- a/zensical.toml +++ b/zensical.toml @@ -20,6 +20,7 @@ nav = [ { "A hub on GitHub" = "getting-started/github.md" }, { "A hub on GitLab" = "getting-started/gitlab.md" }, { "A hub on Gitea or Forgejo" = "getting-started/gitea-forgejo.md" }, + { "A hub on Bitbucket Cloud" = "getting-started/bitbucket.md" }, { "Opt a repository in" = "getting-started/opt-in.md" }, ] }, { "Concepts" = [