diff --git a/docs/agents.md b/docs/agents.md index bf0b73b..203b7c8 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -47,7 +47,8 @@ no lock file, no header, no marker in their files. It never checks a target out and never runs a script, hook, filter or LFS from the hub or a target. It delivers nothing to a repository without the opt-in file, unless `targets.yml` subscribes it (`opt_in: assumed`). There is no pull mode (targets do not -fetch from the hub), no remote pack sources, and no Bitbucket or Azure DevOps driver. +fetch from the hub), no remote pack sources, no Azure DevOps driver, and no hub on +Bitbucket Pipelines: Bitbucket Cloud is a provider a hub elsewhere delivers to. ## Mental model @@ -283,7 +284,7 @@ covers everything under it). At most 64 KiB. `.touchmark/operations.yml` (in the hub; `touchmark schema operations`): `recreate[]` (`target`, `head`: acts while the branch head is that commit), -`forget_declines[]` (`target`, `pr`: acts once), `allow_mass_close` (`max`, `until`), +`forget_declines[]` (`target`, `pr`: acts once; on Bitbucket Cloud while present), `allow_mass_close` (`max`, `until`), `adopt_unmarked` (`until`). Dates are `YYYY-MM-DD`, UTC, inclusive. Environment, one set per provider; `` is the provider id upper-cased with `-` as @@ -458,7 +459,7 @@ Each target ends with one outcome, and a reason for most: | `opened`, `updated`, `unchanged`, `closed` | `updated`: `content`, `rebase`, `recreate`, `body`, `title`, `base-renamed`; `closed`: `no-diff`, `opted-out`, `target-dropped`, `duplicate` | nothing | | `declined` | the declined pull request | nothing; see [memory](concepts/memory.md) to propose again | | `skipped` | `not-opted-in`, `opted-out` (the opt-in file says `enabled: false`), `archived`, `disabled`, `empty`, `mirror`, `pending-deletion`, `prs-disabled`, `sha256`, `unsafe-opt-in`, `private-in-public-hub`, `superseded` | add the opt-in file (or `opt_in: assumed` in the hub), or nothing | -| `blocked` | `edited` (someone pushed to the sync branch), `branch-taken`, `branch-in-use`, `opt-in-invalid`, `marker-invalid`, `rules:`, `permission:`, `cannot-sign`, `archived`, `mass-close` | tick *Rebuild this branch* or add a `recreate` entry; fix the opt-in file; grant the permission; add a signing key; `allow_mass_close` | +| `blocked` | `edited` (someone pushed to the sync branch), `branch-taken`, `branch-in-use`, `opt-in-invalid`, `marker-invalid`, `rules:`, `permission:`, `cannot-sign`, `archived`, `mass-close` | tick *Rebuild this branch* (not on Bitbucket Cloud) or add a `recreate` entry; fix the opt-in file; grant the permission; add a signing key; `allow_mass_close` | | `deferred` | `rate-limit`, `deadline`, `rollout-limit`, `provider-down`, `interrupted`, `cooldown` | nothing: the next run continues | | `failed` | `transient`, `auth`, `access`, `git`, `integrity`, `race`, `secret-exposure`, `internal` | read the report's reason; `auth` and `access` are the credential | diff --git a/docs/concepts/delivery.md b/docs/concepts/delivery.md index 69a2c38..6cf71d0 100644 --- a/docs/concepts/delivery.md +++ b/docs/concepts/delivery.md @@ -102,7 +102,8 @@ The body, in English: `sensitive_paths` from `hub.yml`). Never cut; 3. a table of the changes, by file and pack: sensitive first, then deletions, updates, additions, up to 100 rows; -4. a folded list of the files the repository made its own, with how to take them back; +4. a folded list of the files the repository made its own, with how to take them back + (a plain section on Bitbucket Cloud, which shows HTML as text); 5. blocks for the situation: *previously declined in #N*, *touchmark paused*, *the hub proposes nothing more*; 6. the tick boxes; diff --git a/docs/concepts/memory.md b/docs/concepts/memory.md index cc8abfa..2e302dd 100644 --- a/docs/concepts/memory.md +++ b/docs/concepts/memory.md @@ -58,7 +58,8 @@ current changes, or closes it as `no-diff`. When touchmark first sees a decline, it records it in the marker, adds a tick box to the description and leaves one comment saying what it remembered, when a new pull request -would come, and how to undo it. Two writes, once. +would come, and how to undo it. Two writes, once. (Not on Bitbucket Cloud, where a +declined pull request cannot be edited: see [below](#on-bitbucket-cloud).) To undo a decline: @@ -72,6 +73,42 @@ To refuse a file for good instead, add it to `ignore`, or set `enabled: false` i file (deleting it works too, unless the hub subscribed the repository) to stop receiving anything. +## On Bitbucket Cloud + +A declined pull request on Bitbucket can never be edited or reopened, by anyone. touchmark +writes nothing to it, and remembers the decline from what the marker held while the pull +request was open: + +- **The opt-in state is recorded up front.** Every pull request touchmark opens records + in its marker the state of the opt-in file it is proposed under, which the + acknowledgment records elsewhere. When the file changes while the pull request is + open, the next run records the new state, with one edit of the description even when + the proposed changes stay the same, and even when the target is blocked + (`branch-in-use`, `rules:*`, …): when the run writes nothing else to the pull + request, that edit changes only the marker. A push a branch restriction refuses at + run time is followed by that edit too. +- **A decline** holds while the opt-in file is parsed the same as that recorded state, + and lapses when the team edits `packs` or `ignore`, as an acknowledged decline does + elsewhere. A declined path becoming `local` or `ignored` lifts it too. +- **No comment, no tick box.** touchmark leaves no comment on the declined pull request + (with nothing to record that it did, it would post one every run), and descriptions + carry no tick boxes there: Bitbucket shows the HTML comments they rest on as text. The + report says `declined`, and the description's footnote names the way back. +- **Bringing declined content back** is a `forget_declines` entry, or a change of `packs` + or `ignore`. touchmark cannot mark the pull request revoked, so the entry acts on every + run while it is present: keep it until the pull request it brings is merged or closed. + Removed earlier, the decline is in force again; an open pull request stays (an open pull + request is stronger than memory), and its description then names the decline under + *Previously declined*. +- **A marker without the opt-in state** (one touchmark did not write in full) holds its + decline whatever the opt-in file says, since there is no state to compare with; the + report warns about it on every run and names the `forget_declines` entry that lifts + it. +- **Who closed it** counts as everywhere: Bitbucket names who declined a pull request. + touchmark's own closes write the close into the description in the same edit that + declines it; bots' closes wait for the cooldown, and the third in a row counts as a + decline. + ## Limits - Only the last 50 closed pull requests on the sync branches count. @@ -83,3 +120,10 @@ receiving anything. own: that decline is lost and the content is proposed again. - Closed pull requests of the multi-gitter era, without a marker, are never declines: their content is unknown. +- On Bitbucket Cloud the opt-in state of a decline is the one recorded by the last run + before the decline, whatever that run decided for the target. When the team changes + `packs` or `ignore` and declines the pull request before touchmark runs again, the + decline counts from the earlier state and has lapsed already: the content is proposed + once more, under the new choice. A blocked target does not widen that window: its run + records the state as well. Only a run that writes nothing to the target at all (a + failed or deferred target, `blocked:mass-close`) leaves it open until the next run. diff --git a/docs/guide/operations.md b/docs/guide/operations.md index 39ae2eb..65e1cd4 100644 --- a/docs/guide/operations.md +++ b/docs/guide/operations.md @@ -37,7 +37,7 @@ cleanup to stay safe: | Entry | Acts | Then | |---|---|---| | `recreate` | while the sync branch's head is exactly `head` | a new push to the branch makes it inert | -| `forget_declines` | once: touchmark marks pull request `pr` revoked | inert | +| `forget_declines` | once: touchmark marks pull request `pr` revoked; on Bitbucket Cloud, while the entry is present (see below) | inert | | `allow_mass_close` | up to `max` closes in a run, until the end of `until` (UTC) | inert after the date | | `adopt_unmarked` | until the end of `until` (UTC) | inert after the date | @@ -63,6 +63,15 @@ A target's team can do two of these themselves, from the pull request: - tick **Propose this content again** in a declined pull request: the same as `forget_declines`, or simply reopen it. +On Bitbucket Cloud descriptions carry no tick boxes, and a declined pull request can +never be reopened or edited: a `recreate` entry is the only way to rebuild a paused +branch, and a `forget_declines` entry the only way, besides editing `packs` or `ignore`, +to have declined content proposed again. touchmark cannot mark the declined pull request revoked there, so the entry acts +on every run while it is present: keep it until the pull request it brings is merged or +closed. Removed earlier, the decline is in force again; an open pull request stays, as +memory never closes one, and its description then names the decline under *Previously +declined*. + ## On your machine A maintainer who holds the write key can run the same operations as flags of diff --git a/docs/guide/providers.md b/docs/guide/providers.md index 3e0c928..1afc874 100644 --- a/docs/guide/providers.md +++ b/docs/guide/providers.md @@ -2,7 +2,8 @@ One hub can deliver to GitHub, GitLab, Gitea and Forgejo at once, including self-managed instances. Each platform is a **provider** in `hub.yml`, with its own reader -and writer; `targets.yml` names repositories by provider. +and writer; `targets.yml` names repositories by provider. Bitbucket Cloud is a provider +too, delivered to from a hub on another platform: see [Bitbucket Cloud](#bitbucket-cloud). ## One provider: the shorthand @@ -58,7 +59,8 @@ providers: - `id` is how `targets.yml` names the provider, and the `` of its variables: upper-cased, with `-` as `_`. - `api_url` is needed only when it cannot be derived: `api.github.com`, `/api/v3` - for GitHub Enterprise Server, `/api/v4` for GitLab, `/api/v1` for Gitea and Forgejo. + for GitHub Enterprise Server, `/api/v4` for GitLab, `/api/v1` for Gitea and Forgejo, + `https://api.bitbucket.org/2.0` for Bitbucket Cloud. - `known_authors` are former writers whose pull requests stay the hub's own: a replaced App, a recreated group access token's bot, the account of a multi-gitter setup. touchmark never adds anyone there itself. @@ -124,6 +126,79 @@ on GitHub or GitLab can still deliver to them: add them as providers, and keep t write token in the GitHub environment or the GitLab protected variable, where the probe covers it. +## Bitbucket Cloud + +A provider of type `bitbucket` is Bitbucket Cloud at `https://bitbucket.org` (leave `url` +out). `plan`, `distribute` and `doctor` work on it from a hub on GitHub or GitLab, or +wherever you run touchmark; a hub whose own CI is Bitbucket Pipelines is not supported +yet, and `setup` has nothing for it. Bitbucket Data Center is not supported. + +```yaml +# hub.yml +providers: + - id: bb + type: bitbucket + writer: "{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d}" # the writer account's UUID, in braces +``` + +**Accounts.** Make two bot accounts (Atlassian accounts that only touchmark uses), each +with an API token; app passwords are gone, and repository, project and workspace access +tokens are not supported, since they cannot tell who they are. A token's scopes do not +narrow the repositories it reaches, the account's permissions do: that is why the writer +is an account of its own. + +| | Reader | Writer | +|---|---|---| +| Variable | `TOUCHMARK_BB_READ_TOKEN` | `TOUCHMARK_BB_WRITE_TOKEN` | +| Scopes (all `…:bitbucket`) | `read:user`, `read:workspace`, `read:repository`, `read:pullrequest` | the reader's, plus `write:repository` and `write:pullrequest`; no admin or delete scope | +| Repositories | read access to the targets | write access to the targets only (directly, through a group or the project); not admin, and no access to the hub | + +API tokens expire within a year, and Bitbucket's API shows neither their expiry nor +their scopes: `doctor` reports both as unknown, so keep a reminder to rotate them. + +- **Accounts are UUIDs.** Bitbucket finds no account by its nickname, so `writer`, + `known_authors` and `automation_accounts` name accounts by their UUID in braces, as + `GET https://api.bitbucket.org/2.0/user` shows it. +- **Targets.** `org` is a workspace, `repo` is `workspace/repository`, and web URLs such as + `https://bitbucket.org/acme/billing` work. Bitbucket has no topics: `check` refuses + `topics` on a Bitbucket entry; select repositories with `match` (`acme/svc-*`) or list + them with `repo`. +- **No labels.** Bitbucket pull requests have none: `pr.labels` sets nothing there, and + touchmark records no label as set. +- **Drafts** are Bitbucket's own draft flag (`pr.draft`). +- **Closing.** touchmark closes its own pull request by declining it, after writing the + close into its description in the same edit. Bitbucket can never reopen a declined pull + request, nor change it: a pull request touchmark closed stays closed, and the next + proposal is a new one. +- **Declines** are remembered without writing to the declined pull request: while a pull + request is open, its marker records the state of the opt-in file it is proposed under + (touchmark brings it up to date when the file changes), and a person's decline holds + while the opt-in file is parsed the same. touchmark leaves no comment on it; the report + says `declined`. To have declined content proposed again, edit `packs` or `ignore`, or + add a `forget_declines` entry, which acts while it is present on Bitbucket (see + [Memory of declined pull requests](../concepts/memory.md#on-bitbucket-cloud)). +- **No tick boxes, no HTML.** *Rebuild this branch* and *Propose this content again* rest + on HTML comments, which Bitbucket shows as text: descriptions there carry neither. A + paused pull request asks for a `recreate` entry in `.touchmark/operations.yml` instead, + and the footnote names `forget_declines`. The files the repository made its own are a + plain section, not a folded one. Keep `pr.intro_file` free of HTML too: Bitbucket shows + it as text. +- **Branch restrictions** are readable with admin rights only, which the writer should + not have: `doctor` shows `rules` as unknown, and a push a restriction refuses + ("Permission denied to update branch") is `blocked:rules:protected-branch`. +- **The marker.** Bitbucket shows HTML comments in descriptions as text, so the marker + is a Markdown link reference definition there, which renders as nothing: + `[touchmark]: # "touchmark:v1 hub=… fp=… stream=sync key=… data=…"`, the same payload as + the comment elsewhere. An edit on the website may add backslashes before punctuation + and Windows line endings; touchmark reads the marker either way. +- **Commit author.** Bitbucket links a commit to the account one of whose confirmed + addresses is the author's: touchmark authors the writer's commits with its account's + primary confirmed address, and with `@touchmark.invalid` (an address that can + belong to no one) when the account has none. +- **Pace.** Bitbucket allows an account about 1 000 API requests an hour, so touchmark + reads 15 times a minute by default, with two targets at once, and writes at most once a + second; `limits` overrides it. + ## Moving - **A target moves to another provider on the same host** (a new App, a new group): add diff --git a/docs/reference/compatibility.md b/docs/reference/compatibility.md index 2a8e1e0..efebe96 100644 --- a/docs/reference/compatibility.md +++ b/docs/reference/compatibility.md @@ -31,7 +31,8 @@ changelog before you merge it. (`touchmark schema `): the schemas published on `master` describe the next release and reject fields they do not know. - **What touchmark leaves in repositories:** the marker in pull request bodies - (`touchmark:v1`), the commit trailers (`Touchmark-Hub`, `Touchmark-Stream`, + (`touchmark:v1`, an HTML comment, or a Markdown reference definition on Bitbucket + Cloud), the commit trailers (`Touchmark-Hub`, `Touchmark-Stream`, `Touchmark-Content`, `Touchmark-Hub-Commit`), the control lines of a pull request body, the marker of the plan comment, and the sync branch's name. Every release reads what an earlier one wrote. The marker's payload is frozen: a change to it is a `touchmark:v2` diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md index 1450c99..19c559e 100644 --- a/docs/reference/glossary.md +++ b/docs/reference/glossary.md @@ -38,7 +38,9 @@ history. `touchmark manifest` prints it. **marker** — the HTML comment at the end of a sync pull request's description: ``. It records the -fingerprint and what the pull request carries. +fingerprint and what the pull request carries. On Bitbucket Cloud, which shows HTML +comments as text, the same payload is a Markdown link reference definition: +`[touchmark]: # "touchmark:v1 hub=… fp=… stream=sync key=sha256:… data=…"`. **opt-in file** — `.engineering-assets.yml` at a target's root. Its presence is consent; it can add packs and `ignore` paths, and `enabled: false` in it opts the target out. diff --git a/docs/reference/hub.md b/docs/reference/hub.md index 1e56ae2..c6f7a02 100644 --- a/docs/reference/hub.md +++ b/docs/reference/hub.md @@ -104,11 +104,11 @@ In a hub pull request, `providers` and `ca_file` are read from the default branc | `security.private_targets_in_public_hub` | one of `skip`, `deliver` | What a public hub does with non-public targets: skip them, or deliver and let their names reach public CI logs. Default: skip. | | `providers` | list of objects | Platforms this hub delivers to. Targets refer to them by id. | | `providers[].id` | string, required | Name targets.yml uses for this provider, and the <ID> of its TOUCHMARK_<ID>_\* variables. | -| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, required | — | -| `providers[].url` | string | Web URL of the instance. Default for github: https://github.com; for gitlab: https://gitlab.com. Required for gitea and forgejo. A target targets.yml names by web URL belongs to the provider whose url it lies under. | +| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, required | github: github.com, GHE.com or GitHub Enterprise Server; gitlab; gitea; forgejo; bitbucket: Bitbucket Cloud (plan, distribute and doctor; setup does not set up a hub on it yet). | +| `providers[].url` | string | Web URL of the instance. Default for github: https://github.com; for gitlab: https://gitlab.com; for bitbucket: https://bitbucket.org, the only url it takes without api_url. Required for gitea and forgejo. A target targets.yml names by web URL belongs to the provider whose url it lies under. | | `providers[].api_url` | string | API URL, when it cannot be derived from url. | | `providers[].ca_file` | string | Hub file with extra CA certificates for this instance. | -| `providers[].writer` | string, up to 255 characters | The account that writes to targets, as the platform names it: on GitHub the App's bot, <slug>[bot]; on GitLab, Gitea and Forgejo the username of the service account or bot user. | +| `providers[].writer` | string, up to 255 characters | The account that writes to targets, as the platform names it: on GitHub the App's bot, <slug>[bot]; on GitLab, Gitea and Forgejo the username of the service account or bot user; on Bitbucket the bot account's UUID in braces, {…}. | | `providers[].known_authors` | list of strings, each up to 255 characters | Former writer accounts whose pull requests still count as this hub's own. | | `providers[].automation_accounts` | list of strings, each up to 255 characters | Accounts whose closing of a pull request is automatic, not a team's decision. | | `providers[].sign` | one of `auto`, `always` | auto: sign where it costs nothing and where target rules require it; always: sign every commit. Default: auto. | @@ -127,7 +127,7 @@ In a hub pull request, `providers` and `ca_file` are read from the default branc | `packs..formerly` | list of strings, each up to 64 characters | Former names of this pack. Their history counts as this pack's own. | | `writer` | string, up to 255 characters | Single-provider shorthand: the writer account, as the platform names it (on GitHub the App's bot, <slug>[bot]). Without platform, the provider's type and URL come from the CI environment, or locally from the hub's origin remote. Not allowed with providers. | | `sign` | one of `auto`, `always` | Single-provider shorthand: commit signing. Not allowed with providers. | -| `platform` | one of `github`, `gitlab`, `gitea`, `forgejo` | Single-provider shorthand: the provider type, which is also its id. Not allowed with providers. | +| `platform` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket` | Single-provider shorthand: the provider type, which is also its id. Not allowed with providers. | | `base_url` | string | Single-provider shorthand: the provider URL, for self-managed instances. Needs platform; not allowed with providers. | diff --git a/docs/reference/operations.md b/docs/reference/operations.md index 5738b7d..c6524ba 100644 --- a/docs/reference/operations.md +++ b/docs/reference/operations.md @@ -21,7 +21,11 @@ A `target` is `[PROVIDER:]PATH`, the provider being the `id` of an entry of `pro writes the path alone. Every entry limits itself, so a forgotten one does nothing: a branch head that moved, a -decline already revoked, a date that passed. `check` warns about entries past their date. +decline already revoked, a date that passed. The exception is a `forget_declines` entry +for a Bitbucket Cloud target: a declined pull request cannot be edited there, so it is +never marked revoked, and the entry acts while it is present (see +[Memory of declined pull requests](../concepts/memory.md#on-bitbucket-cloud)). `check` +warns about entries past their date. `plan` on a hub pull request shows each entry's effect: `applies`, `none`, `expired` or `unknown`. In CI the matching flags of `distribute` exit 2. See [One-off operations](../guide/operations.md). @@ -36,7 +40,7 @@ decline already revoked, a date that passed. `check` warns about entries past th | `recreate` | list of objects | Rebuild a paused sync branch from scratch. Commits others added to it are dropped. An entry acts only while the branch head is the given commit. | | `recreate[].target` | string, up to 512 characters, required | One target repository: <provider>:<path> or <path>, as touchmark prints it. | | `recreate[].head` | string, required | Full commit id of the sync branch head to replace: 40 or 64 lowercase hex digits. | -| `forget_declines` | list of objects | Propose again the content of a pull request the target closed without merging. touchmark marks that pull request revoked, so the entry acts once. | +| `forget_declines` | list of objects | Propose again the content of a pull request the target closed without merging. touchmark marks that pull request revoked, so the entry acts once; on Bitbucket Cloud, where a declined pull request cannot be edited, the entry acts while it is present. | | `forget_declines[].target` | string, up to 512 characters, required | One target repository: <provider>:<path> or <path>, as touchmark prints it. | | `forget_declines[].pr` | integer ≥ 1, required | Number of the declined pull request. | | `allow_mass_close` | object | Lift the guard against closing many pull requests in one run, up to max closes, until the date. | diff --git a/docs/reference/output.md b/docs/reference/output.md index 43e7b25..0920bb5 100644 --- a/docs/reference/output.md +++ b/docs/reference/output.md @@ -87,7 +87,7 @@ and the branch head before and after. | `assume_opt_in` | boolean | plan --assume-opt-in: for this report only, every target without an opt-in file, or whose opt-in file is not a regular file or is too large, counts as opted in; one whose opt-in file says enabled: false stays opted out. | | `providers` | list of objects, required | The hub's providers, in hub.yml order. | | `providers[].id` | string, required | — | -| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, required | — | +| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, required | — | | `providers[].host` | string, not empty, required | The provider's host, with a port when it has one. | | `providers[].reader` | string | Login of the read account. | | `providers[].writer` | string | Login of the write account. | @@ -199,7 +199,7 @@ hub, for each provider's write identity, and for each target. See | `hub_checks[].detail` | string | What the check found; absent for a target the report does not name. | | `providers` | list of objects, required | The hub's providers, in hub.yml order. | | `providers[].id` | string, required | — | -| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, required | — | +| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, required | — | | `providers[].host` | string, not empty, required | The provider's host, with a port when it has one. | | `providers[].writer` | string | Login of the writer hub.yml names. | | `providers[].self` | string | Login the write credential acts as. | diff --git a/docs/reference/targets.md b/docs/reference/targets.md index 4d780cf..50216ef 100644 --- a/docs/reference/targets.md +++ b/docs/reference/targets.md @@ -190,8 +190,8 @@ either way. | `targets[].provider` | string | Id of a provider in hub.yml. | | `targets[].packs` | list of strings, each up to 64 characters | — | | `targets[].opt_in` | one of `required`, `assumed` | required (the default): a repository the entry selects gets nothing until it adds the opt-in file. assumed: it counts as opted in without one, with the packs targets.yml gives it, until its opt-in file says enabled: false. One entry with assumed is enough to subscribe a repository. | -| `targets[].org` | string, up to 512 characters | Every repository in a GitHub organisation or a GitLab, Gitea or Forgejo namespace (a synonym of group), named by its path or its web URL. An entry has exactly one of `repo`, `org` and `group`. | -| `targets[].topics` | list of strings | Only repositories with all of these topics. Only with `org` or `group`. | +| `targets[].org` | string, up to 512 characters | Every repository in a GitHub organisation, a GitLab, Gitea or Forgejo namespace or a Bitbucket workspace (a synonym of group), named by its path or its web URL. An entry has exactly one of `repo`, `org` and `group`. | +| `targets[].topics` | list of strings | Only repositories with all of these topics. Bitbucket has no topics: check refuses them on its entries; use match. Only with `org` or `group`. | | `targets[].subgroups` | boolean | Include repositories in nested groups. Default: true. Only with `org` or `group`. | | `targets[].forks` | boolean | Include forks. Default: false. Only with `org` or `group`. | | `targets[].match` | list of strings, each up to 512 characters | Only repositories whose full path, without the provider, matches at least one of these patterns: \* within one segment, \*\* any number of segments, ? one character, compared ignoring case, such as acme/svc-\* or platform/\*\*/api. Quote a pattern that starts with \*, which YAML reads as an alias. Only with `org` or `group`. | diff --git a/internal/cli/distribute_deps.go b/internal/cli/distribute_deps.go index 7ec9c07..5f30a3e 100644 --- a/internal/cli/distribute_deps.go +++ b/internal/cli/distribute_deps.go @@ -32,8 +32,8 @@ import ( type distributeDriver func(config.ResolvedProvider, auth.Credential, *httpx.Client) (platform.Writer, error) // The write drivers of distribute by provider type: Gitea and Forgejo, -// GitLab and GitHub (drivers.go). Tests install the fake -// platform here; tests that do must not run in parallel. +// GitLab, GitHub and Bitbucket Cloud (drivers.go). Tests install the fake platform here; tests that do must +// not run in parallel. var distributeDrivers = map[string]distributeDriver{} // writeSecrets are what distribute reads from the environment for each diff --git a/internal/cli/drivers.go b/internal/cli/drivers.go index 2584f24..a2bc36e 100644 --- a/internal/cli/drivers.go +++ b/internal/cli/drivers.go @@ -1,15 +1,16 @@ package cli import ( + "github.com/bedrock-python/touchmark/internal/platform/bitbucket" "github.com/bedrock-python/touchmark/internal/platform/gitea" "github.com/bedrock-python/touchmark/internal/platform/github" "github.com/bedrock-python/touchmark/internal/platform/gitlab" ) // init registers the platform drivers this build has: Gitea and Forgejo, one -// driver for both, GitLab and GitHub (github.com, GHE.com and GitHub -// Enterprise Server). Tests that install their own drivers replace the maps -// whole and restore them afterwards. +// driver for both, GitLab, GitHub (github.com, GHE.com and GitHub +// Enterprise Server) and Bitbucket Cloud. Tests that install their own +// drivers replace the maps whole and restore them afterwards. func init() { for _, typ := range []string{"gitea", "forgejo"} { planDrivers[typ] = gitea.NewReader @@ -19,4 +20,6 @@ func init() { distributeDrivers["gitlab"] = gitlab.NewWriter planDrivers["github"] = github.NewReader distributeDrivers["github"] = github.NewWriter + planDrivers["bitbucket"] = bitbucket.NewReader + distributeDrivers["bitbucket"] = bitbucket.NewWriter } diff --git a/internal/cli/drivers_test.go b/internal/cli/drivers_test.go index ee59b40..7c54133 100644 --- a/internal/cli/drivers_test.go +++ b/internal/cli/drivers_test.go @@ -3,10 +3,10 @@ package cli import "testing" // TestDriversRegistered: this build plans and distributes on every -// platform of v1: Gitea and Forgejo, one driver for both, -// GitLab and GitHub. +// platform it supports: Gitea and Forgejo, one driver for both, GitLab, +// GitHub and Bitbucket Cloud. func TestDriversRegistered(t *testing.T) { - for _, typ := range []string{"gitea", "forgejo", "gitlab", "github"} { + for _, typ := range []string{"gitea", "forgejo", "gitlab", "github", "bitbucket"} { if planDrivers[typ] == nil || distributeDrivers[typ] == nil { t.Errorf("no %s driver: plan %v, distribute %v", typ, planDrivers[typ] != nil, distributeDrivers[typ] != nil) } diff --git a/internal/cli/plan.go b/internal/cli/plan.go index bc6d8b4..42db517 100644 --- a/internal/cli/plan.go +++ b/internal/cli/plan.go @@ -33,10 +33,10 @@ import ( type planDriver func(config.ResolvedProvider, auth.Credential, *httpx.Client) (platform.Reader, error) // The drivers of plan by provider type, and a snapshot source that replaces -// plan's own. This build has the Gitea and Forgejo, GitLab and GitHub -// drivers (drivers.go). plan's own source is distribute's: a private git -// repository per target (sharedDeps), so that plan and distribute --dry-run -// decide alike. Tests install the fake platform here; a planSnapshots that +// plan's own. This build has the Gitea and Forgejo, GitLab, GitHub and +// Bitbucket Cloud drivers (drivers.go). plan's own source is distribute's: +// a private git repository per target (sharedDeps), so that plan and +// distribute --dry-run decide alike. Tests install the fake platform here; a planSnapshots that // is only a snapshot.Source (the fake in memory mode) keeps the in-memory // plan (distribute.Plan). Tests that install them must not run in // parallel. @@ -50,9 +50,9 @@ var ( var gitVersion = func(ctx context.Context) ([3]int, error) { return gitx.New("").Version(ctx) } // basicUsers are the users a token goes with in HTTP Basic headers (GitHub, -// GitLab and Gitea git over HTTPS, GitLab's job token): the base64 forms of -// "user:token" are masked too. -var basicUsers = []string{"x-access-token", "oauth2", hubch.GitLabUser} +// GitLab, Gitea and Bitbucket git over HTTPS, GitLab's job token): the +// base64 forms of "user:token" are masked too. +var basicUsers = []string{"x-access-token", "oauth2", hubch.GitLabUser, "x-bitbucket-api-token-auth"} // Suffixes of the credential variables after a provider's prefix. var ( diff --git a/internal/cli/setup.go b/internal/cli/setup.go index 0fd2776..da75938 100644 --- a/internal/cli/setup.go +++ b/internal/cli/setup.go @@ -97,6 +97,9 @@ func runSetup(ctx context.Context, e *env, o *options, s *setupOptions) (err err switch platform { case "gitea", "forgejo": return giteaRefusal(platform) + case "bitbucket": + return configErrorf("setup does not set up bitbucket yet: a hub on Bitbucket Pipelines comes in a later release; " + + "a hub on GitHub or GitLab delivers to Bitbucket Cloud as one of its providers") case "github", "gitlab": default: return usageErrorf("unknown platform %q: setup sets up github or gitlab", platform) diff --git a/internal/cli/setup_test.go b/internal/cli/setup_test.go index ea4cde3..da1a77c 100644 --- a/internal/cli/setup_test.go +++ b/internal/cli/setup_test.go @@ -45,7 +45,8 @@ func TestSetupRefusals(t *testing.T) { want string }{ {"no-platform", nil, []string{"setup"}, "missing the platform"}, - {"unknown", nil, []string{"setup", "bitbucket", "--hub", h.dir}, `unknown platform "bitbucket"`}, + {"unknown", nil, []string{"setup", "azure-devops", "--hub", h.dir}, `unknown platform "azure-devops"`}, + {"bitbucket", nil, []string{"setup", "bitbucket", "--hub", h.dir}, "setup does not set up bitbucket yet"}, {"gitea", nil, []string{"setup", "gitea", "--hub", h.dir}, "every branch the secrets"}, {"forgejo", nil, []string{"setup", "forgejo"}, "setup does not set up forgejo"}, {"flags-first", nil, []string{"setup", "--hub", h.dir, "gitea", "--dry-run"}, "setup does not set up gitea"}, diff --git a/internal/config/check.go b/internal/config/check.go index 870346a..6503d15 100644 --- a/internal/config/check.go +++ b/internal/config/check.go @@ -46,6 +46,7 @@ func check(hub *Hub, targets *Targets, known map[string]bool) ([]Warning, []erro c.providers() c.exclude() c.match() + c.topics() c.optIn() return c.warns, c.errs } @@ -247,23 +248,50 @@ func (c *checker) match() { } // flatType returns the type of the provider with id when its platform has no -// nested namespaces (github, gitea, forgejo): a repository path there is -// owner/name. It returns "" for any other type, and for a provider hub.yml -// does not list (the implicit one, whose type the CI tells). +// nested namespaces (github, gitea, forgejo, bitbucket): a repository path +// there is owner/name. It returns "" for any other type, and for a provider +// hub.yml does not list (the implicit one, whose type the CI tells). func (c *checker) flatType(id string) string { + switch typ := c.providerType(id); typ { + case "github", "gitea", "forgejo", "bitbucket": + return typ + } + return "" +} + +// providerType returns the type of the provider with id, "" for a provider +// hub.yml does not list. +func (c *checker) providerType(id string) string { for _, p := range c.hub.Providers { - if p.ID != id { - continue - } - switch p.Type { - case "github", "gitea", "forgejo": + if p.ID == id { return p.Type } - return "" } return "" } +// topics refuses topics on an entry whose provider is Bitbucket Cloud, +// whose repositories have none: the entry would select nothing, and the +// driver refuses the selector. +func (c *checker) topics() { + res := resolver{hub: c.hub, targets: c.targets} + for i := range c.targets.Targets { + e := &c.targets.Targets[i] + if len(e.Topics) == 0 || entryKind(e) == "repo" { + continue + } + ns, err := entryRef(e) + if err != nil { + continue + } + id := res.provider(firstNonEmpty(e.Provider, ns.Provider)) + if c.providerType(id) == "bitbucket" { + c.errorf(TargetsFile, "targets[%d].topics: provider %s is Bitbucket, whose repositories have no topics; "+ + "select them with match: (paths like %s/svc-*) or list them with repo:", i, id, ns.Path) + } + } +} + // optIn warns about a repo entry whose opt_in: required says nothing: an // org or group entry with opt_in: assumed may select the same repository, // and one entry with assumed is enough to subscribe it (Targets.Assumed). diff --git a/internal/config/check_test.go b/internal/config/check_test.go index af6f795..4ba9f75 100644 --- a/internal/config/check_test.go +++ b/internal/config/check_test.go @@ -160,6 +160,35 @@ func TestCheck(t *testing.T) { `targets.yml: exclude[0]: "acme"`, }, }, + { + name: "topics on bitbucket", + hub: &Hub{ID: "acme-eng", Providers: []Provider{ + {ID: "bb", Type: "bitbucket"}, {ID: "gh", Type: "github"}, + }}, + targets: &Targets{ + Defaults: Defaults{Provider: "bb"}, + Targets: []Entry{ + {Org: "acme", Topics: []string{"python"}}, + {Org: "gh:acme", Topics: []string{"python"}}, + {Org: "acme", Provider: "gh", Topics: []string{"python"}}, + {Org: "bb:acme", Match: []string{"acme/svc-*"}}, + {Group: "acme", Provider: "bb", Topics: []string{"go"}}, + }, + }, + errors: []string{ + "targets.yml: targets[0].topics: provider bb is Bitbucket, whose repositories have no topics; select them with match: (paths like acme/svc-*) or list them with repo:", + "targets.yml: targets[4].topics: provider bb is Bitbucket", + }, + }, + { + name: "bitbucket paths are workspace/repository", + hub: &Hub{ID: "acme-eng", Providers: []Provider{{ID: "bb", Type: "bitbucket"}}}, + targets: &Targets{ + Targets: []Entry{{Org: "acme"}}, + Exclude: []string{"acme/legacy/old"}, + }, + warnings: []string{"targets.yml: exclude[0]: acme/legacy/old has 3 path segments, and a bitbucket repository path is owner/name, so it excludes nothing"}, + }, { name: "listed and excluded", hub: selectHub(), diff --git a/internal/config/config.go b/internal/config/config.go index bc35256..a848a64 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -97,7 +97,7 @@ type Security struct { type Provider struct { ID string `yaml:"id"` - Type string `yaml:"type"` // github | gitlab | gitea | forgejo + Type string `yaml:"type"` // github | gitlab | gitea | forgejo | bitbucket URL string `yaml:"url"` APIURL string `yaml:"api_url"` CAFile string `yaml:"ca_file"` diff --git a/internal/config/hub.go b/internal/config/hub.go index ec29194..5933f90 100644 --- a/internal/config/hub.go +++ b/internal/config/hub.go @@ -26,12 +26,12 @@ const ( // Enumerations of hub.yml. var ( - providerTypes = []string{"github", "gitlab", "gitea", "forgejo"} + providerTypes = []string{"github", "gitlab", "gitea", "forgejo", "bitbucket"} linkHubModes = []string{"auto", "always", "never"} isolationModes = []string{"platform", "external", "none"} privateTargets = []string{"skip", "deliver"} signModes = []string{"auto", "always"} - defaultURLs = map[string]string{"github": "https://github.com", "gitlab": "https://gitlab.com"} + defaultURLs = map[string]string{"github": "https://github.com", "gitlab": "https://gitlab.com", "bitbucket": bitbucketURL} shorthandFields = []string{"writer", "sign", "platform", "base_url"} ) @@ -253,6 +253,9 @@ func checkProvider(pr *Provider, f providerFields, p *problems) { } else if pr.Type == "gitea" || pr.Type == "forgejo" { p.errorf(f.name("url"), "required for %s", pr.Type) } + if pr.Type == "bitbucket" && f.has("url") && checkURL(pr.URL) == nil && !f.has("api_url") && !isBitbucketCloud(pr.URL) { + p.errorf(f.name("url"), "%s", bitbucketURLError) + } if f.has("api_url") { if err := checkURL(pr.APIURL); err != nil { p.errorf(f.name("api_url"), "%v", err) diff --git a/internal/config/providers.go b/internal/config/providers.go index 105d3c2..41151dc 100644 --- a/internal/config/providers.go +++ b/internal/config/providers.go @@ -17,6 +17,26 @@ import ( // the prefix of the short names a hub with one provider accepts. const envPrefixBase = "TOUCHMARK_" +// Bitbucket Cloud: the web URL, which is the default url of a provider of +// type bitbucket, and its REST base. Bitbucket Data Center has another API +// and is not a bitbucket provider. +const ( + bitbucketURL = "https://bitbucket.org" + bitbucketAPIURL = "https://api.bitbucket.org/2.0" +) + +// bitbucketURLError is the complaint about a bitbucket provider at another +// url without an api_url. +const bitbucketURLError = "a provider of type bitbucket is Bitbucket Cloud, at https://bitbucket.org (leave url out); " + + "Bitbucket Data Center is not supported yet. Elsewhere (a test server) set api_url too" + +// isBitbucketCloud reports whether raw, a checked URL, is Bitbucket Cloud's +// web URL: https://bitbucket.org, with no path. +func isBitbucketCloud(raw string) bool { + u, err := url.Parse(strings.TrimRight(raw, "/")) + return err == nil && strings.EqualFold(u.Scheme, "https") && strings.EqualFold(u.Host, "bitbucket.org") && u.Path == "" +} + // ResolvedProvider is a provider from hub.yml (or the implicit one) with // everything a driver needs besides credentials. type ResolvedProvider struct { @@ -25,7 +45,8 @@ type ResolvedProvider struct { Host string // APIURL is the REST base: https://api.github.com, https://api. // for *.ghe.com, /api/v3 for GitHub Enterprise Server, - // /api/v4 for GitLab, /api/v1 for Gitea and Forgejo; + // /api/v4 for GitLab, /api/v1 for Gitea and Forgejo, + // https://api.bitbucket.org/2.0 for Bitbucket Cloud; // Provider.APIURL when set. On GitHub Actions for the hub's own host, // GITHUB_API_URL wins over the derived URL, not over Provider.APIURL: // an explicit api_url means the same in CI and locally. @@ -72,7 +93,8 @@ type ResolvedProvider struct { // // Every provider is checked again, with ParseHub's defaults, so a hub built // in code resolves like a parsed one: a valid id, no duplicate ids, a known -// type, the public URL of github and gitlab when URL is empty, https URLs +// type, the public URL of github, gitlab and bitbucket when URL is empty, +// an api_url for a bitbucket provider anywhere but Bitbucket Cloud, https URLs // (http only for localhost), sign auto. A nil hub is a legacy hub. Error // messages never quote environment values. A nil getenv reads the process // environment. @@ -254,6 +276,9 @@ func resolveProvider(p Provider, env ciEnv) (ResolvedProvider, error) { return ResolvedProvider{}, fmt.Errorf("api_url: %w", err) } } + if p.Type == "bitbucket" && p.APIURL == "" && !isBitbucketCloud(p.URL) { + return ResolvedProvider{}, fmt.Errorf("url: %s", bitbucketURLError) + } setDefault(&p.Sign, defaultSign) u, err := url.Parse(p.URL) if err != nil { @@ -299,6 +324,9 @@ func derivedAPIURL(typ, webURL, hostname, host string) string { return webURL + "/api/v3" case "gitlab": return webURL + "/api/v4" + case "bitbucket": + // resolveProvider asks for api_url anywhere else. + return bitbucketAPIURL } return webURL + "/api/v1" } diff --git a/internal/config/providers_test.go b/internal/config/providers_test.go index 7ceaecf..1d5298b 100644 --- a/internal/config/providers_test.go +++ b/internal/config/providers_test.go @@ -37,6 +37,9 @@ func TestResolveProviders(t *testing.T) { {ID: "tea", Type: "gitea", URL: "https://gitea.example.com"}, {ID: "cb", Type: "forgejo", URL: "https://codeberg.org"}, {ID: "local", Type: "gitea", URL: "http://localhost:3000"}, + {ID: "bb", Type: "bitbucket"}, + {ID: "bb-url", Type: "bitbucket", URL: "https://Bitbucket.org/"}, + {ID: "bb-test", Type: "bitbucket", URL: "http://localhost:8080", APIURL: "http://localhost:8080/2.0"}, }} want := map[string]resolvedURLs{ "gh": {"https://github.com", "github.com", "https://api.github.com", "https://api.github.com/graphql", "TOUCHMARK_GH_"}, @@ -50,6 +53,9 @@ func TestResolveProviders(t *testing.T) { "tea": {"https://gitea.example.com", "gitea.example.com", "https://gitea.example.com/api/v1", "", "TOUCHMARK_TEA_"}, "cb": {"https://codeberg.org", "codeberg.org", "https://codeberg.org/api/v1", "", "TOUCHMARK_CB_"}, "local": {"http://localhost:3000", "localhost:3000", "http://localhost:3000/api/v1", "", "TOUCHMARK_LOCAL_"}, + "bb": {"https://bitbucket.org", "bitbucket.org", "https://api.bitbucket.org/2.0", "", "TOUCHMARK_BB_"}, + "bb-url": {"https://Bitbucket.org", "bitbucket.org", "https://api.bitbucket.org/2.0", "", "TOUCHMARK_BB_URL_"}, + "bb-test": {"http://localhost:8080", "localhost:8080", "http://localhost:8080/2.0", "", "TOUCHMARK_BB_TEST_"}, } // The CI environment does not matter for declared providers, except // GITHUB_API_URL on the hub's own host (below). @@ -292,8 +298,18 @@ func TestResolveProvidersErrors(t *testing.T) { }, { name: "bad type", - hub: &Hub{Providers: []Provider{{ID: "bb", Type: "bitbucket"}}}, - want: `hub.yml: providers[0]: type: "bitbucket" must be one of github, gitlab, gitea, forgejo`, + hub: &Hub{Providers: []Provider{{ID: "ado", Type: "azure-devops"}}}, + want: `hub.yml: providers[0]: type: "azure-devops" must be one of github, gitlab, gitea, forgejo, bitbucket`, + }, + { + name: "bitbucket elsewhere without api_url", + hub: &Hub{Providers: []Provider{{ID: "bb", Type: "bitbucket", URL: "https://bitbucket.example.com"}}}, + want: "hub.yml: providers[0]: url: a provider of type bitbucket is Bitbucket Cloud, at https://bitbucket.org (leave url out); Bitbucket Data Center is not supported yet", + }, + { + name: "bitbucket under a path", + hub: &Hub{Providers: []Provider{{ID: "bb", Type: "bitbucket", URL: "https://bitbucket.org/acme"}}}, + want: "hub.yml: providers[0]: url: a provider of type bitbucket is Bitbucket Cloud", }, { name: "gitea without url", diff --git a/internal/config/testdata/hub/invalid/provider-bad-type.yml b/internal/config/testdata/hub/invalid/provider-bad-type.yml index 2b7935e..b3d995b 100644 --- a/internal/config/testdata/hub/invalid/provider-bad-type.yml +++ b/internal/config/testdata/hub/invalid/provider-bad-type.yml @@ -1,6 +1,6 @@ -# error: providers[0].type: "bitbucket" must be one of +# error: providers[0].type: "azure-devops" must be one of version: 1 id: acme-eng providers: - - id: bb - type: bitbucket + - id: ado + type: azure-devops diff --git a/internal/config/testdata/hub/invalid/provider-bitbucket-elsewhere.yml b/internal/config/testdata/hub/invalid/provider-bitbucket-elsewhere.yml new file mode 100644 index 0000000..3986336 --- /dev/null +++ b/internal/config/testdata/hub/invalid/provider-bitbucket-elsewhere.yml @@ -0,0 +1,7 @@ +# error: providers[0].url: a provider of type bitbucket is Bitbucket Cloud +version: 1 +id: acme-eng +providers: + - id: bb + type: bitbucket + url: https://bitbucket.example.com diff --git a/internal/config/testdata/hub/invalid/shorthand-bad-platform.yml b/internal/config/testdata/hub/invalid/shorthand-bad-platform.yml index 3a62a01..ea220d9 100644 --- a/internal/config/testdata/hub/invalid/shorthand-bad-platform.yml +++ b/internal/config/testdata/hub/invalid/shorthand-bad-platform.yml @@ -1,4 +1,4 @@ -# error: platform: "bitbucket" must be one of +# error: platform: "azure-devops" must be one of version: 1 id: acme-eng -platform: bitbucket +platform: azure-devops diff --git a/internal/config/testdata/hub/invalid/shorthand-bitbucket-elsewhere.yml b/internal/config/testdata/hub/invalid/shorthand-bitbucket-elsewhere.yml new file mode 100644 index 0000000..9a6f11f --- /dev/null +++ b/internal/config/testdata/hub/invalid/shorthand-bitbucket-elsewhere.yml @@ -0,0 +1,5 @@ +# error: base_url: a provider of type bitbucket is Bitbucket Cloud +version: 1 +id: acme-eng +platform: bitbucket +base_url: https://bitbucket.example.com diff --git a/internal/config/testdata/hub/valid/bitbucket.yml b/internal/config/testdata/hub/valid/bitbucket.yml new file mode 100644 index 0000000..0c9e449 --- /dev/null +++ b/internal/config/testdata/hub/valid/bitbucket.yml @@ -0,0 +1,14 @@ +version: 1 +id: acme-eng +providers: + - id: bb + type: bitbucket + writer: "{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d}" + known_authors: ["{0b7e5a2c-9d4f-4e1b-8a3c-6f5d2e1c0b9a}"] + - id: bb-cloud + type: bitbucket + url: https://bitbucket.org/ + - id: bb-test + type: bitbucket + url: http://localhost:8080 + api_url: http://localhost:8080/2.0 diff --git a/internal/config/testdata/hub/valid/shorthand-bitbucket.yml b/internal/config/testdata/hub/valid/shorthand-bitbucket.yml new file mode 100644 index 0000000..93905d4 --- /dev/null +++ b/internal/config/testdata/hub/valid/shorthand-bitbucket.yml @@ -0,0 +1,4 @@ +version: 1 +id: acme-eng +platform: bitbucket +writer: "{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d}" diff --git a/internal/config/urls.go b/internal/config/urls.go index 9a9804f..c810f57 100644 --- a/internal/config/urls.go +++ b/internal/config/urls.go @@ -149,9 +149,10 @@ func (t *Targets) HasURLs() bool { // entry names its provider with provider:, which then must be one of them. // An implicit provider (the CI's own platform, for a hub.yml without // providers) is the hub's only one: its targets are written as bare paths. -// On GitHub, Gitea and Forgejo, which have no nested namespaces, a -// repository URL must name owner/name and an organisation URL one -// namespace; on GitLab a URL with a /-/ segment points inside a project and +// On GitHub, Gitea, Forgejo and Bitbucket, which have no nested +// namespaces, a repository URL must name owner/name (workspace/repository on +// Bitbucket, as in https://bitbucket.org/acme/billing) and an organisation +// URL one namespace; on GitLab a URL with a /-/ segment points inside a project and // is refused. // // The result is a copy: targets is left as it is, and returned as it is @@ -305,14 +306,14 @@ func (r urlResolver) list(indexes []int) string { } // checkURLShape checks the path segments a URL of kind k names on a -// platform of type typ: GitHub, Gitea and Forgejo have no nested +// platform of type typ: GitHub, Gitea, Forgejo and Bitbucket have no nested // namespaces, so an exclude URL of more than owner/name, without a "**" // that could span nothing, points inside a repository (/tree/main) and // would exclude nothing; a GitLab URL with a /-/ segment points inside a // project (a file, a merge request), not at it. func checkURLShape(typ string, segs []string, k urlKind) error { switch typ { - case "github", "gitea", "forgejo": + case "github", "gitea", "forgejo", "bitbucket": switch { case k == urlRepo && len(segs) != 2: return fmt.Errorf("a %s repository URL names owner/name, like https://example.com/acme/billing; this one has %d path segments after the provider's url", typ, len(segs)) diff --git a/internal/config/urls_test.go b/internal/config/urls_test.go index ef500d6..db0fab9 100644 --- a/internal/config/urls_test.go +++ b/internal/config/urls_test.go @@ -337,3 +337,42 @@ func TestRefRefusesURL(t *testing.T) { t.Errorf("Selectors of a URL: %v", err) } } + +// TestResolveURLsBitbucket: a Bitbucket Cloud provider takes the web URLs of +// bitbucket.org, workspace/repository and workspace only. +func TestResolveURLsBitbucket(t *testing.T) { + provs := resolvedProviders(t, `version: 1 +id: acme-eng +providers: + - id: bb + type: bitbucket + - id: gh + type: github +`) + for _, tc := range []struct { + entry Entry + want string // the resolved value, or a substring of the error after "!" + }{ + {Entry{Repo: "https://bitbucket.org/acme/billing"}, "bb:acme/billing"}, + {Entry{Repo: "https://bitbucket.org/acme/billing.git"}, "bb:acme/billing"}, + {Entry{Org: "https://bitbucket.org/acme/"}, "bb:acme"}, + {Entry{Repo: "https://bitbucket.org/acme/billing/src/main/README.md"}, "!a bitbucket repository URL names owner/name"}, + {Entry{Org: "https://bitbucket.org/acme/billing"}, "!a bitbucket organisation URL names one owner"}, + {Entry{Repo: "https://api.bitbucket.org/acme/billing"}, "!is not under the url of any provider"}, + } { + got, err := ResolveURLs(&Targets{Targets: []Entry{tc.entry}}, provs) + if want, ok := strings.CutPrefix(tc.want, "!"); ok { + if err == nil || !strings.Contains(err.Error(), want) { + t.Errorf("%+v: error %v, want one with %q", tc.entry, err, want) + } + continue + } + if err != nil { + t.Errorf("%+v: %v", tc.entry, err) + continue + } + if v := selectorValue(&got.Targets[0]); v != tc.want { + t.Errorf("%+v: %q, want %q", tc.entry, v, tc.want) + } + } +} diff --git a/internal/decide/memory.go b/internal/decide/memory.go index 560f87d..5ac5649 100644 --- a/internal/decide/memory.go +++ b/internal/decide/memory.go @@ -76,6 +76,29 @@ type MemoryConfig struct { Automation map[string]bool // CloserKnown is Caps.CloserKnown. CloserKnown bool + // ClosedImmutable is Caps.ClosedImmutable: a closed pull request can + // never be edited (Bitbucket Cloud's declined pull requests), so + // memory asks for no write to one. Declines are then read as follows + // (see BuildMemory): + // - a decline whose marker holds optin, the hash of the opt-in file + // touchmark wrote into the marker while the pull request was open + // (on every create and content edit, and on any edit that finds + // it stale), counts as acked with that optin: in force while the + // opt-in file hashes the same, lapsed once it changes, as after an + // ack written elsewhere; + // - a decline whose marker holds no optin (a marker touchmark did + // not write in full) is in force whatever the opt-in file says, + // and is listed in Memory.Unanchored so that the report says how to + // lift it: no state of the file was recorded to compare with, and a + // decline that lapsed silently would propose the content again; + // - a pull request a forget_declines entry names is no memory while + // the entry is present (Memory.Forgotten): its revocation cannot be + // written into the pull request, so the entry does not act once; + // - ticked repropose controls are ignored: such a platform shows no + // tick box (Caps.BodyControls), and no ack adds one. + // + // ToAck and ToRevoke are always empty then. + ClosedImmutable bool // Window is how many closed own PRs are considered, newest first (50; // a value that is not positive means 50). Window int @@ -129,8 +152,12 @@ type Decline struct { Key string Changes []marker.Change Complete bool // marker.Data.ChangesComplete + // Acked is set when the opt-in state the decline holds under is known: + // an ack was written into its marker, or, on a platform whose closed + // pull requests are immutable (MemoryConfig.ClosedImmutable), the + // marker recorded optin while the pull request was open. Acked bool - OptIn string // marker.Data.OptIn at ack time ("" before ack) + OptIn string // marker.Data.OptIn of an acked decline ("" otherwise) ClosedAt time.Time } @@ -157,7 +184,7 @@ type MemoryInput struct { // Forget lists PR numbers operations.yml forget_declines names. Forget []int64 // Repropose lists declined PRs whose "Propose this content again" - // checkbox is ticked. + // checkbox is ticked (ignored with Config.ClosedImmutable). Repropose map[int64]bool // Now is the run's clock. What memory holds does not depend on it: // Cooldown takes the time it compares with. @@ -193,8 +220,18 @@ type Memory struct { // repropose ticked). Auto-closes named there are revoked too, which // ends their cooldown. A PR in ToRevoke is already out of every other // list, so a run that writes the revocation decides as the next run - // will. + // will. Always empty with Config.ClosedImmutable. ToRevoke []int64 + // Forgotten are, with Config.ClosedImmutable, the closes a + // forget_declines entry names: no memory while the entry is present, + // and out of every other list (an auto-close among them ends its + // cooldown). Nothing is written to them. + Forgotten []int64 + // Unanchored are, with Config.ClosedImmutable, the declines in force + // whose marker records no opt-in state (see MemoryConfig): they hold + // until a forget_declines entry names them or a path of theirs becomes + // local or ignored, and the report says so. + Unanchored []int64 // Auto are auto-closes among the window, newest first. Revoked ones // (and those in ToRevoke) are left out: a revoked PR is not memory. Auto []AutoClose @@ -211,11 +248,16 @@ type Memory struct { // - named by Forget, or its repropose checkbox ticked: ToRevoke, and no // memory from this run on (revoking a lapsed decline makes the revocation // permanent, even if the opt-in file later returns to the acked state); +// with Config.ClosedImmutable, named by Forget: Forgotten, and no memory +// while the entry is present (a ticked checkbox counts for nothing +// there); // - CloseAuto: an entry of Auto; it is a decline as well when its marker // is acked or it is the third or later of a run (see Memory.Declines); // - declines are then in force (Declines, and ToAck while unacked) or // Lapsed: acked with an optin other than OptIn, or a path of their -// Changes LocalOrIgnored. +// Changes LocalOrIgnored. With Config.ClosedImmutable a decline whose +// marker holds optin counts as acked with it, and one in force without +// it is Unanchored instead of ToAck. // // A run of an auto-close is itself and the consecutive older entries of // Auto with the same key: an auto-close with another key ends it, while a @@ -233,6 +275,7 @@ func BuildMemory(in MemoryInput) Memory { for _, n := range in.Forget { forget[n] = true } + immutable := in.Config.ClosedImmutable var m Memory // kept are the closes that carry memory, in window order, with whether // the class was CloseAuto. @@ -255,7 +298,10 @@ func BuildMemory(in MemoryInput) Memory { switch { case class == CloseSelf, o.Marker.Data.Revoked: continue - case forget[n] || in.Repropose[n]: + case immutable && forget[n]: + m.Forgotten = append(m.Forgotten, n) + continue + case !immutable && (forget[n] || in.Repropose[n]): m.ToRevoke = append(m.ToRevoke, n) continue } @@ -282,10 +328,13 @@ func BuildMemory(in MemoryInput) Memory { Key: c.o.Marker.Key, Changes: slices.Clone(data.Changes), Complete: data.ChangesComplete, - Acked: data.Ack, + // On a platform whose closed pull requests are immutable, the + // optin the open pull request's marker held stands for the ack + // no one can write. + Acked: data.Ack || (immutable && data.OptIn != ""), ClosedAt: c.o.PR.ClosedAt, } - if data.Ack { + if d.Acked { d.OptIn = data.OptIn } if memLapsed(d, in) { @@ -293,7 +342,11 @@ func BuildMemory(in MemoryInput) Memory { continue } m.Declines = append(m.Declines, d) - if !d.Acked { + switch { + case d.Acked: + case immutable: + m.Unanchored = append(m.Unanchored, n) + default: m.ToAck = append(m.ToAck, n) } } diff --git a/internal/decide/memory_immutable_test.go b/internal/decide/memory_immutable_test.go new file mode 100644 index 0000000..7139dd8 --- /dev/null +++ b/internal/decide/memory_immutable_test.go @@ -0,0 +1,175 @@ +package decide + +import ( + "slices" + "testing" +) + +// memImmutable is memInput on a platform whose closed pull requests are +// immutable (Bitbucket Cloud), with the current opt-in hash optIn. +func memImmutable(own []OwnPR, optIn string) MemoryInput { + in := memInput(own) + in.OptIn = optIn + in.Config.ClosedImmutable = true + return in +} + +// TestMemoryClosedImmutable: where a declined pull request can never be +// edited, the optin its marker held while open stands for the ack: the +// decline holds while the opt-in file hashes the same and lapses when it +// changes; one without optin holds whatever the file says and is +// unanchored; a forget_declines entry lifts a decline while it is present; +// a ticked repropose control counts for nothing; and memory never asks to +// write to a closed pull request. +func TestMemoryClosedImmutable(t *testing.T) { + a := []Pair{memCreate(memA, memV1)} + declined := memPR(t, 7, a, &memPerson, 0) // optin memH1, written while open + bare := memPR(t, 7, a, &memPerson, 0, memNoOptIn) + cases := []struct { + name string + in MemoryInput + // want is memNumbers of the memory; verdict is memVerdict's for a. + want, verdict string + acked bool + }{ + { + name: "in force while the opt-in file is the same", + in: memImmutable([]OwnPR{declined}, memH1), + want: "Declines [7] Lapsed [] ToAck [] ToRevoke [] Auto []", verdict: "declined", acked: true, + }, + { + name: "lapsed once the opt-in file changes", + in: memImmutable([]OwnPR{declined}, memH2), + want: "Declines [] Lapsed [7] ToAck [] ToRevoke [] Auto []", verdict: "new PR", + }, + { + name: "in force when the current state is unknown", + in: memImmutable([]OwnPR{declined}, ""), + want: "Declines [7] Lapsed [] ToAck [] ToRevoke [] Auto []", verdict: "declined", acked: true, + }, + { + name: "without optin: in force whatever the opt-in file says", + in: memImmutable([]OwnPR{bare}, memH2), + want: "Declines [7] Lapsed [] ToAck [] ToRevoke [] Auto [] Forgotten [] Unanchored [7]", verdict: "declined", + }, + { + name: "without optin: lapsed when a path becomes local", + in: func() MemoryInput { + in := memImmutable([]OwnPR{bare}, memH1) + in.LocalOrIgnored = func(p string) bool { return p == memA } + return in + }(), + want: "Declines [] Lapsed [7] ToAck [] ToRevoke [] Auto []", verdict: "new PR", + }, + { + name: "a forget_declines entry lifts it while present", + in: func() MemoryInput { + in := memImmutable([]OwnPR{bare}, memH1) + in.Forget = []int64{7} + return in + }(), + want: "Declines [] Lapsed [] ToAck [] ToRevoke [] Auto [] Forgotten [7] Unanchored []", verdict: "new PR", + }, + { + name: "a ticked repropose control counts for nothing", + in: func() MemoryInput { + in := memImmutable([]OwnPR{declined}, memH1) + in.Repropose = map[int64]bool{7: true} + return in + }(), + want: "Declines [7] Lapsed [] ToAck [] ToRevoke [] Auto []", verdict: "declined", acked: true, + }, + { + name: "the writer's own close is no memory", + in: memImmutable([]OwnPR{memPR(t, 7, a, &memWriter, 0)}, memH1), + want: "Declines [] Lapsed [] ToAck [] ToRevoke [] Auto []", verdict: "new PR", + }, + { + name: "touchmark's close is no memory", + in: memImmutable([]OwnPR{memPR(t, 7, a, &memWriter, 0, memSelfClosed)}, memH1), + want: "Declines [] Lapsed [] ToAck [] ToRevoke [] Auto []", verdict: "new PR", + }, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + m := BuildMemory(c.in) + if got := memNumbers(m); got != c.want { + t.Errorf("memory %s\nwant %s", got, c.want) + } + if got, _ := memVerdict(m, a, c.in.Now); got != c.verdict { + t.Errorf("verdict %q, want %q", got, c.verdict) + } + if len(m.Declines) == 1 && m.Declines[0].Acked != c.acked { + t.Errorf("decline %+v, want acked %v", m.Declines[0], c.acked) + } + d := DecideTarget(TargetInput{Stream: StreamSync, D: a, Key: Key(StreamSync, a), B: memX, DefaultBranch: "main", + Branch: Branch{Name: "touchmark/acme-eng"}, Own: c.in.Own, Memory: m, Now: c.in.Now}) + for _, s := range d.Steps { + if s.PR != 0 && s.Kind != StepCreatePR { + t.Errorf("a write to closed #%d: %s", s.PR, s.Kind) + } + } + }) + } +} + +// TestMemoryClosedImmutableSame: on a platform whose closed pull requests +// can be edited, the same marker of an unacked decline asks for the ack, +// whatever optin it holds, and a forget_declines entry revokes it: the +// platform decides, not the marker. +func TestMemoryClosedImmutableSame(t *testing.T) { + a := []Pair{memCreate(memA, memV1)} + in := memInput([]OwnPR{memPR(t, 7, a, &memPerson, 0)}) + in.OptIn = memH2 + if got, want := memNumbers(BuildMemory(in)), "Declines [7] Lapsed [] ToAck [7] ToRevoke [] Auto []"; got != want { + t.Errorf("memory %s\nwant %s", got, want) + } + in.Forget = []int64{7} + if got, want := memNumbers(BuildMemory(in)), "Declines [] Lapsed [] ToAck [] ToRevoke [7] Auto []"; got != want { + t.Errorf("forgotten: memory %s\nwant %s", got, want) + } +} + +// TestMemoryClosedImmutableEscalation: auto-closes count as before from +// the closer and the marker: the first two defer the content, the third +// counts as a decline, which holds with the optin of its marker and +// lapses when the opt-in file changes; nothing is written to any of them. +func TestMemoryClosedImmutableEscalation(t *testing.T) { + a := []Pair{memCreate(memA, memV1)} + key := Key(StreamSync, a) + var own []OwnPR + for i, n := range []int64{9, 8, 7} { + own = append(own, memPR(t, n, a, &memStale, 20-i)) + } + m := BuildMemory(memImmutable(own[2:], memH1)) + if until, declined := m.Cooldown(key, memT0.Add(19*memDay), 0); until.IsZero() || declined { + t.Errorf("one auto-close: until %v, declined %v", until, declined) + } + m = BuildMemory(memImmutable(own, memH1)) + if want := "Declines [9] Lapsed [] ToAck [] ToRevoke [] Auto [9 8 7]"; memNumbers(m) != want { + t.Errorf("three auto-closes: %s\nwant %s", memNumbers(m), want) + } + if _, declined := m.Cooldown(key, memT0.Add(100*memDay), 0); !declined { + t.Error("the third auto-close does not count as a decline") + } + d := DecideTarget(TargetInput{Stream: StreamSync, D: a, Key: key, B: memX, DefaultBranch: "main", + Branch: Branch{Name: "touchmark/acme-eng"}, Own: own, Memory: m, CooldownDeclined: true, Now: memT0.Add(100 * memDay)}) + if d.Outcome != OutcomeDeclined || d.PR != 9 || len(d.Steps) != 0 { + t.Errorf("decision %s #%d, steps %v", d.Outcome, d.PR, d.Steps) + } + m = BuildMemory(memImmutable(own, memH2)) + if want := "Declines [] Lapsed [9] ToAck [] ToRevoke [] Auto [9 8 7]"; memNumbers(m) != want { + t.Errorf("after an opt-in change: %s\nwant %s", memNumbers(m), want) + } + if until, declined := m.Cooldown(key, memT0.Add(100*memDay), 0); !until.IsZero() || declined { + t.Errorf("after an opt-in change: until %v, declined %v", until, declined) + } + // A forget_declines entry ends the cooldown of an auto-close it names, + // while present. + in := memImmutable(own[2:], memH1) + in.Forget = []int64{7} + m = BuildMemory(in) + if len(m.Auto) != 0 || !slices.Equal(m.Forgotten, []int64{7}) { + t.Errorf("a forgotten auto-close: %s", memNumbers(m)) + } +} diff --git a/internal/decide/memory_test.go b/internal/decide/memory_test.go index 4c454f5..11b618a 100644 --- a/internal/decide/memory_test.go +++ b/internal/decide/memory_test.go @@ -109,6 +109,7 @@ func memAcked(optIn string) func(*marker.Data) { func memRevoked(d *marker.Data) { d.Revoked = true } func memIncomplete(d *marker.Data) { d.Changes, d.ChangesComplete = nil, false } func memSelfClosed(d *marker.Data) { d.Closed = &marker.Closed{By: "touchmark", Reason: ReasonNoDiff} } +func memNoOptIn(d *marker.Data) { d.OptIn = "" } // memState returns o in another state: open or merged. func memState(o OwnPR, s platform.PRState) OwnPR { @@ -1033,6 +1034,10 @@ func TestMemoryProperties(t *testing.T) { blobs := []string{memV1, memV2, memX} closers := []*platform.Account{nil, &memPerson, &memStale, &memWriter, &memKnown, &memRunner} r := rand.New(rand.NewPCG(3, 5)) + // r2 draws what only the platforms with immutable closed pull requests + // read (markers without optin), so that r draws the inputs it drew + // before they existed. + r2 := rand.New(rand.NewPCG(7, 11)) for run := range runs { var own []OwnPR pairs := map[int64][]Pair{} @@ -1062,6 +1067,9 @@ func TestMemoryProperties(t *testing.T) { if r.IntN(8) == 0 { edits = append(edits, memIncomplete) } + if r2.IntN(6) == 0 { + edits = append(edits, memNoOptIn) + } o := memPR(t, n, d, closers[r.IntN(len(closers))], 100-i, edits...) o.PR.BaseExists = r.IntN(10) > 0 switch r.IntN(8) { @@ -1095,9 +1103,14 @@ func TestMemoryProperties(t *testing.T) { } } slices.Sort(in.Forget) - m := BuildMemory(in) - if err := memCheck(in, m, pairs); err != nil { - t.Fatalf("run %d: %v\ninput: %+v\nmemory: %s", run, err, in, memNumbers(m)) + // The same input on a platform whose closed pull requests are + // immutable, and on the others. + for _, immutable := range []bool{false, true} { + in.Config.ClosedImmutable = immutable + m := BuildMemory(in) + if err := memCheck(in, m, pairs); err != nil { + t.Fatalf("run %d, closed immutable %v: %v\ninput: %+v\nmemory: %s", run, immutable, err, in, memNumbers(m)) + } } } } @@ -1116,7 +1129,7 @@ func memCheck(in MemoryInput, m Memory, pairs map[int64][]Pair) error { closed = append(closed, o) } } - type where struct{ decline, lapsed, revoke, auto, ack bool } + type where struct{ decline, lapsed, revoke, auto, ack, forgotten, unanchored bool } got := map[int64]*where{} at := func(n int64) *where { if got[n] == nil { @@ -1139,6 +1152,19 @@ func memCheck(in MemoryInput, m Memory, pairs map[int64][]Pair) error { for _, n := range m.ToAck { at(n).ack = true } + for _, n := range m.Forgotten { + at(n).forgotten = true + } + for _, n := range m.Unanchored { + at(n).unanchored = true + } + immutable := in.Config.ClosedImmutable + if immutable && len(m.ToAck)+len(m.ToRevoke) > 0 { + return fmt.Errorf("closed pull requests are immutable, yet memory asks to write ack %v, revoke %v", m.ToAck, m.ToRevoke) + } + if !immutable && len(m.Forgotten)+len(m.Unanchored) > 0 { + return fmt.Errorf("closed pull requests can be edited, yet memory lists forgotten %v, unanchored %v", m.Forgotten, m.Unanchored) + } inWindow := map[int64]bool{} var order []int64 for _, o := range closed { @@ -1153,14 +1179,19 @@ func memCheck(in MemoryInput, m Memory, pairs map[int64][]Pair) error { return fmt.Errorf("#%d holds no memory, yet it is in %+v", n, *w) } continue - case slices.Contains(in.Forget, n) || in.Repropose[n]: + case immutable && slices.Contains(in.Forget, n): + if *w != (where{forgotten: true}) { + return fmt.Errorf("#%d is forgotten, yet it is in %+v", n, *w) + } + continue + case !immutable && (slices.Contains(in.Forget, n) || in.Repropose[n]): if *w != (where{revoke: true}) { return fmt.Errorf("#%d is to revoke, yet it is in %+v", n, *w) } continue } - if w.revoke { - return fmt.Errorf("#%d is in ToRevoke unasked", n) + if w.revoke || w.forgotten { + return fmt.Errorf("#%d is in ToRevoke or Forgotten unasked", n) } if (class == CloseAuto) != w.auto { return fmt.Errorf("#%d: class %v, in Auto %v", n, class, w.auto) @@ -1174,10 +1205,22 @@ func memCheck(in MemoryInput, m Memory, pairs map[int64][]Pair) error { if (candidate && w.decline == w.lapsed) || (!candidate && (w.decline || w.lapsed)) { return fmt.Errorf("#%d: a decline %v, in Declines %v, in Lapsed %v", n, candidate, w.decline, w.lapsed) } - if w.ack != (w.decline && !data.Ack) { - return fmt.Errorf("#%d: in ToAck %v, in force %v, acked %v", n, w.ack, w.decline, data.Ack) + // Where no one can write an ack, the optin the marker held while + // open stands for one. + acked := data.Ack || (immutable && data.OptIn != "") + if w.ack != (w.decline && !acked && !immutable) || w.unanchored != (w.decline && !acked && immutable) { + return fmt.Errorf("#%d: in ToAck %v, unanchored %v, in force %v, acked %v", n, w.ack, w.unanchored, w.decline, acked) + } + if i := slices.IndexFunc(m.Declines, func(d Decline) bool { return d.PR == n }); i >= 0 { + want := "" + if acked { + want = data.OptIn + } + if d := m.Declines[i]; d.Acked != acked || d.OptIn != want { + return fmt.Errorf("#%d: decline %+v, want acked %v with %q", n, d, acked, want) + } } - lapse := data.Ack && in.OptIn != "" && data.OptIn != in.OptIn + lapse := acked && in.OptIn != "" && data.OptIn != in.OptIn for _, c := range data.Changes { lapse = lapse || in.LocalOrIgnored(c.Path) } @@ -1209,7 +1252,7 @@ func memCheck(in MemoryInput, m Memory, pairs map[int64][]Pair) error { for _, a := range m.Auto { auto = append(auto, a.PR) } - for _, list := range [][]int64{declines, m.Lapsed, m.ToAck, m.ToRevoke, auto} { + for _, list := range [][]int64{declines, m.Lapsed, m.ToAck, m.ToRevoke, auto, m.Forgotten, m.Unanchored} { if !inOrder(list) { return fmt.Errorf("a list is out of window order: %v", list) } @@ -1243,7 +1286,11 @@ func memNumbers(m Memory) string { for _, a := range m.Auto { auto = append(auto, a.PR) } - return fmt.Sprintf("Declines %v Lapsed %v ToAck %v ToRevoke %v Auto %v", declines, m.Lapsed, m.ToAck, m.ToRevoke, auto) + s := fmt.Sprintf("Declines %v Lapsed %v ToAck %v ToRevoke %v Auto %v", declines, m.Lapsed, m.ToAck, m.ToRevoke, auto) + if len(m.Forgotten)+len(m.Unanchored) > 0 { + s += fmt.Sprintf(" Forgotten %v Unanchored %v", m.Forgotten, m.Unanchored) + } + return s } // memSameNumbers compares the PR numbers of every list of two memories. @@ -1256,5 +1303,7 @@ func memClone(m Memory) Memory { m.ToAck = slices.Clone(m.ToAck) m.ToRevoke = slices.Clone(m.ToRevoke) m.Auto = slices.Clone(m.Auto) + m.Forgotten = slices.Clone(m.Forgotten) + m.Unanchored = slices.Clone(m.Unanchored) return m } diff --git a/internal/decide/target.go b/internal/decide/target.go index 4dd5042..bb8ec9a 100644 --- a/internal/decide/target.go +++ b/internal/decide/target.go @@ -82,6 +82,13 @@ const ( // StepConsumeRecreate unticks the recreate control and writes // recreate_for = Expect into the open PR's marker, before the push. StepConsumeRecreate + // StepRefreshMarker writes the current opt-in state into the marker of + // an open own PR no other step writes to: one body-only edit, the rest + // of the body and of the marker kept. DecideTarget never asks for it: + // the caller adds it where closed PRs are immutable, whose memory reads + // a decline's opt-in state from the marker the PR had while open + // (MemoryConfig.ClosedImmutable). + StepRefreshMarker ) // String returns the step name for plans, messages and tests. @@ -107,6 +114,8 @@ func (k StepKind) String() string { return "revoke" case StepConsumeRecreate: return "consume-recreate" + case StepRefreshMarker: + return "refresh-marker" } return "unknown" } @@ -236,7 +245,10 @@ type TargetDecision struct { // Memory upkeep, always first: StepRevoke for each Memory.ToRevoke, then // StepAck and StepComment for each Memory.ToAck (Reason // CommentAutoDeclined for an auto-close in Memory.Auto, OutcomeDeclined -// otherwise). +// otherwise). These are the only steps that write to a closed PR; on a +// platform whose closed PRs are immutable memory lists none of them +// (MemoryConfig.ClosedImmutable), so no write to a closed PR is planned +// there. // // 1. An open PR in MarkerInvalid → blocked:marker-invalid (no other write), // unless operations.yml lets touchmark adopt it (see below): an adopted diff --git a/internal/decide/target_test.go b/internal/decide/target_test.go index 0e853eb..d477b2f 100644 --- a/internal/decide/target_test.go +++ b/internal/decide/target_test.go @@ -768,7 +768,7 @@ func TestStepKindString(t *testing.T) { want := map[StepKind]string{ StepPush: "push", StepRecreateBranch: "recreate-branch", StepDeleteBranch: "delete-branch", StepCreatePR: "create-pr", StepEditPR: "edit-pr", StepClosePR: "close-pr", StepComment: "comment", StepAck: "ack", StepRevoke: "revoke", - StepConsumeRecreate: "consume-recreate", 0: "unknown", 99: "unknown", + StepConsumeRecreate: "consume-recreate", StepRefreshMarker: "refresh-marker", 0: "unknown", 99: "unknown", } for k, w := range want { if got := k.String(); got != w { diff --git a/internal/distribute/body.go b/internal/distribute/body.go index b0a1be9..585a357 100644 --- a/internal/distribute/body.go +++ b/internal/distribute/body.go @@ -253,14 +253,37 @@ func draftPrefix(c platform.Caps) string { return c.DraftPrefix } +// prLabels returns the labels p's pull requests get: pr.labels of hub.yml, +// none on a platform without labels (Caps.NoLabels), so that the core never +// asks for them there, nor records them in the marker as set. +func (p *provider) prLabels(hub *config.Hub) []string { + if p.caps.NoLabels { + return nil + } + return hub.PR.Labels +} + +// createsLabels reports whether a first pull request of p may create a +// label: hub.yml sets labels and the platform has them. +func (p *provider) createsLabels(hub *config.Hub) bool { return len(p.prLabels(hub)) > 0 } + +// markerFrame is the frame of the marker line on p (Caps.Marker). +func (p *provider) markerFrame() marker.Frame { + if p.caps.Marker == platform.MarkerInRefDef { + return marker.FrameRefDef + } + return marker.FrameComment +} + // reportDecision writes w's decision into the target's report line: the // outcome, reason and pull request, and for a branch touchmark leaves alone // (blocked:edited, blocked:branch-taken) why (decide.Branch.Detail). It // renders the description when the decision writes one, and applies field // ownership (decide.PlanPREdit) to every StepEditPR: an edit that changes -// nothing and rewrites no content is skipped (Work.idle), and an unchanged -// target whose edit writes is reported as updated:title when only the title -// changes, else updated:body. +// nothing, rewrites no content and finds the marker's optin current +// (Work.staleOptIn) is skipped (Work.idle), and an unchanged target whose +// edit writes is reported as updated:title when only the title changes, +// else updated:body. func (r *run) reportDecision(w *Work, prs prSet, reads branchReads) error { res := &w.t.res d := w.Decision @@ -290,16 +313,17 @@ func (r *run) reportDecision(w *Work, prs prSet, reads branchReads) error { if !ok { return fmt.Errorf("the decision edits #%d, which is not an open pull request of touchmark", s.PR) } - desired := decide.DesiredPR{Title: r.hub.PR.Title, Body: human, Labels: r.hub.PR.Labels, Base: s.Base, + desired := decide.DesiredPR{Title: r.hub.PR.Title, Body: human, Labels: w.t.prov.prLabels(r.hub), Base: s.Base, DraftPrefix: draftPrefix(w.t.prov.caps)} edit, _, writes := decide.PlanPREdit(o.PR, o.Marker, desired) - if !writes && !s.Content { + stale := w.staleOptIn(o.Marker) + if !writes && !s.Content && !stale { w.idle[i] = true continue } if changed != "body" { changed = "body" - if titleOnly(edit, o.Marker, human) && !s.Content { + if titleOnly(edit, o.Marker, human) && !s.Content && !stale { changed = "title" } } diff --git a/internal/distribute/commit.go b/internal/distribute/commit.go index 8dc1729..0cacce9 100644 --- a/internal/distribute/commit.go +++ b/internal/distribute/commit.go @@ -1,6 +1,7 @@ package distribute import ( + "cmp" "context" "errors" "fmt" @@ -9,6 +10,7 @@ import ( "net" "path" "slices" + "strings" "time" "github.com/bedrock-python/touchmark/internal/config" @@ -415,11 +417,21 @@ func (r *run) commitPerson(p *provider) gitx.Person { // email private: GitHub "+@users.noreply.", GitLab // "-@users.noreply.", Gitea and Forgejo // "@noreply.". host loses its port. +// +// Bitbucket Cloud has no such address: it links a commit to the account one +// of whose confirmed addresses is the author's, and the driver gives the +// account's primary confirmed address when GET /2.0/user/emails shows one. +// Without it, the author is "@touchmark.invalid", the account's UUID +// without braces under a domain that can never exist (RFC 6761, .invalid): +// the commit then names the bot by its UUID, links to no account, and +// can never be taken for someone else's. func noReplyEmail(typ, host, id, login string) string { if h, _, err := net.SplitHostPort(host); err == nil { host = h } switch typ { + case "bitbucket": + return strings.Trim(cmp.Or(id, login), "{}") + "@touchmark.invalid" case "gitlab": if id != "" { return id + "-" + login + "@users.noreply." + host diff --git a/internal/distribute/doctor.go b/internal/distribute/doctor.go index f5b3f49..883b925 100644 --- a/internal/distribute/doctor.go +++ b/internal/distribute/doctor.go @@ -519,12 +519,12 @@ func listMarkers(items []string) string { } // markerFingerprints returns the fingerprints of the valid v1 markers in -// body whose hub id is id, in the order they appear. +// body, in either frame, whose hub id is id, in the order they appear. func markerFingerprints(body, id string) []string { var out []string for line := range strings.SplitSeq(body, "\n") { line = strings.TrimRight(line, " \t\r") - if !strings.HasPrefix(line, " + FrameComment Frame = iota + // FrameRefDef is a Markdown link reference definition whose title is + // the payload, which renderers hide too, for platforms that escape HTML + // in descriptions (Bitbucket Cloud), where a comment would show as + // text: + // + // [touchmark]: # "touchmark:v1 hub=… fp=… stream=… key=… data=…" + FrameRefDef +) + +// Parts of the frames. +const ( + // commentOpen is what a comment frame puts before the payload. + commentOpen = "") are not marker lines. +func IsLine(line string) bool { + _, ok := asComment(line) + return ok +} + +// asComment returns line in the comment frame, which Parse and Find read: +// line itself when it starts with "". +// A reference definition without its closing quote is still a marker line +// (ok is true), but its comment form never ends in " -->", so Parse refuses +// it even when the payload itself ends in " -->". ok is false for any other +// line. Nothing is trimmed: the callers trim the trailing blanks and +// carriage return they accept. +// +// A reference definition is read after unescape: Bitbucket Cloud returns an +// edited description with backslashes before punctuation ("\[touchmark\]", +// "\_"), which Markdown drops when it renders. +func asComment(line string) (string, bool) { + if strings.HasPrefix(line, commentPrefix) { + return line, true + } + if !strings.HasPrefix(line, "[") && !strings.HasPrefix(line, `\[`) { + return "", false + } + rest, ok := strings.CutPrefix(unescape(line), refDefOpen) + if !ok || !strings.HasPrefix(rest, payloadPrefix) { + return "", false + } + if payload, ok := strings.CutSuffix(rest, refDefClose); ok { + return commentOpen + payload + commentEnd, true + } + // No closing quote: the comment form ends in a quote, never in " -->", + // whatever the payload ends in, so Parse refuses it while Find and + // Strip still see a marker line. + return commentOpen + rest + refDefClose, true +} + +// unescape undoes Markdown's backslash escapes in s: a backslash before +// an ASCII punctuation character is dropped (CommonMark, "Backslash +// escapes"); every other byte stays, a backslash before anything else +// included. +func unescape(s string) string { + if !strings.Contains(s, `\`) { + return s + } + var b strings.Builder + b.Grow(len(s)) + for i := 0; i < len(s); i++ { + if s[i] == '\\' && i+1 < len(s) && isASCIIPunct(s[i+1]) { + i++ + } + b.WriteByte(s[i]) + } + return b.String() +} + +// isASCIIPunct reports whether c is one of CommonMark's ASCII punctuation +// characters: !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~. +func isASCIIPunct(c byte) bool { + return '!' <= c && c <= '/' || ':' <= c && c <= '@' || '[' <= c && c <= '`' || '{' <= c && c <= '~' +} diff --git a/internal/marker/frame_test.go b/internal/marker/frame_test.go new file mode 100644 index 0000000..b6a607b --- /dev/null +++ b/internal/marker/frame_test.go @@ -0,0 +1,277 @@ +package marker + +import ( + "fmt" + "math/rand/v2" + "strings" + "testing" +) + +// refDef returns line, a marker in the comment frame, as a reference +// definition, as frame writes it. +func refDef(line string) string { return frame(line, FrameRefDef) } + +// escapeAll puts a backslash before every ASCII punctuation character of +// s: the most an editor can add (Bitbucket adds some, "\[", "\_", "\{"). +func escapeAll(s string) string { + var b strings.Builder + for i := 0; i < len(s); i++ { + if isASCIIPunct(s[i]) { + b.WriteByte('\\') + } + b.WriteByte(s[i]) + } + return b.String() +} + +func TestEncodeFrameRefDef(t *testing.T) { + for _, v := range goldenVectors() { + t.Run(v.name, func(t *testing.T) { + comment := mustEncode(t, v.m) + line, err := EncodeFrame(v.m, FrameRefDef) + if err != nil { + t.Fatal(err) + } + payload := strings.TrimSuffix(strings.TrimPrefix(comment, "") + if want := `[touchmark]: # "` + payload + `"`; line != want { + t.Fatalf("EncodeFrame = %.120q…, want %.120q…", line, want) + } + if strings.ContainsAny(payload, "\"\r\n") { + t.Fatal("the payload holds a quote or a line break, which would end the title") + } + got, err := Parse(line) + if err != nil { + t.Fatalf("Parse(refdef): %v", err) + } + equalMarkers(t, got, written(v.m)) + if again, _ := EncodeFrame(v.m, FrameComment); again != comment { + t.Error("EncodeFrame(FrameComment) differs from Encode") + } + }) + } +} + +// TestEncodeFrameGolden pins the reference definition frame around the +// payload of a golden vector. +func TestEncodeFrameGolden(t *testing.T) { + v := goldenVectors()[0] + line, err := EncodeFrame(v.m, FrameRefDef) + if err != nil { + t.Fatal(err) + } + if *update { + golden(t, "encode/"+v.name+".refdef", line+"\n") + return + } + pinned := strings.TrimSuffix(readGolden(t, "encode/"+v.name+".refdef"), "\n") + if !strings.HasPrefix(pinned, `[touchmark]: # "touchmark:v1 hub=`) || !strings.HasSuffix(pinned, `"`) { + t.Fatalf("golden line %.60q… is not a reference definition", pinned) + } + got, err := Parse(pinned) + if err != nil { + t.Fatalf("Parse(golden line): %v", err) + } + equalMarkers(t, got, written(v.m)) + comment := strings.TrimSuffix(readGolden(t, "encode/"+v.name+".marker"), "\n") + if refDef(comment) != pinned { + t.Error("the golden reference definition does not frame the golden comment's payload") + } +} + +func TestParseRefDef(t *testing.T) { + m := sampleMarker() + comment := mustEncode(t, m) + line := refDef(comment) + want := written(m) + accepted := []struct{ name, line string }{ + {"as written", line}, + {"brackets escaped", strings.Replace(strings.Replace(line, "[", `\[`, 1), "]", `\]`, 1)}, + {"every punctuation escaped", escapeAll(line)}, + {"hash and quotes escaped", strings.NewReplacer("#", `\#`, `"`, `\"`).Replace(line)}, + } + for _, c := range accepted { + t.Run(c.name, func(t *testing.T) { + got, err := Parse(c.line) + if err != nil { + t.Fatalf("Parse: %v", err) + } + equalMarkers(t, got, want) + }) + } + refused := []struct{ name, line string }{ + {"no closing quote", strings.TrimSuffix(line, `"`)}, + {"no closing quote, comment end", strings.TrimSuffix(line, `"`) + commentEnd}, + {"no closing quote, comment end escaped", escapeAll(strings.TrimSuffix(line, `"`) + commentEnd)}, + {"trailing blank", line + " "}, + {"CR", line + "\r"}, + {"single quotes", strings.Replace(strings.Replace(line, `# "`, `# '`, 1), `"`, `'`, 1)}, + {"another label", strings.Replace(line, "[touchmark]", "[touchmark2]", 1)}, + {"another destination", strings.Replace(line, `]: # "`, `]: #x "`, 1)}, + {"indented", " " + line}, + {"escaped backslash before the label", `\\` + line}, + {"letter escaped", strings.Replace(line, "hub=", `h\ub=`, 1)}, + {"comment inside", strings.Replace(line, `"touchmark:v1 `, `"", true}, + {"- [ ] Rebuild this branch", false}, + {`[touchmark]: # "something else"`, false}, + {`[touchmark]: https://example.com "touchmark:v1"`, false}, + {"[touchmark]", false}, + {" " + line, false}, + {"see " + line, false}, + {"", false}, + {`\`, false}, + {"[", false}, + } + for _, c := range cases { + if got := IsLine(c.line); got != c.want { + t.Errorf("IsLine(%.60q) = %v, want %v", c.line, got, c.want) + } + } +} + +func TestUnescape(t *testing.T) { + cases := []struct{ in, want string }{ + {"", ""}, + {"plain", "plain"}, + {`\[a\]`, "[a]"}, + {`a\_b\*c\#d\(e\)f`, "a_b*c#d(e)f"}, + {`\\`, `\`}, + {`\\\[`, `\[`}, + {`\a\1\ \é`, `\a\1\ \é`}, + {`trailing\`, `trailing\`}, + {`\{\}\~\!\/\:\@\` + "`", "{}~!/:@`"}, + } + for _, c := range cases { + if got := unescape(c.in); got != c.want { + t.Errorf("unescape(%q) = %q, want %q", c.in, got, c.want) + } + } + for c := 0; c < 128; c++ { + want := strings.ContainsRune("!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~", rune(c)) + if got := isASCIIPunct(byte(c)); got != want { + t.Errorf("isASCIIPunct(%q) = %v, want %v", rune(c), got, want) + } + } +} + +func TestFindRefDef(t *testing.T) { + l := newFindLines(t) + fps := []string{ourFP, prevFP} + ours, ours2 := parsed(t, l.ours), parsed(t, l.ours2) + rd := refDef(l.ours) + cases := []struct { + name string + body string + want Status + marker Marker + }{ + {"ours last", "text\n\n" + rd, Found, ours}, + {"CRLF", "text\r\n\r\n" + rd + "\r\n", Found, ours}, + {"escaped by a web edit, CRLF", "text\\_with\\_escapes\r\n\r\n" + escapeAll(rd) + " \r\n", Found, ours}, + {"comment after refdef", rd + "\n" + l.ours2, Found, ours2}, + {"refdef after comment", l.ours2 + "\n" + rd, Found, ours}, + {"foreign refdef", "text\n" + refDef(l.foreign), Foreign, Marker{}}, + {"v2 refdef", refDef(l.v2), Foreign, Marker{}}, + {"broken refdef", "text\n" + strings.TrimSuffix(rd, `"`), Invalid, Marker{}}, + {"tampered refdef", refDef(l.tampered), Invalid, Marker{}}, + {"indented refdef", " " + rd, None, Marker{}}, + {"refdef in a quote", "> " + rd, None, Marker{}}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + m, status := Find(c.body, fps) + if status != c.want { + t.Fatalf("status = %v, want %v", status, c.want) + } + equalMarkers(t, m, c.marker) + }) + } +} + +func TestStripRefDef(t *testing.T) { + l := newFindLines(t) + rd := refDef(l.ours) + cases := []struct{ name, body, want string }{ + {"refdef last", "text\n\n" + rd, "text"}, + {"CRLF", "a\r\nb\r\n\r\n" + rd + "\r\n", "a\r\nb"}, + {"escaped", "a\\_b\n\n" + escapeAll(rd), "a\\_b"}, + {"both frames", "a\n" + l.ours + "\n" + rd + "\nb", "a\nb"}, + {"another reference stays", "a\n\n[docs]: https://example.com \"Docs\"\n" + rd, "a\n\n[docs]: https://example.com \"Docs\""}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + if got := Strip(c.body); got != c.want { + t.Errorf("Strip = %q, want %q", got, c.want) + } + }) + } +} + +// TestEncodeFrameMeasuresTheFramedLine: a marker whose comment fits +// MaxLine but whose reference definition, 8 bytes longer, does not, loses +// its changes in the reference definition frame only. +func TestEncodeFrameMeasuresTheFramedLine(t *testing.T) { + extra := len(refDefOpen) + len(refDefClose) - len(commentOpen) - len(commentEnd) + r := rand.New(rand.NewPCG(5, 6)) + const letters = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789+/" + random := func(n int) string { + b := make([]byte, n) + for i := range b { + b[i] = letters[r.IntN(len(letters))] + } + return string(b) + } + base := sampleMarker() + base.Data.Changes = []Change{{Path: random(40), Mode: "100644", To: fmt.Sprintf("%016x", r.Uint64())}} + fixed := []string{random(MaxString), random(MaxString), random(MaxString)} + tail := random(MaxString) + for n := 1000; n <= MaxString; n++ { + m := base + m.Data.Packs = append(append([]string{}, fixed...), tail[:n]) + comment, err := Encode(m) + if err != nil { + t.Fatal(err) + } + if len(comment) <= MaxLine-extra || len(comment) > MaxLine { + continue + } + if got := parsed(t, comment); len(got.Data.Changes) != 1 { + t.Fatalf("the comment lost its changes at %d bytes", len(comment)) + } + line, err := EncodeFrame(m, FrameRefDef) + if err != nil { + t.Fatal(err) + } + if len(line) > MaxLine { + t.Fatalf("a reference definition of %d bytes, more than %d", len(line), MaxLine) + } + equalMarkers(t, parsed(t, line), dropped(m)) + return + } + t.Fatal("fixture: no pack length put the comment within the last bytes of MaxLine") +} diff --git a/internal/marker/fuzz_test.go b/internal/marker/fuzz_test.go index 3bf049a..9789dfe 100644 --- a/internal/marker/fuzz_test.go +++ b/internal/marker/fuzz_test.go @@ -15,6 +15,8 @@ func seedLines(t testing.TB) []string { for _, v := range goldenVectors() { line := mustEncode(t, v.m) lines = append(lines, line, line[:len(line)/2], strings.Replace(line, "v1", "v2", 1), line+"\r") + rd := refDef(line) + lines = append(lines, rd, escapeAll(rd), rd[:len(rd)/2], rd+"\r", strings.TrimSuffix(rd, refDefClose)+commentEnd) } lines = append(lines, lineWithData(gz(t, bytes.Repeat([]byte{' '}, 8<<20))), @@ -22,6 +24,9 @@ func seedLines(t testing.TB) []string { lineWithJSON(t, "{}"), "", "", + `[touchmark]: # "touchmark:v1 hub= fp= stream= key= data="`, + `\[touchmark\]: \# "touchmark:v1\"`, + `[touchmark]: # "touchmark:v1 hub= fp= stream= key= data= -->`, ) return lines } @@ -39,6 +44,9 @@ func FuzzParse(f *testing.F) { if err != nil { return } + if !strings.HasPrefix(line, commentPrefix) && !strings.HasSuffix(unescape(line), refDefClose) { + t.Fatalf("accepted a reference definition without its closing quote: %.80q", line) + } if m.FP16 != FP16(m.Data.FP) || m.Hub != m.Data.Hub || m.Stream != m.Data.Stream { t.Fatalf("header does not match the data: %s", describeMarker(m)) } @@ -89,6 +97,22 @@ func FuzzEncode(f *testing.F) { if len(line) > MaxLine { t.Fatalf("line of %d bytes", len(line)) } + rd, err := EncodeFrame(m, FrameRefDef) + switch { + case err != nil && !errors.Is(err, errTooLarge): + t.Fatalf("EncodeFrame(FrameRefDef) of an encodable marker: %v", err) + case err != nil: + case len(rd) > MaxLine: + t.Fatalf("reference definition of %d bytes", len(rd)) + default: + back, err := Parse(rd) + if err != nil { + t.Fatalf("Parse(EncodeFrame(m, FrameRefDef)): %v", err) + } + if !reflect.DeepEqual(back, written(m)) && !reflect.DeepEqual(back, dropped(m)) { + t.Fatalf("reference definition round trip differs\n got %s", describeMarker(back)) + } + } got, err := Parse(line) if err != nil { t.Fatalf("Parse(Encode(m)): %v\nline %.300q", err, line) @@ -137,7 +161,7 @@ func FuzzFind(f *testing.F) { t.Fatal("Strip is not idempotent") } for line := range strings.SplitSeq(stripped, "\n") { - if strings.HasPrefix(line, commentPrefix) { + if IsLine(line) { t.Fatalf("Strip left a marker line: %.80q", line) } } @@ -155,5 +179,9 @@ func FuzzFind(f *testing.F) { if Strip(appended) != stripped { t.Fatalf("Strip(body + marker) = %q, want %q", Strip(appended), stripped) } + appended = body + "\r\n" + escapeAll(refDef(last)) + " \r\n" + if got, status := Find(appended, fps); status != Found || !reflect.DeepEqual(got, want) { + t.Fatalf("appended reference definition: status %v", status) + } }) } diff --git a/internal/marker/marker.go b/internal/marker/marker.go index 7fbeafe..13beee3 100644 --- a/internal/marker/marker.go +++ b/internal/marker/marker.go @@ -3,6 +3,11 @@ // // // +// or, where a platform escapes HTML in descriptions (Bitbucket Cloud), the +// same payload in a Markdown link reference definition (FrameRefDef): +// +// [touchmark]: # "touchmark:v1 hub= fp=<16 hex> stream= key=sha256:<64 hex> data=" +// // The marker is the only memory touchmark has: what the PR carried, who // the hub is, and what the engine wrote last. Everything in a PR body is // untrusted input, so parsing is strict and bounded. @@ -28,7 +33,10 @@ import ( const ( // Version is the marker format version ("touchmark:v1"). Version = 1 - // MaxLine is the largest marker comment accepted or written. + // MaxLine is the largest marker line written, in either frame, and the + // largest accepted in the comment frame. A reference definition is + // accepted up to twice as long before its backslash escapes are undone + // (an escape per byte at most), and up to MaxLine after. MaxLine = 16 << 10 // MaxData is the largest decompressed JSON accepted. MaxData = 64 << 10 @@ -68,14 +76,19 @@ type Data struct { // header carries only FP16(FP). FP string `json:"fp"` // HubRepo is omitted when the hub is private and the target public. - HubRepo string `json:"hub_repo,omitempty"` - DecidedAt string `json:"decided_at"` - ContentCommit string `json:"content_commit"` - Base string `json:"base"` - OptIn string `json:"optin"` - Engine string `json:"engine"` - Packs []string `json:"packs"` - Changes []Change `json:"changes"` + HubRepo string `json:"hub_repo,omitempty"` + DecidedAt string `json:"decided_at"` + ContentCommit string `json:"content_commit"` + Base string `json:"base"` + // OptIn is the hash of the opt-in file (config.OptIn.Hash): the one the + // content was proposed under, written with the content, and the one an + // ack records. Where closed pull requests are immutable + // (platform.Caps.ClosedImmutable) touchmark keeps it current while the + // pull request is open, and memory reads it as the ack of a decline. + OptIn string `json:"optin"` + Engine string `json:"engine"` + Packs []string `json:"packs"` + Changes []Change `json:"changes"` // ChangesComplete is false when Changes was dropped to fit the limits; // memory then matches only by the exact key. ChangesComplete bool `json:"changes_complete"` @@ -139,18 +152,23 @@ func FP16(fingerprint string) string { return hex.EncodeToString(sum[:8]) } -// Encode returns the marker comment line (no trailing newline). +// Encode returns the marker line in the comment frame (no trailing +// newline): EncodeFrame(m, FrameComment). +func Encode(m Marker) (string, error) { return EncodeFrame(m, FrameComment) } + +// EncodeFrame returns the marker line in frame f (no trailing newline). // // It fills Hub, FP16 and Stream from Data, validates every field with the // rules of Parse (and every string as valid UTF-8, which JSON needs to // round-trip), and enforces the limits: when there are more than MaxChanges // changes, the JSON exceeds MaxJSON or the line exceeds MaxLine, it drops // Changes and sets ChangesComplete to false; if the JSON still exceeds -// MaxJSON or the line MaxLine it fails. Output is deterministic: JSON with -// empty arrays for nil slices, gzip at the best compression level without a -// timestamp or name, standard padded base64. Parse accepts every line -// Encode returns and yields the marker it encoded. -func Encode(m Marker) (string, error) { +// MaxJSON or the line MaxLine it fails; the line is measured in frame f. +// Output is deterministic: JSON with empty arrays for nil slices, gzip at +// the best compression level without a timestamp or name, standard padded +// base64. Parse accepts every line EncodeFrame returns and yields the +// marker it encoded. +func EncodeFrame(m Marker, f Frame) (string, error) { m.Hub, m.Stream, m.FP16 = m.Data.Hub, m.Data.Stream, FP16(m.Data.FP) if err := checkHeader(m); err != nil { return "", fmt.Errorf("encode marker: %w", err) @@ -166,11 +184,13 @@ func Encode(m Marker) (string, error) { if err != nil { return "", fmt.Errorf("encode marker: %w", err) } + line = frame(line, f) if (size > MaxJSON || len(line) > MaxLine) && len(d.Changes) > 0 { d.Changes, d.ChangesComplete = nil, false if line, size, err = encodeLine(m, d); err != nil { return "", fmt.Errorf("encode marker: %w", err) } + line = frame(line, f) } switch { case size > MaxJSON: @@ -231,9 +251,11 @@ func wire(d Data) Data { return d } -// Parse parses one marker comment strictly: exact prefix and attribute -// order separated by single spaces, base64 that decodes (standard alphabet, -// padded, canonical), one gzip member without a name, comment, extra field +// Parse parses one marker line strictly, in either frame: a reference +// definition is read as the comment it frames (see IsLine), its backslash +// escapes undone, and the comment is parsed. Strictly means: exact prefix +// and attribute order separated by single spaces, base64 that decodes +// (standard alphabet, padded, canonical), one gzip member without a name, comment, extra field // or time and with nothing after it, that decompresses to at most MaxData // bytes (read through a limited reader), UTF-8 JSON with exactly the keys of // Data (compared case-sensitively, none repeated, no unknown fields, none @@ -257,6 +279,14 @@ func Parse(line string) (Marker, error) { } func parse(line string) (Marker, error) { + if !strings.HasPrefix(line, commentPrefix) { + if len(line) > 2*MaxLine { + return Marker{}, fmt.Errorf("the line is %d bytes, more than %d", len(line), 2*MaxLine) + } + if c, ok := asComment(line); ok { + line = c + } + } if len(line) > MaxLine { return Marker{}, fmt.Errorf("the line is %d bytes, more than %d", len(line), MaxLine) } diff --git a/internal/marker/testdata/encode/sync-open.refdef b/internal/marker/testdata/encode/sync-open.refdef new file mode 100644 index 0000000..c2b56d8 --- /dev/null +++ b/internal/marker/testdata/encode/sync-open.refdef @@ -0,0 +1 @@ +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5xSzYrcPBB8laXP2ln9y/btO3zklktyG4ah1WqNzfgP2zNhCXn3oDWbncCeAgYjuqpLqqqfcIdGCVi3hXGABtbXkUBAe4vQANLAzzxeQECeoYFLt7W3eKBpeAlKG+t8qHbweeF5+htRyC88XrqReenGyzOuK28rCEhMXeJ0xg0aMFmTwlinikP26KIlkzSrLLGOFYXk2WWLBgTQNG48bmeahqErXKxV1GSSZZe9DKrStUEbHfkUuMq1xH0OAiKuDA3USbLKGk205JLnkCtZK9TRkE2OfQ6yUrWWIGCat24sjrSonW8cB5RRkU6GbXbSq6ArU1t00VNIFddZSqW0NsZa57wPoarqGjFGolSusHsBDciDPhSFGem6QnMEvPD45gz1eEsMJwHU4njhMj3Cf1/+//r922EoS0LAKEklzSYXbRCgpPTWgoAqG1Ioo+aQLHoFJ3GEeZmGeVtfpj7tG2qUKpI2ia3Le4Dle8QufO/4xw5/FCgePFr1Rlpp6QqJWqbrYW1BwLvv77nsO4Jzn81OH68tyc49bwzNttxYwNZtPZ9XLmFTOy3cPJWCPj3U6ulPreKUXj/yIpkzs5T/+gcBPUbu113+CJ9U+SQA6QpNxn5lAQvfpyunhzMtjBuf87RAM976/tfvAAAA//+h3bfxcAMAAA==" diff --git a/internal/platform/bitbucket/bitbucket.go b/internal/platform/bitbucket/bitbucket.go new file mode 100644 index 0000000..3724380 --- /dev/null +++ b/internal/platform/bitbucket/bitbucket.go @@ -0,0 +1,106 @@ +// Package bitbucket is the platform driver for Bitbucket Cloud +// (bitbucket.org, REST API 2.0). Bitbucket Data Center has another API and +// is not served here. +// +// NewReader serves plan; NewWriter serves distribute and doctor: it opens, +// edits, declines and comments on pull requests and pushes over git, and +// reports the writer's access for doctor (platform.Checker). +// +// Authentication is an API token of a bot account (app passwords are gone: +// their creation closed on 2025-09-09). The API gets +// "Authorization: Bearer "; git over HTTPS gets Basic +// "x-bitbucket-api-token-auth:" (REST intro, "API tokens"; +// support.atlassian.com/bitbucket-cloud, "Using API tokens"). A token's +// scopes do not narrow the repositories it reaches: the account's +// permissions do. Access tokens of a repository, project or workspace are +// left for later: GET /2.0/user does not accept them (BCLOUD-23528), so the +// identity cannot learn who it is. +// +// What the driver relies on, from the OpenAPI description +// (https://api.bitbucket.org/swagger.json), the REST intro +// (https://developer.atlassian.com/cloud/bitbucket/rest/intro/) and +// anonymous answers of public repositories: +// - accounts are known by their uuid, in braces, which never changes; +// nickname is neither unique nor free of spaces, and there is no +// lookup by name, so a login in hub.yml is a uuid (or an Atlassian +// account id), and Account.ID and Account.Login are the uuid; +// - a workspace is the namespace: no nested groups, no topics, no +// archiving; a repository is workspace/slug, its uuid is immutable; +// - collections are pages of values with an opaque next link, absent on +// the last page; q= filters (BBQL) and sort= orders them, and on pull +// request listings a q= makes the state parameter ignored, so the state +// goes into q; +// - GET …/src/{commit}/{path}?format=meta describes a path (commit_file +// or commit_directory, attributes link, executable, subrepository, +// binary, lfs, and size), and without format returns the raw bytes; +// Git LFS content is a redirect to another host; there is no blob id; +// - a pull request's states are OPEN, MERGED, DECLINED and SUPERSEDED, +// draft is a flag, closed_by names who merged or declined it, and its +// commit hashes are short (12 digits); there are no labels; +// - limits are per account and hour (1 000 requests to +// /2.0/repositories/*), answered with 429 and x-ratelimit-* headers +// whose reset is in seconds; +// - only open pull requests can be changed (PUT …/pullrequests/{id}), and a +// declined one can never be reopened; a pull request is closed by +// declining it (POST …/decline); +// - an account's permission on a repository (admin, write, read) is +// GET /2.0/user/workspaces/{workspace}/permissions/repositories, filtered +// by the repository's uuid; branch restrictions are readable with admin +// rights only, so a push meets them ("Permission denied to update +// branch …"). +// +// The driver never retries or paces (internal/throttle does), never knows +// about packs or markers, and masks secrets in its errors. +package bitbucket + +import ( + "errors" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/config" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +// NewReader returns the read driver for provider p (type "bitbucket") with +// credential c (an auth.Token, an API token; a zero credential reads +// anonymously, public repositories only) over client, which sends the token +// only to p's API host. +func NewReader(p config.ResolvedProvider, c auth.Credential, client *httpx.Client) (platform.Reader, error) { + cl, err := newClient(p, c, client) + if err != nil { + return nil, err + } + return &reader{c: cl}, nil +} + +// NewWriter returns the write driver for provider p with credential c, an +// API token of the bot account that writes (a write identity is never +// anonymous). Target checks the account's permission on the repository +// (write or admin) and returns a TargetWriter bound to it; Bitbucket cannot +// mint narrower tokens, so Close only retires the TargetWriter: later calls +// through it fail with ClassAuth. Self also reads the account's primary +// confirmed address (GET /2.0/user/emails), the author of its commits. +func NewWriter(p config.ResolvedProvider, c auth.Credential, client *httpx.Client) (platform.Writer, error) { + if c.Kind == 0 { + return nil, errors.New("bitbucket: the write driver needs a token") + } + cl, err := newClient(p, c, client) + if err != nil { + return nil, err + } + cl.wantEmail = true + return &writer{reader: reader{c: cl}}, nil +} + +// reader is platform.Reader over one identity. +type reader struct{ c *client } + +// writer is platform.Writer: a reader that can mint TargetWriters. +type writer struct{ reader } + +var ( + _ platform.Reader = (*reader)(nil) + _ platform.Writer = (*writer)(nil) + _ platform.Checker = (*writer)(nil) +) diff --git a/internal/platform/bitbucket/check.go b/internal/platform/bitbucket/check.go new file mode 100644 index 0000000..d0baa24 --- /dev/null +++ b/internal/platform/bitbucket/check.go @@ -0,0 +1,95 @@ +package bitbucket + +import ( + "context" + "fmt" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// writerScopes are the scopes of the writer's API token: its account and +// workspaces (GET /2.0/user, /2.0/user/emails, the sweep's +// /2.0/user/workspaces), repositories (reads, the permission listing, +// pushes) and pull requests (reads, creating, editing, declining, +// commenting). +const writerScopes = "read:user:bitbucket, read:workspace:bitbucket, read:repository:bitbucket, write:repository:bitbucket, " + + "read:pullrequest:bitbucket and write:pullrequest:bitbucket" + +// Check reports the writer's checks for doctor (platform.Checker). +// +// With no repository, the identity's: "token-expiry" and "scopes" are +// unknown, since Bitbucket's API shows neither an API token's expiry nor +// its scopes (the account's settings at id.atlassian.com do), and so is +// "2fa", which it does not show either; the account itself is read first +// (GET /2.0/user), so a refused token fails the call. +// +// With repositories, per repository: "access" from the writer's permission +// on it (write ok; admin warns: more than it needs; read or none fails), +// fail when it does not see the repository; "rules" unknown: branch +// restrictions are readable with admin rights only, so a push meets them. +func (w *writer) Check(ctx context.Context, repos []platform.Repo, _ []string) ([]platform.Finding, error) { + if len(repos) == 0 { + return w.identityChecks(ctx) + } + var out []platform.Finding + for _, r := range repos { + fs, err := w.repoChecks(ctx, r) + if err != nil { + return nil, err + } + out = append(out, fs...) + } + return out, nil +} + +// identityChecks are the checks of the writer's token and account. +func (w *writer) identityChecks(ctx context.Context) ([]platform.Finding, error) { + if _, err := w.c.selfAccount(ctx); err != nil { + return nil, err + } + return []platform.Finding{ + {Check: "token-expiry", Status: platform.FindingUnknown, + Detail: "Bitbucket's API does not show when an API token expires (at most a year after it was made); the account's API tokens page at id.atlassian.com does"}, + {Check: "scopes", Status: platform.FindingUnknown, + Detail: "Bitbucket's API does not show an API token's scopes; the writer's token needs " + writerScopes + ", and no admin or delete scope"}, + {Check: "2fa", Status: platform.FindingUnknown, + Detail: "Bitbucket's API does not show whether an account uses two-step verification"}, + }, nil +} + +// repoChecks are the checks of one repository. +func (w *writer) repoChecks(ctx context.Context, r platform.Repo) ([]platform.Finding, error) { + const op = "check" + find := func(check string, status platform.FindingStatus, format string, args ...any) platform.Finding { + return platform.Finding{Repo: r.Path, Check: check, Status: status, Detail: fmt.Sprintf(format, args...)} + } + rules := find("rules", platform.FindingUnknown, "branch restrictions need admin to read; a push meets them") + repo, err := w.c.targetRepo(ctx, op, r) + switch { + case err == nil: + case stops(err): + return nil, err + case platform.ClassOf(err) == platform.ClassNotFound: + return []platform.Finding{find("access", platform.FindingFail, "the writer does not see this repository")}, nil + default: + return []platform.Finding{find("access", platform.FindingUnknown, "%v", err), rules}, nil + } + ws, _, _ := splitRepoPath(repo.FullName) + perm, err := w.c.permission(ctx, op, ws, repo.UUID) + var access platform.Finding + switch { + case err != nil && stops(err): + return nil, err + case err != nil: + access = find("access", platform.FindingUnknown, "%v", err) + case perm == permAdmin: + access = find("access", platform.FindingWarn, "the writer administers the repository: write is enough, "+ + "and a leaked token could change its settings and branch restrictions") + case perm == permWrite: + access = find("access", platform.FindingOK, "the writer may push and write pull requests") + default: + access = find("access", platform.FindingFail, "the writer has %s permission, not write: it may not push; "+ + "give it write access to the repository (directly, or through a group or the project)", perm) + } + return []platform.Finding{access, rules}, nil +} diff --git a/internal/platform/bitbucket/check_test.go b/internal/platform/bitbucket/check_test.go new file mode 100644 index 0000000..4e98da7 --- /dev/null +++ b/internal/platform/bitbucket/check_test.go @@ -0,0 +1,114 @@ +package bitbucket + +import ( + "net/http" + "strings" + "testing" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +func TestCheckIdentity(t *testing.T) { + f := newWriterFixture(t) + fs, err := f.writer.Check(t.Context(), nil, nil) + if err != nil { + t.Fatal(err) + } + got := map[string]platform.Finding{} + for _, x := range fs { + got[x.Check] = x + } + for _, check := range []string{"token-expiry", "scopes", "2fa"} { + x, ok := got[check] + if !ok || x.Status != platform.FindingUnknown || x.Repo != "" || x.Detail == "" { + t.Errorf("%s: %+v; want unknown, with the reason", check, x) + } + } + if len(fs) != 3 || !strings.Contains(got["scopes"].Detail, "write:pullrequest:bitbucket") { + t.Errorf("findings %+v", fs) + } + if w := f.writes(); len(w) != 0 { + t.Errorf("writes %+v", w) + } + + s := newAPIServer(t) + w, err := NewWriter(s.provider(), auth.Credential{Kind: auth.Token, Token: testToken(t)}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + s.json("/user", http.StatusUnauthorized, errorBody("Token is invalid, expired, or not supported for this endpoint.")) + _, err = w.(platform.Checker).Check(t.Context(), nil, nil) + wantClass(t, "Check with a refused token", err, platform.ClassAuth, nil) +} + +func TestCheckRepository(t *testing.T) { + for name, tc := range map[string]struct { + perm string + status int // of the permission listing, 0 for 200 + want platform.FindingStatus + detail string + }{ + "write": {permWrite, 0, platform.FindingOK, "may push"}, + "admin": {permAdmin, 0, platform.FindingWarn, "administers"}, + "read": {permRead, 0, platform.FindingFail, "read permission"}, + "none": {"", 0, platform.FindingFail, "none permission"}, + "refused": {"", http.StatusForbidden, platform.FindingUnknown, "403"}, + } { + t.Run(name, func(t *testing.T) { + f := newWriterFixture(t) + f.repoRoute() + if tc.status != 0 { + f.json(permsPath, tc.status, errorBody("The requesting user does not have access to the workspace.")) + } else { + f.permRoute(tc.perm) + } + fs, err := f.writer.Check(t.Context(), []platform.Repo{apiRepoFixture}, []string{syncBranch}) + if err != nil { + t.Fatal(err) + } + if len(fs) != 2 { + t.Fatalf("findings %+v", fs) + } + access, rules := fs[0], fs[1] + if access.Check != "access" || access.Repo != "acme/api" || access.Status != tc.want || !strings.Contains(access.Detail, tc.detail) { + t.Errorf("access %+v; want %s with %q", access, tc.want, tc.detail) + } + if rules.Check != "rules" || rules.Repo != "acme/api" || rules.Status != platform.FindingUnknown || + rules.Detail != "branch restrictions need admin to read; a push meets them" { + t.Errorf("rules %+v", rules) + } + }) + } +} + +func TestCheckRepositoryErrors(t *testing.T) { + t.Run("hidden", func(t *testing.T) { + f := newWriterFixture(t) + f.json("/repositories/acme/"+uuidPath(repoUUID), http.StatusNotFound, errorBody(noRepoMessage)) + fs, err := f.writer.Check(t.Context(), []platform.Repo{apiRepoFixture}, nil) + if err != nil || len(fs) != 1 || fs[0].Check != "access" || fs[0].Status != platform.FindingFail { + t.Errorf("Check = %+v, %v; want access failing", fs, err) + } + }) + t.Run("server error", func(t *testing.T) { + f := newWriterFixture(t) + f.json("/repositories/acme/"+uuidPath(repoUUID), http.StatusInternalServerError, errorBody("Something went wrong")) + fs, err := f.writer.Check(t.Context(), []platform.Repo{apiRepoFixture}, nil) + if err != nil || len(fs) != 2 || fs[0].Status != platform.FindingUnknown || fs[1].Check != "rules" { + t.Errorf("Check = %+v, %v; want access unknown", fs, err) + } + }) + for name, status := range map[string]int{"rate limited": http.StatusTooManyRequests, "refused token": http.StatusUnauthorized} { + t.Run(name, func(t *testing.T) { + f := newWriterFixture(t) + f.repoRoute() + f.json(permsPath, status, errorBody("no")) + _, err := f.writer.Check(t.Context(), []platform.Repo{apiRepoFixture}, nil) + if err == nil { + t.Error("Check: no error; a rate limit or a refused credential fails the call") + } + }) + } +} diff --git a/internal/platform/bitbucket/client.go b/internal/platform/bitbucket/client.go new file mode 100644 index 0000000..0654c57 --- /dev/null +++ b/internal/platform/bitbucket/client.go @@ -0,0 +1,396 @@ +package bitbucket + +import ( + "context" + "encoding/base64" + "errors" + "fmt" + "net/http" + "net/url" + "strings" + "sync" + "unicode" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/config" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" + "github.com/bedrock-python/touchmark/internal/redact" +) + +// basicUser is the user name of git's Basic credentials for an API token +// of an account (support.atlassian.com/bitbucket-cloud, "Using API tokens": +// the git user of an API token is x-bitbucket-api-token-auth, or the +// account's username, which touchmark does not know). It is one of the +// users whose Basic form the CLI registers for masking. +const basicUser = "x-bitbucket-api-token-auth" + +// Page lengths asked for: the API's global bounds are 10 and 100 (REST +// intro, "Pagination"); the pull request listings are taken to stop at 50 +// (an assumption to confirm live; a smaller page only costs pages). +const ( + repoPageLen = 100 + prPageLen = 50 +) + +// maxRedirects bounds the redirects one GET follows. +const maxRedirects = 3 + +// snippetLimit bounds the body kept from a failed raw request, in bytes, as +// httpx keeps for JSON requests. +const snippetLimit = 1 << 10 + +// client is the HTTP side of one driver: one provider under one identity. +// It is safe for concurrent use. +type client struct { + http *httpx.Client + // api is the REST base without a trailing slash + // (https://api.bitbucket.org/2.0); apiURL is api parsed: redirects + // and pagination links are followed only below it. + api string + apiURL *url.URL + // web is the provider's web URL without a trailing slash; git remotes + // are under it. host is the provider host (Repo.Host). + web, host string + token string // "" when anonymous + auth *httpx.Auth + masks *redact.Registry + + // wantEmail makes selfAccount read the account's primary address, for + // the commits of a writer. + wantEmail bool + + mu sync.Mutex + self *platform.Account // cached by selfAccount + // commits maps " " to the full commit id + // (fullCommit): a short id of a commit that exists never names another. + commits map[string]string +} + +// newClient checks the provider and the credential and builds the client. +func newClient(p config.ResolvedProvider, cred auth.Credential, hc *httpx.Client) (*client, error) { + if p.Type != "bitbucket" { + return nil, fmt.Errorf("bitbucket: provider %s has type %q, not bitbucket", p.ID, p.Type) + } + if hc == nil { + return nil, errors.New("bitbucket: no HTTP client") + } + api, apiURL, err := baseURL("api_url", p.APIURL) + if err != nil { + return nil, fmt.Errorf("bitbucket: provider %s: %w", p.ID, err) + } + web, webURL, err := baseURL("url", p.URL) + if err != nil { + return nil, fmt.Errorf("bitbucket: provider %s: %w", p.ID, err) + } + host := strings.ToLower(p.Host) + if host == "" { + host = strings.ToLower(webURL.Host) + } + c := &client{ + http: hc, + api: api, + apiURL: apiURL, + web: web, + host: host, + masks: redact.New(), + commits: map[string]string{}, + } + switch cred.Kind { + case 0: + if cred.Token != "" || cred.AppID != "" || len(cred.AppKey) > 0 { + return nil, errors.New("bitbucket: a credential without a kind") + } + case auth.Token: + if cred.Token == "" { + return nil, errors.New("bitbucket: an empty token") + } + c.token = cred.Token + c.masks.Add(cred.Token, basicUser) + c.auth = &httpx.Auth{Hosts: []string{strings.ToLower(apiURL.Host)}, Header: c.apiHeader} + case auth.App: + return nil, fmt.Errorf("bitbucket: provider %s: a GitHub App cannot sign in to Bitbucket; use an API token of a bot account", p.ID) + default: + return nil, fmt.Errorf("bitbucket: provider %s: unknown credential kind %d", p.ID, cred.Kind) + } + return c, nil +} + +// baseURL checks an http(s) base URL without credentials, query or +// fragment, and returns it without a trailing slash. +func baseURL(what, raw string) (string, *url.URL, error) { + s := strings.TrimRight(raw, "/") + u, err := url.Parse(s) + switch { + case s == "": + return "", nil, fmt.Errorf("%s is empty", what) + case err != nil: + return "", nil, fmt.Errorf("%s does not parse", what) + case u.Scheme != "https" && u.Scheme != "http": + return "", nil, fmt.Errorf("%s is not an http or https URL", what) + case u.Host == "" || u.User != nil || u.RawQuery != "" || u.Fragment != "": + return "", nil, fmt.Errorf("%s must be a URL with a host and without credentials, query or fragment", what) + } + return s, u, nil +} + +// apiHeader is the Authorization header of API requests: an API token is +// a bearer token of the REST API (REST intro, "API tokens"). +func (c *client) apiHeader(context.Context) (string, error) { return "Bearer " + c.token, nil } + +// gitHeader is the Authorization header of git over HTTPS. +func (c *client) gitHeader(ctx context.Context) (string, error) { + if err := ctx.Err(); err != nil { + return "", err + } + return "Basic " + base64.StdEncoding.EncodeToString([]byte(basicUser+":"+c.token)), nil +} + +// mask replaces the credential's forms in s. +func (c *client) mask(s string) string { return c.masks.Replace(s) } + +// endpoint joins the API base and path segments, each escaped. +func (c *client) endpoint(segments ...string) string { + var b strings.Builder + b.WriteString(c.api) + for _, s := range segments { + b.WriteByte('/') + b.WriteString(url.PathEscape(s)) + } + return b.String() +} + +// pathSegments escapes each segment of a slash-separated path (a file +// path, a branch name) and joins them with slashes, which the API reads as +// they are: the branch API's own links keep the slashes of a branch name +// ("…/refs/branches/issue-9.3/AUI-5343-assistive-class" in the OpenAPI +// description). +func pathSegments(p string) []string { return strings.Split(p, "/") } + +// call sends one API request of op and decodes a 2xx JSON response into +// out (nil skips decoding). Every error is a masked *platform.Error, but a +// canceled context's. +// +// A GET follows a redirect to the same origin below the API base, at most +// maxRedirects times, with the same credential (a renamed repository may +// answer with one; whether it does is to be confirmed live). httpx follows +// no redirect of an authenticated request itself. +func (c *client) call(ctx context.Context, op, method, u string, query url.Values, in, out any) (*httpx.Response, error) { + if len(query) > 0 { + u += "?" + query.Encode() + } + for hops := 0; ; hops++ { + resp, err := c.http.JSON(ctx, method, u, c.auth, in, out) + if err == nil { + return resp, nil + } + if method == http.MethodGet && hops < maxRedirects { + if next, ok := c.redirect(u, err); ok { + u = next + continue + } + } + return resp, c.apiError(op, err) + } +} + +// get is call with GET. +func (c *client) get(ctx context.Context, op, u string, query url.Values, out any) (*httpx.Response, error) { + return c.call(ctx, op, http.MethodGet, u, query, nil, out) +} + +// redirect returns where the redirect err answered to a request for from +// leads, when it is one to follow (see below). +func (c *client) redirect(from string, err error) (string, bool) { + var se *httpx.StatusError + if !errors.As(err, &se) { + return "", false + } + return c.follow(from, se.Status, se.Header) +} + +// follow returns where a response of status with header to a request for +// from leads, when it is a redirect to follow: a 301, 302, 303, 307 or 308 +// whose Location, resolved against from, is on the API's scheme, host and +// port, without credentials, and below the API base. +func (c *client) follow(from string, status int, header http.Header) (string, bool) { + switch status { + case http.StatusMovedPermanently, http.StatusFound, http.StatusSeeOther, + http.StatusTemporaryRedirect, http.StatusPermanentRedirect: + default: + return "", false + } + loc := strings.TrimSpace(header.Get("Location")) + base, perr := url.Parse(from) + if loc == "" || perr != nil { + return "", false + } + to, perr := base.Parse(loc) + if perr != nil || !c.belowAPI(to) { + return "", false + } + to.Fragment, to.RawFragment = "", "" + return to.String(), true +} + +// belowAPI reports whether u is on the API's scheme, host and port, without +// credentials, and below the API base. +func (c *client) belowAPI(u *url.URL) bool { + if u.User != nil || !sameOrigin(u, c.apiURL) { + return false + } + root := strings.TrimRight(c.apiURL.Path, "/") + "/" + return strings.HasPrefix(u.Path, root) && !strings.Contains(u.Path, "/../") && !strings.HasSuffix(u.Path, "/..") +} + +// sameOrigin reports whether a and b have the same scheme, host and port +// (the scheme's default when absent), ignoring case. +func sameOrigin(a, b *url.URL) bool { + port := func(u *url.URL) string { + if p := u.Port(); p != "" { + return p + } + if strings.EqualFold(u.Scheme, "https") { + return "443" + } + return "80" + } + return strings.EqualFold(a.Scheme, b.Scheme) && strings.EqualFold(a.Hostname(), b.Hostname()) && port(a) == port(b) +} + +// errOffsite says the API redirected a raw read away from itself: Git LFS +// content lives on Atlassian's media platform (the OpenAPI description of +// GET …/src/{commit}/{path}). +var errOffsite = errors.New("the content is served from another host") + +// raw reads the body of u, a GET outside JSON (a file's raw content). A +// redirect below the API base is followed as call follows it; one +// elsewhere is errOffsite. A non-2xx answer is classified as call +// classifies it. +func (c *client) raw(ctx context.Context, op, u string) ([]byte, error) { + for hops := 0; ; hops++ { + req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil) + if err != nil { + return nil, &platform.Error{Op: op, Class: platform.ClassInvalid, Err: c.masked(err)} + } + req.Header.Set("Accept", "*/*") + resp, err := c.http.Do(req, c.auth) + if err != nil { + return nil, c.apiError(op, err) + } + if resp.Status >= 200 && resp.Status <= 299 { + return resp.Body, nil + } + if next, ok := c.follow(u, resp.Status, resp.Header); ok && hops < maxRedirects { + u = next + continue + } + if resp.Status >= 300 && resp.Status <= 399 { + return nil, &platform.Error{Op: op, Class: platform.ClassUnknown, Status: resp.Status, + Err: fmt.Errorf("GET %s: %d: %w", c.mask(withoutQuery(u)), resp.Status, errOffsite)} + } + return nil, c.statusError(op, &httpx.StatusError{ + Method: http.MethodGet, URL: withoutQuery(u), Status: resp.Status, + Header: resp.Header, Snippet: snippet(resp.Body), + }) + } +} + +// withoutQuery returns u without its query and fragment, as httpx shows +// URLs in errors. +func withoutQuery(u string) string { + if i := strings.IndexAny(u, "?#"); i >= 0 { + return u[:i] + } + return u +} + +// snippet returns at most snippetLimit bytes of body on one line. +func snippet(body []byte) string { + if len(body) > snippetLimit { + body = body[:snippetLimit] + } + return strings.Join(strings.FieldsFunc(strings.ToValidUTF8(string(body), "?"), func(r rune) bool { + return unicode.IsSpace(r) || unicode.IsControl(r) + }), " ") +} + +// page is one page of a paginated collection (REST intro, "Pagination"): +// values, and next, the opaque link to the following page, absent on the +// last. +type page[T any] struct { + Values []T `json:"values"` + Next string `json:"next"` +} + +// listAll reads a paginated collection of T at u with query and calls each +// for every item, in order, following the next links, which must stay +// below the API base. complete is false when the listing was capped at +// maxPages. An error of a page after the first comes with the page in err +// (see pageError), so a caller can tell a failed listing from a failed +// first request. +func listAll[T any](ctx context.Context, c *client, op, u string, query url.Values, maxPages int, each func(T) error) (complete bool, err error) { + next := u + if len(query) > 0 { + next += "?" + query.Encode() + } + for n := 1; n <= maxPages; n++ { + var p page[T] + if _, err := c.get(ctx, op, next, nil, &p); err != nil { + if n > 1 { + return false, &pageError{page: n, err: err} + } + return false, err + } + for _, it := range p.Values { + if err := each(it); err != nil { + return false, err + } + } + if p.Next == "" { + return true, nil + } + to, perr := url.Parse(p.Next) + if perr != nil || !c.belowAPI(to) { + return false, shapeError(op, "a next page link outside the API: %s", c.mask(withoutQuery(p.Next))) + } + next = to.String() + } + return false, nil +} + +// pageError is the failure of a page after the first of a listing. +type pageError struct { + page int + err error +} + +func (e *pageError) Error() string { return fmt.Sprintf("page %d: %v", e.page, e.err) } +func (e *pageError) Unwrap() error { return e.err } + +// laterPage reports whether err is the failure of a page after the first. +func laterPage(err error) bool { + var pe *pageError + return errors.As(err, &pe) +} + +// splitRepoPath splits "workspace/slug". Bitbucket has no nested +// namespaces: a path of another shape cannot name a repository (ok false). +func splitRepoPath(p string) (workspace, slug string, ok bool) { + workspace, slug, found := strings.Cut(p, "/") + if !found || workspace == "" || slug == "" || strings.Contains(slug, "/") || + strings.ContainsAny(p, "\x00?#") || workspace == "." || workspace == ".." || slug == "." || slug == ".." { + return "", "", false + } + return workspace, slug, true +} + +// repoPath returns the workspace and slug of r, or a ClassInvalid error for +// op. +func repoPath(op string, r platform.Repo) (workspace, slug string, err error) { + workspace, slug, ok := splitRepoPath(r.Path) + if !ok { + return "", "", invalid(op, "%q is not a workspace/repository path", r.Path) + } + return workspace, slug, nil +} diff --git a/internal/platform/bitbucket/errors.go b/internal/platform/bitbucket/errors.go new file mode 100644 index 0000000..bc4f690 --- /dev/null +++ b/internal/platform/bitbucket/errors.go @@ -0,0 +1,362 @@ +package bitbucket + +import ( + "context" + "crypto/tls" + "crypto/x509" + "encoding/json" + "errors" + "fmt" + "maps" + "math" + "net/http" + "slices" + "strconv" + "strings" + "time" + "unicode" + + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +// maxRetryAfter bounds a wait read from response headers: a broken or +// hostile header must not park the provider for days. +const maxRetryAfter = time.Hour + +// now is the clock of Retry-After dates and reset times (tests replace it). +var now = time.Now + +// maskedError carries a masked message over the original error, which +// errors.Is and errors.As still see. +type maskedError struct { + msg string + err error +} + +func (e *maskedError) Error() string { return e.msg } +func (e *maskedError) Unwrap() error { return e.err } + +// masked returns err with its message masked. +func (c *client) masked(err error) error { + return &maskedError{msg: c.mask(err.Error()), err: err} +} + +// apiError classifies err, the failure of an API request of op, into a +// platform.Error: a non-2xx status by its code, a timeout as transient, a +// request httpx refused as invalid, a certificate the client does not +// trust, a response over the size bound or of an unexpected shape as +// unknown (none of them gets better by retrying), any other transport +// failure as transient. A canceled context stays unclassified. Messages +// never hold the credential nor headers. +func (c *client) apiError(op string, err error) error { + var se *httpx.StatusError + if errors.As(err, &se) { + return c.statusError(op, se) + } + class := platform.ClassTransient + switch { + case errors.Is(err, context.Canceled): + return fmt.Errorf("%s: %w", op, c.masked(err)) + case errors.Is(err, context.DeadlineExceeded): + case errors.Is(err, httpx.ErrRefused): + class = platform.ClassInvalid + case errors.Is(err, httpx.ErrTooLarge), certificateError(err), decodeError(err): + class = platform.ClassUnknown + } + return &platform.Error{Op: op, Class: class, Err: c.masked(err)} +} + +// certificateError reports whether err is a TLS certificate failure. +func certificateError(err error) bool { + var ( + verify *tls.CertificateVerificationError + unknown x509.UnknownAuthorityError + invalid x509.CertificateInvalidError + hostError x509.HostnameError + ) + return errors.As(err, &verify) || errors.As(err, &unknown) || errors.As(err, &invalid) || errors.As(err, &hostError) +} + +// decodeError reports whether err is a JSON body that does not decode. +func decodeError(err error) bool { + var ( + syntax *json.SyntaxError + typ *json.UnmarshalTypeError + ) + return errors.As(err, &syntax) || errors.As(err, &typ) || errors.Is(err, errShape) +} + +// errShape is wrapped by errors about responses that decode but lack what +// the driver depends on. +var errShape = errors.New("unexpected response") + +// shapeError is an error of op about a response that lacks what the driver +// depends on: the API changed shape. +func shapeError(op, format string, args ...any) error { + return &platform.Error{Op: op, Class: platform.ClassUnknown, + Err: fmt.Errorf("%w: "+format, append([]any{errShape}, args...)...)} +} + +// statusError classifies a non-2xx response of op. +// +// Bitbucket's error body is {"type": "error", "error": {"message": …, +// "detail": …, "data": {…}}} (the error schema of the OpenAPI description; +// a missing repository answers "You may not have access to this repository +// or it no longer exists in this workspace. …", a missing file "No such +// file or directory: ", both with 404, seen anonymously). A rate +// limit is 429, or a message about a rate limit ("Rate limit for this +// resource has been exceeded"), with x-ratelimit-limit, -remaining and +// -reset (seconds) headers (support.atlassian.com/bitbucket-cloud, "API +// request limits"). +func (c *client) statusError(op string, se *httpx.StatusError) error { + m := parseMessage(se.Snippet) + detail := m.text() + if detail == "" { + detail = se.Snippet + } + msg := c.mask(describe(se, detail)) + e := &platform.Error{Op: op, Class: statusClass(se.Status), Status: se.Status} + switch { + case se.Status == http.StatusNotFound || se.Status == http.StatusGone: + e.Err = ¬FoundError{message: m.Error.Message, err: fmt.Errorf("%s: %w", msg, platform.ErrNotFound)} + return e + case se.Status == http.StatusTooManyRequests, se.Status == http.StatusServiceUnavailable: + e.RetryAfter = retryAfter(se.Header, now()) + case limitMessage(detail) || se.Status == http.StatusForbidden && rateLimited(se.Header): + // A limit answered with another status, by its message or a proxy's + // headers. + e.Class, e.RetryAfter = platform.ClassRateLimited, retryAfter(se.Header, now()) + } + e.Err = errors.New(msg) + return e +} + +// notFoundError is the error of a 404 (or 410): Bitbucket's message ("" when +// the body is no Bitbucket error), over the ErrNotFound it wraps. +type notFoundError struct { + message string + err error +} + +func (e *notFoundError) Error() string { return e.err.Error() } +func (e *notFoundError) Unwrap() error { return e.err } + +// missingMessage returns Bitbucket's message of a 404 in err; ok is false +// when err is no 404 of the API. +func missingMessage(err error) (message string, ok bool) { + var nf *notFoundError + if !errors.As(err, &nf) { + return "", false + } + return nf.message, true +} + +// statusClass maps an HTTP status to the class the core reacts to. +func statusClass(status int) platform.Class { + switch { + case status == http.StatusUnauthorized: + return platform.ClassAuth + case status == http.StatusForbidden: + return platform.ClassPermission + case status == http.StatusNotFound, status == http.StatusGone: + // 410: a deleted workspace (the 410 of GET /repositories/{workspace}). + return platform.ClassNotFound + case status == http.StatusConflict: + return platform.ClassConflict + case status == http.StatusTooManyRequests: + return platform.ClassRateLimited + case status == http.StatusRequestTimeout, status == http.StatusTooEarly: + return platform.ClassTransient + case status == http.StatusNotImplemented: + return platform.ClassUnsupported + case status >= 500: + // 555 is the API's own timeout of a deep directory listing. + return platform.ClassTransient + case status >= 400: + return platform.ClassInvalid + } + // An unexpected redirect or informational status. + return platform.ClassUnknown +} + +// apiMessage is Bitbucket's error body. +type apiMessage struct { + Type string `json:"type"` + Error struct { + Message string `json:"message"` + Detail string `json:"detail"` + // Fields are messages about fields of a write's request, by field, + // each a list of messages or one ({"reviewers": ["Malformed + // reviewers list"]}, as Renovate reads them; the OpenAPI description + // leaves them out). + Fields map[string]json.RawMessage `json:"fields"` + } `json:"error"` +} + +// text returns the message, its detail and the messages of its fields +// ("reviewers: Malformed reviewers list"), by field name, on one line. +func (m apiMessage) text() string { + parts := make([]string, 0, 2+len(m.Error.Fields)) + if m.Error.Message != "" { + parts = append(parts, m.Error.Message) + } + if m.Error.Detail != "" && m.Error.Detail != m.Error.Message { + parts = append(parts, m.Error.Detail) + } + for _, k := range slices.Sorted(maps.Keys(m.Error.Fields)) { + var list []string + if json.Unmarshal(m.Error.Fields[k], &list) != nil { + var one string + if json.Unmarshal(m.Error.Fields[k], &one) != nil { + continue + } + list = []string{one} + } + parts = append(parts, k+": "+strings.Join(list, ", ")) + } + return strings.Join(strings.FieldsFunc(strings.Join(parts, "; "), func(r rune) bool { + return unicode.IsSpace(r) || unicode.IsControl(r) + }), " ") +} + +// parseMessage decodes an error body; a body of another shape gives the +// zero apiMessage. +func parseMessage(snippet string) apiMessage { + var m apiMessage + if json.Unmarshal([]byte(snippet), &m) != nil { + return apiMessage{} + } + return m +} + +// describe returns "METHOD URL: status text: detail". +func describe(se *httpx.StatusError, detail string) string { + var b strings.Builder + fmt.Fprintf(&b, "%s %s: %d", se.Method, se.URL, se.Status) + if text := http.StatusText(se.Status); text != "" { + b.WriteString(" " + text) + } + if detail != "" { + b.WriteString(": " + detail) + } + return b.String() +} + +// limitMessage reports whether an error message says a rate limit was hit. +func limitMessage(s string) bool { + s = strings.ToLower(s) + return strings.Contains(s, "rate limit") && strings.Contains(s, "exceeded") +} + +// rateLimited reports whether a response's headers say a rate limit was +// hit: a Retry-After, or no requests left in X-RateLimit-Remaining. +func rateLimited(h http.Header) bool { + return h.Get("Retry-After") != "" || strings.TrimSpace(h.Get("X-RateLimit-Remaining")) == "0" +} + +// retryAfter reads how long to wait from a response's headers, in order: +// Retry-After (seconds or an HTTP date), then X-RateLimit-Reset, which +// Bitbucket gives in seconds until the window resets (a Unix time when it +// is large, as other platforms send it). It is 0 when none is usable, and +// at most maxRetryAfter. +func retryAfter(h http.Header, at time.Time) time.Duration { + d, ok := time.Duration(0), false + if v := strings.TrimSpace(h.Get("Retry-After")); v != "" { + if secs, err := strconv.ParseInt(v, 10, 64); err == nil { + d, ok = seconds(secs), true + } else if t, err := http.ParseTime(v); err == nil { + d, ok = t.Sub(at), true + } + } + if !ok { + if n, err := strconv.ParseInt(strings.TrimSpace(h.Get("X-RateLimit-Reset")), 10, 64); err == nil { + // A Unix time is far above any sensible number of seconds. + if n > 1_000_000_000 { + d, ok = time.Unix(n, 0).Sub(at), true + } else { + d, ok = seconds(n), true + } + } + } + switch { + case !ok || d < 0: + return 0 + case d > maxRetryAfter: + return maxRetryAfter + } + return d +} + +// seconds converts a count of seconds, saturating instead of overflowing. +func seconds(n int64) time.Duration { + if n < 0 { + return -1 + } + if n > math.MaxInt64/int64(time.Second) { + return maxRetryAfter + } + return time.Duration(n) * time.Second +} + +// closedError is the error of a TargetWriter used after Close. +func closedError(op string) error { + return &platform.Error{Op: op, Class: platform.ClassAuth, Err: errors.New("the target writer is closed")} +} + +// invalid is a ClassInvalid error of op: the request was wrong. +func invalid(op, format string, args ...any) error { + return &platform.Error{Op: op, Class: platform.ClassInvalid, Err: fmt.Errorf(format, args...)} +} + +// notFound is a ClassNotFound error of op that wraps ErrNotFound. +func notFound(op, format string, args ...any) error { + return &platform.Error{Op: op, Class: platform.ClassNotFound, Err: fmt.Errorf(format+": %w", append(args, platform.ErrNotFound)...)} +} + +// unknown is a ClassUnknown error of op that says what happened (format +// and args) and, when err is not nil, what err says, with err's status but +// without the ErrNotFound err may wrap: a 404 that must not count as a +// missing file. +func unknown(op string, err error, format string, args ...any) error { + msg := fmt.Sprintf(format, args...) + if err != nil { + msg += ": " + detail(err) + } + return &platform.Error{Op: op, Class: platform.ClassUnknown, Status: statusOf(err), Err: errors.New(msg)} +} + +// detail returns what err says without its class: the message of the +// *platform.Error in it, without the ErrNotFound suffix. +func detail(err error) string { + var pe *platform.Error + if errors.As(err, &pe) && pe.Err != nil { + return strings.TrimSuffix(pe.Err.Error(), ": "+platform.ErrNotFound.Error()) + } + return err.Error() +} + +// statusOf returns the HTTP status of a classified error, 0 for none. +func statusOf(err error) int { + var pe *platform.Error + if errors.As(err, &pe) { + return pe.Status + } + return 0 +} + +// stops reports whether err ends a call at once, whatever it already +// found: the context ended, the platform limits the rate or refuses the +// credential. +func stops(err error) bool { + if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) { + return true + } + c := platform.ClassOf(err) + return c == platform.ClassRateLimited || c == platform.ClassAuth +} + +// fatal reports whether err must fail a listing instead of making it +// incomplete: it stops the call, or the platform failed for now (the caller +// retries the whole listing). +func fatal(err error) bool { return stops(err) || platform.ClassOf(err) == platform.ClassTransient } diff --git a/internal/platform/bitbucket/errors_test.go b/internal/platform/bitbucket/errors_test.go new file mode 100644 index 0000000..82c8619 --- /dev/null +++ b/internal/platform/bitbucket/errors_test.go @@ -0,0 +1,165 @@ +package bitbucket + +import ( + "encoding/base64" + "net/http" + "net/url" + "strconv" + "strings" + "testing" + "time" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +func TestStatusClasses(t *testing.T) { + f := newFixture(t) + for _, tc := range []struct { + status int + header map[string]string + msg string + class platform.Class + wait time.Duration + }{ + {http.StatusBadRequest, nil, "Field \".summary.raw\" does not support filtering", platform.ClassInvalid, 0}, + {http.StatusUnauthorized, nil, "Token is invalid, expired, or not supported for this endpoint.", platform.ClassAuth, 0}, + {http.StatusForbidden, nil, "Your credentials lack one or more required privilege scopes.", platform.ClassPermission, 0}, + {http.StatusForbidden, nil, "Rate limit for this resource has been exceeded", platform.ClassRateLimited, 0}, + {http.StatusForbidden, map[string]string{"X-RateLimit-Remaining": "0", "X-RateLimit-Reset": "60"}, "Forbidden", platform.ClassRateLimited, time.Minute}, + {http.StatusNotFound, nil, noRepoMessage, platform.ClassNotFound, 0}, + {http.StatusConflict, nil, "conflict", platform.ClassConflict, 0}, + {http.StatusGone, nil, "This workspace has been deleted", platform.ClassNotFound, 0}, + {http.StatusTooManyRequests, map[string]string{"X-RateLimit-Limit": "1000", "X-RateLimit-Remaining": "0", "X-RateLimit-Reset": "394"}, + "Rate limit for this resource has been exceeded", platform.ClassRateLimited, 394 * time.Second}, + {http.StatusTooManyRequests, map[string]string{"Retry-After": "30", "X-RateLimit-Reset": "394"}, "Too many", platform.ClassRateLimited, 30 * time.Second}, + {http.StatusTooManyRequests, map[string]string{"X-RateLimit-Reset": "999999"}, "Too many", platform.ClassRateLimited, time.Hour}, + {http.StatusInternalServerError, nil, "Something went wrong", platform.ClassTransient, 0}, + {http.StatusServiceUnavailable, map[string]string{"Retry-After": "5"}, "Unavailable", platform.ClassTransient, 5 * time.Second}, + {555, nil, "timeout", platform.ClassTransient, 0}, + {http.StatusNotImplemented, nil, "no", platform.ClassUnsupported, 0}, + } { + path := "/repositories/acme/s" + strconv.Itoa(tc.status) + strconv.Itoa(len(tc.header)) + f.handle(http.MethodGet, path, func(w http.ResponseWriter, _ *http.Request) { + for k, v := range tc.header { + w.Header().Set(k, v) + } + writeJSON(w, tc.status, errorBody(tc.msg)) + }) + _, err := f.reader.Repo(t.Context(), strings.TrimPrefix(path, "/repositories/")) + what := strconv.Itoa(tc.status) + " " + tc.msg + wantClass(t, what, err, tc.class, nil) + if err == nil { + continue + } + if !strings.Contains(err.Error(), tc.msg) { + t.Errorf("%s: the error %q lacks the API's message", what, err) + } + if d := retryAfterOf(err); d != tc.wait { + t.Errorf("%s: RetryAfter %v, want %v", what, d, tc.wait) + } + if tc.status == http.StatusNotFound && !isNotFound(err) { + t.Errorf("%s: %v is no ErrNotFound", what, err) + } + } +} + +func TestTransportErrors(t *testing.T) { + t.Run("a body that does not decode", func(t *testing.T) { + f := newFixture(t) + f.handle(http.MethodGet, "/repositories/acme/api", func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte("maintenance")) + }) + _, err := f.reader.Repo(t.Context(), "acme/api") + wantClass(t, "HTML instead of JSON", err, platform.ClassUnknown, nil) + }) + t.Run("a server that is gone", func(t *testing.T) { + f := newFixture(t) + f.srv.Close() + _, err := f.reader.Repo(t.Context(), "acme/api") + wantClass(t, "a closed server", err, platform.ClassTransient, nil) + }) +} + +// TestTokenMasked: the token never shows in an error, raw or in git's Basic +// form, even when the API echoes it. +func TestTokenMasked(t *testing.T) { + f := newFixture(t) + basic := base64.StdEncoding.EncodeToString([]byte("x-bitbucket-api-token-auth:" + f.token)) + echo := "token " + f.token + " basic " + basic + " escaped " + url.QueryEscape(f.token) + f.json("/repositories/acme/api", http.StatusUnauthorized, errorBody(echo)) + f.json("/user", http.StatusForbidden, map[string]any{"type": "error", "error": map[string]any{"message": "no", "detail": echo}}) + f.handle(http.MethodGet, "/repositories/acme/api/src/"+headCommit+"/"+optIn, func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("format") == "meta" { + writeJSON(w, http.StatusOK, map[string]any{"path": optIn, "type": "commit_file", "attributes": []string{}, "size": 3}) + return + } + w.WriteHeader(http.StatusInternalServerError) + _, _ = w.Write([]byte("upstream said " + echo)) + }) + _, err1 := f.reader.Repo(t.Context(), "acme/api") + _, err2 := f.reader.Self(t.Context()) + _, err3 := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 100) + for i, err := range []error{err1, err2, err3} { + if err == nil { + t.Fatalf("error %d: none", i) + } + msg := err.Error() + if strings.Contains(msg, f.token) || strings.Contains(msg, basic) || strings.Contains(msg, url.QueryEscape(f.token)) { + t.Errorf("error %d shows the token: %s", i, msg) + } + if !strings.Contains(msg, "***") { + t.Errorf("error %d: %s; want the token masked", i, msg) + } + } + wantClass(t, "a raw read that fails", err3, platform.ClassTransient, nil) +} + +func TestRedirects(t *testing.T) { + f := newFixture(t) + f.handle(http.MethodGet, "/repositories/acme/old-name", func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, "/2.0/repositories/acme/api", http.StatusMovedPermanently) + }) + f.json("/repositories/acme/api", http.StatusOK, repo(repoUUID, "acme/api")) + got, err := f.reader.Repo(t.Context(), "acme/old-name") + if err != nil || got.Path != "acme/api" || got.ID != repoUUID { + t.Errorf("Repo(old name) = %+v, %v; want the repository under its canonical path", got, err) + } + if calls := f.requests(http.MethodGet, "/repositories/acme/api"); len(calls) != 1 || calls[0].Auth != "Bearer "+f.token { + t.Errorf("calls %+v; want the redirect followed with the credential", calls) + } + f.handle(http.MethodGet, "/repositories/acme/away", func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, "https://evil.example.com/2.0/repositories/acme/api", http.StatusFound) + }) + f.handle(http.MethodGet, "/repositories/acme/up", func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, "/elsewhere", http.StatusFound) + }) + for _, p := range []string{"acme/away", "acme/up"} { + _, err := f.reader.Repo(t.Context(), p) + wantClass(t, p, err, platform.ClassUnknown, nil) + } +} + +func TestBlobID(t *testing.T) { + // git hash-object of "hello\n" and of an empty file. + for content, want := range map[string]string{ + "hello\n": "ce013625030ba8dba906f756967f9e9ca394464a", + "": "e69de29bb2d1d6434b8b29ae775ad8c2e48c5391", + } { + if got := blobID([]byte(content)); got != want { + t.Errorf("blobID(%q) = %s, want %s", content, got, want) + } + } +} + +// TestMessageFields: the messages of a write's fields join the message, by +// field name, whether a field has a list of messages or one. +func TestMessageFields(t *testing.T) { + m := parseMessage(`{"type": "error", "error": {"message": "Bad request", "fields": {` + + `"reviewers": ["Malformed reviewers list", "x is the author and cannot be included as a reviewer."], ` + + `"destination": "branch not found", "odd": {"a": 1}}}}`) + want := "Bad request; destination: branch not found; reviewers: Malformed reviewers list, x is the author and cannot be included as a reviewer." + if got := m.text(); got != want { + t.Errorf("text = %q\nwant %q", got, want) + } +} diff --git a/internal/platform/bitbucket/files.go b/internal/platform/bitbucket/files.go new file mode 100644 index 0000000..828c476 --- /dev/null +++ b/internal/platform/bitbucket/files.go @@ -0,0 +1,296 @@ +package bitbucket + +import ( + "bytes" + "cmp" + "context" + "crypto/sha1" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "net/url" + "slices" + "strings" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// File modes of regular files. +const ( + modeFile = "100644" + modeExecutable = "100755" +) + +// missingFileMessage starts the message of a 404 of the source API for a +// path that a commit lacks ("No such file or directory: ", seen +// anonymously); a missing repository or commit says something else. +const missingFileMessage = "no such file or directory" + +// apiEntry is GET …/src/{commit}/{path}?format=meta: a commit_file or a +// commit_directory. +type apiEntry struct { + Type string `json:"type"` + Path string `json:"path"` + Size *int64 `json:"size"` + Attributes attributes `json:"attributes"` +} + +// attributes are a file's modifiers: link, executable, subrepository, +// binary, lfs. The OpenAPI description types them as one string; the API +// sends a list ([] for a plain file), so both are read. +type attributes []string + +func (a *attributes) UnmarshalJSON(b []byte) error { + b = bytes.TrimSpace(b) + switch { + case bytes.Equal(b, []byte("null")): + *a = nil + return nil + case len(b) > 0 && b[0] == '"': + var s string + if err := json.Unmarshal(b, &s); err != nil { + return err + } + *a = attributes{s} + return nil + } + var list []string + if err := json.Unmarshal(b, &list); err != nil { + return err + } + *a = list + return nil +} + +// has reports whether a holds the attribute name. +func (a attributes) has(name string) bool { return slices.Contains(a, name) } + +// ReadFile returns a regular file of r at ref ("" is the default branch +// head): its kind, size and mode from the source API's metadata, then its +// raw bytes, at the commit ref resolves to, so that both answers describe +// the same tree. The blob id is computed here (the API gives none). +// Symlinks, submodules, directories and Git LFS files are ErrNotRegular, a +// file over max bytes ErrTooLarge. +// +// Only a path that an existing commit lacks is ErrNotFound: the core takes a +// missing opt-in file for an opt-out and closes the target's pull requests. +// A missing repository, ref or commit, a 404 the source API gives for +// anything else, and a file that the metadata shows and the raw read does +// not find are ClassUnknown. An empty repository has no file: ErrNotFound. +func (d *reader) ReadFile(ctx context.Context, r platform.Repo, ref, filePath string, max int64) (platform.File, error) { + const op = "read file" + ws, slug, err := repoPath(op, r) + if err != nil { + return platform.File{}, err + } + if err := checkTreePath(filePath); err != nil { + return platform.File{}, invalid(op, "%v", err) + } + if max < 0 { + return platform.File{}, invalid(op, "negative size limit %d", max) + } + commit, err := d.c.commitOf(ctx, op, r, ws, slug, ref) + if err != nil { + return platform.File{}, err + } + e, err := d.c.meta(ctx, op, ws, slug, commit, filePath) + switch { + case err != nil && platform.ClassOf(err) == platform.ClassNotFound: + return platform.File{}, d.c.missingPath(ctx, op, r, ws, slug, commit, filePath, err) + case err != nil: + return platform.File{}, err + } + what := "" + switch { + case e.Type == "commit_directory": + what = "a directory" + case e.Type != "commit_file": + what = "a " + cmp.Or(e.Type, "thing without a type") + case e.Attributes.has("link"): + what = "a symlink" + case e.Attributes.has("subrepository"): + what = "a submodule" + case e.Attributes.has("lfs"): + what = "a Git LFS file" + } + if what != "" { + return platform.File{}, fmt.Errorf("%s: %s: %s is %s: %w", op, r.Path, filePath, what, platform.ErrNotRegular) + } + switch { + case e.Path != "" && e.Path != filePath: + return platform.File{}, shapeError(op, "%s: the metadata describes %s", filePath, e.Path) + case e.Size == nil || *e.Size < 0: + return platform.File{}, shapeError(op, "%s: the metadata has no size", filePath) + case *e.Size > max: + return platform.File{}, fmt.Errorf("%s: %s: %s has %d bytes, more than %d: %w", op, r.Path, filePath, *e.Size, max, platform.ErrTooLarge) + } + content, err := d.c.raw(ctx, op, d.c.srcURL(ws, slug, commit, filePath)) + switch { + case errors.Is(err, errOffsite): + return platform.File{}, fmt.Errorf("%s: %s: %s: the API serves its content from another host, as it serves Git LFS files: %w", + op, r.Path, filePath, platform.ErrNotRegular) + case err != nil && platform.ClassOf(err) == platform.ClassNotFound: + // Never ErrNotFound: the metadata of the same commit has just shown it. + return platform.File{}, unknown(op, err, "%s: %s at %s: the metadata shows it, its content is not found", r.Path, filePath, commit) + case err != nil: + return platform.File{}, err + case int64(len(content)) != *e.Size: + return platform.File{}, shapeError(op, "%s: %d bytes of content, the metadata says %d", filePath, len(content), *e.Size) + } + mode := modeFile + if e.Attributes.has("executable") { + mode = modeExecutable + } + return platform.File{Path: filePath, Mode: mode, OID: blobID(content), Content: content}, nil +} + +// commitOf resolves ref of r to a full commit id: a full id is taken as it +// is (missingPath checks that it exists when a path is missing), a branch +// through the branch API, and "" through the default branch. A ref the API +// does not find, or a repository it does not find, is ClassUnknown: never a +// missing file. A repository without a main branch is empty: ErrNotFound. +// When the default branch r names is gone, the repository's current one is +// read once more (the default branch was renamed in between). +func (c *client) commitOf(ctx context.Context, op string, r platform.Repo, ws, slug, ref string) (string, error) { + if isHexOID(ref) { + return strings.ToLower(ref), nil + } + if ref != "" { + hash, err := c.branchHead(ctx, op, ws, slug, ref) + if err != nil && platform.ClassOf(err) == platform.ClassNotFound { + return "", unknown(op, err, "%s: ref %s is not found", r.Path, ref) + } + return hash, err + } + branch := r.DefaultBranch + for range 2 { + if branch != "" { + hash, err := c.branchHead(ctx, op, ws, slug, branch) + if err == nil || platform.ClassOf(err) != platform.ClassNotFound { + return hash, err + } + } + // No default branch, or not where r says: the repository tells. + repo, err := c.getRepo(ctx, op, ws, slug) + switch { + case err != nil && platform.ClassOf(err) == platform.ClassNotFound: + return "", unknown(op, err, "%s: the repository is not found", r.Path) + case err != nil: + return "", err + case repo.MainBranch == nil || repo.MainBranch.Name == "": + return "", notFound(op, "%s: the repository is empty", r.Path) + case repo.MainBranch.Name == branch: + return "", unknown(op, nil, "%s: the branch API does not find the default branch %s", r.Path, branch) + } + branch = repo.MainBranch.Name + } + return "", unknown(op, nil, "%s: the default branch moved while it was read", r.Path) +} + +// branchHead returns the head commit of branch in ws/slug: GET +// …/refs/branches/{name}, the name's slashes kept. +func (c *client) branchHead(ctx context.Context, op, ws, slug, branch string) (string, error) { + var b apiBranch + if _, err := c.get(ctx, op, c.endpoint(append([]string{"repositories", ws, slug, "refs", "branches"}, pathSegments(branch)...)...), nil, &b); err != nil { + return "", err + } + if b.Target == nil || !isHexOID(b.Target.Hash) { + return "", shapeError(op, "branch %s: no head commit", branch) + } + return strings.ToLower(b.Target.Hash), nil +} + +// srcURL is the source API's URL of filePath at commit. +func (c *client) srcURL(ws, slug, commit, filePath string) string { + return c.endpoint(append([]string{"repositories", ws, slug, "src", commit}, pathSegments(filePath)...)...) +} + +// meta reads the metadata of filePath at commit. +func (c *client) meta(ctx context.Context, op, ws, slug, commit, filePath string) (*apiEntry, error) { + var e apiEntry + if _, err := c.get(ctx, op, c.srcURL(ws, slug, commit, filePath), url.Values{"format": {"meta"}}, &e); err != nil { + return nil, err + } + return &e, nil +} + +// missingPath judges err, a 404 of the metadata of filePath at commit. A +// path is missing only at a commit that exists: the 404 says so ("No such +// file or directory"), or the commit API finds the commit; else the +// repository or the commit is gone, which is ClassUnknown. A missing path +// is ErrNotFound, but one that leads through a symlink, which git never +// follows, is ErrNotRegular, as on the other platforms: the directories on +// the way are read until one is missing or no directory. +func (c *client) missingPath(ctx context.Context, op string, r platform.Repo, ws, slug, commit, filePath string, err error) error { + msg, _ := missingMessage(err) + if !strings.HasPrefix(strings.ToLower(msg), missingFileMessage) { + _, cerr := c.get(ctx, op, c.endpoint("repositories", ws, slug, "commit", commit), nil, nil) + switch { + case cerr != nil && platform.ClassOf(cerr) == platform.ClassNotFound: + return unknown(op, err, "%s: %s at %s: neither the file nor the commit is found", r.Path, filePath, commit) + case cerr != nil: + return cerr + } + } + segs := strings.Split(filePath, "/") + for i := 1; i < len(segs); i++ { + dir := strings.Join(segs[:i], "/") + e, derr := c.meta(ctx, op, ws, slug, commit, dir) + switch { + case derr != nil && platform.ClassOf(derr) == platform.ClassNotFound: + return notFound(op, "%s: %s at %s: %s is missing", r.Path, filePath, commit, dir) + case derr != nil: + return derr + case e.Type == "commit_directory": + continue + case e.Type == "commit_file" && e.Attributes.has("link"): + return fmt.Errorf("%s: %s: %s leads through a symlink at %s: %w", op, r.Path, filePath, dir, platform.ErrNotRegular) + } + return notFound(op, "%s: %s at %s: %s is no directory", r.Path, filePath, commit, dir) + } + return notFound(op, "%s: %s at %s", r.Path, filePath, commit) +} + +// checkTreePath accepts a repository-relative path: no leading slash, no +// empty, "." or ".." segment, no NUL. +func checkTreePath(p string) error { + if p == "" { + return errors.New("empty path") + } + if strings.IndexByte(p, 0) >= 0 { + return fmt.Errorf("path %q contains NUL", p) + } + for seg := range strings.SplitSeq(p, "/") { + if seg == "" || seg == "." || seg == ".." { + return fmt.Errorf("path %q has an empty, \".\" or \"..\" segment", p) + } + } + return nil +} + +// blobID is git's SHA-1 id of a blob with content: Bitbucket hosts SHA-1 +// repositories only. +func blobID(content []byte) string { + h := sha1.New() + fmt.Fprintf(h, "blob %d\x00", len(content)) + h.Write(content) + return hex.EncodeToString(h.Sum(nil)) +} + +// isHexOID reports whether s is a full SHA-1 object id. +func isHexOID(s string) bool { return len(s) == 40 && isHex(s) } + +// isHex reports whether s is a non-empty string of hexadecimal digits. +func isHex(s string) bool { + if s == "" { + return false + } + for i := 0; i < len(s); i++ { + c := s[i] + if (c < '0' || c > '9') && (c < 'a' || c > 'f') && (c < 'A' || c > 'F') { + return false + } + } + return true +} diff --git a/internal/platform/bitbucket/files_test.go b/internal/platform/bitbucket/files_test.go new file mode 100644 index 0000000..c3f96a8 --- /dev/null +++ b/internal/platform/bitbucket/files_test.go @@ -0,0 +1,322 @@ +package bitbucket + +import ( + "crypto/sha1" + "encoding/hex" + "fmt" + "net/http" + "strings" + "testing" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// headCommit is the head of the default branch of acme/api in these tests. +const headCommit = "be4cdd0b1f9566847c274d0d05b9549881e0ae4a" + +// optIn is the opt-in file the tests read. +const optIn = ".engineering-assets.yml" + +// apiRepoFixture is the repository the file tests read. +var apiRepoFixture = platform.Repo{ID: repoUUID, Path: "acme/api", DefaultBranch: "main", ObjectFormat: "sha1"} + +// branch declares GET …/refs/branches/{name} of acme/api with its head. +func (s *apiServer) branch(name, hash string) { + s.json("/repositories/acme/api/refs/branches/"+name, http.StatusOK, map[string]any{ + "name": name, "type": "branch", "default_merge_strategy": "merge_commit", + "merge_strategies": []string{"merge_commit", "squash", "fast_forward"}, + "target": map[string]any{"hash": hash, "type": "commit", "date": "2026-09-30T10:00:00+00:00", + "message": "chore: sync", "parents": []any{}}, + "links": map[string]any{"html": map[string]any{"href": "https://bitbucket.org/acme/api/branch/" + name}}, + }) +} + +// srcLinks are the links of an entry of the source API. +func srcLinks(commit, path string) map[string]any { + self := "https://api.bitbucket.org/2.0/repositories/acme/api/src/" + commit + "/" + path + return map[string]any{ + "self": map[string]any{"href": self}, + "meta": map[string]any{"href": self + "?format=meta"}, + } +} + +// file declares the metadata and the raw content of path at commit of +// acme/api: attributes as the API lists them, content as the raw read +// returns it (the metadata's size is its length unless size is set). +func (s *apiServer) file(commit, path string, attrs any, content string, size ...int) { + n := len(content) + if len(size) > 0 { + n = size[0] + } + s.handle(http.MethodGet, "/repositories/acme/api/src/"+commit+"/"+path, func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("format") == "meta" { + writeJSON(w, http.StatusOK, map[string]any{ + "path": path, "type": "commit_file", "attributes": attrs, "escaped_path": path, "size": n, + "mimetype": "text/plain", "has_malware_warning": false, "links": srcLinks(commit, path), + "commit": map[string]any{"hash": commit, "type": "commit"}, + }) + return + } + w.Header().Set("Content-Type", "text/plain") + w.Header().Set("Content-Disposition", "attachment") + _, _ = w.Write([]byte(content)) + }) +} + +// dir declares the metadata of a directory at commit of acme/api. +func (s *apiServer) dir(commit, path string) { + s.json("/repositories/acme/api/src/"+commit+"/"+path, http.StatusOK, map[string]any{ + "path": path, "type": "commit_directory", "links": srcLinks(commit, path+"/"), + "commit": map[string]any{"hash": commit, "type": "commit"}, + }) +} + +// missing declares a path that commit of acme/api lacks. +func (s *apiServer) missing(commit, path string) { + s.json("/repositories/acme/api/src/"+commit+"/"+path, http.StatusNotFound, errorBody(noFileMessage+path)) +} + +// gitBlob is git's id of a blob with content. +func gitBlob(content string) string { + h := sha1.New() + fmt.Fprintf(h, "blob %d\x00%s", len(content), content) + return hex.EncodeToString(h.Sum(nil)) +} + +func TestReadFile(t *testing.T) { + f := newFixture(t) + f.branch("main", headCommit) + content := "version: 1\npacks: [agents]\n" + f.file(headCommit, optIn, []string{}, content) + got, err := f.reader.ReadFile(t.Context(), apiRepoFixture, "", optIn, 1<<20) + if err != nil { + t.Fatal(err) + } + want := platform.File{Path: optIn, Mode: "100644", OID: gitBlob(content), Content: []byte(content)} + if got.Path != want.Path || got.Mode != want.Mode || got.OID != want.OID || string(got.Content) != content { + t.Errorf("ReadFile = %+v\nwant %+v", got, want) + } + calls := f.requests(http.MethodGet, "/repositories/acme/api/src/"+headCommit+"/"+optIn) + if len(calls) != 2 || calls[0].Query.Get("format") != "meta" || calls[1].Query.Get("format") != "" { + t.Errorf("calls %+v; want the metadata, then the raw content, at the branch's head commit", calls) + } +} + +func TestReadFileModes(t *testing.T) { + f := newFixture(t) + c := headCommit + f.file(c, "bin/run.sh", []string{"executable"}, "#!/bin/sh\n") + f.file(c, "string-attr", "executable", "x") + f.file(c, "binary", []string{"binary"}, "\x00\x01") + for path, mode := range map[string]string{"bin/run.sh": "100755", "string-attr": "100755", "binary": "100644"} { + got, err := f.reader.ReadFile(t.Context(), apiRepoFixture, c, path, 100) + if err != nil || got.Mode != mode { + t.Errorf("%s: %+v, %v; want mode %s", path, got, err, mode) + } + } + if calls := f.requests(http.MethodGet, "/repositories/acme/api/refs/branches/main"); len(calls) != 0 { + t.Error("a full commit id was resolved through the branch API") + } +} + +func TestReadFileNotRegular(t *testing.T) { + f := newFixture(t) + c := headCommit + f.file(c, "link", []string{"link"}, "target") + f.file(c, "sub", []string{"subrepository"}, "0123456789abcdef0123456789abcdef01234567") + f.file(c, "big.bin", []string{"lfs", "binary"}, "") + f.dir(c, "docs") + f.json("/repositories/acme/api/src/"+c+"/odd", http.StatusOK, map[string]any{"path": "odd", "type": "commit_symlink"}) + for _, path := range []string{"link", "sub", "big.bin", "docs", "odd"} { + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, c, path, 100) + wantClass(t, path, err, platform.ClassUnknown, platform.ErrNotRegular) + } + if calls := f.requests("", ""); len(calls) != 5 { + t.Errorf("%d requests; want the metadata only, never the content", len(calls)) + } +} + +func TestReadFileLFSRedirect(t *testing.T) { + f := newFixture(t) + f.handle(http.MethodGet, "/repositories/acme/api/src/"+headCommit+"/model.bin", func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("format") == "meta" { + writeJSON(w, http.StatusOK, map[string]any{"path": "model.bin", "type": "commit_file", "attributes": []string{}, "size": 3}) + return + } + http.Redirect(w, r, "https://api.media.atlassian.com/file/0123/binary", http.StatusFound) + }) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, "model.bin", 100) + wantClass(t, "a redirect elsewhere", err, platform.ClassUnknown, platform.ErrNotRegular) +} + +func TestReadFileTooLarge(t *testing.T) { + f := newFixture(t) + f.file(headCommit, optIn, []string{}, strings.Repeat("x", 11)) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 10) + wantClass(t, "over the limit", err, platform.ClassUnknown, platform.ErrTooLarge) + if calls := f.requests("", ""); len(calls) != 1 { + t.Errorf("%d requests; want the metadata only", len(calls)) + } + if _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 11); err != nil { + t.Errorf("at the limit: %v", err) + } +} + +func TestReadFileMissing(t *testing.T) { + t.Run("a missing file", func(t *testing.T) { + f := newFixture(t) + f.branch("main", headCommit) + f.missing(headCommit, optIn) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, "", optIn, 100) + wantClass(t, "a missing file", err, platform.ClassNotFound, platform.ErrNotFound) + if calls := f.requests("", ""); len(calls) != 2 { + t.Errorf("%d requests; want the branch and the metadata", len(calls)) + } + }) + t.Run("a missing nested file", func(t *testing.T) { + f := newFixture(t) + f.missing(headCommit, "a/b/c.yml") + f.dir(headCommit, "a") + f.missing(headCommit, "a/b") + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, "a/b/c.yml", 100) + wantClass(t, "a missing directory", err, platform.ClassNotFound, platform.ErrNotFound) + }) + t.Run("through a symlink", func(t *testing.T) { + f := newFixture(t) + f.missing(headCommit, "conf/touchmark.yml") + f.file(headCommit, "conf", []string{"link"}, "../elsewhere") + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, "conf/touchmark.yml", 100) + wantClass(t, "through a symlink", err, platform.ClassUnknown, platform.ErrNotRegular) + }) + t.Run("through a file", func(t *testing.T) { + f := newFixture(t) + f.missing(headCommit, "README.md/x") + f.file(headCommit, "README.md", []string{}, "# api\n") + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, "README.md/x", 100) + wantClass(t, "through a file", err, platform.ClassNotFound, platform.ErrNotFound) + }) + t.Run("another 404 at a commit that exists", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/src/"+headCommit+"/"+optIn, http.StatusNotFound, errorBody("Not found")) + f.json("/repositories/acme/api/commit/"+headCommit, http.StatusOK, map[string]any{"hash": headCommit, "type": "commit"}) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 100) + wantClass(t, "a commit that exists", err, platform.ClassNotFound, platform.ErrNotFound) + }) + t.Run("another 404 at a commit that is gone", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/src/"+headCommit+"/"+optIn, http.StatusNotFound, errorBody(noRepoMessage)) + f.json("/repositories/acme/api/commit/"+headCommit, http.StatusNotFound, errorBody(noRepoMessage)) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 100) + wantClass(t, "a commit that is gone", err, platform.ClassUnknown, nil) + if platform.ClassOf(err) == platform.ClassNotFound || isNotFound(err) { + t.Errorf("%v: a missing commit must never read as a missing file", err) + } + }) + t.Run("a missing repository", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/refs/branches/main", http.StatusNotFound, errorBody(noRepoMessage)) + f.json("/repositories/acme/api", http.StatusNotFound, errorBody(noRepoMessage)) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, "", optIn, 100) + wantClass(t, "a missing repository", err, platform.ClassUnknown, nil) + if isNotFound(err) { + t.Errorf("%v: a missing repository must never read as a missing file", err) + } + }) + t.Run("a missing ref", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/refs/branches/feature/x", http.StatusNotFound, errorBody("Branch \"feature/x\" not found")) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, "feature/x", optIn, 100) + wantClass(t, "a missing ref", err, platform.ClassUnknown, nil) + if isNotFound(err) { + t.Errorf("%v: a missing ref must never read as a missing file", err) + } + }) + t.Run("the default branch is gone", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/refs/branches/main", http.StatusNotFound, errorBody("Branch \"main\" not found")) + f.json("/repositories/acme/api", http.StatusOK, repo(repoUUID, "acme/api")) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, "", optIn, 100) + wantClass(t, "a default branch without a head", err, platform.ClassUnknown, nil) + if isNotFound(err) { + t.Errorf("%v: must never read as a missing file", err) + } + }) + t.Run("the default branch was renamed", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/refs/branches/master", http.StatusNotFound, errorBody("Branch \"master\" not found")) + f.json("/repositories/acme/api", http.StatusOK, repo(repoUUID, "acme/api")) + f.branch("main", headCommit) + f.file(headCommit, optIn, []string{}, "version: 1\n") + r := apiRepoFixture + r.DefaultBranch = "master" + if _, err := f.reader.ReadFile(t.Context(), r, "", optIn, 100); err != nil { + t.Errorf("ReadFile: %v; want the file at the current main branch", err) + } + }) + t.Run("an empty repository", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api", http.StatusOK, repo(repoUUID, "acme/api", with("mainbranch", nil))) + r := apiRepoFixture + r.DefaultBranch, r.Empty = "", true + _, err := f.reader.ReadFile(t.Context(), r, "", optIn, 100) + wantClass(t, "an empty repository", err, platform.ClassNotFound, platform.ErrNotFound) + }) +} + +// isNotFound reports whether err says the file is missing, as the core +// asks. +func isNotFound(err error) bool { + return platform.ClassOf(err) == platform.ClassNotFound +} + +func TestReadFileInconsistent(t *testing.T) { + t.Run("the content is gone", func(t *testing.T) { + f := newFixture(t) + f.handle(http.MethodGet, "/repositories/acme/api/src/"+headCommit+"/"+optIn, func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("format") == "meta" { + writeJSON(w, http.StatusOK, map[string]any{"path": optIn, "type": "commit_file", "attributes": []string{}, "size": 3}) + return + } + writeJSON(w, http.StatusNotFound, errorBody(noRepoMessage)) + }) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 100) + wantClass(t, "content gone after the metadata", err, platform.ClassUnknown, nil) + if isNotFound(err) { + t.Errorf("%v: must never read as a missing file", err) + } + }) + t.Run("the size differs", func(t *testing.T) { + f := newFixture(t) + f.file(headCommit, optIn, []string{}, "abc", 4) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 100) + wantClass(t, "a size that differs", err, platform.ClassUnknown, nil) + }) + t.Run("another path", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/src/"+headCommit+"/"+optIn, http.StatusOK, + map[string]any{"path": "elsewhere.yml", "type": "commit_file", "attributes": []string{}, "size": 3}) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 100) + wantClass(t, "metadata of another path", err, platform.ClassUnknown, nil) + }) +} + +func TestReadFileInvalid(t *testing.T) { + f := newFixture(t) + for _, tc := range []struct { + r platform.Repo + path string + max int64 + }{ + {platform.Repo{Path: "acme"}, optIn, 1}, + {apiRepoFixture, "", 1}, + {apiRepoFixture, "a//b", 1}, + {apiRepoFixture, "../x", 1}, + {apiRepoFixture, optIn, -1}, + } { + _, err := f.reader.ReadFile(t.Context(), tc.r, "", tc.path, tc.max) + wantClass(t, fmt.Sprintf("%s %q %d", tc.r.Path, tc.path, tc.max), err, platform.ClassInvalid, nil) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("%d requests for invalid reads", len(calls)) + } +} diff --git a/internal/platform/bitbucket/helpers_test.go b/internal/platform/bitbucket/helpers_test.go new file mode 100644 index 0000000..3ec5660 --- /dev/null +++ b/internal/platform/bitbucket/helpers_test.go @@ -0,0 +1,357 @@ +package bitbucket + +import ( + "bytes" + "crypto/rand" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "io" + "net/http" + "net/http/httptest" + "net/url" + "strconv" + "strings" + "sync" + "testing" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/config" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +// The fixtures below follow the OpenAPI description of the REST API 2.0 +// (https://api.bitbucket.org/swagger.json) and answers of public +// repositories read anonymously (atlassian/atlaskit-mk-2, 2026-10-08): +// every field the driver reads, and a few it ignores, as the API sends +// them. + +// apiServer is an httptest server that answers the routes a test declares +// and records every request. An undeclared route fails the test, and so +// does any write unless the test is the writer's (allowWrites), whose +// writes are declared routes too. Routes are paths under /2.0 as sent, +// escaped ("/users/%7B…%7D"). +type apiServer struct { + t *testing.T + srv *httptest.Server + mu sync.Mutex + routes map[string]http.HandlerFunc + calls []apiCall + // allowWrites lets the writer's tests write (their declared routes). + allowWrites bool +} + +// apiCall is one request the server received. +type apiCall struct { + Method, Path string // Path escaped, under /2.0 + Query url.Values + Auth string // Authorization + Body []byte +} + +func newAPIServer(t *testing.T) *apiServer { + t.Helper() + s := &apiServer{t: t, routes: map[string]http.HandlerFunc{}} + s.srv = httptest.NewServer(http.HandlerFunc(s.serve)) + t.Cleanup(s.srv.Close) + // The read driver never writes. + t.Cleanup(func() { + if s.allowWrites { + return + } + for _, c := range s.writes() { + t.Errorf("the reader wrote: %s %s", c.Method, c.Path) + } + }) + return s +} + +// base is the server's web URL; the API is under base + "/2.0". +func (s *apiServer) base() string { return s.srv.URL } + +func (s *apiServer) serve(w http.ResponseWriter, r *http.Request) { + body, _ := io.ReadAll(r.Body) + r.Body = io.NopCloser(bytes.NewReader(body)) + p := r.URL.EscapedPath() + s.mu.Lock() + s.calls = append(s.calls, apiCall{Method: r.Method, Path: p, Query: r.URL.Query(), Auth: r.Header.Get("Authorization"), Body: body}) + h := s.routes[r.Method+" "+p] + s.mu.Unlock() + if h == nil { + s.t.Errorf("unexpected request %s %s?%s", r.Method, p, r.URL.RawQuery) + writeJSON(w, http.StatusNotFound, errorBody("Resource not found")) + return + } + h(w, r) +} + +// apiPath returns path under /2.0. +func apiPath(path string) string { return "/2.0" + path } + +// handle declares a route: method and an escaped path under /2.0. +func (s *apiServer) handle(method, path string, h http.HandlerFunc) { + s.mu.Lock() + defer s.mu.Unlock() + s.routes[method+" "+apiPath(path)] = h +} + +// json declares a GET route that answers status with v as JSON. +func (s *apiServer) json(path string, status int, v any) { + s.handle(http.MethodGet, path, func(w http.ResponseWriter, _ *http.Request) { writeJSON(w, status, v) }) +} + +// pages declares a GET route that answers items per a page, with the +// page's next link (absolute, the query kept, page=n+1) on every page but +// the last, as the API paginates. +func (s *apiServer) pages(path string, per int, items []any) { + s.handle(http.MethodGet, path, func(w http.ResponseWriter, r *http.Request) { + servePage(w, r, per, items) + }) +} + +// servePage writes the page of items r asks for. +func servePage(w http.ResponseWriter, r *http.Request, per int, items []any) { + q := r.URL.Query() + n, _ := strconv.Atoi(q.Get("page")) + if n <= 0 { + n = 1 + } + from := min((n-1)*per, len(items)) + to := min(from+per, len(items)) + body := map[string]any{"values": items[from:to], "pagelen": per, "page": n, "size": len(items)} + if to < len(items) { + q.Set("page", strconv.Itoa(n+1)) + body["next"] = fmt.Sprintf("http://%s%s?%s", r.Host, r.URL.EscapedPath(), q.Encode()) + } + writeJSON(w, http.StatusOK, body) +} + +// requests returns the recorded calls of method to path under /2.0 ("" +// matches all). +func (s *apiServer) requests(method, path string) []apiCall { + s.mu.Lock() + defer s.mu.Unlock() + var out []apiCall + for _, c := range s.calls { + if (method == "" || c.Method == method) && (path == "" || c.Path == apiPath(path)) { + out = append(out, c) + } + } + return out +} + +// writes returns the recorded requests that are not GETs. +func (s *apiServer) writes() []apiCall { + var out []apiCall + for _, c := range s.requests("", "") { + if c.Method != http.MethodGet { + out = append(out, c) + } + } + return out +} + +// reset forgets the recorded calls. +func (s *apiServer) reset() { + s.mu.Lock() + s.calls = nil + s.mu.Unlock() +} + +func writeJSON(w http.ResponseWriter, status int, v any) { + w.Header().Set("Content-Type", "application/json; charset=utf-8") + w.WriteHeader(status) + _ = json.NewEncoder(w).Encode(v) +} + +// errorBody is Bitbucket's error body. +func errorBody(msg string) map[string]any { + return map[string]any{"type": "error", "error": map[string]any{"message": msg}} +} + +// Messages of 404s, as the API sends them anonymously. +const ( + noRepoMessage = "You may not have access to this repository or it no longer exists in this workspace. " + + "If you think this repository exists and you have access, make sure you are authenticated." + noFileMessage = "No such file or directory: " +) + +// testToken returns a token made at run time, as long as a real one. +func testToken(t *testing.T) string { + t.Helper() + b := make([]byte, 16) + if _, err := rand.Read(b); err != nil { + t.Fatal(err) + } + return "ATATT3x-" + hex.EncodeToString(b) +} + +// provider returns the resolved Bitbucket provider at the server. +func (s *apiServer) provider() config.ResolvedProvider { + u, _ := url.Parse(s.base()) + return config.ResolvedProvider{ + Provider: config.Provider{ID: "bb", Type: "bitbucket", URL: s.base(), APIURL: s.base() + "/2.0"}, + Host: strings.ToLower(u.Host), + APIURL: s.base() + "/2.0", + EnvPrefix: "TOUCHMARK_BB_", + } +} + +// fixture is a reader with a token over one test server. +type fixture struct { + *apiServer + token string + reader *reader +} + +func newFixture(t *testing.T) *fixture { + t.Helper() + s := newAPIServer(t) + tok := testToken(t) + r, err := NewReader(s.provider(), auth.Credential{Kind: auth.Token, Token: tok}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + return &fixture{apiServer: s, token: tok, reader: r.(*reader)} +} + +// Fixed ids. +const ( + repoUUID = "{6380b4e9-6ac5-4dd4-a8e0-65f09cabe4c8}" + forkUUID = "{0d3c2b1a-9f8e-4d7c-b6a5-948372615041}" + botUUID = "{583f7ec5-ed93-49a9-b449-cfc4556cd7f8}" + personUUID = "{bd1ac3ae-5a4e-416a-beab-37f06dd4cc93}" + otherUUID = "{b2457ec9-dd07-49b5-817e-9c9168905a3d}" + syncBranch = "touchmark/acme-eng" +) + +// uuidPath is a uuid as a path segment: braces escaped. +func uuidPath(uuid string) string { return url.PathEscape(uuid) } + +// account is a user as pull requests and GET /users/{uuid} send it. +func account(uuid, nickname string) map[string]any { + return map[string]any{ + "display_name": nickname, + "links": map[string]any{ + "self": map[string]any{"href": "https://api.bitbucket.org/2.0/users/" + uuidPath(uuid)}, + "avatar": map[string]any{"href": "https://avatar-management--avatars.us-west-2.prod.public.atl-paas.net/initials/H-5.png"}, + "html": map[string]any{"href": "https://bitbucket.org/" + uuidPath(uuid) + "/"}, + }, + "type": "user", "uuid": uuid, "account_id": "5a8383aa89ef572e36a586ac", "nickname": nickname, + } +} + +// appUser is an app user (an access token's bot). +func appUser(uuid, name string) map[string]any { + return map[string]any{"display_name": name, "type": "app_user", "uuid": uuid, + "account_id": "712020:1c2d3e4f-5a6b-4c7d-8e9f-0a1b2c3d4e5f", "kind": "access_token", "links": map[string]any{}} +} + +// repoOpt changes a repository fixture. +type repoOpt func(map[string]any) + +func with(k string, v any) repoOpt { return func(m map[string]any) { m[k] = v } } + +// repo is a repository as GET /repositories/{workspace}/{slug} sends it. +func repo(uuid, full string, opts ...repoOpt) map[string]any { + ws, slug, _ := strings.Cut(full, "/") + m := map[string]any{ + "type": "repository", "full_name": full, "name": slug, "slug": slug, "description": "", "scm": "git", + "website": "", "is_private": true, "fork_policy": "no_public_forks", "uuid": uuid, + "owner": map[string]any{"display_name": ws, "type": "team", "uuid": "{02b941e3-cfaa-40f9-9a58-cec53e20bdc3}", "username": ws}, + "workspace": map[string]any{"type": "workspace", "uuid": "{02b941e3-cfaa-40f9-9a58-cec53e20bdc3}", "name": ws, "slug": ws}, + "project": map[string]any{"type": "project", "key": "PROJ", "uuid": "{8b56daff-dbc7-4cae-a7a3-1228c526906b}", "name": "Project"}, + "created_on": "2026-09-01T10:00:00.000000+00:00", "updated_on": "2026-09-02T10:00:00.000000+00:00", + "size": 191411, "language": "python", "mainbranch": map[string]any{"name": "main", "type": "branch"}, + "parent": nil, "enforced_signed_commits": nil, "has_issues": false, "has_wiki": false, + "override_settings": map[string]any{"default_merge_strategy": false, "branching_model": false}, + "links": map[string]any{ + "self": map[string]any{"href": "https://api.bitbucket.org/2.0/repositories/" + full}, + "html": map[string]any{"href": "https://bitbucket.org/" + full}, + }, + } + for _, o := range opts { + o(m) + } + return m +} + +// repoRef is the short repository of a pull request's endpoint. +func repoRef(uuid, full string) map[string]any { + return map[string]any{"type": "repository", "full_name": full, "name": full[strings.IndexByte(full, '/')+1:], "uuid": uuid, + "links": map[string]any{"html": map[string]any{"href": "https://bitbucket.org/" + full}}} +} + +// prSpec describes a pull request fixture. +type prSpec struct { + id int64 + state string // OPEN (default), MERGED, DECLINED, SUPERSEDED + author map[string]any + closedBy map[string]any + source string + sourceRepo map[string]any // default the destination repository + target string // default "main" + hash string // the source commit, 12 digits by default + title string + body string + draft bool + noSummary bool // a listing without summary and description +} + +// pr is a pull request as the pull request APIs send it. +func pr(p prSpec) map[string]any { + state := cmpStr(p.state, "OPEN") + dest := repoRef(repoUUID, "acme/api") + src := p.sourceRepo + if src == nil { + src = dest + } + hash := cmpStr(p.hash, fmt.Sprintf("%012x", 0xa6dcc7402000+p.id)) + m := map[string]any{ + "comment_count": 0, "task_count": 0, "type": "pullrequest", "id": p.id, "title": cmpStr(p.title, "sync"), + "state": state, "draft": p.draft, "merge_commit": nil, "close_source_branch": false, "reason": "", + "author": p.author, "closed_by": nil, + "created_on": "2026-09-05T08:00:00.000000+00:00", "updated_on": "2026-09-10T12:00:00.123456+00:00", + "destination": map[string]any{ + "branch": map[string]any{"name": cmpStr(p.target, "main")}, + "commit": map[string]any{"hash": "d2391895ff0b", "type": "commit"}, + "repository": dest, + }, + "source": map[string]any{ + "branch": map[string]any{"name": p.source, "links": map[string]any{}, "sync_strategies": []string{"merge_commit", "rebase"}}, + "commit": map[string]any{"hash": hash, "type": "commit"}, + "repository": src, + }, + "links": map[string]any{"html": map[string]any{"href": fmt.Sprintf("https://bitbucket.org/acme/api/pull-requests/%d", p.id)}}, + } + if state != "OPEN" && p.closedBy != nil { + m["closed_by"] = p.closedBy + } + if !p.noSummary { + m["description"] = p.body + m["summary"] = map[string]any{"type": "rendered", "raw": p.body, "markup": "markdown", "html": "

" + p.body + "

"} + } + return m +} + +func cmpStr(a, b string) string { + if a != "" { + return a + } + return b +} + +// wantClass checks the class of err (and, with sentinel, that it wraps it). +func wantClass(t *testing.T, what string, err error, class platform.Class, sentinel error) { + t.Helper() + switch { + case err == nil: + t.Errorf("%s: no error, want class %v", what, class) + case platform.ClassOf(err) != class: + t.Errorf("%s: class %v of %v, want %v", what, platform.ClassOf(err), err, class) + case sentinel != nil && !errors.Is(err, sentinel): + t.Errorf("%s: %v does not wrap %v", what, err, sentinel) + } +} diff --git a/internal/platform/bitbucket/probe.go b/internal/platform/bitbucket/probe.go new file mode 100644 index 0000000..3054e87 --- /dev/null +++ b/internal/platform/bitbucket/probe.go @@ -0,0 +1,197 @@ +package bitbucket + +import ( + "context" + "errors" + "net/url" + "regexp" + "strconv" + "strings" + "time" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// Capabilities of Bitbucket Cloud. +const ( + // maxBody is the body budget. The API documents no limit on a pull + // request's description (Renovate assumes 250 000 characters); 60 000 + // bytes keeps parity with the other platforms until a live check + // finds the real bound. + maxBody = 60000 +) + +// limits are the default pacing. The repository API allows 1 000 requests +// an hour per account (about 16 a minute), and a run reads several times +// per target: 15 reads a minute, two targets and two git fetches at once, +// and at least a second between two writes, keep a run under the limit. +var limits = platform.Limits{Reads: 2, GitReads: 2, ReadsPerMinute: 15, MinInterval: time.Second} + +// branchRestrictions is the check the writer cannot make upfront: branch +// restrictions are readable with repository:admin only, so a push meets +// them ("Permission denied to update branch …"). +const branchRestrictions = "branch-restrictions" + +// Account ids: a uuid in braces, and an Atlassian account id (24 hex +// digits, or ":"). +var ( + uuidRe = regexp.MustCompile(`^\{[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}\}$`) + accountIDRe = regexp.MustCompile(`^(?:[0-9a-f]{24}|[0-9]{1,10}:[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})$`) +) + +// apiAccount is an account as the API reports it: type "user" for a +// person or a bot account with API tokens, "app_user" for an app or an +// access token's bot, "team" for a workspace. +type apiAccount struct { + Type string `json:"type"` + UUID string `json:"uuid"` + AccountID string `json:"account_id"` + Nickname string `json:"nickname"` + DisplayName string `json:"display_name"` + // Kind is the kind of an app_user (the OpenAPI description names it + // without listing values). + Kind string `json:"kind"` +} + +// Probe reports the capabilities of Bitbucket Cloud, which has one version: +// it asks the API nothing. +func (d *reader) Probe(context.Context) (platform.Caps, error) { + return platform.Caps{ + Flavor: "bitbucket", + MaxBody: maxBody, + Draft: platform.DraftNative, + // Pull requests have no labels. + NoLabels: true, + // closed_by names who declined or merged a pull request. + CloserKnown: true, + // A declined pull request can never be changed again. + ClosedImmutable: true, + // The API escapes HTML in descriptions: an HTML comment would show. + Marker: platform.MarkerInRefDef, + RuntimeOnly: []string{branchRestrictions}, + Limits: limits, + }, nil +} + +// Self is the account the credential acts as: GET /2.0/user, which takes +// the API token of an account. +func (d *reader) Self(ctx context.Context) (platform.Account, error) { + return d.c.selfAccount(ctx) +} + +// selfAccount returns the credential's account, looked up once. +func (c *client) selfAccount(ctx context.Context) (platform.Account, error) { + const op = "get account" + c.mu.Lock() + if c.self != nil { + a := *c.self + c.mu.Unlock() + return a, nil + } + c.mu.Unlock() + if c.token == "" { + return platform.Account{}, &platform.Error{Op: op, Class: platform.ClassAuth, Err: errors.New("no credential: the reader is anonymous")} + } + var u apiAccount + if _, err := c.get(ctx, op, c.endpoint("user"), nil, &u); err != nil { + return platform.Account{}, err + } + if !uuidRe.MatchString(u.UUID) { + return platform.Account{}, shapeError(op, "GET /user: no uuid") + } + a := toAccount(&u) + if c.wantEmail { + email, err := c.primaryEmail(ctx, op) + if err != nil { + return platform.Account{}, err + } + a.Email = email + } + c.mu.Lock() + c.self = &a + c.mu.Unlock() + return a, nil +} + +// maxEmailPages bounds the listing of the account's addresses. +const maxEmailPages = 5 + +// apiEmail is an address of GET /2.0/user/emails (the OpenAPI description +// does not describe its answer; these are the fields the API sends). +type apiEmail struct { + Email string `json:"email"` + IsPrimary bool `json:"is_primary"` + IsConfirmed bool `json:"is_confirmed"` +} + +// primaryEmail returns the primary address of the credential's account +// when it is confirmed (GET /2.0/user/emails, scope read:user:bitbucket, the +// scope GET /2.0/user needs too): the commit author touchmark gives the +// writer's commits, which Bitbucket links to the account by that address. +// "" when the account has none or the listing fails for another reason +// than a rate limit, a refused credential or the end of ctx, which fail the +// call: the core then authors commits with a placeholder address. +func (c *client) primaryEmail(ctx context.Context, op string) (string, error) { + found := "" + q := url.Values{"pagelen": {strconv.Itoa(repoPageLen)}} + _, err := listAll(ctx, c, op, c.endpoint("user", "emails"), q, maxEmailPages, func(e apiEmail) error { + if found == "" && e.IsPrimary && e.IsConfirmed && validEmail(e.Email) { + found = e.Email + } + return nil + }) + switch { + case err != nil && stops(err): + return "", err + case err != nil: + return "", nil + } + return found, nil +} + +// validEmail reports whether s can be a commit author's address: one "@" +// between non-empty parts, and nothing git would refuse or misread (no +// angle brackets, blanks or control characters). +func validEmail(s string) bool { + local, domain, ok := strings.Cut(s, "@") + return ok && local != "" && domain != "" && !strings.Contains(domain, "@") && + !strings.ContainsFunc(s, func(r rune) bool { return r <= ' ' || r == '<' || r == '>' || r == 0x7f }) +} + +// Lookup resolves a login to its account: a uuid in braces or an Atlassian +// account id (GET /2.0/users/{selected_user} takes both). Bitbucket finds no +// account by its nickname, so anything else is ClassInvalid; an id the API +// does not know is ErrNotFound. +func (d *reader) Lookup(ctx context.Context, login string) (platform.Account, error) { + const op = "look up account" + if !uuidRe.MatchString(login) && !accountIDRe.MatchString(login) { + return platform.Account{}, invalid(op, "%q: Bitbucket logins are account UUIDs in braces, such as {3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d}, "+ + "as GET /2.0/user shows them (a nickname names no account)", login) + } + var u apiAccount + if _, err := d.c.get(ctx, op, d.c.endpoint("users", login), nil, &u); err != nil { + return platform.Account{}, err + } + if !uuidRe.MatchString(u.UUID) { + return platform.Account{}, shapeError(op, "GET /users/%s: no uuid", login) + } + return toAccount(&u), nil +} + +// toAccount converts an API account: its uuid is both its id and its login. +// An app user is a bot; a user is a person or a bot account with API tokens, +// which the API does not tell apart; a workspace (team) or an account of +// another type is unknown. +func toAccount(u *apiAccount) platform.Account { + if u == nil || u.UUID == "" { + return platform.Account{} + } + a := platform.Account{ID: u.UUID, Login: u.UUID, Kind: platform.KindUnknown} + switch u.Type { + case "user": + a.Kind = platform.KindUser + case "app_user": + a.Kind = platform.KindBot + } + return a +} diff --git a/internal/platform/bitbucket/probe_test.go b/internal/platform/bitbucket/probe_test.go new file mode 100644 index 0000000..d2b13f9 --- /dev/null +++ b/internal/platform/bitbucket/probe_test.go @@ -0,0 +1,167 @@ +package bitbucket + +import ( + "net/http" + "slices" + "strings" + "testing" + "time" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/config" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +func TestProbe(t *testing.T) { + f := newFixture(t) + caps, err := f.reader.Probe(t.Context()) + if err != nil { + t.Fatal(err) + } + want := platform.Caps{ + Flavor: "bitbucket", MaxBody: 60000, Draft: platform.DraftNative, CloserKnown: true, NoLabels: true, ClosedImmutable: true, + Marker: platform.MarkerInRefDef, RuntimeOnly: []string{"branch-restrictions"}, + Limits: platform.Limits{Reads: 2, GitReads: 2, ReadsPerMinute: 15, MinInterval: time.Second}, + } + if caps.Flavor != want.Flavor || caps.MaxBody != want.MaxBody || caps.Draft != want.Draft || !caps.CloserKnown || + caps.Marker != want.Marker || !slices.Equal(caps.RuntimeOnly, want.RuntimeOnly) || caps.Limits != want.Limits || + caps.LabelsByID || caps.QuickActions || caps.WorkflowPerm || caps.Commit.API || !caps.NoLabels || !caps.ClosedImmutable || + caps.BodyControls() { + t.Errorf("Probe = %+v\nwant %+v", caps, want) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("Probe sent %d requests; Bitbucket Cloud has one version", len(calls)) + } +} + +func TestSelf(t *testing.T) { + f := newFixture(t) + me := account(botUUID, "touchmark bot") + me["account_status"] = "active" + me["has_2fa_enabled"] = nil + me["is_staff"] = false + me["created_on"] = "2026-09-01T10:00:00.000000+00:00" + f.json("/user", http.StatusOK, me) + for range 2 { + a, err := f.reader.Self(t.Context()) + if err != nil { + t.Fatal(err) + } + if a != (platform.Account{ID: botUUID, Login: botUUID, Kind: platform.KindUser}) { + t.Errorf("Self = %+v; want the uuid as id and login, a user", a) + } + } + calls := f.requests(http.MethodGet, "/user") + if len(calls) != 1 { + t.Fatalf("%d requests of /user; want one, then the cache", len(calls)) + } + if calls[0].Auth != "Bearer "+f.token { + t.Errorf("Authorization %q; want the token as a bearer token", calls[0].Auth) + } +} + +func TestSelfAppUser(t *testing.T) { + f := newFixture(t) + f.json("/user", http.StatusOK, appUser(botUUID, "touchmark")) + a, err := f.reader.Self(t.Context()) + if err != nil || a.Kind != platform.KindBot || a.ID != botUUID { + t.Errorf("Self = %+v, %v; want a bot", a, err) + } +} + +func TestSelfErrors(t *testing.T) { + t.Run("anonymous", func(t *testing.T) { + s := newAPIServer(t) + r, err := NewReader(s.provider(), auth.Credential{}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + _, err = r.Self(t.Context()) + wantClass(t, "Self", err, platform.ClassAuth, nil) + }) + t.Run("refused token", func(t *testing.T) { + f := newFixture(t) + f.json("/user", http.StatusUnauthorized, errorBody("Token is invalid, expired, or not supported for this endpoint.")) + _, err := f.reader.Self(t.Context()) + wantClass(t, "Self", err, platform.ClassAuth, nil) + if err != nil && !strings.Contains(err.Error(), "Token is invalid") { + t.Errorf("the error %q lacks the API's message", err) + } + }) + t.Run("no uuid", func(t *testing.T) { + f := newFixture(t) + f.json("/user", http.StatusOK, map[string]any{"type": "user", "nickname": "x"}) + _, err := f.reader.Self(t.Context()) + wantClass(t, "Self", err, platform.ClassUnknown, nil) + }) +} + +func TestLookup(t *testing.T) { + f := newFixture(t) + f.json("/users/"+uuidPath(personUUID), http.StatusOK, account(personUUID, "Wilson Mendes Neto")) + f.json("/users/5b57c56fdfe79e2c947cd85f", http.StatusOK, account(personUUID, "Wilson Mendes Neto")) + f.json("/users/"+uuidPath(otherUUID), http.StatusNotFound, errorBody(otherUUID+" not found")) + for _, login := range []string{personUUID, "5b57c56fdfe79e2c947cd85f"} { + a, err := f.reader.Lookup(t.Context(), login) + if err != nil || a.ID != personUUID || a.Login != personUUID || a.Kind != platform.KindUser { + t.Errorf("Lookup(%s) = %+v, %v; want the account by its uuid", login, a, err) + } + } + _, err := f.reader.Lookup(t.Context(), otherUUID) + wantClass(t, "an unknown uuid", err, platform.ClassNotFound, platform.ErrNotFound) + f.reset() + for _, login := range []string{"touchmark-bot", "Wilson Mendes Neto", "", "{not-a-uuid}", "../user"} { + _, err := f.reader.Lookup(t.Context(), login) + wantClass(t, "Lookup("+login+")", err, platform.ClassInvalid, nil) + if err != nil && !strings.Contains(err.Error(), "Bitbucket logins are account UUIDs") { + t.Errorf("Lookup(%q): %v; want the message to say logins are UUIDs", login, err) + } + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("a login that is no uuid sent %d requests", len(calls)) + } +} + +func TestNewReaderAndWriter(t *testing.T) { + s := newAPIServer(t) + client := httpx.New(httpx.Options{}) + tok := auth.Credential{Kind: auth.Token, Token: testToken(t)} + other := s.provider() + other.Type = "gitea" + noAPI := s.provider() + noAPI.APIURL = "" + for name, tc := range map[string]struct { + p config.ResolvedProvider + c auth.Credential + want string + }{ + "another type": {other, tok, `type "gitea", not bitbucket`}, + "an app": {s.provider(), auth.Credential{Kind: auth.App, AppID: "1", AppKey: []byte("k")}, "a GitHub App cannot sign in to Bitbucket"}, + "empty token": {s.provider(), auth.Credential{Kind: auth.Token}, "an empty token"}, + "no api url": {noAPI, tok, "api_url is empty"}, + } { + if _, err := NewReader(tc.p, tc.c, client); err == nil || !strings.Contains(err.Error(), tc.want) { + t.Errorf("%s: NewReader: %v; want %q", name, err, tc.want) + } + } + if _, err := NewReader(s.provider(), tok, nil); err == nil { + t.Error("NewReader without an HTTP client: no error") + } + for name, tc := range map[string]struct { + p config.ResolvedProvider + c auth.Credential + want string + }{ + "anonymous": {s.provider(), auth.Credential{}, "the write driver needs a token"}, + "an app": {s.provider(), auth.Credential{Kind: auth.App, AppID: "1", AppKey: []byte("k")}, "a GitHub App cannot sign in to Bitbucket"}, + "no api": {noAPI, tok, "api_url is empty"}, + } { + if w, err := NewWriter(tc.p, tc.c, client); err == nil || w != nil || !strings.Contains(err.Error(), tc.want) { + t.Errorf("%s: NewWriter = %v, %v; want %q", name, w, err, tc.want) + } + } + if w, err := NewWriter(s.provider(), tok, client); err != nil || w == nil { + t.Errorf("NewWriter = %v, %v; want a writer", w, err) + } +} diff --git a/internal/platform/bitbucket/pulls.go b/internal/platform/bitbucket/pulls.go new file mode 100644 index 0000000..319dcd9 --- /dev/null +++ b/internal/platform/bitbucket/pulls.go @@ -0,0 +1,545 @@ +package bitbucket + +import ( + "cmp" + "context" + "errors" + "fmt" + "net/url" + "slices" + "strconv" + "strings" + "time" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// Bounds of listings. +const ( + // maxPRPages bounds a listing of one repository's pull requests. PRs + // has no way to say it saw only part: past the bound it fails. + maxPRPages = 200 + // maxSweepPages bounds the open pull requests of one author in one + // workspace. + maxSweepPages = 200 + // maxWorkspacePages bounds the workspaces of the identity. + maxWorkspacePages = 50 +) + +// Pull request states (the state enum of the OpenAPI description). A +// declined pull request cannot be reopened; a superseded one was replaced +// by another and is closed too. +const ( + stateOpen = "OPEN" + stateMerged = "MERGED" + stateDeclined = "DECLINED" + stateSuperseded = "SUPERSEDED" +) + +// allStates are every state, which a listing with q= must name itself: the +// state parameter is ignored next to q. +var allStates = []string{stateOpen, stateMerged, stateDeclined, stateSuperseded} + +// apiPR is a pull request as the API reports it, in lists and alone. +// summary.raw is the description as typed; description is the same text +// (sent by the API, not in the OpenAPI description). +type apiPR struct { + ID int64 `json:"id"` + Title string `json:"title"` + State string `json:"state"` + Draft bool `json:"draft"` + Summary *apiText `json:"summary"` + Description *string `json:"description"` + Author *apiAccount `json:"author"` + ClosedBy *apiAccount `json:"closed_by"` + Source apiPREndpoint `json:"source"` + Destination apiPREndpoint `json:"destination"` + CreatedOn *time.Time `json:"created_on"` + UpdatedOn *time.Time `json:"updated_on"` + Links apiLinks `json:"links"` + // Reviewers are sent by GET …/pullrequests/{id} only (nil when absent, + // as in listings); EditPR sends them back. + Reviewers []apiAccount `json:"reviewers"` +} + +// apiText is rendered markup: raw is what was typed. +type apiText struct { + Raw *string `json:"raw"` +} + +// apiPREndpoint is the source or destination of a pull request: its branch, +// its commit (a short hash) and its repository (gone for the source of a +// deleted fork). +type apiPREndpoint struct { + Branch *apiBranch `json:"branch"` + Commit *apiCommit `json:"commit"` + Repository *apiRepoRef `json:"repository"` +} + +// apiWorkspaceAccess is an entry of GET /2.0/user/workspaces. +type apiWorkspaceAccess struct { + Workspace *struct { + Slug string `json:"slug"` + } `json:"workspace"` +} + +// check reports what the driver depends on that a pull request lacks. +func (p *apiPR) check(op string) error { + switch { + case p.ID <= 0: + return shapeError(op, "a pull request without an id") + case !slices.Contains(allStates, p.State): + return shapeError(op, "pull request #%d has state %q", p.ID, p.State) + case p.Destination.Repository == nil || !uuidRe.MatchString(p.Destination.Repository.UUID): + return shapeError(op, "pull request #%d lacks its destination repository", p.ID) + case p.Destination.Branch == nil || p.Destination.Branch.Name == "": + return shapeError(op, "pull request #%d lacks its destination branch", p.ID) + case p.CreatedOn == nil: + return shapeError(op, "pull request #%d has no created_on", p.ID) + } + return nil +} + +// head returns the source branch of p, "" when the API names none. +func (p *apiPR) head() string { + if p.Source.Branch == nil { + return "" + } + return p.Source.Branch.Name +} + +// authorID returns the uuid of p's author, "" when there is none (a +// deleted account). +func (p *apiPR) authorID() string { + if p.Author == nil { + return "" + } + return p.Author.UUID +} + +// body returns the description and whether the API sent it. +func (p *apiPR) body() (string, bool) { + switch { + case p.Summary != nil && p.Summary.Raw != nil: + return *p.Summary.Raw, true + case p.Description != nil: + return *p.Description, true + } + return "", false +} + +// toPR converts an API pull request; BaseExists and HeadSHA are left to the +// caller (fill). Declined and superseded pull requests are closed, merged +// ones merged; closed_by names who did it. The API has no time of closing: +// ClosedAt is updated_on, which a later change of a closed pull request +// (a comment) moves on, so it is never earlier than the close. +func (c *client) toPR(p *apiPR) platform.PR { + body, _ := p.body() + pr := platform.PR{ + Number: p.ID, + URL: p.Links.HTML.Href, + State: platform.Open, + Draft: p.Draft, + Head: p.head(), + Base: p.Destination.Branch.Name, + RepoID: p.Destination.Repository.UUID, + Title: p.Title, + Body: body, + Labels: []string{}, + Author: toAccount(p.Author), + CreatedAt: p.CreatedOn.UTC(), + } + if pr.URL == "" && p.Destination.Repository.FullName != "" { + pr.URL = c.web + "/" + p.Destination.Repository.FullName + "/pull-requests/" + strconv.FormatInt(p.ID, 10) + } + if p.Source.Repository != nil && uuidRe.MatchString(p.Source.Repository.UUID) && pr.Head != "" { + pr.HeadRepoID = p.Source.Repository.UUID + } + switch p.State { + case stateMerged: + pr.State = platform.Merged + case stateDeclined, stateSuperseded: + pr.State = platform.Closed + } + if pr.State != platform.Open { + if by := toAccount(p.ClosedBy); by.ID != "" { + pr.ClosedBy = &by + } + if p.UpdatedOn != nil { + pr.ClosedAt = p.UpdatedOn.UTC() + } + } + return pr +} + +// prQuery returns the BBQL filter of pull requests from heads in states: +// `source.branch.name IN ("a", "b") AND state IN ("OPEN")`. The escaping +// of BBQL strings is not documented: a head with a quote, a backslash or a +// control character drops the branch filter, and the caller, which +// filters every listing anyway, reads them all. +func prQuery(heads, states []string) string { + quote := func(list []string) string { + parts := make([]string, len(list)) + for i, s := range list { + parts[i] = `"` + s + `"` + } + return "(" + strings.Join(parts, ", ") + ")" + } + q := "state IN " + quote(states) + for _, h := range heads { + if strings.ContainsFunc(h, func(r rune) bool { return r == '"' || r == '\\' || r < 0x20 || r == 0x7f }) { + return q + } + } + return "source.branch.name IN " + quote(heads) + " AND " + q +} + +// PRs returns, newest first, the pull requests of r from heads by authors +// in every state and the open ones by anyone: one listing of r's pull +// requests filtered by source branch and every state in q=, sorted by +// descending id, filtered again here. A pull request's head is its source +// branch in whatever repository: those from forks are included with their +// own HeadRepoID. The listing is complete or the call fails. +// +// BaseExists reads the branch API for a base other than r's default branch +// (once per base). HeadSHA is the full id of the source commit of an open +// pull request from r itself, from the commit API (the pull request shows +// 12 digits; a short id once expanded is remembered); "" for the others, +// which the core reads no head of. +func (d *reader) PRs(ctx context.Context, r platform.Repo, heads []string, authors []platform.Account) ([]platform.PR, error) { + const op = "list pull requests" + ws, slug, err := repoPath(op, r) + if err != nil { + return nil, err + } + wanted := setOf(heads) + if len(wanted) == 0 { + return []platform.PR{}, nil + } + ids := accountIDs(authors) + q := url.Values{ + "q": {prQuery(sortedKeys(wanted), allStates)}, + "sort": {"-id"}, + "pagelen": {strconv.Itoa(prPageLen)}, + } + var found []apiPR + seen := map[int64]bool{} + complete, err := listAll(ctx, d.c, op, d.c.endpoint("repositories", ws, slug, "pullrequests"), q, maxPRPages, func(p apiPR) error { + if err := p.check(op); err != nil { + return err + } + if !seen[p.ID] && wanted[p.head()] && (p.State == stateOpen || ids[p.authorID()]) { + seen[p.ID] = true + found = append(found, p) + } + return nil + }) + if err != nil { + return nil, err + } + if !complete { + return nil, &platform.Error{Op: op, Class: platform.ClassUnknown, + Err: fmt.Errorf("%s has more pull requests than %d pages; touchmark does not act on a partial list", r.Path, maxPRPages)} + } + out := make([]platform.PR, 0, len(found)) + bases := map[string]bool{} + for i := range found { + p := &found[i] + if err := d.c.fillBody(ctx, op, ws, slug, p); err != nil { + return nil, err + } + pr := d.c.toPR(p) + if pr.BaseExists, err = d.c.baseExists(ctx, op, r, ws, slug, pr.Base, bases); err != nil { + return nil, err + } + if pr.State == platform.Open && pr.HeadRepoID != "" && pr.HeadRepoID == pr.RepoID && p.Source.Commit != nil { + if pr.HeadSHA, err = d.c.fullCommit(ctx, op, ws, slug, pr.RepoID, p.Source.Commit.Hash); err != nil { + return nil, err + } + } + out = append(out, pr) + } + slices.SortFunc(out, func(a, b platform.PR) int { return cmp.Compare(b.Number, a.Number) }) + return out, nil +} + +// fillBody reads a pull request alone when its listing left its +// description out (a listing is taken to carry summary; to be confirmed +// live), so that the marker in it is never missed. +func (c *client) fillBody(ctx context.Context, op, ws, slug string, p *apiPR) error { + if _, ok := p.body(); ok { + return nil + } + var full apiPR + if _, err := c.get(ctx, op, c.endpoint("repositories", ws, slug, "pullrequests", strconv.FormatInt(p.ID, 10)), nil, &full); err != nil { + return err + } + if err := full.check(op); err != nil { + return err + } + if _, ok := full.body(); !ok || full.ID != p.ID { + return shapeError(op, "pull request #%d has no description", p.ID) + } + *p = full + return nil +} + +// baseExists reports whether branch, a pull request's base in r, exists: +// r's default branch does; another is looked up once per call (known holds +// the answers). A pull request whose base was deleted stays with its +// branch's name. +func (c *client) baseExists(ctx context.Context, op string, r platform.Repo, ws, slug, branch string, known map[string]bool) (bool, error) { + if branch == r.DefaultBranch { + return true, nil + } + if v, ok := known[branch]; ok { + return v, nil + } + _, err := c.get(ctx, op, c.endpoint(append([]string{"repositories", ws, slug, "refs", "branches"}, pathSegments(branch)...)...), nil, nil) + switch { + case err == nil: + known[branch] = true + case platform.ClassOf(err) == platform.ClassNotFound: + known[branch] = false + default: + return false, err + } + return known[branch], nil +} + +// fullCommit returns the full id of the commit whose id starts with short +// in ws/slug (the repository with uuid repo): GET …/commit/{short}. A +// commit the API does not find, or an answer that is no full id with that +// prefix, gives "" (the head stays unknown); a failure that stops the call +// or is transient is returned. +func (c *client) fullCommit(ctx context.Context, op, ws, slug, repo, short string) (string, error) { + short = strings.ToLower(strings.TrimSpace(short)) + switch { + case !isHex(short) || len(short) < 7: + return "", nil + case isHexOID(short): + return short, nil + } + key := repo + " " + short + c.mu.Lock() + full, ok := c.commits[key] + c.mu.Unlock() + if ok { + return full, nil + } + var cm apiCommit + _, err := c.get(ctx, op, c.endpoint("repositories", ws, slug, "commit", short), nil, &cm) + switch { + case err != nil && fatal(err): + return "", err + case err != nil: + return "", nil + } + full = strings.ToLower(cm.Hash) + if !isHexOID(full) || !strings.HasPrefix(full, short) { + return "", nil + } + c.mu.Lock() + c.commits[key] = full + c.mu.Unlock() + return full, nil +} + +// OpenPRsBy lists the open pull requests by authors from heads in every +// repository this identity sees, for the stale sweep: for each workspace of +// the identity (GET /2.0/user/workspaces) and each author, the open pull +// requests the author made there (GET +// /2.0/workspaces/{workspace}/pullrequests/{uuid}, filtered by source +// branch and state in q=), filtered again here; each repository is read +// once, for the report and the default branch. HeadSHA stays "": the sweep +// closes pull requests and reads no heads. +// +// A listing that fails, on any page, with a rate limit, a refused +// credential, a transient failure or the end of ctx fails the call, so the +// core can pause or retry; any other failure of a listing (an author the +// workspace does not know, a missing scope) makes the result incomplete, as +// does a listing capped at its bound. A repository or pull request gone +// since the listing is left out; one that cannot be read otherwise makes +// the result incomplete. +func (d *reader) OpenPRsBy(ctx context.Context, authors []platform.Account, heads []string) (platform.Swept, error) { + const op = "list open pull requests" + ids := accountIDs(authors) + wanted := setOf(heads) + out := platform.Swept{PRs: []platform.RepoPR{}, Complete: true} + if len(ids) == 0 || len(wanted) == 0 { + return out, nil + } + if d.c.token == "" { + return platform.Swept{}, &platform.Error{Op: op, Class: platform.ClassAuth, Err: errAnonymousSweep} + } + workspaces, complete, err := d.c.workspaces(ctx, op) + switch { + case err != nil && fatal(err): + return platform.Swept{}, err + case err != nil: + out.Complete = false + } + out.Complete = out.Complete && complete + type hit struct { + ws string + pr apiPR + } + hits := map[string]hit{} + q := url.Values{ + "q": {prQuery(sortedKeys(wanted), []string{stateOpen})}, + "sort": {"-id"}, + "pagelen": {strconv.Itoa(prPageLen)}, + } + for _, ws := range workspaces { + for _, id := range sortedKeys(ids) { + if !uuidRe.MatchString(id) { + out.Complete = false + continue + } + complete, err := listAll(ctx, d.c, op, d.c.endpoint("workspaces", ws, "pullrequests", id), q, maxSweepPages, func(p apiPR) error { + if err := p.check(op); err != nil { + return err + } + if p.State == stateOpen && wanted[p.head()] && ids[p.authorID()] { + hits[p.Destination.Repository.UUID+"#"+strconv.FormatInt(p.ID, 10)] = hit{ws: ws, pr: p} + } + return nil + }) + switch { + case err == nil: + out.Complete = out.Complete && complete + case fatal(err): + return platform.Swept{}, err + default: + out.Complete = false + } + } + } + keys := make([]string, 0, len(hits)) + for k := range hits { + keys = append(keys, k) + } + slices.Sort(keys) + repos := map[string]*apiRepo{} + bases := map[string]map[string]bool{} + for _, k := range keys { + h := hits[k] + p := h.pr + uuid := p.Destination.Repository.UUID + ws := h.ws + if w, _, ok := splitRepoPath(p.Destination.Repository.FullName); ok { + ws = w + } + repo, ok := repos[uuid] + if !ok { + repo, err = d.c.getRepo(ctx, op, ws, uuid) + if err == nil && repo.UUID != uuid { + err = shapeError(op, "repository %s answers with uuid %s", uuid, repo.UUID) + } + switch { + case err == nil: + case platform.ClassOf(err) == platform.ClassNotFound: + repo = nil + case stops(err): + return platform.Swept{}, err + default: + out.Complete = false + repo = nil + } + repos[uuid] = repo + } + if repo == nil { + continue + } + rws, rslug, _ := splitRepoPath(repo.FullName) + r := d.c.toRepo(repo) + if bases[uuid] == nil { + bases[uuid] = map[string]bool{} + } + err := d.c.fillBody(ctx, op, rws, rslug, &p) + var pr platform.PR + if err == nil { + pr = d.c.toPR(&p) + pr.BaseExists, err = d.c.baseExists(ctx, op, r, rws, rslug, pr.Base, bases[uuid]) + } + switch { + case err == nil: + case platform.ClassOf(err) == platform.ClassNotFound: + continue // gone since the listing + case stops(err): + return platform.Swept{}, err + default: + out.Complete = false + continue + } + out.PRs = append(out.PRs, platform.RepoPR{Repo: r, PR: pr}) + } + slices.SortFunc(out.PRs, func(a, b platform.RepoPR) int { + if c := strings.Compare(strings.ToLower(a.Repo.Path), strings.ToLower(b.Repo.Path)); c != 0 { + return c + } + if c := strings.Compare(a.Repo.ID, b.Repo.ID); c != 0 { + return c + } + return cmp.Compare(b.PR.Number, a.PR.Number) + }) + return out, nil +} + +// errAnonymousSweep is why an anonymous reader cannot sweep: it has no +// workspaces and no pull requests of its own. +var errAnonymousSweep = errors.New("no credential: the reader is anonymous, and only an account has workspaces to sweep") + +// workspaces returns the slugs of the workspaces the identity reaches, +// sorted; complete is false when the listing was capped or a later page +// failed. +func (c *client) workspaces(ctx context.Context, op string) ([]string, bool, error) { + set := map[string]bool{} + q := url.Values{"pagelen": {strconv.Itoa(repoPageLen)}} + complete, err := listAll(ctx, c, op, c.endpoint("user", "workspaces"), q, maxWorkspacePages, func(a apiWorkspaceAccess) error { + if a.Workspace == nil || a.Workspace.Slug == "" || strings.ContainsAny(a.Workspace.Slug, "/\x00?#") { + return shapeError(op, "a workspace without a slug") + } + set[a.Workspace.Slug] = true + return nil + }) + if err != nil && (!laterPage(err) || fatal(err)) { + return nil, false, err + } + if err != nil { + complete = false + } + return sortedKeys(set), complete, nil +} + +// accountIDs returns the set of the accounts' ids. +func accountIDs(accounts []platform.Account) map[string]bool { + ids := map[string]bool{} + for _, a := range accounts { + if a.ID != "" { + ids[a.ID] = true + } + } + return ids +} + +// setOf returns the set of the non-empty strings of list. +func setOf(list []string) map[string]bool { + out := map[string]bool{} + for _, s := range list { + if s != "" { + out[s] = true + } + } + return out +} + +// sortedKeys returns the keys of m in order. +func sortedKeys(m map[string]bool) []string { + out := make([]string, 0, len(m)) + for k := range m { + out = append(out, k) + } + slices.Sort(out) + return out +} diff --git a/internal/platform/bitbucket/pulls_test.go b/internal/platform/bitbucket/pulls_test.go new file mode 100644 index 0000000..af137e1 --- /dev/null +++ b/internal/platform/bitbucket/pulls_test.go @@ -0,0 +1,350 @@ +package bitbucket + +import ( + "errors" + "net/http" + "strings" + "testing" + "time" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +// The accounts of the pull request tests: the writer (a bot account with an +// API token, type user) and a person. +var ( + botAccount = platform.Account{ID: botUUID, Login: botUUID, Kind: platform.KindUser} + personAccount = platform.Account{ID: personUUID, Login: personUUID, Kind: platform.KindUser} +) + +// fullHash is the full id of the source commit of pull request id in pr's +// fixtures, whose short form pr sends. +func fullHash(short string) string { return short + strings.Repeat("0", 40-len(short)) } + +// commitRoute declares GET …/commit/{short} of acme/api with its full id. +func (s *apiServer) commitRoute(short string) { + s.json("/repositories/acme/api/commit/"+short, http.StatusOK, map[string]any{ + "hash": fullHash(short), "type": "commit", "date": "2026-09-30T10:00:00+00:00", "message": "chore: sync", + }) +} + +func TestPRs(t *testing.T) { + f := newFixture(t) + bot, person := account(botUUID, "touchmark bot"), account(personUUID, "Wilson Mendes Neto") + fork := repoRef(forkUUID, "someone/api") + items := []any{ + pr(prSpec{id: 9, author: bot, source: syncBranch, body: "sync\n\n[touchmark]: # \"touchmark:v1 …\"", draft: true}), + pr(prSpec{id: 8, author: person, source: syncBranch, sourceRepo: fork, body: "from a fork"}), + pr(prSpec{id: 7, state: "DECLINED", author: bot, closedBy: person, source: syncBranch, body: "declined"}), + pr(prSpec{id: 6, state: "MERGED", author: bot, closedBy: person, source: syncBranch, target: "release"}), + pr(prSpec{id: 5, state: "SUPERSEDED", author: bot, closedBy: bot, source: syncBranch, target: "gone"}), + pr(prSpec{id: 4, state: "DECLINED", author: person, closedBy: person, source: syncBranch}), + pr(prSpec{id: 3, author: bot, source: "feature/other"}), + } + f.pages("/repositories/acme/api/pullrequests", 3, items) + f.commitRoute("a6dcc7402009") + f.json("/repositories/acme/api/refs/branches/release", http.StatusOK, map[string]any{"name": "release", "type": "branch", + "target": map[string]any{"hash": headCommit, "type": "commit"}}) + f.json("/repositories/acme/api/refs/branches/gone", http.StatusNotFound, errorBody("Branch \"gone\" not found")) + + got, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch, ""}, []platform.Account{botAccount}) + if err != nil { + t.Fatal(err) + } + var numbers []int64 + byNumber := map[int64]platform.PR{} + for _, p := range got { + numbers = append(numbers, p.Number) + byNumber[p.Number] = p + } + if len(numbers) != 5 || numbers[0] != 9 || numbers[1] != 8 || numbers[2] != 7 || numbers[3] != 6 || numbers[4] != 5 { + t.Fatalf("PRs = %v; want 9, 8, 7, 6, 5: the bot's in every state and anyone's open ones, newest first", numbers) + } + open := byNumber[9] + if open.State != platform.Open || !open.Draft || open.Head != syncBranch || open.Base != "main" || !open.BaseExists || + open.RepoID != repoUUID || open.HeadRepoID != repoUUID || open.HeadSHA != fullHash("a6dcc7402009") || + open.Author != botAccount || open.ClosedBy != nil || !open.ClosedAt.IsZero() || + open.Body != "sync\n\n[touchmark]: # \"touchmark:v1 …\"" || open.URL != "https://bitbucket.org/acme/api/pull-requests/9" || + len(open.Labels) != 0 || open.Labels == nil || !open.CreatedAt.Equal(time.Date(2026, 9, 5, 8, 0, 0, 0, time.UTC)) { + t.Errorf("open = %+v", open) + } + if fromFork := byNumber[8]; fromFork.HeadRepoID != forkUUID || fromFork.HeadSHA != "" || fromFork.State != platform.Open { + t.Errorf("from a fork = %+v; want its own head repository and no head commit read", fromFork) + } + declined := byNumber[7] + if declined.State != platform.Closed || declined.ClosedBy == nil || *declined.ClosedBy != personAccount || + !declined.ClosedAt.Equal(time.Date(2026, 9, 10, 12, 0, 0, 123456000, time.UTC)) || declined.HeadSHA != "" { + t.Errorf("declined = %+v; want closed, by the person, at updated_on", declined) + } + if merged := byNumber[6]; merged.State != platform.Merged || !merged.BaseExists || merged.ClosedBy == nil { + t.Errorf("merged = %+v", merged) + } + if superseded := byNumber[5]; superseded.State != platform.Closed || superseded.BaseExists || superseded.ClosedBy == nil || + *superseded.ClosedBy != botAccount { + t.Errorf("superseded = %+v; want closed, its deleted base reported", superseded) + } + + calls := f.requests(http.MethodGet, "/repositories/acme/api/pullrequests") + if len(calls) != 3 { + t.Fatalf("%d pages read; want 3", len(calls)) + } + q := calls[0].Query + wantQ := `source.branch.name IN ("touchmark/acme-eng") AND state IN ("OPEN", "MERGED", "DECLINED", "SUPERSEDED")` + if q.Get("q") != wantQ || q.Get("sort") != "-id" || q.Get("pagelen") != "50" || q.Has("state") { + t.Errorf("query %v; want q %s, sorted by descending id, the state in q", q, wantQ) + } + if n := len(f.requests(http.MethodGet, "/repositories/acme/api/refs/branches/main")); n != 0 { + t.Errorf("the default branch was looked up %d times; it exists", n) + } + + // A second listing reads the full id of the head from memory. + f.reset() + if _, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch}, []platform.Account{botAccount}); err != nil { + t.Fatal(err) + } + if n := len(f.requests(http.MethodGet, "/repositories/acme/api/commit/a6dcc7402009")); n != 0 { + t.Errorf("the short id was expanded again (%d requests)", n) + } +} + +func TestPRsNoHeads(t *testing.T) { + f := newFixture(t) + got, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{""}, []platform.Account{botAccount}) + if err != nil || got == nil || len(got) != 0 { + t.Errorf("PRs without heads = %v, %v; want an empty list", got, err) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("%d requests without heads", len(calls)) + } +} + +func TestPRsQuoting(t *testing.T) { + f := newFixture(t) + f.pages("/repositories/acme/api/pullrequests", 50, []any{ + pr(prSpec{id: 2, author: account(botUUID, "bot"), source: `odd"name`, hash: strings.Repeat("c", 40)}), + pr(prSpec{id: 1, author: account(botUUID, "bot"), source: "other"}), + }) + got, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{`odd"name`}, []platform.Account{botAccount}) + if err != nil || len(got) != 1 || got[0].Number != 2 || got[0].HeadSHA != strings.Repeat("c", 40) { + t.Errorf("PRs = %+v, %v; want #2, its full head taken as it is", got, err) + } + q := f.requests(http.MethodGet, "/repositories/acme/api/pullrequests")[0].Query.Get("q") + if q != `state IN ("OPEN", "MERGED", "DECLINED", "SUPERSEDED")` { + t.Errorf("q %s; want no branch filter for a name BBQL cannot hold safely", q) + } +} + +func TestPRsBody(t *testing.T) { + f := newFixture(t) + f.pages("/repositories/acme/api/pullrequests", 50, []any{ + pr(prSpec{id: 4, state: "MERGED", author: account(botUUID, "bot"), source: syncBranch, noSummary: true}), + }) + f.json("/repositories/acme/api/pullrequests/4", http.StatusOK, + pr(prSpec{id: 4, state: "MERGED", author: account(botUUID, "bot"), source: syncBranch, body: "the marker"})) + got, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch}, []platform.Account{botAccount}) + if err != nil || len(got) != 1 || got[0].Body != "the marker" { + t.Errorf("PRs = %+v, %v; want the description of the pull request read alone", got, err) + } +} + +func TestPRsErrors(t *testing.T) { + t.Run("a missing repository", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/pullrequests", http.StatusNotFound, errorBody(noRepoMessage)) + _, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch}, nil) + wantClass(t, "PRs", err, platform.ClassNotFound, nil) + }) + t.Run("a later page fails", func(t *testing.T) { + f := newFixture(t) + items := []any{ + pr(prSpec{id: 2, author: account(botUUID, "bot"), source: syncBranch}), + pr(prSpec{id: 1, author: account(botUUID, "bot"), source: syncBranch}), + } + f.handle(http.MethodGet, "/repositories/acme/api/pullrequests", func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("page") == "2" { + writeJSON(w, http.StatusBadGateway, errorBody("Bad gateway")) + return + } + servePage(w, r, 1, items) + }) + _, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch}, nil) + wantClass(t, "PRs", err, platform.ClassTransient, nil) + }) + t.Run("an unknown state", func(t *testing.T) { + f := newFixture(t) + p := pr(prSpec{id: 1, author: account(botUUID, "bot"), source: syncBranch}) + p["state"] = "QUEUED" + f.pages("/repositories/acme/api/pullrequests", 50, []any{p}) + _, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch}, nil) + wantClass(t, "PRs", err, platform.ClassUnknown, nil) + }) + t.Run("a head commit the API does not find", func(t *testing.T) { + f := newFixture(t) + f.pages("/repositories/acme/api/pullrequests", 50, []any{pr(prSpec{id: 1, author: account(botUUID, "bot"), source: syncBranch})}) + f.json("/repositories/acme/api/commit/a6dcc7402001", http.StatusNotFound, errorBody("Commit not found")) + got, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch}, nil) + if err != nil || len(got) != 1 || got[0].HeadSHA != "" { + t.Errorf("PRs = %+v, %v; want the pull request with its head unknown", got, err) + } + }) + t.Run("the commit API limits the rate", func(t *testing.T) { + f := newFixture(t) + f.pages("/repositories/acme/api/pullrequests", 50, []any{pr(prSpec{id: 1, author: account(botUUID, "bot"), source: syncBranch})}) + f.handle(http.MethodGet, "/repositories/acme/api/commit/a6dcc7402001", func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Retry-After", "30") + writeJSON(w, http.StatusTooManyRequests, errorBody("Rate limit for this resource has been exceeded")) + }) + _, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch}, nil) + wantClass(t, "PRs", err, platform.ClassRateLimited, nil) + }) +} + +// sweepFixture declares two workspaces, acme and labs, where the bot has +// open pull requests. +func sweepFixture(t *testing.T) *fixture { + f := newFixture(t) + bot := account(botUUID, "touchmark bot") + f.pages("/user/workspaces", 1, []any{ + map[string]any{"administrator": false, "type": "workspace_access", + "workspace": map[string]any{"type": "workspace_base", "slug": "labs", "uuid": "{470c176d-3574-44ea-bb41-89e8638bcca4}"}}, + map[string]any{"administrator": false, "type": "workspace_access", + "workspace": map[string]any{"type": "workspace_base", "slug": "acme", "uuid": "{02b941e3-cfaa-40f9-9a58-cec53e20bdc3}"}}, + }) + labsUUID := "{9c8b7a6d-5e4f-4a3b-9c2d-1e0f9a8b7c6d}" + labsPR := pr(prSpec{id: 3, author: bot, source: syncBranch, body: "labs", target: "develop"}) + labsPR["destination"].(map[string]any)["repository"] = repoRef(labsUUID, "labs/tool") + labsPR["source"].(map[string]any)["repository"] = repoRef(labsUUID, "labs/tool") + f.pages("/workspaces/acme/pullrequests/"+uuidPath(botUUID), 50, []any{ + pr(prSpec{id: 12, author: bot, source: syncBranch, body: "twelve"}), + pr(prSpec{id: 11, author: bot, source: "feature/x"}), + pr(prSpec{id: 10, author: bot, source: syncBranch, noSummary: true}), + }) + f.pages("/workspaces/labs/pullrequests/"+uuidPath(botUUID), 50, []any{labsPR}) + f.json("/repositories/acme/"+uuidPath(repoUUID), http.StatusOK, repo(repoUUID, "acme/api")) + f.json("/repositories/labs/"+uuidPath(labsUUID), http.StatusOK, repo(labsUUID, "labs/tool", with("is_private", false))) + f.json("/repositories/acme/api/pullrequests/10", http.StatusOK, pr(prSpec{id: 10, author: bot, source: syncBranch, body: "ten"})) + f.json("/repositories/labs/tool/refs/branches/develop", http.StatusNotFound, errorBody("Branch \"develop\" not found")) + return f +} + +func TestOpenPRsBy(t *testing.T) { + f := sweepFixture(t) + got, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{botAccount}, []string{syncBranch}) + if err != nil { + t.Fatal(err) + } + if !got.Complete || len(got.PRs) != 3 { + t.Fatalf("OpenPRsBy = %+v; want three pull requests, complete", got) + } + want := []struct { + path string + number int64 + body string + base bool + }{{"acme/api", 12, "twelve", true}, {"acme/api", 10, "ten", true}, {"labs/tool", 3, "labs", false}} + for i, w := range want { + rp := got.PRs[i] + if rp.Repo.Path != w.path || rp.PR.Number != w.number || rp.PR.Body != w.body || rp.PR.BaseExists != w.base || + rp.PR.State != platform.Open || rp.PR.RepoID != rp.Repo.ID || rp.PR.HeadRepoID != rp.Repo.ID || rp.Repo.DefaultBranch != "main" { + t.Errorf("PRs[%d] = %s #%d %+v; want %+v", i, rp.Repo.Path, rp.PR.Number, rp.PR, w) + } + } + if got.PRs[2].Repo.Visibility != "public" || got.PRs[0].Repo.Visibility != "private" { + t.Errorf("visibility %s, %s", got.PRs[0].Repo.Visibility, got.PRs[2].Repo.Visibility) + } + if n := len(f.requests(http.MethodGet, "/repositories/acme/"+uuidPath(repoUUID))); n != 1 { + t.Errorf("acme/api read %d times; want once", n) + } + calls := f.requests(http.MethodGet, "/workspaces/acme/pullrequests/"+uuidPath(botUUID)) + if len(calls) != 1 || calls[0].Query.Get("q") != `source.branch.name IN ("touchmark/acme-eng") AND state IN ("OPEN")` { + t.Errorf("calls %+v; want the open pull requests from the sync branch", calls) + } +} + +func TestOpenPRsByIncomplete(t *testing.T) { + t.Run("an author a workspace does not know", func(t *testing.T) { + f := sweepFixture(t) + f.json("/workspaces/labs/pullrequests/"+uuidPath(botUUID), http.StatusNotFound, errorBody("No such user")) + got, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{botAccount}, []string{syncBranch}) + if err != nil || got.Complete || len(got.PRs) != 2 { + t.Errorf("OpenPRsBy = %+v, %v; want acme's pull requests, incomplete", got, err) + } + }) + t.Run("a repository that cannot be read", func(t *testing.T) { + f := sweepFixture(t) + f.json("/repositories/acme/"+uuidPath(repoUUID), http.StatusForbidden, errorBody("Forbidden")) + got, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{botAccount}, []string{syncBranch}) + if err != nil || got.Complete || len(got.PRs) != 1 { + t.Errorf("OpenPRsBy = %+v, %v; want labs' pull request, incomplete", got, err) + } + }) + t.Run("a repository gone since the listing", func(t *testing.T) { + f := sweepFixture(t) + f.json("/repositories/acme/"+uuidPath(repoUUID), http.StatusNotFound, errorBody(noRepoMessage)) + got, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{botAccount}, []string{syncBranch}) + if err != nil || !got.Complete || len(got.PRs) != 1 { + t.Errorf("OpenPRsBy = %+v, %v; want labs' pull request, complete", got, err) + } + }) + t.Run("an author id that is no uuid", func(t *testing.T) { + f := sweepFixture(t) + got, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{botAccount, {ID: "5b57c56fdfe79e2c947cd85f"}}, []string{syncBranch}) + if err != nil || got.Complete || len(got.PRs) != 3 { + t.Errorf("OpenPRsBy = %+v, %v; want the bot's, incomplete", got, err) + } + }) +} + +func TestOpenPRsByErrors(t *testing.T) { + t.Run("rate limited", func(t *testing.T) { + f := sweepFixture(t) + f.handle(http.MethodGet, "/workspaces/labs/pullrequests/"+uuidPath(botUUID), func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("X-RateLimit-Remaining", "0") + w.Header().Set("X-RateLimit-Reset", "120") + writeJSON(w, http.StatusTooManyRequests, errorBody("Rate limit for this resource has been exceeded")) + }) + _, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{botAccount}, []string{syncBranch}) + wantClass(t, "OpenPRsBy", err, platform.ClassRateLimited, nil) + if d := retryAfterOf(err); d != 120*time.Second { + t.Errorf("RetryAfter %v; want x-ratelimit-reset's 120 s", d) + } + }) + t.Run("no workspaces", func(t *testing.T) { + f := newFixture(t) + f.json("/user/workspaces", http.StatusForbidden, errorBody("Your credentials lack one or more required privilege scopes.")) + got, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{botAccount}, []string{syncBranch}) + if err != nil || got.Complete || len(got.PRs) != 0 { + t.Errorf("OpenPRsBy = %+v, %v; want nothing, incomplete", got, err) + } + }) + t.Run("anonymous", func(t *testing.T) { + s := newAPIServer(t) + r, err := NewReader(s.provider(), auth.Credential{}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + _, err = r.OpenPRsBy(t.Context(), []platform.Account{botAccount}, []string{syncBranch}) + wantClass(t, "anonymous", err, platform.ClassAuth, nil) + }) + t.Run("nothing to look for", func(t *testing.T) { + f := newFixture(t) + got, err := f.reader.OpenPRsBy(t.Context(), nil, []string{syncBranch}) + if err != nil || !got.Complete || got.PRs == nil { + t.Errorf("OpenPRsBy without authors = %+v, %v", got, err) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("%d requests", len(calls)) + } + }) +} + +// retryAfterOf returns the RetryAfter of err's *platform.Error. +func retryAfterOf(err error) time.Duration { + var pe *platform.Error + if errors.As(err, &pe) { + return pe.RetryAfter + } + return 0 +} diff --git a/internal/platform/bitbucket/repos.go b/internal/platform/bitbucket/repos.go new file mode 100644 index 0000000..f2fd703 --- /dev/null +++ b/internal/platform/bitbucket/repos.go @@ -0,0 +1,213 @@ +package bitbucket + +import ( + "context" + "errors" + "net/url" + "slices" + "strconv" + "strings" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// maxRepoPages bounds a workspace listing: 20 000 repositories at 100 a +// page. A larger one resolves incompletely. +const maxRepoPages = 200 + +// apiRepo is a repository as the API reports it. +type apiRepo struct { + UUID string `json:"uuid"` + FullName string `json:"full_name"` + IsPrivate bool `json:"is_private"` + MainBranch *apiBranch `json:"mainbranch"` + Parent *apiRepoRef `json:"parent"` + Links apiLinks `json:"links"` +} + +// apiRepoRef is the short repository of a fork's parent or of a pull +// request's source or destination. +type apiRepoRef struct { + UUID string `json:"uuid"` + FullName string `json:"full_name"` +} + +// apiBranch is a branch: a repository's main branch, a pull request's +// source or destination branch, or GET …/refs/branches/{name}, where target +// is its head commit. +type apiBranch struct { + Name string `json:"name"` + Target *apiCommit `json:"target"` +} + +// apiCommit is a commit; hash is full in the commit and branch APIs and +// short (12 digits) in pull requests. +type apiCommit struct { + Hash string `json:"hash"` +} + +// apiLinks are the links of a resource that the driver reads. +type apiLinks struct { + HTML struct { + Href string `json:"href"` + } `json:"html"` +} + +// check reports what the driver depends on that a repository lacks. +func (r *apiRepo) check(op string) error { + if !uuidRe.MatchString(r.UUID) || r.FullName == "" { + return shapeError(op, "a repository without uuid or full_name") + } + if _, _, ok := splitRepoPath(r.FullName); !ok { + return shapeError(op, "repository full_name %q is not workspace/slug", r.FullName) + } + return nil +} + +// toRepo converts an API repository. A repository without a main branch is +// taken for empty (nothing was pushed yet; to be confirmed live). Bitbucket +// archives nothing and always has pull requests; a fork has a parent. +func (c *client) toRepo(r *apiRepo) platform.Repo { + visibility := "public" + if r.IsPrivate { + visibility = "private" + } + web := r.Links.HTML.Href + if web == "" { + web = c.web + "/" + r.FullName + } + out := platform.Repo{ + Host: c.host, + ID: r.UUID, + Path: r.FullName, + WebURL: web, + Visibility: visibility, + ObjectFormat: "sha1", + Fork: r.Parent != nil, + Empty: r.MainBranch == nil || r.MainBranch.Name == "", + } + if r.MainBranch != nil { + out.DefaultBranch = r.MainBranch.Name + } + return out +} + +// Repo returns one repository by its workspace/slug path. The API takes +// paths ignoring case (slugs are lowercase); the result carries the +// canonical full_name. +func (d *reader) Repo(ctx context.Context, path string) (platform.Repo, error) { + const op = "get repository" + ws, slug, ok := splitRepoPath(path) + if !ok { + return platform.Repo{}, notFound(op, "repository %q: Bitbucket paths are workspace/repository", path) + } + r, err := d.c.getRepo(ctx, op, ws, slug) + if err != nil { + return platform.Repo{}, err + } + return d.c.toRepo(r), nil +} + +// getRepo reads GET /2.0/repositories/{workspace}/{slug}; slug may be the +// repository's uuid in braces. +func (c *client) getRepo(ctx context.Context, op, ws, slug string) (*apiRepo, error) { + var r apiRepo + if _, err := c.get(ctx, op, c.endpoint("repositories", ws, slug), nil, &r); err != nil { + return nil, err + } + if err := r.check(op); err != nil { + return nil, err + } + return &r, nil +} + +// Resolve lists the repositories sel selects, sorted by path ignoring case. +// A namespace is a workspace (GET /2.0/repositories/{workspace}, every +// repository the identity sees there); Bitbucket has no nested namespaces, +// so Subgroups changes nothing, and no topics, so a selector with topics is +// ClassInvalid (check refuses such an entry before). Forks only with +// sel.Forks. +// +// The listing is incomplete when it is capped or a later page fails; a +// failure of the first page, a rate limit, an auth error or a transient +// failure is returned instead, so the caller can retry or pause. +func (d *reader) Resolve(ctx context.Context, sel platform.Selector) (platform.Resolved, error) { + const op = "resolve" + if sel.Repo != "" { + r, err := d.Repo(ctx, sel.Repo) + if err != nil { + return platform.Resolved{}, err + } + return platform.Resolved{Repos: []platform.Repo{r}, Complete: true}, nil + } + switch { + case sel.Namespace == "": + return platform.Resolved{}, invalid(op, "the selector names no repository and no namespace") + case len(sel.Topics) > 0: + return platform.Resolved{}, &platform.Error{Op: op, Class: platform.ClassInvalid, Err: errTopics} + case strings.ContainsAny(sel.Namespace, "/\x00?#") || sel.Namespace == "." || sel.Namespace == "..": + return platform.Resolved{}, notFound(op, "workspace %q: Bitbucket has no nested namespaces", sel.Namespace) + } + var found []apiRepo + q := url.Values{"pagelen": {strconv.Itoa(repoPageLen)}} + complete, err := listAll(ctx, d.c, op, d.c.endpoint("repositories", sel.Namespace), q, maxRepoPages, func(r apiRepo) error { + if err := r.check(op); err != nil { + return err + } + found = append(found, r) + return nil + }) + if err != nil { + if !laterPage(err) || fatal(err) { + return platform.Resolved{}, err + } + complete = false + } + out := platform.Resolved{Repos: []platform.Repo{}, Complete: complete} + seen := map[string]bool{} + for i := range found { + r := &found[i] + if seen[r.UUID] || r.Parent != nil && !sel.Forks { + continue + } + seen[r.UUID] = true + out.Repos = append(out.Repos, d.c.toRepo(r)) + } + sortRepos(out.Repos) + return out, nil +} + +// errTopics is why a selector with topics is refused. +var errTopics = errors.New("repositories on Bitbucket have no topics; select them with match (workspace/svc-*) or list them by repo") + +// sortRepos orders repositories by path ignoring case, then path, then id. +func sortRepos(repos []platform.Repo) { + slices.SortFunc(repos, func(a, b platform.Repo) int { + if c := strings.Compare(strings.ToLower(a.Path), strings.ToLower(b.Path)); c != 0 { + return c + } + if c := strings.Compare(a.Path, b.Path); c != 0 { + return c + } + return strings.Compare(a.ID, b.ID) + }) +} + +// Remote is the repository's git URL under the provider's web URL +// (https://bitbucket.org/{full_name}.git), never a host the API names, with +// the credential's Basic header (none when anonymous). +func (d *reader) Remote(_ context.Context, r platform.Repo) (platform.Remote, error) { + const op = "get remote" + if r.Host != "" && !strings.EqualFold(r.Host, d.c.host) { + return platform.Remote{}, invalid(op, "repository %s is on %s, not %s", r.Path, r.Host, d.c.host) + } + ws, slug, err := repoPath(op, r) + if err != nil { + return platform.Remote{}, err + } + rem := platform.Remote{URL: d.c.web + "/" + url.PathEscape(ws) + "/" + url.PathEscape(slug) + ".git"} + if d.c.token != "" { + rem.Header = d.c.gitHeader + } + return rem, nil +} diff --git a/internal/platform/bitbucket/repos_test.go b/internal/platform/bitbucket/repos_test.go new file mode 100644 index 0000000..39c9b6a --- /dev/null +++ b/internal/platform/bitbucket/repos_test.go @@ -0,0 +1,205 @@ +package bitbucket + +import ( + "encoding/base64" + "fmt" + "net/http" + "strings" + "testing" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +func TestRepo(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api", http.StatusOK, repo(repoUUID, "acme/api")) + f.json("/repositories/acme/fork", http.StatusOK, repo(forkUUID, "acme/fork", with("is_private", false), + with("parent", repoRef(repoUUID, "acme/api")), with("mainbranch", map[string]any{"name": "develop", "type": "branch"}))) + f.json("/repositories/acme/empty", http.StatusOK, repo("{11111111-2222-4333-8444-555555555555}", "acme/empty", with("mainbranch", nil))) + f.json("/repositories/acme/gone", http.StatusNotFound, errorBody(noRepoMessage)) + + got, err := f.reader.Repo(t.Context(), "acme/api") + if err != nil { + t.Fatal(err) + } + want := platform.Repo{Host: f.provider().Host, ID: repoUUID, Path: "acme/api", DefaultBranch: "main", + WebURL: "https://bitbucket.org/acme/api", Visibility: "private", ObjectFormat: "sha1"} + if fmt.Sprint(got) != fmt.Sprint(want) { + t.Errorf("Repo = %+v\nwant %+v", got, want) + } + fork, err := f.reader.Repo(t.Context(), "acme/fork") + if err != nil || !fork.Fork || fork.Visibility != "public" || fork.DefaultBranch != "develop" || fork.Empty { + t.Errorf("fork = %+v, %v", fork, err) + } + empty, err := f.reader.Repo(t.Context(), "acme/empty") + if err != nil || !empty.Empty || empty.DefaultBranch != "" { + t.Errorf("a repository without a main branch = %+v, %v; want empty", empty, err) + } + _, err = f.reader.Repo(t.Context(), "acme/gone") + wantClass(t, "a missing repository", err, platform.ClassNotFound, platform.ErrNotFound) + if err != nil && !strings.Contains(err.Error(), "no longer exists") { + t.Errorf("the error %q lacks the API's message", err) + } + f.reset() + for _, p := range []string{"acme", "acme/a/b", "/api", "acme/..", ""} { + _, err := f.reader.Repo(t.Context(), p) + wantClass(t, "Repo("+p+")", err, platform.ClassNotFound, platform.ErrNotFound) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("paths that name no repository sent %d requests", len(calls)) + } +} + +func TestRepoShape(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api", http.StatusOK, repo("", "acme/api")) + _, err := f.reader.Repo(t.Context(), "acme/api") + wantClass(t, "a repository without uuid", err, platform.ClassUnknown, nil) +} + +// workspaceRepos are the repositories of workspace acme, out of order, with +// a fork. +func workspaceRepos() []any { + return []any{ + repo("{00000000-0000-4000-8000-000000000003}", "acme/web"), + repo("{00000000-0000-4000-8000-000000000001}", "acme/API"), + repo(forkUUID, "acme/api-fork", with("parent", repoRef(repoUUID, "other/api"))), + repo("{00000000-0000-4000-8000-000000000002}", "acme/billing"), + repo("{00000000-0000-4000-8000-000000000004}", "acme/zeta"), + } +} + +func TestResolve(t *testing.T) { + f := newFixture(t) + f.pages("/repositories/acme", 2, workspaceRepos()) + got, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme", Subgroups: true}) + if err != nil { + t.Fatal(err) + } + var paths []string + for _, r := range got.Repos { + paths = append(paths, r.Path) + } + if !got.Complete || strings.Join(paths, " ") != "acme/API acme/billing acme/web acme/zeta" { + t.Errorf("Resolve = %v, complete %v; want the four repositories that are no fork, sorted ignoring case", paths, got.Complete) + } + calls := f.requests(http.MethodGet, "/repositories/acme") + if len(calls) != 3 || calls[0].Query.Get("pagelen") != "100" || calls[2].Query.Get("page") != "3" { + t.Errorf("calls %+v; want three pages, the next links followed", calls) + } + withForks, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme", Forks: true}) + if err != nil || len(withForks.Repos) != 5 { + t.Errorf("with forks: %d repositories, %v; want five", len(withForks.Repos), err) + } +} + +func TestResolveRepo(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api", http.StatusOK, repo(repoUUID, "acme/api", with("parent", repoRef(forkUUID, "other/api")))) + got, err := f.reader.Resolve(t.Context(), platform.Selector{Repo: "acme/api", Topics: []string{"ignored"}}) + if err != nil || !got.Complete || len(got.Repos) != 1 || got.Repos[0].ID != repoUUID { + t.Errorf("Resolve(repo) = %+v, %v; want the repository, a fork too", got, err) + } +} + +func TestResolveErrors(t *testing.T) { + t.Run("topics", func(t *testing.T) { + f := newFixture(t) + _, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme", Topics: []string{"python"}}) + wantClass(t, "topics", err, platform.ClassInvalid, nil) + if err != nil && !strings.Contains(err.Error(), "no topics") { + t.Errorf("error %q; want it to say Bitbucket has no topics", err) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("%d requests for a selector with topics", len(calls)) + } + }) + t.Run("no namespace", func(t *testing.T) { + f := newFixture(t) + _, err := f.reader.Resolve(t.Context(), platform.Selector{}) + wantClass(t, "empty", err, platform.ClassInvalid, nil) + _, err = f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme/sub"}) + wantClass(t, "nested", err, platform.ClassNotFound, platform.ErrNotFound) + }) + t.Run("missing workspace", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/gone", http.StatusNotFound, errorBody("No workspace with identifier 'gone'.")) + _, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "gone"}) + wantClass(t, "a missing workspace", err, platform.ClassNotFound, platform.ErrNotFound) + }) + t.Run("a later page fails", func(t *testing.T) { + f := newFixture(t) + f.handle(http.MethodGet, "/repositories/acme", func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("page") == "2" { + writeJSON(w, http.StatusForbidden, errorBody("Forbidden")) + return + } + servePage(w, r, 2, workspaceRepos()) + }) + got, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme"}) + if err != nil || got.Complete || len(got.Repos) != 2 { + t.Errorf("Resolve = %+v, %v; want the first page, incomplete", got, err) + } + }) + t.Run("a later page is rate limited", func(t *testing.T) { + f := newFixture(t) + f.handle(http.MethodGet, "/repositories/acme", func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("page") == "2" { + w.Header().Set("X-RateLimit-Limit", "1000") + w.Header().Set("X-RateLimit-Remaining", "0") + w.Header().Set("X-RateLimit-Reset", "394") + writeJSON(w, http.StatusTooManyRequests, errorBody("Rate limit for this resource has been exceeded")) + return + } + servePage(w, r, 2, workspaceRepos()) + }) + _, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme"}) + wantClass(t, "rate limited", err, platform.ClassRateLimited, nil) + }) + t.Run("a next link elsewhere", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme", http.StatusOK, map[string]any{ + "values": workspaceRepos()[:1], "pagelen": 1, "next": "https://evil.example.com/2.0/repositories/acme?page=2", + }) + _, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme"}) + wantClass(t, "a next link to another host", err, platform.ClassUnknown, nil) + if len(f.requests("", "")) != 1 { + t.Error("the driver followed a next link to another host") + } + }) +} + +func TestRemote(t *testing.T) { + f := newFixture(t) + r := platform.Repo{Host: f.provider().Host, ID: repoUUID, Path: "acme/api"} + rem, err := f.reader.Remote(t.Context(), r) + if err != nil { + t.Fatal(err) + } + if rem.URL != f.base()+"/acme/api.git" || rem.Header == nil { + t.Fatalf("Remote = %+v; want the web URL with .git and a header", rem) + } + h, err := rem.Header(t.Context()) + want := "Basic " + base64.StdEncoding.EncodeToString([]byte("x-bitbucket-api-token-auth:"+f.token)) + if err != nil || h != want { + t.Errorf("header %q, %v; want git's Basic form of an API token", h, err) + } + if _, err := f.reader.Remote(t.Context(), platform.Repo{Host: "bitbucket.example.com", Path: "acme/api"}); err == nil { + t.Error("a repository of another host got a remote") + } + if _, err := f.reader.Remote(t.Context(), platform.Repo{Path: "acme"}); err == nil { + t.Error("a path that is no workspace/repository got a remote") + } + anon, err := NewReader(f.provider(), auth.Credential{}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + if rem, err := anon.Remote(t.Context(), r); err != nil || rem.Header != nil { + t.Errorf("anonymous Remote = %+v, %v; want no header", rem, err) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("Remote sent %d requests", len(calls)) + } +} diff --git a/internal/platform/bitbucket/target.go b/internal/platform/bitbucket/target.go new file mode 100644 index 0000000..4a65346 --- /dev/null +++ b/internal/platform/bitbucket/target.go @@ -0,0 +1,544 @@ +package bitbucket + +import ( + "context" + "errors" + "fmt" + "net/http" + "net/url" + "strconv" + "strings" + "sync" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// Repository permissions of an account (the permission enum of the +// OpenAPI description's repository_permission). +const ( + permAdmin = "admin" + permWrite = "write" + permRead = "read" + permNone = "none" +) + +// maxPermPages bounds the listing of the writer's repository permissions in +// a workspace when the API does not filter it by repository. +const maxPermPages = 50 + +// apiRepoPermission is an entry of GET +// /2.0/user/workspaces/{workspace}/permissions/repositories. +type apiRepoPermission struct { + Permission string `json:"permission"` + Repository *apiRepoRef `json:"repository"` +} + +// permission returns the identity's permission on the repository with uuid +// in workspace ws: admin, write or read, and none when the listing does not +// name the repository (the API lists the repositories the account was +// given access to, not public ones it merely sees). The listing is +// filtered by the repository's uuid (q=repository.uuid="{…}"); when the API +// refuses the filter (400), the workspace's whole listing is read, at most +// maxPermPages pages. Entries are matched by uuid whatever the filter did. +func (c *client) permission(ctx context.Context, op, ws, uuid string) (string, error) { + u := c.endpoint("user", "workspaces", ws, "permissions", "repositories") + found := "" + each := func(p apiRepoPermission) error { + if p.Repository != nil && p.Repository.UUID == uuid && found == "" { + found = strings.ToLower(p.Permission) + } + return nil + } + q := url.Values{"q": {`repository.uuid="` + uuid + `"`}, "pagelen": {strconv.Itoa(repoPageLen)}} + complete, err := listAll(ctx, c, op, u, q, maxPermPages, each) + if err != nil && platform.ClassOf(err) == platform.ClassInvalid && !laterPage(err) { + complete, err = listAll(ctx, c, op, u, url.Values{"pagelen": {strconv.Itoa(repoPageLen)}}, maxPermPages, each) + } + switch { + case err != nil: + return "", err + case found != "": + return found, nil + case !complete: + return "", &platform.Error{Op: op, Class: platform.ClassUnknown, + Err: fmt.Errorf("the writer's permissions in %s span more than %d pages, and repository %s is not among those read", ws, maxPermPages, uuid)} + } + return permNone, nil +} + +// mayWrite reports whether a permission allows pushes and pull requests. +func mayWrite(perm string) bool { return perm == permWrite || perm == permAdmin } + +// Target checks that the writer may write to r and returns a TargetWriter +// bound to it, before the target's writes. The repository is read as the +// writer, by its uuid (a rename does not matter): one the writer cannot see +// is ClassNotFound. Its permission there must be write or admin (Bitbucket +// has one right for pushes and pull requests, and none of its own for CI +// files, so Workflows needs nothing more); otherwise the error is +// ClassPermission with Rule "contents" or "pull-requests". +func (w *writer) Target(ctx context.Context, r platform.Repo, need platform.Perms) (platform.TargetWriter, error) { + const op = "target" + if _, err := w.c.selfAccount(ctx); err != nil { + return nil, err + } + if _, err := w.Remote(ctx, r); err != nil { + return nil, err + } + repo, err := w.c.targetRepo(ctx, op, r) + if err != nil { + return nil, err + } + ws, slug, _ := splitRepoPath(repo.FullName) + perm, err := w.c.permission(ctx, op, ws, repo.UUID) + if err != nil { + return nil, err + } + switch { + case need.Contents && !mayWrite(perm): + return nil, &platform.Error{Op: op, Class: platform.ClassPermission, Status: http.StatusForbidden, Rule: "contents", + Err: fmt.Errorf("the writer has %s permission on %s, not write: it may not push", perm, repo.FullName)} + case need.PRs && !mayWrite(perm): + return nil, &platform.Error{Op: op, Class: platform.ClassPermission, Status: http.StatusForbidden, Rule: "pull-requests", + Err: fmt.Errorf("the writer has %s permission on %s, not write: it may not write pull requests", perm, repo.FullName)} + } + pr := w.c.toRepo(repo) + rem, err := w.Remote(ctx, pr) + if err != nil { + return nil, err + } + return &target{c: w.c, ws: ws, slug: slug, repo: pr, remote: rem.URL}, nil +} + +// targetRepo reads r as the identity: by its uuid when r has one (the path +// may be stale after a rename), else by its path. A repository that now +// answers with another uuid is ClassNotFound: r is gone. +func (c *client) targetRepo(ctx context.Context, op string, r platform.Repo) (*apiRepo, error) { + ws, slug, err := repoPath(op, r) + if err != nil { + return nil, err + } + if uuidRe.MatchString(r.ID) { + slug = r.ID + } + repo, err := c.getRepo(ctx, op, ws, slug) + if err != nil { + return nil, err + } + if r.ID != "" && repo.UUID != r.ID { + return nil, notFound(op, "%s is another repository (%s) now, not %s", r.Path, repo.UUID, r.ID) + } + return repo, nil +} + +// target is a platform.TargetWriter. The writer's token is not narrowed: +// Close retires the TargetWriter, and later calls through it fail with +// ClassAuth, as after a revoked per-target token elsewhere. +type target struct { + c *client + ws, slug string + repo platform.Repo + remote string + + mu sync.Mutex + closed bool +} + +// live returns closedError(op) after Close. +func (t *target) live(op string) error { + t.mu.Lock() + defer t.mu.Unlock() + if t.closed { + return closedError(op) + } + return nil +} + +// Remote is the repository's URL with the writer's Basic header +// ("x-bitbucket-api-token-auth:"), which fails with ClassAuth after +// Close. +func (t *target) Remote() platform.Remote { + return platform.Remote{URL: t.remote, Header: func(ctx context.Context) (string, error) { + if err := t.live("git credentials"); err != nil { + return "", err + } + return t.c.gitHeader(ctx) + }} +} + +// Close retires the TargetWriter; closing twice is fine. +func (t *target) Close() error { + t.mu.Lock() + t.closed = true + t.mu.Unlock() + return nil +} + +// apiBranchRef names a branch in a pull request's source or destination. +type apiBranchRef struct { + Branch struct { + Name string `json:"name"` + } `json:"branch"` +} + +// branchRef returns the endpoint of branch name. +func branchRef(name string) *apiBranchRef { + var b apiBranchRef + b.Branch.Name = name + return &b +} + +// apiAccountRef names an account by its uuid. +type apiAccountRef struct { + UUID string `json:"uuid"` +} + +// createPR is the body of POST …/pullrequests: from a branch of the +// repository itself, never closing the source branch on merge (people +// decide what happens to touchmark's branch; the next run deletes it). +type createPR struct { + Title string `json:"title"` + Description string `json:"description"` + Source *apiBranchRef `json:"source"` + Destination *apiBranchRef `json:"destination"` + CloseSourceBranch bool `json:"close_source_branch"` + Draft bool `json:"draft"` +} + +// editPR is the body of PUT …/pullrequests/{id}. Title, description and +// draft are always sent, as they are when unchanged; reviewers too when the +// pull request showed them, since a PUT without them is said to clear them +// (Renovate's experience); the destination only to change it. +type editPR struct { + Title string `json:"title"` + Description *string `json:"description,omitempty"` + Destination *apiBranchRef `json:"destination,omitempty"` + Reviewers *[]apiAccountRef `json:"reviewers,omitempty"` + Draft bool `json:"draft"` +} + +// comment is the body of POST …/pullrequests/{id}/comments. +type comment struct { + Content struct { + Raw string `json:"raw"` + } `json:"content"` +} + +// CreatePR opens a pull request from np.Head in the repository itself to +// np.Base, a draft when np.Draft (Bitbucket's own flag). Bitbucket has no +// labels: np.Labels are ignored (Caps.NoLabels). +// +// An open pull request from the same head in the repository itself, to +// whatever base, is returned with ErrExists before anything is written. +// When the POST is refused (400 or 409) and such a pull request is open +// then (one opened meanwhile), it is returned with ErrExists too; otherwise +// the refusal stands. A POST whose answer was lost is the core's to +// reconcile: it lists the pull requests of the head before it tries again. +func (t *target) CreatePR(ctx context.Context, np platform.NewPR) (platform.PR, error) { + const op = "create pull request" + if err := t.live(op); err != nil { + return platform.PR{}, err + } + switch { + case np.Head == "" || np.Base == "": + return platform.PR{}, invalid(op, "head and base are required") + case np.Head == np.Base: + return platform.PR{}, invalid(op, "head and base are both %q", np.Head) + case strings.TrimSpace(np.Title) == "": + return platform.PR{}, invalid(op, "the title is empty") + } + open, err := t.openFrom(ctx, op, np) + if err != nil { + return platform.PR{}, err + } + if open != nil { + return t.exists(ctx, op, open, np.Head, 0) + } + body := createPR{Title: np.Title, Description: np.Body, Source: branchRef(np.Head), Destination: branchRef(np.Base), Draft: np.Draft} + var p apiPR + _, err = t.c.call(ctx, op, http.MethodPost, t.prsURL(), nil, body, &p) + if err != nil { + if c := platform.ClassOf(err); c == platform.ClassInvalid || c == platform.ClassConflict { + return t.existing(ctx, op, np, err) + } + return platform.PR{}, err + } + if err := p.check(op); err != nil { + return platform.PR{}, err + } + return t.finish(ctx, op, &p) +} + +// existing returns the open pull request from np.Head in the repository +// itself with ErrExists, after refused, a 400 or 409 of CreatePR's POST; +// refused alone when none is open. +func (t *target) existing(ctx context.Context, op string, np platform.NewPR, refused error) (platform.PR, error) { + match, err := t.openFrom(ctx, op, np) + if err != nil { + return platform.PR{}, errors.Join(refused, err) + } + if match == nil { + return platform.PR{}, refused + } + return t.exists(ctx, op, match, np.Head, statusOf(refused)) +} + +// openFrom returns the open pull request from np.Head in the repository +// itself (source and destination), the one to np.Base first; nil when none +// is open. One from a fork's branch of that name is not from np.Head. +func (t *target) openFrom(ctx context.Context, op string, np platform.NewPR) (*apiPR, error) { + var match *apiPR + q := url.Values{ + "q": {prQuery([]string{np.Head}, []string{stateOpen})}, + "sort": {"-id"}, + "pagelen": {strconv.Itoa(prPageLen)}, + } + complete, err := listAll(ctx, t.c, op, t.prsURL(), q, maxPRPages, func(p apiPR) error { + if err := p.check(op); err != nil { + return err + } + if !t.ownOpen(&p, np.Head) { + return nil + } + if match == nil || p.Destination.Branch.Name == np.Base && match.Destination.Branch.Name != np.Base { + match = &p + } + return nil + }) + switch { + case err != nil: + return nil, err + case !complete && match == nil: + return nil, &platform.Error{Op: op, Class: platform.ClassUnknown, + Err: fmt.Errorf("%s has more open pull requests from %s than %d pages", t.repo.Path, np.Head, maxPRPages)} + } + return match, nil +} + +// ownOpen reports whether p is an open pull request from head in the +// repository itself. +func (t *target) ownOpen(p *apiPR, head string) bool { + return p.State == stateOpen && p.head() == head && + p.Source.Repository != nil && p.Source.Repository.UUID == t.repo.ID && + p.Destination.Repository.UUID == t.repo.ID +} + +// exists returns p, the open pull request from head, with the ErrExists of +// CreatePR; status is the HTTP status that told it (0 when a listing did). +func (t *target) exists(ctx context.Context, op string, p *apiPR, head string, status int) (platform.PR, error) { + pr, err := t.finish(ctx, op, p) + if err != nil { + return platform.PR{}, err + } + return pr, &platform.Error{Op: op, Class: platform.ClassConflict, Status: status, + Err: fmt.Errorf("#%d is open from %s: %w", p.ID, head, platform.ErrExists)} +} + +// finish converts a pull request the API answered with, as PRs reports +// it: its description read alone when the answer left it out, BaseExists, +// and the full id of its head for an open pull request from the +// repository itself. +func (t *target) finish(ctx context.Context, op string, p *apiPR) (platform.PR, error) { + if err := t.c.fillBody(ctx, op, t.ws, t.slug, p); err != nil { + return platform.PR{}, err + } + pr := t.c.toPR(p) + var err error + if pr.BaseExists, err = t.c.baseExists(ctx, op, t.repo, t.ws, t.slug, pr.Base, map[string]bool{}); err != nil { + return platform.PR{}, err + } + if pr.State == platform.Open && pr.HeadRepoID != "" && pr.HeadRepoID == pr.RepoID && p.Source.Commit != nil { + if pr.HeadSHA, err = t.c.fullCommit(ctx, op, t.ws, t.slug, pr.RepoID, p.Source.Commit.Hash); err != nil { + return platform.PR{}, err + } + } + return pr, nil +} + +// EditPR changes a pull request. It reads the pull request first: only an +// open one can be changed on Bitbucket. +// +// - Open: title, body and base go in one PUT (the current title, body, +// draft flag and reviewers sent back with them, so that nothing else +// changes), skipped when it would change nothing; then, for State +// Closed, the pull request is declined (POST …/decline). The PUT comes +// first: once declined, a pull request can never be changed again, and +// the body touchmark closes its own pull requests with carries the +// marker's "closed". A failure between the two leaves an open pull +// request with the new body, which the next run decides on again. +// - Declined or superseded: State Closed alone changes nothing; State +// Open is ClassUnsupported (Bitbucket cannot reopen a declined pull +// request), and so is any change of title, body or base. +// - Merged: any change is ClassConflict, as a pull request merged since +// the core read it. +// +// State Merged is ClassUnsupported: touchmark merges nothing. Labels are +// ignored (Caps.NoLabels); the draft flag never changes. A PUT refused +// because of the reviewers sent back (400 naming reviewers: one left the +// workspace or was deactivated) is sent once more without them. +func (t *target) EditPR(ctx context.Context, number int64, e platform.PREdit) (platform.PR, error) { + const op = "edit pull request" + if err := t.live(op); err != nil { + return platform.PR{}, err + } + switch { + case number <= 0: + return platform.PR{}, notFound(op, "pull request #%d", number) + case e.Title != nil && strings.TrimSpace(*e.Title) == "": + return platform.PR{}, invalid(op, "the title is empty") + case e.State != nil && *e.State == platform.Merged: + return platform.PR{}, &platform.Error{Op: op, Class: platform.ClassUnsupported, Err: errors.New("touchmark merges no pull request")} + case e.State != nil && *e.State != platform.Open && *e.State != platform.Closed: + return platform.PR{}, invalid(op, "state %q: only open and closed can be set", *e.State) + case e.Base != nil && *e.Base == "": + return platform.PR{}, invalid(op, "the base is empty") + } + cur, err := t.getPR(ctx, op, number) + if err != nil { + return platform.PR{}, err + } + body, _ := cur.body() + changes := e.Title != nil && *e.Title != cur.Title || e.Body != nil && *e.Body != body || + e.Base != nil && *e.Base != cur.Destination.Branch.Name + switch cur.State { + case stateMerged: + if changes || e.State != nil { + return platform.PR{}, conflict(op, "#%d is merged: it can no longer be changed", number) + } + return t.finish(ctx, op, cur) + case stateDeclined, stateSuperseded: + switch { + case e.State != nil && *e.State == platform.Open: + return platform.PR{}, &platform.Error{Op: op, Class: platform.ClassUnsupported, + Err: fmt.Errorf("#%d: Bitbucket cannot reopen a declined pull request", number)} + case changes: + return platform.PR{}, &platform.Error{Op: op, Class: platform.ClassUnsupported, + Err: fmt.Errorf("#%d is %s: Bitbucket changes open pull requests only", number, strings.ToLower(cur.State))} + } + return t.finish(ctx, op, cur) + } + if changes { + put := editPR{Title: cur.Title, Draft: cur.Draft} + if e.Title != nil { + put.Title = *e.Title + } + switch { + case e.Body != nil: + put.Description = e.Body + case cur.Summary != nil && cur.Summary.Raw != nil || cur.Description != nil: + put.Description = &body + } + if e.Base != nil && *e.Base != cur.Destination.Branch.Name { + put.Destination = branchRef(*e.Base) + } + if cur.Reviewers != nil { + refs := make([]apiAccountRef, 0, len(cur.Reviewers)) + for _, r := range cur.Reviewers { + if r.UUID != "" { + refs = append(refs, apiAccountRef{UUID: r.UUID}) + } + } + put.Reviewers = &refs + } + got, err := t.put(ctx, op, number, put) + if err != nil && put.Reviewers != nil && len(*put.Reviewers) > 0 && reviewersRefused(err) { + put.Reviewers = nil + got, err = t.put(ctx, op, number, put) + } + if err != nil { + return platform.PR{}, err + } + cur = got + } + if e.State != nil && *e.State == platform.Closed { + // The answer of a decline is not described: the pull request is + // read again. + if _, err := t.c.call(ctx, op, http.MethodPost, t.prURL(number, "decline"), nil, nil, nil); err != nil { + return platform.PR{}, err + } + got, err := t.getPR(ctx, op, number) + if err != nil { + return platform.PR{}, err + } + if got.State == stateOpen { + return platform.PR{}, conflict(op, "Bitbucket did not decline #%d (it is %s)", number, got.State) + } + cur = got + } + return t.finish(ctx, op, cur) +} + +// reviewersRefused reports whether err, a refused PUT, is about its +// reviewers (a 400 whose message or fields name them). +func reviewersRefused(err error) bool { + return statusOf(err) == http.StatusBadRequest && strings.Contains(strings.ToLower(err.Error()), "reviewer") +} + +// getPR reads pull request number alone (with its reviewers). +func (t *target) getPR(ctx context.Context, op string, number int64) (*apiPR, error) { + var p apiPR + if _, err := t.c.get(ctx, op, t.prURL(number), nil, &p); err != nil { + return nil, err + } + if err := p.check(op); err != nil { + return nil, err + } + if p.ID != number { + return nil, shapeError(op, "GET pull request #%d answers #%d", number, p.ID) + } + return &p, nil +} + +// put sends one PUT of pull request number and returns the pull request it +// answers with. +func (t *target) put(ctx context.Context, op string, number int64, body editPR) (*apiPR, error) { + var got apiPR + if _, err := t.c.call(ctx, op, http.MethodPut, t.prURL(number), nil, body, &got); err != nil { + return nil, err + } + if err := got.check(op); err != nil { + return nil, err + } + return &got, nil +} + +// Comment adds a comment to pull request number (POST …/comments with the +// text as raw Markdown). Bitbucket takes comments on declined and merged +// pull requests too. +func (t *target) Comment(ctx context.Context, number int64, body string) error { + const op = "comment" + if err := t.live(op); err != nil { + return err + } + switch { + case number <= 0: + return notFound(op, "pull request #%d", number) + case strings.TrimSpace(body) == "": + return invalid(op, "the comment is empty") + } + var c comment + c.Content.Raw = body + _, err := t.c.call(ctx, op, http.MethodPost, t.prURL(number, "comments"), nil, c, nil) + return err +} + +// EnsureLabels does nothing: Bitbucket pull requests have no labels +// (Caps.NoLabels), and the core asks for none. +func (t *target) EnsureLabels(context.Context, []string) ([]string, error) { + if err := t.live("ensure labels"); err != nil { + return nil, err + } + return nil, nil +} + +func (t *target) prsURL() string { + return t.c.endpoint("repositories", t.ws, t.slug, "pullrequests") +} + +func (t *target) prURL(number int64, more ...string) string { + return t.c.endpoint(append([]string{"repositories", t.ws, t.slug, "pullrequests", strconv.FormatInt(number, 10)}, more...)...) +} + +// conflict is a ClassConflict error of op. +func conflict(op, format string, args ...any) error { + return &platform.Error{Op: op, Class: platform.ClassConflict, Err: fmt.Errorf(format, args...)} +} diff --git a/internal/platform/bitbucket/target_test.go b/internal/platform/bitbucket/target_test.go new file mode 100644 index 0000000..b9a0906 --- /dev/null +++ b/internal/platform/bitbucket/target_test.go @@ -0,0 +1,677 @@ +package bitbucket + +import ( + "encoding/base64" + "encoding/json" + "errors" + "net/http" + "slices" + "strings" + "testing" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +// writerFixture is a writer with a token over one test server that takes +// the writes its routes declare. GET /user and /user/emails answer for the +// bot account. +type writerFixture struct { + *apiServer + token string + writer *writer +} + +// botEmail is the bot account's primary confirmed address. +const botEmail = "touchmark-bot@acme.example" + +// Paths of acme/api's pull requests and of the writer's permissions in acme. +const ( + prsPath = "/repositories/acme/api/pullrequests" + permsPath = "/user/workspaces/acme/permissions/repositories" +) + +func newWriterFixture(t *testing.T) *writerFixture { + t.Helper() + s := newAPIServer(t) + s.allowWrites = true + tok := testToken(t) + w, err := NewWriter(s.provider(), auth.Credential{Kind: auth.Token, Token: tok}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + s.json("/user", http.StatusOK, account(botUUID, "touchmark bot")) + s.pages("/user/emails", 10, []any{ + map[string]any{"type": "email", "email": "old@acme.example", "is_primary": false, "is_confirmed": true}, + map[string]any{"type": "email", "email": botEmail, "is_primary": true, "is_confirmed": true}, + }) + return &writerFixture{apiServer: s, token: tok, writer: w.(*writer)} +} + +// repoRoute declares GET /repositories/acme/{uuid} of acme/api. +func (f *writerFixture) repoRoute() { + f.json("/repositories/acme/"+uuidPath(repoUUID), http.StatusOK, repo(repoUUID, "acme/api")) +} + +// permRoute declares the writer's permission on acme/api ("" for none: +// the listing does not name it). +func (f *writerFixture) permRoute(perm string) { + var items []any + if perm != "" { + items = append(items, map[string]any{"type": "repository_permission", "permission": perm, + "user": account(botUUID, "touchmark bot"), "repository": repoRef(repoUUID, "acme/api")}) + } + f.pages(permsPath, 10, items) +} + +// target returns the TargetWriter of acme/api, the writer having write +// permission, with the recorded calls forgotten. +func (f *writerFixture) target(t *testing.T) *target { + t.Helper() + f.repoRoute() + f.permRoute(permWrite) + tw, err := f.writer.Target(t.Context(), apiRepoFixture, platform.Perms{Contents: true, PRs: true}) + if err != nil { + t.Fatal(err) + } + f.reset() + return tw.(*target) +} + +// body decodes the JSON body of a recorded request. +func body(t *testing.T, c apiCall) map[string]any { + t.Helper() + var m map[string]any + if err := json.Unmarshal(c.Body, &m); err != nil { + t.Fatalf("%s %s: body %q: %v", c.Method, c.Path, c.Body, err) + } + return m +} + +// writeLog returns the writes recorded, as "METHOD path". +func (s *apiServer) writeLog() []string { + var out []string + for _, c := range s.writes() { + out = append(out, c.Method+" "+strings.TrimPrefix(c.Path, "/2.0")) + } + return out +} + +func TestWriterSelfEmail(t *testing.T) { + f := newWriterFixture(t) + a, err := f.writer.Self(t.Context()) + if err != nil { + t.Fatal(err) + } + if a != (platform.Account{ID: botUUID, Login: botUUID, Email: botEmail, Kind: platform.KindUser}) { + t.Errorf("Self = %+v; want the primary confirmed address", a) + } + for name, tc := range map[string]struct { + status int + items []any + want string + class platform.Class + }{ + "unconfirmed primary": {http.StatusOK, []any{map[string]any{"email": "x@acme.example", "is_primary": true, "is_confirmed": false}}, "", 0}, + "not an address": {http.StatusOK, []any{map[string]any{"email": "a @c", "is_primary": true, "is_confirmed": true}}, "", 0}, + "forbidden": {http.StatusForbidden, nil, "", 0}, + "rate limited": {http.StatusTooManyRequests, nil, "", platform.ClassRateLimited}, + } { + t.Run(name, func(t *testing.T) { + f := newWriterFixture(t) + if tc.status == http.StatusOK { + f.pages("/user/emails", 10, tc.items) + } else { + f.json("/user/emails", tc.status, errorBody("no")) + } + a, err := f.writer.Self(t.Context()) + if tc.class != 0 { + wantClass(t, "Self", err, tc.class, nil) + return + } + if err != nil || a.Email != tc.want || a.ID != botUUID { + t.Errorf("Self = %+v, %v; want email %q", a, err, tc.want) + } + }) + } +} + +func TestTarget(t *testing.T) { + for name, tc := range map[string]struct { + perm string + need platform.Perms + class platform.Class + rule string + }{ + "write": {permWrite, platform.Perms{Contents: true, PRs: true}, 0, ""}, + "admin": {permAdmin, platform.Perms{Contents: true, PRs: true, Workflows: true}, 0, ""}, + "read": {permRead, platform.Perms{Contents: true, PRs: true}, platform.ClassPermission, "contents"}, + "none": {"", platform.Perms{Contents: true}, platform.ClassPermission, "contents"}, + "read, pull requests": {permRead, platform.Perms{PRs: true}, platform.ClassPermission, "pull-requests"}, + "read, nothing to need": {permRead, platform.Perms{}, 0, ""}, + } { + t.Run(name, func(t *testing.T) { + f := newWriterFixture(t) + f.repoRoute() + f.permRoute(tc.perm) + tw, err := f.writer.Target(t.Context(), apiRepoFixture, tc.need) + if tc.class != 0 { + wantClass(t, "Target", err, tc.class, nil) + var pe *platform.Error + if !asError(err, &pe) || pe.Rule != tc.rule || pe.Status != http.StatusForbidden { + t.Errorf("Target = %v; want rule %q", err, tc.rule) + } + return + } + if err != nil || tw == nil { + t.Fatalf("Target = %v, %v", tw, err) + } + calls := f.requests(http.MethodGet, permsPath) + if len(calls) != 1 || calls[0].Query.Get("q") != `repository.uuid="`+repoUUID+`"` { + t.Errorf("permission requests %+v; want one filtered by the uuid", calls) + } + }) + } +} + +func TestTargetErrors(t *testing.T) { + t.Run("hidden repository", func(t *testing.T) { + f := newWriterFixture(t) + f.json("/repositories/acme/"+uuidPath(repoUUID), http.StatusNotFound, errorBody(noRepoMessage)) + _, err := f.writer.Target(t.Context(), apiRepoFixture, platform.Perms{Contents: true}) + wantClass(t, "Target", err, platform.ClassNotFound, platform.ErrNotFound) + }) + t.Run("another repository under the path", func(t *testing.T) { + f := newWriterFixture(t) + f.json("/repositories/acme/api", http.StatusOK, repo(otherUUID, "acme/api")) + r := apiRepoFixture + r.ID = "42" + _, err := f.writer.Target(t.Context(), r, platform.Perms{Contents: true}) + wantClass(t, "Target", err, platform.ClassNotFound, platform.ErrNotFound) + }) + t.Run("another host", func(t *testing.T) { + f := newWriterFixture(t) + r := apiRepoFixture + r.Host = "bitbucket.example.com" + _, err := f.writer.Target(t.Context(), r, platform.Perms{Contents: true}) + wantClass(t, "Target", err, platform.ClassInvalid, nil) + }) + t.Run("refused token", func(t *testing.T) { + s := newAPIServer(t) + w, err := NewWriter(s.provider(), auth.Credential{Kind: auth.Token, Token: testToken(t)}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + s.json("/user", http.StatusUnauthorized, errorBody("Token is invalid, expired, or not supported for this endpoint.")) + _, err = w.Target(t.Context(), apiRepoFixture, platform.Perms{Contents: true}) + wantClass(t, "Target", err, platform.ClassAuth, nil) + }) + t.Run("filter refused", func(t *testing.T) { + f := newWriterFixture(t) + f.repoRoute() + var items []any + for i := range 25 { + items = append(items, map[string]any{"permission": permRead, "repository": repoRef("{00000000-0000-4000-8000-0000000000"+twoDigits(i)+"}", "acme/r")}) + } + items = append(items, map[string]any{"permission": permWrite, "repository": repoRef(repoUUID, "acme/api")}) + f.handle(http.MethodGet, permsPath, func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("q") != "" { + writeJSON(w, http.StatusBadRequest, errorBody(`Field "repository.uuid" does not support filtering`)) + return + } + servePage(w, r, 10, items) + }) + if _, err := f.writer.Target(t.Context(), apiRepoFixture, platform.Perms{Contents: true}); err != nil { + t.Fatalf("Target: %v", err) + } + if n := len(f.requests(http.MethodGet, permsPath)); n != 4 { + t.Errorf("%d permission requests; want the refused filter and three pages", n) + } + }) + t.Run("permission listing fails", func(t *testing.T) { + f := newWriterFixture(t) + f.repoRoute() + f.json(permsPath, http.StatusForbidden, errorBody("The requesting user does not have access to the workspace.")) + _, err := f.writer.Target(t.Context(), apiRepoFixture, platform.Perms{Contents: true}) + wantClass(t, "Target", err, platform.ClassPermission, nil) + }) +} + +// twoDigits formats i with two digits. +func twoDigits(i int) string { return string([]byte{'0' + byte(i/10), '0' + byte(i%10)}) } + +// asError is errors.As for *platform.Error. +func asError(err error, pe **platform.Error) bool { return errors.As(err, pe) } + +func TestTargetRemote(t *testing.T) { + f := newWriterFixture(t) + tg := f.target(t) + rem := tg.Remote() + if rem.URL != f.base()+"/acme/api.git" { + t.Errorf("URL %q", rem.URL) + } + h, err := rem.Header(t.Context()) + want := "Basic " + base64.StdEncoding.EncodeToString([]byte("x-bitbucket-api-token-auth:"+f.token)) + if err != nil || h != want { + t.Errorf("Header = %q, %v; want the API token's Basic credentials", h, err) + } + if err := tg.Close(); err != nil { + t.Fatal(err) + } + if err := tg.Close(); err != nil { + t.Errorf("a second Close: %v", err) + } + _, err = rem.Header(t.Context()) + wantClass(t, "Header after Close", err, platform.ClassAuth, nil) + _, err = tg.CreatePR(t.Context(), platform.NewPR{Head: syncBranch, Base: "main", Title: "x"}) + wantClass(t, "CreatePR after Close", err, platform.ClassAuth, nil) + _, err = tg.EditPR(t.Context(), 1, platform.PREdit{}) + wantClass(t, "EditPR after Close", err, platform.ClassAuth, nil) + wantClass(t, "Comment after Close", tg.Comment(t.Context(), 1, "x"), platform.ClassAuth, nil) + _, err = tg.EnsureLabels(t.Context(), nil) + wantClass(t, "EnsureLabels after Close", err, platform.ClassAuth, nil) + if n := len(f.requests("", "")); n != 0 { + t.Errorf("%d requests after Close", n) + } +} + +// listing declares GET of acme/api's pull requests (items) and POST, which +// answers created. +func (f *writerFixture) prRoutes(items []any, status int, created any) { + f.pages(prsPath, 10, items) + f.handle(http.MethodPost, prsPath, func(w http.ResponseWriter, _ *http.Request) { writeJSON(w, status, created) }) +} + +func TestCreatePR(t *testing.T) { + for _, draft := range []bool{false, true} { + f := newWriterFixture(t) + tg := f.target(t) + bot := account(botUUID, "touchmark bot") + created := pr(prSpec{id: 12, author: bot, source: syncBranch, title: "chore: sync", body: "body\n\n[touchmark]: # \"x\"", draft: draft}) + fork := pr(prSpec{id: 3, author: account(personUUID, "someone"), source: syncBranch, sourceRepo: repoRef(forkUUID, "someone/api")}) + f.prRoutes([]any{fork}, http.StatusCreated, created) + f.commitRoute("a6dcc740200c") + got, err := tg.CreatePR(t.Context(), platform.NewPR{Head: syncBranch, Base: "main", Title: "chore: sync", + Body: "body\n\n[touchmark]: # \"x\"", Labels: []string{"engineering-assets"}, Draft: draft}) + if err != nil { + t.Fatal(err) + } + if got.Number != 12 || got.State != platform.Open || got.Draft != draft || got.Head != syncBranch || got.Base != "main" || + !got.BaseExists || got.HeadSHA != fullHash("a6dcc740200c") || got.Author.ID != botUUID || len(got.Labels) != 0 || + got.Body != "body\n\n[touchmark]: # \"x\"" { + t.Errorf("CreatePR = %+v", got) + } + posts := f.requests(http.MethodPost, prsPath) + if len(posts) != 1 { + t.Fatalf("%d POSTs", len(posts)) + } + b := body(t, posts[0]) + want := map[string]any{ + "title": "chore: sync", "description": "body\n\n[touchmark]: # \"x\"", + "source": map[string]any{"branch": map[string]any{"name": syncBranch}}, + "destination": map[string]any{"branch": map[string]any{"name": "main"}}, + "close_source_branch": false, "draft": draft, + } + if js, wantJS := mustJSON(t, b), mustJSON(t, want); js != wantJS { + t.Errorf("POST body %s\nwant %s", js, wantJS) + } + lists := f.requests(http.MethodGet, prsPath) + if len(lists) != 1 || lists[0].Query.Get("q") != `source.branch.name IN ("`+syncBranch+`") AND state IN ("OPEN")` { + t.Errorf("listings %+v", lists) + } + } +} + +// mustJSON returns v as canonical JSON. +func mustJSON(t *testing.T, v any) string { + t.Helper() + b, err := json.Marshal(v) + if err != nil { + t.Fatal(err) + } + return string(b) +} + +func TestCreatePRExists(t *testing.T) { + bot := account(botUUID, "touchmark bot") + np := platform.NewPR{Head: syncBranch, Base: "main", Title: "chore: sync", Body: "b"} + t.Run("open before", func(t *testing.T) { + f := newWriterFixture(t) + tg := f.target(t) + f.pages(prsPath, 10, []any{ + pr(prSpec{id: 7, author: bot, source: syncBranch, target: "release"}), + pr(prSpec{id: 5, author: bot, source: syncBranch}), + }) + f.commitRoute("a6dcc7402005") + got, err := tg.CreatePR(t.Context(), np) + wantClass(t, "CreatePR", err, platform.ClassConflict, platform.ErrExists) + if got.Number != 5 { + t.Errorf("CreatePR returned #%d; want #5, the one to main", got.Number) + } + if w := f.writes(); len(w) != 0 { + t.Errorf("writes %+v", w) + } + }) + t.Run("opened meanwhile", func(t *testing.T) { + f := newWriterFixture(t) + tg := f.target(t) + listed := 0 + f.handle(http.MethodGet, prsPath, func(w http.ResponseWriter, r *http.Request) { + listed++ + var items []any + if listed > 1 { + items = append(items, pr(prSpec{id: 6, author: bot, source: syncBranch})) + } + servePage(w, r, 10, items) + }) + f.handle(http.MethodPost, prsPath, func(w http.ResponseWriter, _ *http.Request) { + writeJSON(w, http.StatusBadRequest, errorBody("There are no changes to be pulled")) + }) + f.commitRoute("a6dcc7402006") + got, err := tg.CreatePR(t.Context(), np) + wantClass(t, "CreatePR", err, platform.ClassConflict, platform.ErrExists) + var pe *platform.Error + if got.Number != 6 || !asError(err, &pe) || pe.Status != http.StatusBadRequest { + t.Errorf("CreatePR = #%d, %v", got.Number, err) + } + }) + t.Run("refused", func(t *testing.T) { + f := newWriterFixture(t) + tg := f.target(t) + f.prRoutes(nil, http.StatusBadRequest, errorBody("source branch not found: "+f.token)) + _, err := tg.CreatePR(t.Context(), np) + wantClass(t, "CreatePR", err, platform.ClassInvalid, nil) + if err != nil && strings.Contains(err.Error(), f.token) { + t.Errorf("the error shows the token: %v", err) + } + if n := len(f.requests(http.MethodGet, prsPath)); n != 2 { + t.Errorf("%d listings; want one before and one after the refusal", n) + } + }) + t.Run("invalid", func(t *testing.T) { + f := newWriterFixture(t) + tg := f.target(t) + for _, np := range []platform.NewPR{{Base: "main", Title: "t"}, {Head: "x", Base: "x", Title: "t"}, {Head: "x", Base: "main", Title: " "}} { + _, err := tg.CreatePR(t.Context(), np) + wantClass(t, "CreatePR", err, platform.ClassInvalid, nil) + } + if n := len(f.requests("", "")); n != 0 { + t.Errorf("%d requests", n) + } + }) +} + +// editFixture is a TargetWriter over pull request #4 of acme/api, open +// (or in state) from the sync branch, with a reviewer, and the routes of +// its edits: PUT answers the body it was sent applied to the pull request, +// POST …/decline declines it. +type editFixture struct { + *writerFixture + tg *target + // cur is the pull request as the server holds it. + cur map[string]any + // putStatus, when set, answers PUTs; putBody is its body. + putStatus []int + putBody any + // keepOpen makes the decline answer 200 without declining. + keepOpen bool +} + +const editPath = prsPath + "/4" + +func newEditFixture(t *testing.T, state string) *editFixture { + t.Helper() + wf := newWriterFixture(t) + f := &editFixture{writerFixture: wf, tg: wf.target(t)} + f.cur = pr(prSpec{id: 4, state: state, author: account(botUUID, "touchmark bot"), closedBy: account(botUUID, "touchmark bot"), + source: syncBranch, title: "chore: sync", body: "old body\r\n\r\n[touchmark]: # \"old\""}) + f.cur["reviewers"] = []any{account(personUUID, "Reviewer")} + f.handle(http.MethodGet, editPath, func(w http.ResponseWriter, _ *http.Request) { writeJSON(w, http.StatusOK, f.cur) }) + f.handle(http.MethodPut, editPath, func(w http.ResponseWriter, r *http.Request) { + if len(f.putStatus) > 0 { + status := f.putStatus[0] + f.putStatus = f.putStatus[1:] + if status != http.StatusOK { + writeJSON(w, status, f.putBody) + return + } + } + var in map[string]any + if err := json.NewDecoder(r.Body).Decode(&in); err != nil { + t.Errorf("PUT body: %v", err) + } + f.cur["title"] = in["title"] + if d, ok := in["description"].(string); ok { + f.cur["description"] = d + f.cur["summary"] = map[string]any{"raw": d, "markup": "markdown", "html": ""} + } + if dst, ok := in["destination"].(map[string]any); ok { + f.cur["destination"].(map[string]any)["branch"] = dst["branch"] + } + writeJSON(w, http.StatusOK, f.cur) + }) + f.handle(http.MethodPost, editPath+"/decline", func(w http.ResponseWriter, _ *http.Request) { + if !f.keepOpen { + f.cur["state"] = "DECLINED" + f.cur["closed_by"] = account(botUUID, "touchmark bot") + } + writeJSON(w, http.StatusOK, f.cur) + }) + f.commitRoute("a6dcc7402004") + f.branch("release", headCommit) + return f +} + +func ptr[T any](v T) *T { return &v } + +func TestEditPRFields(t *testing.T) { + f := newEditFixture(t, "OPEN") + got, err := f.tg.EditPR(t.Context(), 4, platform.PREdit{Title: ptr("chore: new"), Body: ptr("new body"), Base: ptr("release"), + AddLabels: []string{"engineering-assets"}}) + if err != nil { + t.Fatal(err) + } + if got.Title != "chore: new" || got.Body != "new body" || got.Base != "release" || !got.BaseExists || got.State != platform.Open || + got.HeadSHA != fullHash("a6dcc7402004") || len(got.Labels) != 0 { + t.Errorf("EditPR = %+v", got) + } + if w := f.writeLog(); !slices.Equal(w, []string{"PUT " + editPath}) { + t.Fatalf("writes %q", w) + } + b := body(t, f.requests(http.MethodPut, editPath)[0]) + want := map[string]any{ + "title": "chore: new", "description": "new body", "draft": false, + "destination": map[string]any{"branch": map[string]any{"name": "release"}}, + "reviewers": []any{map[string]any{"uuid": personUUID}}, + } + if js, wantJS := mustJSON(t, b), mustJSON(t, want); js != wantJS { + t.Errorf("PUT body %s\nwant %s", js, wantJS) + } +} + +func TestEditPRKeepsWhatItDoesNotChange(t *testing.T) { + f := newEditFixture(t, "OPEN") + f.cur["draft"] = true + if _, err := f.tg.EditPR(t.Context(), 4, platform.PREdit{Body: ptr("new body")}); err != nil { + t.Fatal(err) + } + b := body(t, f.requests(http.MethodPut, editPath)[0]) + if b["title"] != "chore: sync" || b["draft"] != true || b["destination"] != nil || b["description"] != "new body" { + t.Errorf("PUT body %v; want the title and draft flag kept, no destination", b) + } + // Nothing to change: no PUT. + f.reset() + if _, err := f.tg.EditPR(t.Context(), 4, platform.PREdit{Title: ptr("chore: sync"), Body: ptr("new body"), Base: ptr("main")}); err != nil { + t.Fatal(err) + } + if w := f.writeLog(); len(w) != 0 { + t.Errorf("an edit that changes nothing wrote %q", w) + } +} + +func TestEditPRClose(t *testing.T) { + f := newEditFixture(t, "OPEN") + body := "closed by touchmark\n\n[touchmark]: # \"closed\"" + got, err := f.tg.EditPR(t.Context(), 4, platform.PREdit{Body: &body, State: ptr(platform.Closed)}) + if err != nil { + t.Fatal(err) + } + if got.State != platform.Closed || got.Body != body || got.ClosedBy == nil || got.ClosedBy.ID != botUUID || got.HeadSHA != "" { + t.Errorf("EditPR = %+v", got) + } + // The body first: a declined pull request can no longer be changed. + if w := f.writeLog(); !slices.Equal(w, []string{"PUT " + editPath, "POST " + editPath + "/decline"}) { + t.Errorf("writes %q; want the PUT, then the decline", w) + } + + // A close alone declines. + f = newEditFixture(t, "OPEN") + if _, err := f.tg.EditPR(t.Context(), 4, platform.PREdit{State: ptr(platform.Closed)}); err != nil { + t.Fatal(err) + } + if w := f.writeLog(); !slices.Equal(w, []string{"POST " + editPath + "/decline"}) { + t.Errorf("writes %q", w) + } + + // A decline that leaves the pull request open is a conflict. + f = newEditFixture(t, "OPEN") + f.keepOpen = true + _, err = f.tg.EditPR(t.Context(), 4, platform.PREdit{State: ptr(platform.Closed)}) + wantClass(t, "EditPR", err, platform.ClassConflict, nil) +} + +func TestEditPRClosedPullRequests(t *testing.T) { + for _, state := range []string{"DECLINED", "SUPERSEDED"} { + t.Run(state, func(t *testing.T) { + f := newEditFixture(t, state) + _, err := f.tg.EditPR(t.Context(), 4, platform.PREdit{State: ptr(platform.Open)}) + wantClass(t, "reopen", err, platform.ClassUnsupported, nil) + if err == nil || !strings.Contains(err.Error(), "Bitbucket cannot reopen a declined pull request") { + t.Errorf("reopen: %v", err) + } + for name, e := range map[string]platform.PREdit{ + "body": {Body: ptr("acked")}, + "body and close": {Body: ptr("acked"), State: ptr(platform.Closed)}, + "title": {Title: ptr("chore: new")}, + "base": {Base: ptr("release")}, + } { + _, err := f.tg.EditPR(t.Context(), 4, e) + wantClass(t, name, err, platform.ClassUnsupported, nil) + } + got, err := f.tg.EditPR(t.Context(), 4, platform.PREdit{State: ptr(platform.Closed), Body: ptr("old body\r\n\r\n[touchmark]: # \"old\"")}) + if err != nil || got.State != platform.Closed { + t.Errorf("a close of a closed pull request: %+v, %v; want nothing to do", got, err) + } + if w := f.writeLog(); len(w) != 0 { + t.Errorf("writes %q", w) + } + }) + } + f := newEditFixture(t, "MERGED") + for name, e := range map[string]platform.PREdit{ + "close": {State: ptr(platform.Closed)}, + "body": {Body: ptr("x")}, + } { + _, err := f.tg.EditPR(t.Context(), 4, e) + wantClass(t, "merged: "+name, err, platform.ClassConflict, nil) + } + _, err := f.tg.EditPR(t.Context(), 4, platform.PREdit{State: ptr(platform.Merged)}) + wantClass(t, "set merged", err, platform.ClassUnsupported, nil) + if w := f.writeLog(); len(w) != 0 { + t.Errorf("writes %q", w) + } +} + +func TestEditPRReviewersRefused(t *testing.T) { + f := newEditFixture(t, "OPEN") + f.putStatus = []int{http.StatusBadRequest, http.StatusOK} + f.putBody = map[string]any{"type": "error", "error": map[string]any{"message": "Bad request", + "fields": map[string]any{"reviewers": []string{"Malformed reviewers list"}}}} + if _, err := f.tg.EditPR(t.Context(), 4, platform.PREdit{Body: ptr("new")}); err != nil { + t.Fatal(err) + } + puts := f.requests(http.MethodPut, editPath) + if len(puts) != 2 || body(t, puts[0])["reviewers"] == nil || body(t, puts[1])["reviewers"] != nil { + t.Errorf("PUTs %d; want one with the reviewers, then one without", len(puts)) + } + // Another refusal stands. + f = newEditFixture(t, "OPEN") + f.putStatus = []int{http.StatusBadRequest} + f.putBody = errorBody("the destination branch does not exist") + _, err := f.tg.EditPR(t.Context(), 4, platform.PREdit{Base: ptr("nope")}) + wantClass(t, "EditPR", err, platform.ClassInvalid, nil) + if n := len(f.requests(http.MethodPut, editPath)); n != 1 { + t.Errorf("%d PUTs", n) + } +} + +func TestEditPRErrors(t *testing.T) { + f := newEditFixture(t, "OPEN") + for name, tc := range map[string]struct { + n int64 + e platform.PREdit + class platform.Class + }{ + "no number": {0, platform.PREdit{}, platform.ClassNotFound}, + "blank title": {4, platform.PREdit{Title: ptr(" ")}, platform.ClassInvalid}, + "empty base": {4, platform.PREdit{Base: ptr("")}, platform.ClassInvalid}, + "bad state": {4, platform.PREdit{State: ptr(platform.PRState("draft"))}, platform.ClassInvalid}, + } { + _, err := f.tg.EditPR(t.Context(), tc.n, tc.e) + wantClass(t, name, err, tc.class, nil) + } + f.json(prsPath+"/9", http.StatusNotFound, errorBody("Pull request not found")) + _, err := f.tg.EditPR(t.Context(), 9, platform.PREdit{Body: ptr("x")}) + wantClass(t, "missing", err, platform.ClassNotFound, platform.ErrNotFound) + if w := f.writeLog(); len(w) != 0 { + t.Errorf("writes %q", w) + } +} + +func TestComment(t *testing.T) { + f := newWriterFixture(t) + tg := f.target(t) + f.handle(http.MethodPost, editPath+"/comments", func(w http.ResponseWriter, _ *http.Request) { + writeJSON(w, http.StatusCreated, map[string]any{"id": 1, "type": "pullrequest_comment", "content": map[string]any{"raw": "x"}}) + }) + f.handle(http.MethodPost, prsPath+"/9/comments", func(w http.ResponseWriter, _ *http.Request) { + writeJSON(w, http.StatusNotFound, errorBody("Pull request not found")) + }) + if err := tg.Comment(t.Context(), 4, "touchmark closed this:\n\nno diff"); err != nil { + t.Fatal(err) + } + c := f.requests(http.MethodPost, editPath+"/comments") + if len(c) != 1 || mustJSON(t, body(t, c[0])) != `{"content":{"raw":"touchmark closed this:\n\nno diff"}}` { + t.Errorf("comments %+v", c) + } + wantClass(t, "missing", tg.Comment(t.Context(), 9, "x"), platform.ClassNotFound, platform.ErrNotFound) + wantClass(t, "empty", tg.Comment(t.Context(), 4, " \n"), platform.ClassInvalid, nil) + wantClass(t, "no number", tg.Comment(t.Context(), 0, "x"), platform.ClassNotFound, nil) +} + +func TestEnsureLabels(t *testing.T) { + f := newWriterFixture(t) + tg := f.target(t) + ids, err := tg.EnsureLabels(t.Context(), []string{"engineering-assets"}) + if ids != nil || err != nil { + t.Errorf("EnsureLabels = %v, %v; want nothing: Bitbucket has no labels", ids, err) + } + if n := len(f.requests("", "")); n != 0 { + t.Errorf("%d requests", n) + } +} + +// TestWriterMasksToken: a write's error that echoes the token shows it +// masked, in both of its forms. +func TestWriterMasksToken(t *testing.T) { + f := newEditFixture(t, "OPEN") + basic := base64.StdEncoding.EncodeToString([]byte("x-bitbucket-api-token-auth:" + f.token)) + f.putStatus = []int{http.StatusConflict} + f.putBody = errorBody("echo " + f.token + " and " + basic) + _, err := f.tg.EditPR(t.Context(), 4, platform.PREdit{Body: ptr("x")}) + wantClass(t, "EditPR", err, platform.ClassConflict, nil) + if err == nil || strings.Contains(err.Error(), f.token) || strings.Contains(err.Error(), basic) { + t.Errorf("EditPR: %v", err) + } +} diff --git a/internal/platform/fake/fake.go b/internal/platform/fake/fake.go index 439342b..5578c66 100644 --- a/internal/platform/fake/fake.go +++ b/internal/platform/fake/fake.go @@ -14,7 +14,9 @@ // capabilities of its flavor: title-prefix drafts on // GitLab, Gitea and Forgejo; bodies and comments that would run GitLab // quick actions are refused (git mode runs them and records a violation); -// closers are not reported on Gitea and Forgejo. Branch rulesets, the +// closers are not reported on Gitea and Forgejo; on Bitbucket Cloud labels +// are refused and a pull request closed without merging can never be +// edited or reopened (Caps.ClosedImmutable). Branch rulesets, the // Preflighter of GitHub's driver (WithPreflight) and its API commits // (WithAPICommits, platform.Committer) are modeled in rules.go. // @@ -62,6 +64,8 @@ const ( GitLab Flavor = "gitlab" Gitea Flavor = "gitea" Forgejo Flavor = "forgejo" + // Bitbucket is Bitbucket Cloud. + Bitbucket Flavor = "bitbucket" ) // Tree entry modes. @@ -76,9 +80,12 @@ const ( var Epoch = time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC) // CapsFor returns the capabilities the fake reports for f, with the default -// limits of its platform. Every flavor keeps the marker in the body and has -// no API commit (WithAPICommits adds one). A flavor other than the four known -// ones gets GitHub's capabilities under its own name. +// limits of its platform. Every flavor keeps the marker in the body (as a +// Markdown reference definition on Bitbucket, MarkerInRefDef) and has no +// API commit (WithAPICommits adds one). Bitbucket's are those its driver's +// Probe reports: native drafts, no labels, a known closer, closed pull +// requests immutable, no permission of its own for CI files. A flavor other +// than the five known ones gets GitHub's capabilities under its own name. func CapsFor(f Flavor) platform.Caps { c := platform.Caps{ Flavor: string(f), @@ -102,6 +109,13 @@ func CapsFor(f Flavor) platform.Caps { c.LabelsByID = true c.CloserKnown = false c.Limits = platform.Limits{Reads: 4, GitReads: 2, MinInterval: 250 * time.Millisecond} + case Bitbucket: + c.MaxBody = 60000 + c.NoLabels = true + c.ClosedImmutable = true + c.Marker = platform.MarkerInRefDef + c.RuntimeOnly = []string{"branch-restrictions"} + c.Limits = platform.Limits{Reads: 2, GitReads: 2, ReadsPerMinute: 15, MinInterval: time.Second} default: c.WorkflowPerm = true c.Limits = platform.Limits{Reads: 8, GitReads: 4, WritesPerMinute: 60, WritesPerHour: 450, MinInterval: time.Second} @@ -696,7 +710,9 @@ func (p *Platform) AddPR(repoID string, pr platform.PR) int64 { // SetPRState moves a pull request to state as a person or bot would: // Closed and Merged record closedBy (nil: the platform does not say) and -// at (the clock when zero); Open reopens it and clears both. +// at (the clock when zero); Open reopens it and clears both. With +// Caps.ClosedImmutable no one can reopen a pull request closed without +// merging: that is a setup error. func (p *Platform) SetPRState(repoID string, number int64, state platform.PRState, closedBy *platform.Account, at time.Time) { p.mu.Lock() defer p.mu.Unlock() @@ -706,6 +722,10 @@ func (p *Platform) SetPRState(repoID string, number int64, state platform.PRStat } switch state { case platform.Open: + if p.caps.ClosedImmutable && ps.pr.State == platform.Closed { + p.setupf("SetPRState(%s, #%d): a declined pull request cannot be reopened", repoID, number) + return + } ps.pr.State, ps.pr.ClosedBy, ps.pr.ClosedAt = platform.Open, nil, time.Time{} if tip := p.headTip(ps.pr); tip != "" { ps.pr.HeadSHA = tip @@ -1152,6 +1172,8 @@ func (p *Platform) prURL(s *repoState, number int64) string { return base + "/-/merge_requests/" + n case Gitea, Forgejo: return base + "/pulls/" + n + case Bitbucket: + return base + "/pull-requests/" + n } return base + "/pull/" + n } @@ -1241,6 +1263,13 @@ func notFound(op, format string, args ...any) error { Err: fmt.Errorf(format+": %w", append(args, platform.ErrNotFound)...)} } +// unsupported is a write the platform refuses whatever the identity may +// do (a declined pull request on Bitbucket Cloud). +func unsupported(op, format string, args ...any) error { + return &platform.Error{Op: op, Class: platform.ClassUnsupported, Status: http.StatusBadRequest, + Err: fmt.Errorf(format, args...)} +} + func invalid(op, format string, args ...any) error { return &platform.Error{Op: op, Class: platform.ClassInvalid, Status: http.StatusUnprocessableEntity, Err: fmt.Errorf(format, args...)} diff --git a/internal/platform/fake/fake_test.go b/internal/platform/fake/fake_test.go index 701edde..cd6078a 100644 --- a/internal/platform/fake/fake_test.go +++ b/internal/platform/fake/fake_test.go @@ -46,7 +46,7 @@ func TestCapsFor(t *testing.T) { t.Errorf("Probe with an unknown account = %+v, %v; want an auth error", got, err) } } - if c := fake.CapsFor("bitbucket"); c.Flavor != "bitbucket" || !c.WorkflowPerm || c.Draft != platform.DraftNative { + if c := fake.CapsFor("azure-devops"); c.Flavor != "azure-devops" || !c.WorkflowPerm || c.Draft != platform.DraftNative { t.Errorf("CapsFor(unknown) = %+v, want GitHub's under its own name", c) } } diff --git a/internal/platform/fake/gitserve.go b/internal/platform/fake/gitserve.go index 175911c..aaa514c 100644 --- a/internal/platform/fake/gitserve.go +++ b/internal/platform/fake/gitserve.go @@ -83,11 +83,12 @@ func (p *Platform) gitHeader(id string) func(context.Context) (string, error) { } // appTokens reports whether the flavor is GitHub-like: identities are Apps -// whose tokens carry permissions (every flavor but GitLab, Gitea and -// Forgejo). Called with mu held. +// whose tokens carry permissions (every flavor but GitLab, Gitea, Forgejo +// and Bitbucket, whose identities are accounts with tokens of their own). +// Called with mu held. func (p *Platform) appTokens() bool { switch Flavor(p.caps.Flavor) { - case GitLab, Gitea, Forgejo: + case GitLab, Gitea, Forgejo, Bitbucket: return false } return true diff --git a/internal/platform/fake/meter.go b/internal/platform/fake/meter.go index ca337df..b45661c 100644 --- a/internal/platform/fake/meter.go +++ b/internal/platform/fake/meter.go @@ -178,15 +178,18 @@ func (t *target) createWrites(np platform.NewPR) (first, then int) { // editWrites is how many writes the flavor's driver sends for e, before // the edit is known to apply (first) and after (then): the edit, on Gitea -// and Forgejo after a state or base change in a request of its own; then -// labels added in a call of their own but on GitLab, after each missing one -// is created. +// and Forgejo after a state or base change in a request of its own, on +// Bitbucket followed by the decline of a close; then labels added in a call +// of their own but on GitLab, after each missing one is created. func (t *target) editWrites(e platform.PREdit) (first, then int) { first = 1 flavor := Flavor(t.p.caps.Flavor) if (flavor == Gitea || flavor == Forgejo) && (e.State != nil || e.Base != nil) && (e.Title != nil || e.Body != nil) { first++ } + if flavor == Bitbucket && e.State != nil && (e.Title != nil || e.Body != nil || e.Base != nil) { + first++ + } if len(e.AddLabels) > 0 && flavor != GitLab { then = 1 + t.missingLabels(e.AddLabels) } diff --git a/internal/platform/fake/writer.go b/internal/platform/fake/writer.go index c577694..adb7420 100644 --- a/internal/platform/fake/writer.go +++ b/internal/platform/fake/writer.go @@ -203,7 +203,7 @@ func (t *target) createPR(ctx context.Context, np platform.NewPR) (platform.PR, if err := p.checkText(op, np.Body); err != nil { return platform.PR{}, err } - if err := checkLabels(op, np.Labels); err != nil { + if err := t.p.checkLabels(op, np.Labels); err != nil { return platform.PR{}, err } if s.repo.PRsDisabled { @@ -266,6 +266,12 @@ func (t *target) createPR(ctx context.Context, np platform.NewPR) (platform.PR, // writer as the closer), a new base is taken to exist, labels are only // added. Draft never changes. // +// With Caps.ClosedImmutable (Bitbucket Cloud) a PR closed without merging +// is final, as Bitbucket's driver reports it: reopening it, or changing its +// title, body or base, is ClassUnsupported and changes nothing; Closed +// alone changes nothing. An open PR's new body is written before it closes, +// in the same call. +// // In git mode a new base must exist and reopening needs the head branch // (HeadSHA follows it again); after a new base or a reopening the flavor // reacts as after a push, and GitLab runs the body's quick actions. @@ -310,7 +316,17 @@ func (t *target) editPR(ctx context.Context, number int64, e platform.PREdit) (p if e.Base != nil && (*e.Base == "" || *e.Base == pr.Head) { return platform.PR{}, invalid(op, "base %q is empty or the head", *e.Base) } - if err := checkLabels(op, e.AddLabels); err != nil { + if p.caps.ClosedImmutable && pr.State == platform.Closed { + changes := e.Title != nil && p.draftTitle(*e.Title, pr.Draft) != pr.Title || + e.Body != nil && *e.Body != pr.Body || e.Base != nil && *e.Base != pr.Base + switch { + case e.State != nil && *e.State == platform.Open: + return platform.PR{}, unsupported(op, "#%d: a declined pull request cannot be reopened", number) + case changes: + return platform.PR{}, unsupported(op, "#%d is declined: only open pull requests can be changed", number) + } + } + if err := t.p.checkLabels(op, e.AddLabels); err != nil { return platform.PR{}, err } reopen := e.State != nil && *e.State == platform.Open && pr.State == platform.Closed @@ -400,7 +416,7 @@ func (t *target) EnsureLabels(ctx context.Context, names []string) ([]string, er return nil, err } return write(ctx, t, "EnsureLabels", []string{t.repoArg()}, func(s *repoState) ([]string, error) { - if err := checkLabels(op, names); err != nil { + if err := t.p.checkLabels(op, names); err != nil { return nil, err } ids := make([]string, len(names)) @@ -463,7 +479,12 @@ func (p *Platform) checkText(op, text string) error { return nil } -func checkLabels(op string, names []string) error { +// checkLabels refuses blank label names, and any label on a flavor +// without labels (Caps.NoLabels), where the core must ask for none. +func (p *Platform) checkLabels(op string, names []string) error { + if p.caps.NoLabels && len(names) > 0 { + return invalid(op, "labels %q on a platform without labels", names) + } for _, n := range names { if strings.TrimSpace(n) == "" { return invalid(op, "blank label name") diff --git a/internal/platform/fake/writer_test.go b/internal/platform/fake/writer_test.go index c93bdbb..3ae60f3 100644 --- a/internal/platform/fake/writer_test.go +++ b/internal/platform/fake/writer_test.go @@ -5,6 +5,7 @@ import ( "net/http" "strings" "testing" + "time" "github.com/bedrock-python/touchmark/internal/platform" "github.com/bedrock-python/touchmark/internal/platform/fake" @@ -308,3 +309,70 @@ func TestCloseAndRemote(t *testing.T) { _, err = tw2.EnsureLabels(ctx, []string{"x"}) wantClass(t, "after an applied Close fault", err, platform.ClassAuth) } + +// TestBitbucketFlavor: the Bitbucket flavor reports the capabilities of +// Bitbucket's driver, refuses labels, and treats a pull request closed +// without merging as final: an edit that closes writes the body first, +// and afterwards no title, body or base changes and no one reopens it. +func TestBitbucketFlavor(t *testing.T) { + t.Parallel() + ctx := t.Context() + c := fake.CapsFor(fake.Bitbucket) + if c.Flavor != "bitbucket" || !c.NoLabels || !c.ClosedImmutable || c.Marker != platform.MarkerInRefDef || c.BodyControls() || + c.Draft != platform.DraftNative || !c.CloserKnown || c.WorkflowPerm || c.LabelsByID || c.QuickActions || c.MaxBody != 60000 { + t.Errorf("CapsFor(Bitbucket) = %+v", c) + } + for _, f := range []fake.Flavor{fake.GitHub, fake.GitLab, fake.Gitea, fake.Forgejo} { + if c := fake.CapsFor(f); c.ClosedImmutable || c.NoLabels || !c.BodyControls() { + t.Errorf("CapsFor(%s) = %+v", f, c) + } + } + e := newEnv(t, fake.WithFlavor(fake.Bitbucket)) + r := e.repo("acme/api", "README.md", "x") + tw := e.target(r) + _, err := tw.CreatePR(ctx, platform.NewPR{Head: "touchmark/hub", Base: "main", Title: "sync", Labels: []string{"engineering-assets"}}) + wantClass(t, "a pull request with labels", err, platform.ClassInvalid) + pr, err := tw.CreatePR(ctx, platform.NewPR{Head: "touchmark/hub", Base: "main", Title: "sync", Body: "first", Draft: true}) + if err != nil || pr.URL != "https://github.com/acme/api/pull-requests/1" || pr.Title != "sync" || !pr.Draft { + t.Fatalf("CreatePR = %+v, %v", pr, err) + } + _, err = tw.EditPR(ctx, pr.Number, platform.PREdit{AddLabels: []string{"x"}}) + wantClass(t, "a label added", err, platform.ClassInvalid) + _, err = tw.EnsureLabels(ctx, []string{"x"}) + wantClass(t, "EnsureLabels", err, platform.ClassInvalid) + + // The close writes its body first, and the writer is the closer. + closed := platform.Closed + got, err := tw.EditPR(ctx, pr.Number, platform.PREdit{Body: ptr("closed by touchmark"), State: &closed}) + if err != nil || got.State != platform.Closed || got.Body != "closed by touchmark" || got.ClosedBy == nil || got.ClosedBy.ID != e.writer.ID { + t.Fatalf("close = %+v, %v", got, err) + } + open := platform.Open + for name, edit := range map[string]platform.PREdit{ + "reopen": {State: &open}, + "body": {Body: ptr("again")}, + "title": {Title: ptr("renamed")}, + "base": {Base: ptr("develop")}, + "body and open": {Body: ptr("again"), State: &open}, + } { + _, err := tw.EditPR(ctx, pr.Number, edit) + wantRule(t, name, err, platform.ClassUnsupported, http.StatusBadRequest, "") + } + // What the declined pull request already holds, and closing it again, + // change nothing and succeed. + for name, edit := range map[string]platform.PREdit{ + "close again": {State: &closed}, + "the same body": {Body: ptr("closed by touchmark"), Title: ptr("sync"), Base: ptr("main")}, + } { + if _, err := tw.EditPR(ctx, pr.Number, edit); err != nil { + t.Errorf("%s: %v", name, err) + } + } + if after := e.p.PR(r.ID, pr.Number); after.State != platform.Closed || after.Body != "closed by touchmark" || after.Title != "sync" || + after.Base != "main" || len(after.Labels) != 0 { + t.Errorf("after the refused edits: %+v", after) + } + // A person cannot reopen it either. + e.p.SetPRState(r.ID, pr.Number, platform.Open, nil, time.Time{}) + wantSetupErr(t, e.p, "cannot be reopened") +} diff --git a/internal/platform/platform.go b/internal/platform/platform.go index b7c0b05..6eb0074 100644 --- a/internal/platform/platform.go +++ b/internal/platform/platform.go @@ -1,5 +1,5 @@ // Package platform defines what touchmark needs from a code hosting platform -// (GitHub, GitLab, Gitea, Forgejo; later Bitbucket and Azure DevOps): the +// (GitHub, GitLab, Gitea, Forgejo, Bitbucket Cloud; later Azure DevOps): the // Reader and Writer interfaces drivers implement, their capabilities, and one // error model. // @@ -257,6 +257,13 @@ type MarkerStore uint8 const ( MarkerInBody MarkerStore = iota // last line of the description MarkerInProperties // PR properties (Azure DevOps, later) + // MarkerInRefDef is the last line of the description too, with the + // payload of MarkerInBody wrapped in a Markdown link reference + // definition, `[touchmark]: # "touchmark:v1 …"`, which renderers do not + // show: Bitbucket Cloud escapes HTML in descriptions, so an HTML comment + // would show as text (marker.FrameRefDef; the core reads both frames on + // every platform). + MarkerInRefDef ) // Limits are a provider's default pacing (internal/throttle applies them, @@ -273,7 +280,8 @@ type Limits struct { // Caps describes a platform instance, found once per run by Reader.Probe. type Caps struct { - // Flavor is "github", "ghe.com", "ghes", "gitlab", "gitea" or "forgejo". + // Flavor is "github", "ghe.com", "ghes", "gitlab", "gitea", "forgejo" + // or "bitbucket" (Bitbucket Cloud). Flavor string Version string // MaxBody is the body budget in bytes, marker included. @@ -288,16 +296,40 @@ type Caps struct { // are refused or dropped silently (Gitea and Forgejo answer 422 to // names when a pull request is created, and drop unknown ids). LabelsByID bool + // NoLabels is set when the platform has no pull request labels + // (Bitbucket Cloud): the core asks for none (NewPR.Labels, + // PREdit.AddLabels and EnsureLabels stay empty, the marker records + // none set) and reports none, and the driver ignores any it is given. + NoLabels bool // CloserKnown is set when PR.ClosedBy is reliable. CloserKnown bool - Marker MarkerStore - Commit struct{ API, SignedByPlatform, CAS bool } + // ClosedImmutable is set when a pull request closed without merging can + // never be edited again, by anyone (Bitbucket Cloud's declined pull + // requests): the core never plans a write to the body of a closed pull + // request there. Memory then reads a decline from what the marker held + // while the pull request was open (its optin), and a forget_declines + // entry acts while it is present instead of once (see + // decide.MemoryConfig.ClosedImmutable). + ClosedImmutable bool + Marker MarkerStore + Commit struct{ API, SignedByPlatform, CAS bool } // RuntimeOnly lists checks the writer cannot read upfront, e.g. // "push_rules" on GitLab. RuntimeOnly []string Limits Limits } +// BodyControls reports whether pull request descriptions on the platform +// can carry touchmark's tick boxes ("Rebuild this branch", "Propose this +// content again"). Each is a task list item tagged with an HTML comment, +// which renderers hide; a platform that escapes HTML in descriptions would +// show the tag as text, and it is the same platform whose marker is a +// Markdown reference definition (MarkerInRefDef). There the description +// offers no tick box, says to ask for a rebuild or a new proposal through +// .touchmark/operations.yml instead, and the core reads no tick box from +// it. +func (c Caps) BodyControls() bool { return c.Marker != MarkerInRefDef } + // Reader is one provider from hub.yml under one identity. Implementations // are safe for concurrent use: the core inspects targets in parallel. type Reader interface { diff --git a/internal/platform/platform_test.go b/internal/platform/platform_test.go index 366822f..ce0b9db 100644 --- a/internal/platform/platform_test.go +++ b/internal/platform/platform_test.go @@ -220,3 +220,23 @@ func TestFileErrors(t *testing.T) { t.Errorf("empty: %q", msg) } } + +// TestBodyControls: descriptions carry tick boxes unless the platform's +// marker is a reference definition (HTML shows as text there), whatever +// else it says. +func TestBodyControls(t *testing.T) { + for _, c := range []struct { + caps Caps + want bool + }{ + {Caps{Marker: MarkerInBody}, true}, + {Caps{Marker: MarkerInBody, ClosedImmutable: true, NoLabels: true}, true}, + {Caps{Marker: MarkerInProperties}, true}, + {Caps{Marker: MarkerInRefDef}, false}, + {Caps{Marker: MarkerInRefDef, ClosedImmutable: true}, false}, + } { + if got := c.caps.BodyControls(); got != c.want { + t.Errorf("%+v: BodyControls = %v, want %v", c.caps, got, c.want) + } + } +} diff --git a/internal/prbody/controls.go b/internal/prbody/controls.go index a7ab3f5..e3699fd 100644 --- a/internal/prbody/controls.go +++ b/internal/prbody/controls.go @@ -28,7 +28,7 @@ func ControlLine(control string) string { // a closed PR's body). A body that already has a line of control, ticked or // not, and an unknown control leave body as it is. // -// Marker lines are the lines that start with "" + end := " -->" + if !strings.HasPrefix(line, "" line -// are errors too. +// URL, or a Marker that is neither "" nor one marker line in a frame of +// marker.EncodeFrame are errors too. func Render(in Input) (string, error) { r, err := newRenderer(in) if err != nil { @@ -271,6 +287,9 @@ func newRenderer(in Input) (*renderer, error) { if err != nil { return nil, err } + if !in.Caps.BodyControls() { + in.ShowRecreate, in.ShowRepropose = false, false + } r := &renderer{in: in, intro: intro, noun: "pull request", sigil: "#", optIn: "the opt-in file"} if in.Caps.Flavor == "gitlab" { r.noun, r.sigil = "merge request", "!" @@ -644,7 +663,9 @@ func (r *renderer) changesSection() *cuttable { return newCuttable("### Changes", "\n\n| Change | File | Pack |\n|---|---|---|", "", items, len(r.rows), "change", "changes") } -// localSection is the collapsed list of local files. +// localSection is the list of local files: collapsed in a details element, +// or, where the description cannot carry HTML (renderer.html), plain +// Markdown under a heading. func (r *renderer) localSection() *cuttable { total := len(r.local) if total == 0 { @@ -654,17 +675,29 @@ func (r *renderer) localSection() *cuttable { if total == 1 { summary = "1 file here differs from the hub and stays as it is" } - prefix := "
\n" + summary + "\n\n" + - "touchmark does not update files changed in this repository. " + + hints := "touchmark does not update files changed in this repository. " + "To take the hub's version of one, run `touchmark apply --adopt `. " + "To stop seeing it here, add it to `ignore` in " + r.optIn + "." + prefix := "
\n" + summary + "\n\n" + hints + suffix := "\n\n
" + if !r.html() { + prefix, suffix = "### Local files\n\n"+summary+". "+hints, "" + } items := make([]string, 0, min(total, maxRows)) for _, p := range r.local[:min(total, maxRows)] { items = append(items, "\n- "+Code(p)) } - return newCuttable(prefix, "\n", "\n\n
", items, total, "file", "files") + return newCuttable(prefix, "\n", suffix, items, total, "file", "files") } +// html reports whether the description may carry touchmark's HTML: the +// details element of the local files, and the comments that tag the +// controls. A platform that escapes HTML in descriptions shows it as text; +// it is the platform whose marker is a reference definition and whose +// descriptions carry no controls (Caps.BodyControls false: Bitbucket +// Cloud). +func (r *renderer) html() bool { return r.in.Caps.BodyControls() } + // previouslyDeclined is the block for declines that overlap D. func (r *renderer) previouslyDeclined() string { if len(r.declined) == 0 { @@ -712,8 +745,12 @@ func (r *renderer) pausedBlock() *cuttable { items = append(items, item) } suffix := "" - if r.in.ShowRecreate { + switch { + case r.in.ShowRecreate: suffix = "\n\nTo rebuild the branch and drop the other commits, tick **Rebuild this branch** below." + case !r.in.Caps.BodyControls(): + suffix = "\n\nTo rebuild the branch and drop the other commits, ask the hub's maintainers to add a `recreate` entry " + + "with this branch's head to `.touchmark/operations.yml`." } return newCuttable(prefix, "\n", suffix, items, len(r.pending), "change", "changes") } @@ -778,9 +815,22 @@ func (r *renderer) note() string { return "---\n\nIf this " + r.noun + " is closed without merging, touchmark remembers it and does not propose the same changes again; " + "a new one comes when the hub changes these files. " + "Editing `packs` or `ignore` in " + r.optIn + " resets that memory, and `ignore` opts a file out for good. " + + r.forgetHint() + "touchmark rewrites this description, so please comment instead of editing it." } +// forgetHint is the footnote's sentence on the forget_declines entry, where +// nothing in a declined pull request can bring its changes back: its +// description carries no control, or can never be edited once declined. +// "" elsewhere. +func (r *renderer) forgetHint() string { + if r.in.Caps.BodyControls() && !r.in.Caps.ClosedImmutable { + return "" + } + return "To have the same changes proposed again sooner, the hub's maintainers can add a `forget_declines` entry " + + "with the number of this " + r.noun + " to `.touchmark/operations.yml`. " +} + // actionText is how a change reads in the table and the lists. func actionText(c Change) string { switch c.Action { @@ -810,11 +860,19 @@ func moreLine(shown, more int, one, many string) string { } // checkMarker accepts one "" line whose comment ends -// only at the end of the line. +// only at the end of the line, or one `[touchmark]: # "touchmark:…"` +// reference definition whose title ends only at the end of the line (the +// two frames of marker.EncodeFrame). func checkMarker(m string) error { + const refDef = `[touchmark]: # "touchmark:` switch { case strings.ContainsAny(m, "\r\n"): return errors.New("the marker is not one line") + case strings.HasPrefix(m, refDef): + if !strings.HasSuffix(m, `"`) || strings.IndexByte(m[len(refDef):], '"') != len(m)-len(refDef)-1 || + strings.Contains(m, `\`) { + return fmt.Errorf("%.40q… is not a marker reference definition", m) + } case !strings.HasPrefix(m, "") || strings.Index(m, "-->") != len(m)-len("-->"): return fmt.Errorf("%.40q… is not a marker comment", m) } diff --git a/internal/prbody/render_test.go b/internal/prbody/render_test.go index f1d5c05..97534fb 100644 --- a/internal/prbody/render_test.go +++ b/internal/prbody/render_test.go @@ -25,7 +25,7 @@ func base(t testing.TB, flavor string) Input { Packs: []string{"agents", "claude", "base"}, OptInFile: optInName, Caps: capsOf(flavor), - Marker: testMarker(t), + Marker: markerFor(t, flavor), } } @@ -154,7 +154,7 @@ func checkBody(t *testing.T, in Input, body string) { if in.Marker == "" { full = body + "\n\n" + longestMarker for _, line := range strings.Split(body, "\n") { - if strings.HasPrefix(line, "" }, nil, "not a marker"}, {"marker ends early", func(in *Input) { in.Marker = " x -->" }, nil, "not a marker"}, + {"reference definition ends early", func(in *Input) { in.Marker = `[touchmark]: # "touchmark:v1" x "` }, nil, "not a marker"}, + {"reference definition unclosed", func(in *Input) { in.Marker = `[touchmark]: # "touchmark:v1 x` }, nil, "not a marker"}, + {"reference definition escaped", func(in *Input) { in.Marker = `[touchmark]: # "touchmark:v1 hub\=x"` }, nil, "not a marker"}, + {"intro reference definition", func(in *Input) { in.Intro = "Hello\n\n[touchmark]: # \"touchmark:v1 x\"" }, ErrIntro, "line 3"}, {"unknown action", func(in *Input) { in.Changes = []Change{{Path: "a", Action: "rename"}} }, nil, "unknown action"}, {"unknown pending action", func(in *Input) { in.Paused = true diff --git a/internal/prbody/testdata/assumed/bitbucket.md b/internal/prbody/testdata/assumed/bitbucket.md new file mode 100644 index 0000000..ad5dd44 --- /dev/null +++ b/internal/prbody/testdata/assumed/bitbucket.md @@ -0,0 +1,19 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` ([`github.com/acme/engineering-assets`](https://redirect.github.com/acme/engineering-assets)) at commit [`3f2c1ab9d8e7`](https://redirect.github.com/acme/engineering-assets/commit/3f2c1ab9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3). + +The hub subscribed this repository, which has no `.engineering-assets.yml`. To add packs or keep files out of the sync, add `.engineering-assets.yml` with `packs` or `ignore`; to stop these pull requests, add it with `enabled: false`. + +### Changes + +| Change | File | Pack | +|---|---|---| +| create | `AGENTS.md` | `agents` | +| create | `prompts/review.md` | `agents` | + +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. To have the same changes proposed again sooner, the hub's maintainers can add a `forget_declines` entry with the number of this pull request to `.touchmark/operations.yml`. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/declined/bitbucket.md b/internal/prbody/testdata/declined/bitbucket.md new file mode 100644 index 0000000..f4c4f07 --- /dev/null +++ b/internal/prbody/testdata/declined/bitbucket.md @@ -0,0 +1,21 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` ([`github.com/acme/engineering-assets`](https://redirect.github.com/acme/engineering-assets)) at commit [`3f2c1ab9d8e7`](https://redirect.github.com/acme/engineering-assets/commit/3f2c1ab9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3). + +### Changes + +| Change | File | Pack | +|---|---|---| +| update | `AGENTS.md` | `agents` | +| create | `prompts/review.md` | `agents` | + +### Previously declined + +Some of these changes were in #7 and #9, which were closed without merging. This pull request brings the whole current set anyway, because the files of a pack go together. To keep a file as it is here, add it to `ignore` in `.engineering-assets.yml`. + +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. To have the same changes proposed again sooner, the hub's maintainers can add a `forget_declines` entry with the number of this pull request to `.touchmark/operations.yml`. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/huge/bitbucket.md b/internal/prbody/testdata/huge/bitbucket.md new file mode 100644 index 0000000..3aae35c --- /dev/null +++ b/internal/prbody/testdata/huge/bitbucket.md @@ -0,0 +1,232 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` ([`github.com/acme/engineering-assets`](https://redirect.github.com/acme/engineering-assets)) at commit [`3f2c1ab9d8e7`](https://redirect.github.com/acme/engineering-assets/commit/3f2c1ab9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3). + +### ⚠ Sensitive paths + +These files can run code in CI or change how tools and agents work in this repository. Review them first. + +- `.github/workflows/docs.yml`: update, pack `ci` +- `.pre-commit-config.yaml`: create, pack `base` +- `bin/setup`: create, executable, pack `base` + +### Changes + +| Change | File | Pack | +|---|---|---| +| ⚠ update | `.github/workflows/docs.yml` | `ci` | +| ⚠ create | `.pre-commit-config.yaml` | `base` | +| ⚠ create, executable | `bin/setup` | `base` | +| delete | `docs/handbook/chapter-02/page-002.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-014.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-026.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-038.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-050.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-062.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-074.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-086.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-098.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-110.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-122.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-134.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-146.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-158.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-170.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-182.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-194.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-206.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-218.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-230.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-242.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-254.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-266.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-278.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-290.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-302.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-314.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-326.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-338.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-350.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-362.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-374.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-386.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-398.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-410.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-422.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-434.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-446.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-458.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-470.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-482.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-494.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-506.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-518.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-530.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-542.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-554.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-566.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-578.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-590.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-602.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-614.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-626.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-638.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-650.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-662.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-674.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-686.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-698.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-710.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-722.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-734.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-746.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-758.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-770.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-782.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-794.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-806.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-818.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-830.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-842.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-854.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-866.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-878.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-890.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-902.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-914.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-926.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-938.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-950.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-962.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-974.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-986.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-998.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-006.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-018.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-030.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-042.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-054.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-066.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-078.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-090.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-102.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-114.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-126.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-138.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-150.md` | `handbook` | + +…and 903 more. + +### Local files + +150 files here differ from the hub and stay as they are. touchmark does not update files changed in this repository. To take the hub's version of one, run `touchmark apply --adopt `. To stop seeing it here, add it to `ignore` in `.engineering-assets.yml`. + +- `docs/handbook/chapter-00/local-000.md` +- `docs/handbook/chapter-00/local-012.md` +- `docs/handbook/chapter-00/local-024.md` +- `docs/handbook/chapter-00/local-036.md` +- `docs/handbook/chapter-00/local-048.md` +- `docs/handbook/chapter-00/local-060.md` +- `docs/handbook/chapter-00/local-072.md` +- `docs/handbook/chapter-00/local-084.md` +- `docs/handbook/chapter-00/local-096.md` +- `docs/handbook/chapter-00/local-108.md` +- `docs/handbook/chapter-00/local-120.md` +- `docs/handbook/chapter-00/local-132.md` +- `docs/handbook/chapter-00/local-144.md` +- `docs/handbook/chapter-01/local-001.md` +- `docs/handbook/chapter-01/local-013.md` +- `docs/handbook/chapter-01/local-025.md` +- `docs/handbook/chapter-01/local-037.md` +- `docs/handbook/chapter-01/local-049.md` +- `docs/handbook/chapter-01/local-061.md` +- `docs/handbook/chapter-01/local-073.md` +- `docs/handbook/chapter-01/local-085.md` +- `docs/handbook/chapter-01/local-097.md` +- `docs/handbook/chapter-01/local-109.md` +- `docs/handbook/chapter-01/local-121.md` +- `docs/handbook/chapter-01/local-133.md` +- `docs/handbook/chapter-01/local-145.md` +- `docs/handbook/chapter-02/local-002.md` +- `docs/handbook/chapter-02/local-014.md` +- `docs/handbook/chapter-02/local-026.md` +- `docs/handbook/chapter-02/local-038.md` +- `docs/handbook/chapter-02/local-050.md` +- `docs/handbook/chapter-02/local-062.md` +- `docs/handbook/chapter-02/local-074.md` +- `docs/handbook/chapter-02/local-086.md` +- `docs/handbook/chapter-02/local-098.md` +- `docs/handbook/chapter-02/local-110.md` +- `docs/handbook/chapter-02/local-122.md` +- `docs/handbook/chapter-02/local-134.md` +- `docs/handbook/chapter-02/local-146.md` +- `docs/handbook/chapter-03/local-003.md` +- `docs/handbook/chapter-03/local-015.md` +- `docs/handbook/chapter-03/local-027.md` +- `docs/handbook/chapter-03/local-039.md` +- `docs/handbook/chapter-03/local-051.md` +- `docs/handbook/chapter-03/local-063.md` +- `docs/handbook/chapter-03/local-075.md` +- `docs/handbook/chapter-03/local-087.md` +- `docs/handbook/chapter-03/local-099.md` +- `docs/handbook/chapter-03/local-111.md` +- `docs/handbook/chapter-03/local-123.md` +- `docs/handbook/chapter-03/local-135.md` +- `docs/handbook/chapter-03/local-147.md` +- `docs/handbook/chapter-04/local-004.md` +- `docs/handbook/chapter-04/local-016.md` +- `docs/handbook/chapter-04/local-028.md` +- `docs/handbook/chapter-04/local-040.md` +- `docs/handbook/chapter-04/local-052.md` +- `docs/handbook/chapter-04/local-064.md` +- `docs/handbook/chapter-04/local-076.md` +- `docs/handbook/chapter-04/local-088.md` +- `docs/handbook/chapter-04/local-100.md` +- `docs/handbook/chapter-04/local-112.md` +- `docs/handbook/chapter-04/local-124.md` +- `docs/handbook/chapter-04/local-136.md` +- `docs/handbook/chapter-04/local-148.md` +- `docs/handbook/chapter-05/local-005.md` +- `docs/handbook/chapter-05/local-017.md` +- `docs/handbook/chapter-05/local-029.md` +- `docs/handbook/chapter-05/local-041.md` +- `docs/handbook/chapter-05/local-053.md` +- `docs/handbook/chapter-05/local-065.md` +- `docs/handbook/chapter-05/local-077.md` +- `docs/handbook/chapter-05/local-089.md` +- `docs/handbook/chapter-05/local-101.md` +- `docs/handbook/chapter-05/local-113.md` +- `docs/handbook/chapter-05/local-125.md` +- `docs/handbook/chapter-05/local-137.md` +- `docs/handbook/chapter-05/local-149.md` +- `docs/handbook/chapter-06/local-006.md` +- `docs/handbook/chapter-06/local-018.md` +- `docs/handbook/chapter-06/local-030.md` +- `docs/handbook/chapter-06/local-042.md` +- `docs/handbook/chapter-06/local-054.md` +- `docs/handbook/chapter-06/local-066.md` +- `docs/handbook/chapter-06/local-078.md` +- `docs/handbook/chapter-06/local-090.md` +- `docs/handbook/chapter-06/local-102.md` +- `docs/handbook/chapter-06/local-114.md` +- `docs/handbook/chapter-06/local-126.md` +- `docs/handbook/chapter-06/local-138.md` +- `docs/handbook/chapter-07/local-007.md` +- `docs/handbook/chapter-07/local-019.md` +- `docs/handbook/chapter-07/local-031.md` +- `docs/handbook/chapter-07/local-043.md` +- `docs/handbook/chapter-07/local-055.md` +- `docs/handbook/chapter-07/local-067.md` +- `docs/handbook/chapter-07/local-079.md` +- `docs/handbook/chapter-07/local-091.md` +- `docs/handbook/chapter-07/local-103.md` +- `docs/handbook/chapter-07/local-115.md` + +…and 50 more. + +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. To have the same changes proposed again sooner, the hub's maintainers can add a `forget_declines` entry with the number of this pull request to `.touchmark/operations.yml`. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/local/bitbucket.md b/internal/prbody/testdata/local/bitbucket.md new file mode 100644 index 0000000..1dc4da3 --- /dev/null +++ b/internal/prbody/testdata/local/bitbucket.md @@ -0,0 +1,24 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` at commit `3f2c1ab9d8e7`. + +### Changes + +| Change | File | Pack | +|---|---|---| +| update | `AGENTS.md` | `agents` | + +### Local files + +3 files here differ from the hub and stay as they are. touchmark does not update files changed in this repository. To take the hub's version of one, run `touchmark apply --adopt `. To stop seeing it here, add it to `ignore` in `.engineering-assets.yml`. + +- `.github/CODEOWNERS` +- `CONTRIBUTING.md` +- `docs/guide.md` + +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. To have the same changes proposed again sooner, the hub's maintainers can add a `forget_declines` entry with the number of this pull request to `.touchmark/operations.yml`. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/new/bitbucket.md b/internal/prbody/testdata/new/bitbucket.md new file mode 100644 index 0000000..ff5fe66 --- /dev/null +++ b/internal/prbody/testdata/new/bitbucket.md @@ -0,0 +1,30 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` ([`github.com/acme/engineering-assets`](https://redirect.github.com/acme/engineering-assets)) at commit [`3f2c1ab9d8e7`](https://redirect.github.com/acme/engineering-assets/commit/3f2c1ab9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3). + +### ⚠ Sensitive paths + +These files can run code in CI or change how tools and agents work in this repository. Review them first. + +- `.claude/settings.json`: update, pack `claude` +- `scripts/check.sh`: create, executable, pack `base` +- `tools/lint.sh`: chmod +x, pack `base` + +### Changes + +| Change | File | Pack | +|---|---|---| +| ⚠ update | `.claude/settings.json` | `claude` | +| ⚠ chmod +x | `tools/lint.sh` | `base` | +| ⚠ create, executable | `scripts/check.sh` | `base` | +| delete | `prompts/old.md` | `agents` | +| update | `AGENTS.md` | `agents` | +| chmod -x | `docs/guide.md` | `base` | +| create | `prompts/review.md` | `agents` | + +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. To have the same changes proposed again sooner, the hub's maintainers can add a `forget_declines` entry with the number of this pull request to `.touchmark/operations.yml`. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/nothing-more/bitbucket.md b/internal/prbody/testdata/nothing-more/bitbucket.md new file mode 100644 index 0000000..71bbd11 --- /dev/null +++ b/internal/prbody/testdata/nothing-more/bitbucket.md @@ -0,0 +1,20 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` ([`github.com/acme/engineering-assets`](https://redirect.github.com/acme/engineering-assets)) at commit [`3f2c1ab9d8e7`](https://redirect.github.com/acme/engineering-assets/commit/3f2c1ab9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3). + +### Changes + +| Change | File | Pack | +|---|---|---| +| create | `prompts/review.md` | `agents` | + +### Nothing more to sync + +The hub proposes nothing more for this repository: the default branch has the hub's versions of these files, or they are ignored or changed here. This branch has commits touchmark did not make, so touchmark leaves this pull request open. Close it, or keep your commits and merge it. + +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. To have the same changes proposed again sooner, the hub's maintainers can add a `forget_declines` entry with the number of this pull request to `.touchmark/operations.yml`. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/paused/bitbucket.md b/internal/prbody/testdata/paused/bitbucket.md new file mode 100644 index 0000000..9804404 --- /dev/null +++ b/internal/prbody/testdata/paused/bitbucket.md @@ -0,0 +1,32 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` ([`github.com/acme/engineering-assets`](https://redirect.github.com/acme/engineering-assets)) at commit [`3f2c1ab9d8e7`](https://redirect.github.com/acme/engineering-assets/commit/3f2c1ab9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3). + +### ⚠ Sensitive paths + +These files can run code in CI or change how tools and agents work in this repository. Review them first. + +- `.github/workflows/lint.yml`: create, pack `ci` (after a rebuild) + +### Changes + +| Change | File | Pack | +|---|---|---| +| update | `AGENTS.md` | `agents` | +| create | `prompts/review.md` | `agents` | + +### touchmark paused + +This branch has commits touchmark did not make, so touchmark stopped updating it to keep them. The changes above are what the branch holds now. A rebuild from the default branch would bring: + +- ⚠ create `.github/workflows/lint.yml` (pack `ci`) +- update `AGENTS.md` (pack `agents`) + +To rebuild the branch and drop the other commits, ask the hub's maintainers to add a `recreate` entry with this branch's head to `.touchmark/operations.yml`. + +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. To have the same changes proposed again sooner, the hub's maintainers can add a `forget_declines` entry with the number of this pull request to `.touchmark/operations.yml`. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/sensitive-only/bitbucket.md b/internal/prbody/testdata/sensitive-only/bitbucket.md new file mode 100644 index 0000000..3dde12f --- /dev/null +++ b/internal/prbody/testdata/sensitive-only/bitbucket.md @@ -0,0 +1,32 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` ([`github.com/acme/engineering-assets`](https://redirect.github.com/acme/engineering-assets)) at commit [`3f2c1ab9d8e7`](https://redirect.github.com/acme/engineering-assets/commit/3f2c1ab9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3). + +### ⚠ Sensitive paths + +These files can run code in CI or change how tools and agents work in this repository. Review them first. + +- `.gitea/workflows/ci.yml`: create, pack `ci` +- `.github/CODEOWNERS`: delete, pack `base` +- `.github/workflows/ci.yml`: update, pack `ci` +- `deploy/prod.yaml`: update, pack `ops` +- `scripts/release.sh`: chmod +x, pack `base` + +This pull request adds `.gitea/workflows` or `.forgejo/workflows`, which turns off `.github/workflows` in this repository on Gitea and Forgejo. + +### Changes + +| Change | File | Pack | +|---|---|---| +| ⚠ delete | `.github/CODEOWNERS` | `base` | +| ⚠ update | `.github/workflows/ci.yml` | `ci` | +| ⚠ update | `deploy/prod.yaml` | `ops` | +| ⚠ chmod +x | `scripts/release.sh` | `base` | +| ⚠ create | `.gitea/workflows/ci.yml` | `ci` | + +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. To have the same changes proposed again sooner, the hub's maintainers can add a `forget_declines` entry with the number of this pull request to `.touchmark/operations.yml`. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/report/doctor_test.go b/internal/report/doctor_test.go index fc5df1e..13a84dc 100644 --- a/internal/report/doctor_test.go +++ b/internal/report/doctor_test.go @@ -99,7 +99,7 @@ func TestDoctorSchema(t *testing.T) { bad := map[string]func(*Doctor){ "status": func(d *Doctor) { d.HubChecks[0].Status = "red" }, "check": func(d *Doctor) { d.HubChecks[0].Name = "Write Isolation" }, - "provider": func(d *Doctor) { d.Providers[0].Type = "bitbucket" }, + "provider": func(d *Doctor) { d.Providers[0].Type = "azure-devops" }, "summary": func(d *Doctor) { delete(d.Summary, StatusUnknown) }, "schema": func(d *Doctor) { d.Schema = "doctor/v2" }, } diff --git a/schemas/doctor.schema.json b/schemas/doctor.schema.json index 7e46c08..973d6e4 100644 --- a/schemas/doctor.schema.json +++ b/schemas/doctor.schema.json @@ -113,7 +113,7 @@ "required": ["id", "type", "host", "resolve_complete", "checks"], "properties": { "id": { "$ref": "#/$defs/providerId" }, - "type": { "enum": ["github", "gitlab", "gitea", "forgejo"] }, + "type": { "enum": ["github", "gitlab", "gitea", "forgejo", "bitbucket"] }, "host": { "description": "The provider's host, with a port when it has one.", "type": "string", diff --git a/schemas/hub.schema.json b/schemas/hub.schema.json index 0135dff..1d3549d 100644 --- a/schemas/hub.schema.json +++ b/schemas/hub.schema.json @@ -209,6 +209,13 @@ "properties": { "platform": { "enum": ["gitea", "forgejo"] } } }, "then": { "required": ["base_url"] } + }, + { + "if": { + "required": ["platform"], + "properties": { "platform": { "const": "bitbucket" } } + }, + "then": { "properties": { "base_url": { "pattern": "^https://[Bb][Ii][Tt][Bb][Uu][Cc][Kk][Ee][Tt]\\.[Oo][Rr][Gg]/*$" } } } } ], "$defs": { @@ -239,7 +246,8 @@ "pattern": "^[a-z][a-z0-9-]{0,31}$" }, "providerType": { - "enum": ["github", "gitlab", "gitea", "forgejo"] + "description": "github: github.com, GHE.com or GitHub Enterprise Server; gitlab; gitea; forgejo; bitbucket: Bitbucket Cloud (plan, distribute and doctor; setup does not set up a hub on it yet).", + "enum": ["github", "gitlab", "gitea", "forgejo", "bitbucket"] }, "sign": { "description": "auto: sign where it costs nothing and where target rules require it; always: sign every commit. Default: auto.", @@ -279,7 +287,7 @@ }, "type": { "$ref": "#/$defs/providerType" }, "url": { - "description": "Web URL of the instance. Default for github: https://github.com; for gitlab: https://gitlab.com. Required for gitea and forgejo. A target targets.yml names by web URL belongs to the provider whose url it lies under.", + "description": "Web URL of the instance. Default for github: https://github.com; for gitlab: https://gitlab.com; for bitbucket: https://bitbucket.org, the only url it takes without api_url. Required for gitea and forgejo. A target targets.yml names by web URL belongs to the provider whose url it lies under.", "$ref": "#/$defs/url" }, "api_url": { @@ -291,7 +299,7 @@ "$ref": "#/$defs/relativePath" }, "writer": { - "description": "The account that writes to targets, as the platform names it: on GitHub the App's bot, [bot]; on GitLab, Gitea and Forgejo the username of the service account or bot user.", + "description": "The account that writes to targets, as the platform names it: on GitHub the App's bot, [bot]; on GitLab, Gitea and Forgejo the username of the service account or bot user; on Bitbucket the bot account's UUID in braces, {…}.", "$ref": "#/$defs/account" }, "known_authors": { @@ -324,10 +332,21 @@ } } }, - "if": { - "properties": { "type": { "enum": ["gitea", "forgejo"] } } - }, - "then": { "required": ["url"] } + "allOf": [ + { + "if": { + "properties": { "type": { "enum": ["gitea", "forgejo"] } } + }, + "then": { "required": ["url"] } + }, + { + "if": { + "required": ["type", "url"], + "properties": { "type": { "const": "bitbucket" }, "url": { "not": { "pattern": "^https://[Bb][Ii][Tt][Bb][Uu][Cc][Kk][Ee][Tt]\\.[Oo][Rr][Gg]/*$" } } } + }, + "then": { "required": ["api_url"] } + } + ] } } } diff --git a/schemas/operations.schema.json b/schemas/operations.schema.json index 194dcbb..2a712bd 100644 --- a/schemas/operations.schema.json +++ b/schemas/operations.schema.json @@ -28,7 +28,7 @@ } }, "forget_declines": { - "description": "Propose again the content of a pull request the target closed without merging. touchmark marks that pull request revoked, so the entry acts once.", + "description": "Propose again the content of a pull request the target closed without merging. touchmark marks that pull request revoked, so the entry acts once; on Bitbucket Cloud, where a declined pull request cannot be edited, the entry acts while it is present.", "type": ["array", "null"], "items": { "type": "object", diff --git a/schemas/report.schema.json b/schemas/report.schema.json index e2ea28a..8272d96 100644 --- a/schemas/report.schema.json +++ b/schemas/report.schema.json @@ -142,7 +142,7 @@ "required": ["id", "type", "host", "write_check", "resolve_complete"], "properties": { "id": { "$ref": "#/$defs/providerId" }, - "type": { "enum": ["github", "gitlab", "gitea", "forgejo"] }, + "type": { "enum": ["github", "gitlab", "gitea", "forgejo", "bitbucket"] }, "host": { "description": "The provider's host, with a port when it has one.", "type": "string", diff --git a/schemas/targets.schema.json b/schemas/targets.schema.json index e09ffba..bac2d24 100644 --- a/schemas/targets.schema.json +++ b/schemas/targets.schema.json @@ -89,7 +89,7 @@ "required": ["org"], "properties": { "org": { - "description": "Every repository in a GitHub organisation or a GitLab, Gitea or Forgejo namespace (a synonym of group), named by its path or its web URL.", + "description": "Every repository in a GitHub organisation, a GitLab, Gitea or Forgejo namespace or a Bitbucket workspace (a synonym of group), named by its path or its web URL.", "$ref": "#/$defs/namespace" }, "provider": { "$ref": "#/$defs/providerId" }, @@ -134,7 +134,7 @@ } }, "topics": { - "description": "Only repositories with all of these topics.", + "description": "Only repositories with all of these topics. Bitbucket has no topics: check refuses them on its entries; use match.", "type": ["array", "null"], "items": { "type": "string", "pattern": "[^\t\n\u000b\f\r \u00a0\u1680\u2000-\u200a\u2028\u2029\u202f\u205f\u3000\ufeff]" } },