From 96a3eeb9f8b50659618e484c9299bb7a91bc12fa Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Thu, 8 Oct 2026 23:38:18 +0300 Subject: [PATCH 01/20] feat(config): accept Bitbucket Cloud providers A provider of type bitbucket is Bitbucket Cloud: its url defaults to https://bitbucket.org and its API to https://api.bitbucket.org/2.0. Any other url needs an api_url (a test server); Bitbucket Data Center has another API and is not this type. Web URLs such as https://bitbucket.org/acme/billing resolve to the provider, workspace/repository as on the other platforms without nested namespaces, and exclude and match warn about paths deeper than that. Bitbucket has no topics: check refuses topics on an entry whose provider is Bitbucket, with a hint to use match or repo. The JSON Schemas of hub.yml, the plan and doctor reports, and targets.yml follow, and so does the reference. --- docs/reference/hub.md | 8 ++-- docs/reference/output.md | 4 +- docs/reference/targets.md | 4 +- internal/config/check.go | 46 +++++++++++++++---- internal/config/check_test.go | 29 ++++++++++++ internal/config/config.go | 2 +- internal/config/hub.go | 7 ++- internal/config/providers.go | 32 ++++++++++++- internal/config/providers_test.go | 20 +++++++- .../hub/invalid/provider-bad-type.yml | 6 +-- .../invalid/provider-bitbucket-elsewhere.yml | 7 +++ .../hub/invalid/shorthand-bad-platform.yml | 4 +- .../invalid/shorthand-bitbucket-elsewhere.yml | 5 ++ .../config/testdata/hub/valid/bitbucket.yml | 14 ++++++ .../hub/valid/shorthand-bitbucket.yml | 4 ++ internal/config/urls.go | 11 +++-- internal/config/urls_test.go | 39 ++++++++++++++++ schemas/doctor.schema.json | 2 +- schemas/hub.schema.json | 33 ++++++++++--- schemas/report.schema.json | 2 +- schemas/targets.schema.json | 4 +- 21 files changed, 238 insertions(+), 45 deletions(-) create mode 100644 internal/config/testdata/hub/invalid/provider-bitbucket-elsewhere.yml create mode 100644 internal/config/testdata/hub/invalid/shorthand-bitbucket-elsewhere.yml create mode 100644 internal/config/testdata/hub/valid/bitbucket.yml create mode 100644 internal/config/testdata/hub/valid/shorthand-bitbucket.yml diff --git a/docs/reference/hub.md b/docs/reference/hub.md index 1e56ae2..70313e2 100644 --- a/docs/reference/hub.md +++ b/docs/reference/hub.md @@ -104,11 +104,11 @@ In a hub pull request, `providers` and `ca_file` are read from the default branc | `security.private_targets_in_public_hub` | one of `skip`, `deliver` | What a public hub does with non-public targets: skip them, or deliver and let their names reach public CI logs. Default: skip. | | `providers` | list of objects | Platforms this hub delivers to. Targets refer to them by id. | | `providers[].id` | string, required | Name targets.yml uses for this provider, and the <ID> of its TOUCHMARK_<ID>_\* variables. | -| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, required | — | -| `providers[].url` | string | Web URL of the instance. Default for github: https://github.com; for gitlab: https://gitlab.com. Required for gitea and forgejo. A target targets.yml names by web URL belongs to the provider whose url it lies under. | +| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, required | github: github.com, GHE.com or GitHub Enterprise Server; gitlab; gitea; forgejo; bitbucket: Bitbucket Cloud, read only for now (plan reads it; distribute, doctor and setup do not support it yet). | +| `providers[].url` | string | Web URL of the instance. Default for github: https://github.com; for gitlab: https://gitlab.com; for bitbucket: https://bitbucket.org, the only url it takes without api_url. Required for gitea and forgejo. A target targets.yml names by web URL belongs to the provider whose url it lies under. | | `providers[].api_url` | string | API URL, when it cannot be derived from url. | | `providers[].ca_file` | string | Hub file with extra CA certificates for this instance. | -| `providers[].writer` | string, up to 255 characters | The account that writes to targets, as the platform names it: on GitHub the App's bot, <slug>[bot]; on GitLab, Gitea and Forgejo the username of the service account or bot user. | +| `providers[].writer` | string, up to 255 characters | The account that writes to targets, as the platform names it: on GitHub the App's bot, <slug>[bot]; on GitLab, Gitea and Forgejo the username of the service account or bot user; on Bitbucket the bot account's UUID in braces, {…}. | | `providers[].known_authors` | list of strings, each up to 255 characters | Former writer accounts whose pull requests still count as this hub's own. | | `providers[].automation_accounts` | list of strings, each up to 255 characters | Accounts whose closing of a pull request is automatic, not a team's decision. | | `providers[].sign` | one of `auto`, `always` | auto: sign where it costs nothing and where target rules require it; always: sign every commit. Default: auto. | @@ -127,7 +127,7 @@ In a hub pull request, `providers` and `ca_file` are read from the default branc | `packs..formerly` | list of strings, each up to 64 characters | Former names of this pack. Their history counts as this pack's own. | | `writer` | string, up to 255 characters | Single-provider shorthand: the writer account, as the platform names it (on GitHub the App's bot, <slug>[bot]). Without platform, the provider's type and URL come from the CI environment, or locally from the hub's origin remote. Not allowed with providers. | | `sign` | one of `auto`, `always` | Single-provider shorthand: commit signing. Not allowed with providers. | -| `platform` | one of `github`, `gitlab`, `gitea`, `forgejo` | Single-provider shorthand: the provider type, which is also its id. Not allowed with providers. | +| `platform` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket` | Single-provider shorthand: the provider type, which is also its id. Not allowed with providers. | | `base_url` | string | Single-provider shorthand: the provider URL, for self-managed instances. Needs platform; not allowed with providers. | diff --git a/docs/reference/output.md b/docs/reference/output.md index 43e7b25..0920bb5 100644 --- a/docs/reference/output.md +++ b/docs/reference/output.md @@ -87,7 +87,7 @@ and the branch head before and after. | `assume_opt_in` | boolean | plan --assume-opt-in: for this report only, every target without an opt-in file, or whose opt-in file is not a regular file or is too large, counts as opted in; one whose opt-in file says enabled: false stays opted out. | | `providers` | list of objects, required | The hub's providers, in hub.yml order. | | `providers[].id` | string, required | — | -| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, required | — | +| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, required | — | | `providers[].host` | string, not empty, required | The provider's host, with a port when it has one. | | `providers[].reader` | string | Login of the read account. | | `providers[].writer` | string | Login of the write account. | @@ -199,7 +199,7 @@ hub, for each provider's write identity, and for each target. See | `hub_checks[].detail` | string | What the check found; absent for a target the report does not name. | | `providers` | list of objects, required | The hub's providers, in hub.yml order. | | `providers[].id` | string, required | — | -| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, required | — | +| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, required | — | | `providers[].host` | string, not empty, required | The provider's host, with a port when it has one. | | `providers[].writer` | string | Login of the writer hub.yml names. | | `providers[].self` | string | Login the write credential acts as. | diff --git a/docs/reference/targets.md b/docs/reference/targets.md index 4d780cf..50216ef 100644 --- a/docs/reference/targets.md +++ b/docs/reference/targets.md @@ -190,8 +190,8 @@ either way. | `targets[].provider` | string | Id of a provider in hub.yml. | | `targets[].packs` | list of strings, each up to 64 characters | — | | `targets[].opt_in` | one of `required`, `assumed` | required (the default): a repository the entry selects gets nothing until it adds the opt-in file. assumed: it counts as opted in without one, with the packs targets.yml gives it, until its opt-in file says enabled: false. One entry with assumed is enough to subscribe a repository. | -| `targets[].org` | string, up to 512 characters | Every repository in a GitHub organisation or a GitLab, Gitea or Forgejo namespace (a synonym of group), named by its path or its web URL. An entry has exactly one of `repo`, `org` and `group`. | -| `targets[].topics` | list of strings | Only repositories with all of these topics. Only with `org` or `group`. | +| `targets[].org` | string, up to 512 characters | Every repository in a GitHub organisation, a GitLab, Gitea or Forgejo namespace or a Bitbucket workspace (a synonym of group), named by its path or its web URL. An entry has exactly one of `repo`, `org` and `group`. | +| `targets[].topics` | list of strings | Only repositories with all of these topics. Bitbucket has no topics: check refuses them on its entries; use match. Only with `org` or `group`. | | `targets[].subgroups` | boolean | Include repositories in nested groups. Default: true. Only with `org` or `group`. | | `targets[].forks` | boolean | Include forks. Default: false. Only with `org` or `group`. | | `targets[].match` | list of strings, each up to 512 characters | Only repositories whose full path, without the provider, matches at least one of these patterns: \* within one segment, \*\* any number of segments, ? one character, compared ignoring case, such as acme/svc-\* or platform/\*\*/api. Quote a pattern that starts with \*, which YAML reads as an alias. Only with `org` or `group`. | diff --git a/internal/config/check.go b/internal/config/check.go index 870346a..6503d15 100644 --- a/internal/config/check.go +++ b/internal/config/check.go @@ -46,6 +46,7 @@ func check(hub *Hub, targets *Targets, known map[string]bool) ([]Warning, []erro c.providers() c.exclude() c.match() + c.topics() c.optIn() return c.warns, c.errs } @@ -247,23 +248,50 @@ func (c *checker) match() { } // flatType returns the type of the provider with id when its platform has no -// nested namespaces (github, gitea, forgejo): a repository path there is -// owner/name. It returns "" for any other type, and for a provider hub.yml -// does not list (the implicit one, whose type the CI tells). +// nested namespaces (github, gitea, forgejo, bitbucket): a repository path +// there is owner/name. It returns "" for any other type, and for a provider +// hub.yml does not list (the implicit one, whose type the CI tells). func (c *checker) flatType(id string) string { + switch typ := c.providerType(id); typ { + case "github", "gitea", "forgejo", "bitbucket": + return typ + } + return "" +} + +// providerType returns the type of the provider with id, "" for a provider +// hub.yml does not list. +func (c *checker) providerType(id string) string { for _, p := range c.hub.Providers { - if p.ID != id { - continue - } - switch p.Type { - case "github", "gitea", "forgejo": + if p.ID == id { return p.Type } - return "" } return "" } +// topics refuses topics on an entry whose provider is Bitbucket Cloud, +// whose repositories have none: the entry would select nothing, and the +// driver refuses the selector. +func (c *checker) topics() { + res := resolver{hub: c.hub, targets: c.targets} + for i := range c.targets.Targets { + e := &c.targets.Targets[i] + if len(e.Topics) == 0 || entryKind(e) == "repo" { + continue + } + ns, err := entryRef(e) + if err != nil { + continue + } + id := res.provider(firstNonEmpty(e.Provider, ns.Provider)) + if c.providerType(id) == "bitbucket" { + c.errorf(TargetsFile, "targets[%d].topics: provider %s is Bitbucket, whose repositories have no topics; "+ + "select them with match: (paths like %s/svc-*) or list them with repo:", i, id, ns.Path) + } + } +} + // optIn warns about a repo entry whose opt_in: required says nothing: an // org or group entry with opt_in: assumed may select the same repository, // and one entry with assumed is enough to subscribe it (Targets.Assumed). diff --git a/internal/config/check_test.go b/internal/config/check_test.go index af6f795..4ba9f75 100644 --- a/internal/config/check_test.go +++ b/internal/config/check_test.go @@ -160,6 +160,35 @@ func TestCheck(t *testing.T) { `targets.yml: exclude[0]: "acme"`, }, }, + { + name: "topics on bitbucket", + hub: &Hub{ID: "acme-eng", Providers: []Provider{ + {ID: "bb", Type: "bitbucket"}, {ID: "gh", Type: "github"}, + }}, + targets: &Targets{ + Defaults: Defaults{Provider: "bb"}, + Targets: []Entry{ + {Org: "acme", Topics: []string{"python"}}, + {Org: "gh:acme", Topics: []string{"python"}}, + {Org: "acme", Provider: "gh", Topics: []string{"python"}}, + {Org: "bb:acme", Match: []string{"acme/svc-*"}}, + {Group: "acme", Provider: "bb", Topics: []string{"go"}}, + }, + }, + errors: []string{ + "targets.yml: targets[0].topics: provider bb is Bitbucket, whose repositories have no topics; select them with match: (paths like acme/svc-*) or list them with repo:", + "targets.yml: targets[4].topics: provider bb is Bitbucket", + }, + }, + { + name: "bitbucket paths are workspace/repository", + hub: &Hub{ID: "acme-eng", Providers: []Provider{{ID: "bb", Type: "bitbucket"}}}, + targets: &Targets{ + Targets: []Entry{{Org: "acme"}}, + Exclude: []string{"acme/legacy/old"}, + }, + warnings: []string{"targets.yml: exclude[0]: acme/legacy/old has 3 path segments, and a bitbucket repository path is owner/name, so it excludes nothing"}, + }, { name: "listed and excluded", hub: selectHub(), diff --git a/internal/config/config.go b/internal/config/config.go index bc35256..a848a64 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -97,7 +97,7 @@ type Security struct { type Provider struct { ID string `yaml:"id"` - Type string `yaml:"type"` // github | gitlab | gitea | forgejo + Type string `yaml:"type"` // github | gitlab | gitea | forgejo | bitbucket URL string `yaml:"url"` APIURL string `yaml:"api_url"` CAFile string `yaml:"ca_file"` diff --git a/internal/config/hub.go b/internal/config/hub.go index ec29194..5933f90 100644 --- a/internal/config/hub.go +++ b/internal/config/hub.go @@ -26,12 +26,12 @@ const ( // Enumerations of hub.yml. var ( - providerTypes = []string{"github", "gitlab", "gitea", "forgejo"} + providerTypes = []string{"github", "gitlab", "gitea", "forgejo", "bitbucket"} linkHubModes = []string{"auto", "always", "never"} isolationModes = []string{"platform", "external", "none"} privateTargets = []string{"skip", "deliver"} signModes = []string{"auto", "always"} - defaultURLs = map[string]string{"github": "https://github.com", "gitlab": "https://gitlab.com"} + defaultURLs = map[string]string{"github": "https://github.com", "gitlab": "https://gitlab.com", "bitbucket": bitbucketURL} shorthandFields = []string{"writer", "sign", "platform", "base_url"} ) @@ -253,6 +253,9 @@ func checkProvider(pr *Provider, f providerFields, p *problems) { } else if pr.Type == "gitea" || pr.Type == "forgejo" { p.errorf(f.name("url"), "required for %s", pr.Type) } + if pr.Type == "bitbucket" && f.has("url") && checkURL(pr.URL) == nil && !f.has("api_url") && !isBitbucketCloud(pr.URL) { + p.errorf(f.name("url"), "%s", bitbucketURLError) + } if f.has("api_url") { if err := checkURL(pr.APIURL); err != nil { p.errorf(f.name("api_url"), "%v", err) diff --git a/internal/config/providers.go b/internal/config/providers.go index 105d3c2..41151dc 100644 --- a/internal/config/providers.go +++ b/internal/config/providers.go @@ -17,6 +17,26 @@ import ( // the prefix of the short names a hub with one provider accepts. const envPrefixBase = "TOUCHMARK_" +// Bitbucket Cloud: the web URL, which is the default url of a provider of +// type bitbucket, and its REST base. Bitbucket Data Center has another API +// and is not a bitbucket provider. +const ( + bitbucketURL = "https://bitbucket.org" + bitbucketAPIURL = "https://api.bitbucket.org/2.0" +) + +// bitbucketURLError is the complaint about a bitbucket provider at another +// url without an api_url. +const bitbucketURLError = "a provider of type bitbucket is Bitbucket Cloud, at https://bitbucket.org (leave url out); " + + "Bitbucket Data Center is not supported yet. Elsewhere (a test server) set api_url too" + +// isBitbucketCloud reports whether raw, a checked URL, is Bitbucket Cloud's +// web URL: https://bitbucket.org, with no path. +func isBitbucketCloud(raw string) bool { + u, err := url.Parse(strings.TrimRight(raw, "/")) + return err == nil && strings.EqualFold(u.Scheme, "https") && strings.EqualFold(u.Host, "bitbucket.org") && u.Path == "" +} + // ResolvedProvider is a provider from hub.yml (or the implicit one) with // everything a driver needs besides credentials. type ResolvedProvider struct { @@ -25,7 +45,8 @@ type ResolvedProvider struct { Host string // APIURL is the REST base: https://api.github.com, https://api. // for *.ghe.com, /api/v3 for GitHub Enterprise Server, - // /api/v4 for GitLab, /api/v1 for Gitea and Forgejo; + // /api/v4 for GitLab, /api/v1 for Gitea and Forgejo, + // https://api.bitbucket.org/2.0 for Bitbucket Cloud; // Provider.APIURL when set. On GitHub Actions for the hub's own host, // GITHUB_API_URL wins over the derived URL, not over Provider.APIURL: // an explicit api_url means the same in CI and locally. @@ -72,7 +93,8 @@ type ResolvedProvider struct { // // Every provider is checked again, with ParseHub's defaults, so a hub built // in code resolves like a parsed one: a valid id, no duplicate ids, a known -// type, the public URL of github and gitlab when URL is empty, https URLs +// type, the public URL of github, gitlab and bitbucket when URL is empty, +// an api_url for a bitbucket provider anywhere but Bitbucket Cloud, https URLs // (http only for localhost), sign auto. A nil hub is a legacy hub. Error // messages never quote environment values. A nil getenv reads the process // environment. @@ -254,6 +276,9 @@ func resolveProvider(p Provider, env ciEnv) (ResolvedProvider, error) { return ResolvedProvider{}, fmt.Errorf("api_url: %w", err) } } + if p.Type == "bitbucket" && p.APIURL == "" && !isBitbucketCloud(p.URL) { + return ResolvedProvider{}, fmt.Errorf("url: %s", bitbucketURLError) + } setDefault(&p.Sign, defaultSign) u, err := url.Parse(p.URL) if err != nil { @@ -299,6 +324,9 @@ func derivedAPIURL(typ, webURL, hostname, host string) string { return webURL + "/api/v3" case "gitlab": return webURL + "/api/v4" + case "bitbucket": + // resolveProvider asks for api_url anywhere else. + return bitbucketAPIURL } return webURL + "/api/v1" } diff --git a/internal/config/providers_test.go b/internal/config/providers_test.go index 7ceaecf..1d5298b 100644 --- a/internal/config/providers_test.go +++ b/internal/config/providers_test.go @@ -37,6 +37,9 @@ func TestResolveProviders(t *testing.T) { {ID: "tea", Type: "gitea", URL: "https://gitea.example.com"}, {ID: "cb", Type: "forgejo", URL: "https://codeberg.org"}, {ID: "local", Type: "gitea", URL: "http://localhost:3000"}, + {ID: "bb", Type: "bitbucket"}, + {ID: "bb-url", Type: "bitbucket", URL: "https://Bitbucket.org/"}, + {ID: "bb-test", Type: "bitbucket", URL: "http://localhost:8080", APIURL: "http://localhost:8080/2.0"}, }} want := map[string]resolvedURLs{ "gh": {"https://github.com", "github.com", "https://api.github.com", "https://api.github.com/graphql", "TOUCHMARK_GH_"}, @@ -50,6 +53,9 @@ func TestResolveProviders(t *testing.T) { "tea": {"https://gitea.example.com", "gitea.example.com", "https://gitea.example.com/api/v1", "", "TOUCHMARK_TEA_"}, "cb": {"https://codeberg.org", "codeberg.org", "https://codeberg.org/api/v1", "", "TOUCHMARK_CB_"}, "local": {"http://localhost:3000", "localhost:3000", "http://localhost:3000/api/v1", "", "TOUCHMARK_LOCAL_"}, + "bb": {"https://bitbucket.org", "bitbucket.org", "https://api.bitbucket.org/2.0", "", "TOUCHMARK_BB_"}, + "bb-url": {"https://Bitbucket.org", "bitbucket.org", "https://api.bitbucket.org/2.0", "", "TOUCHMARK_BB_URL_"}, + "bb-test": {"http://localhost:8080", "localhost:8080", "http://localhost:8080/2.0", "", "TOUCHMARK_BB_TEST_"}, } // The CI environment does not matter for declared providers, except // GITHUB_API_URL on the hub's own host (below). @@ -292,8 +298,18 @@ func TestResolveProvidersErrors(t *testing.T) { }, { name: "bad type", - hub: &Hub{Providers: []Provider{{ID: "bb", Type: "bitbucket"}}}, - want: `hub.yml: providers[0]: type: "bitbucket" must be one of github, gitlab, gitea, forgejo`, + hub: &Hub{Providers: []Provider{{ID: "ado", Type: "azure-devops"}}}, + want: `hub.yml: providers[0]: type: "azure-devops" must be one of github, gitlab, gitea, forgejo, bitbucket`, + }, + { + name: "bitbucket elsewhere without api_url", + hub: &Hub{Providers: []Provider{{ID: "bb", Type: "bitbucket", URL: "https://bitbucket.example.com"}}}, + want: "hub.yml: providers[0]: url: a provider of type bitbucket is Bitbucket Cloud, at https://bitbucket.org (leave url out); Bitbucket Data Center is not supported yet", + }, + { + name: "bitbucket under a path", + hub: &Hub{Providers: []Provider{{ID: "bb", Type: "bitbucket", URL: "https://bitbucket.org/acme"}}}, + want: "hub.yml: providers[0]: url: a provider of type bitbucket is Bitbucket Cloud", }, { name: "gitea without url", diff --git a/internal/config/testdata/hub/invalid/provider-bad-type.yml b/internal/config/testdata/hub/invalid/provider-bad-type.yml index 2b7935e..b3d995b 100644 --- a/internal/config/testdata/hub/invalid/provider-bad-type.yml +++ b/internal/config/testdata/hub/invalid/provider-bad-type.yml @@ -1,6 +1,6 @@ -# error: providers[0].type: "bitbucket" must be one of +# error: providers[0].type: "azure-devops" must be one of version: 1 id: acme-eng providers: - - id: bb - type: bitbucket + - id: ado + type: azure-devops diff --git a/internal/config/testdata/hub/invalid/provider-bitbucket-elsewhere.yml b/internal/config/testdata/hub/invalid/provider-bitbucket-elsewhere.yml new file mode 100644 index 0000000..3986336 --- /dev/null +++ b/internal/config/testdata/hub/invalid/provider-bitbucket-elsewhere.yml @@ -0,0 +1,7 @@ +# error: providers[0].url: a provider of type bitbucket is Bitbucket Cloud +version: 1 +id: acme-eng +providers: + - id: bb + type: bitbucket + url: https://bitbucket.example.com diff --git a/internal/config/testdata/hub/invalid/shorthand-bad-platform.yml b/internal/config/testdata/hub/invalid/shorthand-bad-platform.yml index 3a62a01..ea220d9 100644 --- a/internal/config/testdata/hub/invalid/shorthand-bad-platform.yml +++ b/internal/config/testdata/hub/invalid/shorthand-bad-platform.yml @@ -1,4 +1,4 @@ -# error: platform: "bitbucket" must be one of +# error: platform: "azure-devops" must be one of version: 1 id: acme-eng -platform: bitbucket +platform: azure-devops diff --git a/internal/config/testdata/hub/invalid/shorthand-bitbucket-elsewhere.yml b/internal/config/testdata/hub/invalid/shorthand-bitbucket-elsewhere.yml new file mode 100644 index 0000000..9a6f11f --- /dev/null +++ b/internal/config/testdata/hub/invalid/shorthand-bitbucket-elsewhere.yml @@ -0,0 +1,5 @@ +# error: base_url: a provider of type bitbucket is Bitbucket Cloud +version: 1 +id: acme-eng +platform: bitbucket +base_url: https://bitbucket.example.com diff --git a/internal/config/testdata/hub/valid/bitbucket.yml b/internal/config/testdata/hub/valid/bitbucket.yml new file mode 100644 index 0000000..0c9e449 --- /dev/null +++ b/internal/config/testdata/hub/valid/bitbucket.yml @@ -0,0 +1,14 @@ +version: 1 +id: acme-eng +providers: + - id: bb + type: bitbucket + writer: "{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d}" + known_authors: ["{0b7e5a2c-9d4f-4e1b-8a3c-6f5d2e1c0b9a}"] + - id: bb-cloud + type: bitbucket + url: https://bitbucket.org/ + - id: bb-test + type: bitbucket + url: http://localhost:8080 + api_url: http://localhost:8080/2.0 diff --git a/internal/config/testdata/hub/valid/shorthand-bitbucket.yml b/internal/config/testdata/hub/valid/shorthand-bitbucket.yml new file mode 100644 index 0000000..93905d4 --- /dev/null +++ b/internal/config/testdata/hub/valid/shorthand-bitbucket.yml @@ -0,0 +1,4 @@ +version: 1 +id: acme-eng +platform: bitbucket +writer: "{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d}" diff --git a/internal/config/urls.go b/internal/config/urls.go index 9a9804f..c810f57 100644 --- a/internal/config/urls.go +++ b/internal/config/urls.go @@ -149,9 +149,10 @@ func (t *Targets) HasURLs() bool { // entry names its provider with provider:, which then must be one of them. // An implicit provider (the CI's own platform, for a hub.yml without // providers) is the hub's only one: its targets are written as bare paths. -// On GitHub, Gitea and Forgejo, which have no nested namespaces, a -// repository URL must name owner/name and an organisation URL one -// namespace; on GitLab a URL with a /-/ segment points inside a project and +// On GitHub, Gitea, Forgejo and Bitbucket, which have no nested +// namespaces, a repository URL must name owner/name (workspace/repository on +// Bitbucket, as in https://bitbucket.org/acme/billing) and an organisation +// URL one namespace; on GitLab a URL with a /-/ segment points inside a project and // is refused. // // The result is a copy: targets is left as it is, and returned as it is @@ -305,14 +306,14 @@ func (r urlResolver) list(indexes []int) string { } // checkURLShape checks the path segments a URL of kind k names on a -// platform of type typ: GitHub, Gitea and Forgejo have no nested +// platform of type typ: GitHub, Gitea, Forgejo and Bitbucket have no nested // namespaces, so an exclude URL of more than owner/name, without a "**" // that could span nothing, points inside a repository (/tree/main) and // would exclude nothing; a GitLab URL with a /-/ segment points inside a // project (a file, a merge request), not at it. func checkURLShape(typ string, segs []string, k urlKind) error { switch typ { - case "github", "gitea", "forgejo": + case "github", "gitea", "forgejo", "bitbucket": switch { case k == urlRepo && len(segs) != 2: return fmt.Errorf("a %s repository URL names owner/name, like https://example.com/acme/billing; this one has %d path segments after the provider's url", typ, len(segs)) diff --git a/internal/config/urls_test.go b/internal/config/urls_test.go index ef500d6..db0fab9 100644 --- a/internal/config/urls_test.go +++ b/internal/config/urls_test.go @@ -337,3 +337,42 @@ func TestRefRefusesURL(t *testing.T) { t.Errorf("Selectors of a URL: %v", err) } } + +// TestResolveURLsBitbucket: a Bitbucket Cloud provider takes the web URLs of +// bitbucket.org, workspace/repository and workspace only. +func TestResolveURLsBitbucket(t *testing.T) { + provs := resolvedProviders(t, `version: 1 +id: acme-eng +providers: + - id: bb + type: bitbucket + - id: gh + type: github +`) + for _, tc := range []struct { + entry Entry + want string // the resolved value, or a substring of the error after "!" + }{ + {Entry{Repo: "https://bitbucket.org/acme/billing"}, "bb:acme/billing"}, + {Entry{Repo: "https://bitbucket.org/acme/billing.git"}, "bb:acme/billing"}, + {Entry{Org: "https://bitbucket.org/acme/"}, "bb:acme"}, + {Entry{Repo: "https://bitbucket.org/acme/billing/src/main/README.md"}, "!a bitbucket repository URL names owner/name"}, + {Entry{Org: "https://bitbucket.org/acme/billing"}, "!a bitbucket organisation URL names one owner"}, + {Entry{Repo: "https://api.bitbucket.org/acme/billing"}, "!is not under the url of any provider"}, + } { + got, err := ResolveURLs(&Targets{Targets: []Entry{tc.entry}}, provs) + if want, ok := strings.CutPrefix(tc.want, "!"); ok { + if err == nil || !strings.Contains(err.Error(), want) { + t.Errorf("%+v: error %v, want one with %q", tc.entry, err, want) + } + continue + } + if err != nil { + t.Errorf("%+v: %v", tc.entry, err) + continue + } + if v := selectorValue(&got.Targets[0]); v != tc.want { + t.Errorf("%+v: %q, want %q", tc.entry, v, tc.want) + } + } +} diff --git a/schemas/doctor.schema.json b/schemas/doctor.schema.json index 7e46c08..973d6e4 100644 --- a/schemas/doctor.schema.json +++ b/schemas/doctor.schema.json @@ -113,7 +113,7 @@ "required": ["id", "type", "host", "resolve_complete", "checks"], "properties": { "id": { "$ref": "#/$defs/providerId" }, - "type": { "enum": ["github", "gitlab", "gitea", "forgejo"] }, + "type": { "enum": ["github", "gitlab", "gitea", "forgejo", "bitbucket"] }, "host": { "description": "The provider's host, with a port when it has one.", "type": "string", diff --git a/schemas/hub.schema.json b/schemas/hub.schema.json index 0135dff..d0dc2dd 100644 --- a/schemas/hub.schema.json +++ b/schemas/hub.schema.json @@ -209,6 +209,13 @@ "properties": { "platform": { "enum": ["gitea", "forgejo"] } } }, "then": { "required": ["base_url"] } + }, + { + "if": { + "required": ["platform"], + "properties": { "platform": { "const": "bitbucket" } } + }, + "then": { "properties": { "base_url": { "pattern": "^https://[Bb][Ii][Tt][Bb][Uu][Cc][Kk][Ee][Tt]\\.[Oo][Rr][Gg]/*$" } } } } ], "$defs": { @@ -239,7 +246,8 @@ "pattern": "^[a-z][a-z0-9-]{0,31}$" }, "providerType": { - "enum": ["github", "gitlab", "gitea", "forgejo"] + "description": "github: github.com, GHE.com or GitHub Enterprise Server; gitlab; gitea; forgejo; bitbucket: Bitbucket Cloud, read only for now (plan reads it; distribute, doctor and setup do not support it yet).", + "enum": ["github", "gitlab", "gitea", "forgejo", "bitbucket"] }, "sign": { "description": "auto: sign where it costs nothing and where target rules require it; always: sign every commit. Default: auto.", @@ -279,7 +287,7 @@ }, "type": { "$ref": "#/$defs/providerType" }, "url": { - "description": "Web URL of the instance. Default for github: https://github.com; for gitlab: https://gitlab.com. Required for gitea and forgejo. A target targets.yml names by web URL belongs to the provider whose url it lies under.", + "description": "Web URL of the instance. Default for github: https://github.com; for gitlab: https://gitlab.com; for bitbucket: https://bitbucket.org, the only url it takes without api_url. Required for gitea and forgejo. A target targets.yml names by web URL belongs to the provider whose url it lies under.", "$ref": "#/$defs/url" }, "api_url": { @@ -291,7 +299,7 @@ "$ref": "#/$defs/relativePath" }, "writer": { - "description": "The account that writes to targets, as the platform names it: on GitHub the App's bot, [bot]; on GitLab, Gitea and Forgejo the username of the service account or bot user.", + "description": "The account that writes to targets, as the platform names it: on GitHub the App's bot, [bot]; on GitLab, Gitea and Forgejo the username of the service account or bot user; on Bitbucket the bot account's UUID in braces, {…}.", "$ref": "#/$defs/account" }, "known_authors": { @@ -324,10 +332,21 @@ } } }, - "if": { - "properties": { "type": { "enum": ["gitea", "forgejo"] } } - }, - "then": { "required": ["url"] } + "allOf": [ + { + "if": { + "properties": { "type": { "enum": ["gitea", "forgejo"] } } + }, + "then": { "required": ["url"] } + }, + { + "if": { + "required": ["type", "url"], + "properties": { "type": { "const": "bitbucket" }, "url": { "not": { "pattern": "^https://[Bb][Ii][Tt][Bb][Uu][Cc][Kk][Ee][Tt]\\.[Oo][Rr][Gg]/*$" } } } + }, + "then": { "required": ["api_url"] } + } + ] } } } diff --git a/schemas/report.schema.json b/schemas/report.schema.json index e2ea28a..8272d96 100644 --- a/schemas/report.schema.json +++ b/schemas/report.schema.json @@ -142,7 +142,7 @@ "required": ["id", "type", "host", "write_check", "resolve_complete"], "properties": { "id": { "$ref": "#/$defs/providerId" }, - "type": { "enum": ["github", "gitlab", "gitea", "forgejo"] }, + "type": { "enum": ["github", "gitlab", "gitea", "forgejo", "bitbucket"] }, "host": { "description": "The provider's host, with a port when it has one.", "type": "string", diff --git a/schemas/targets.schema.json b/schemas/targets.schema.json index e09ffba..bac2d24 100644 --- a/schemas/targets.schema.json +++ b/schemas/targets.schema.json @@ -89,7 +89,7 @@ "required": ["org"], "properties": { "org": { - "description": "Every repository in a GitHub organisation or a GitLab, Gitea or Forgejo namespace (a synonym of group), named by its path or its web URL.", + "description": "Every repository in a GitHub organisation, a GitLab, Gitea or Forgejo namespace or a Bitbucket workspace (a synonym of group), named by its path or its web URL.", "$ref": "#/$defs/namespace" }, "provider": { "$ref": "#/$defs/providerId" }, @@ -134,7 +134,7 @@ } }, "topics": { - "description": "Only repositories with all of these topics.", + "description": "Only repositories with all of these topics. Bitbucket has no topics: check refuses them on its entries; use match.", "type": ["array", "null"], "items": { "type": "string", "pattern": "[^\t\n\u000b\f\r \u00a0\u1680\u2000-\u200a\u2028\u2029\u202f\u205f\u3000\ufeff]" } }, From 9836911242894c43924072d8c235af923a2979a6 Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Thu, 8 Oct 2026 23:39:55 +0300 Subject: [PATCH 02/20] feat(bitbucket): read Bitbucket Cloud repositories, files and pull requests A new driver, internal/platform/bitbucket, serves plan for providers of type bitbucket; its write driver refuses with ClassUnsupported until the writer comes, so distribute and doctor stop at such a provider, and setup and the hub's key store say Bitbucket is not supported yet. The reader authenticates with an API token of a bot account: Bearer for the REST API, Basic x-bitbucket-api-token-auth for git, both masked. Accounts are their uuids in braces (Lookup takes a uuid or an Atlassian account id and refuses anything else: Bitbucket finds no account by name). Probe asks nothing: flavor bitbucket, native drafts, closed_by as the closer, a 60 000-byte body, the marker as a Markdown link reference definition (platform.MarkerInRefDef, whose codec comes with the writer), branch restrictions known at push time only, and 15 reads a minute with two targets at once. Repo and Resolve read workspace/slug and a workspace's repositories through the next links (topics are refused); a repository without a main branch is empty. ReadFile resolves the ref to a commit, reads the path's metadata, then its raw bytes at that commit, and computes the blob id. Symlinks, submodules, directories and Git LFS files are not regular. Only a path an existing commit lacks is ErrNotFound; a missing repository, ref or commit is ClassUnknown, never an opt-out. PRs lists the sync branches' pull requests in every state through q= (sorted by descending id), maps OPEN, MERGED, DECLINED and SUPERSEDED, takes the closer from closed_by and ClosedAt from updated_on, checks bases other than the default branch, and expands the short head of an open pull request through the commit API. OpenPRsBy sweeps every workspace of the identity by author. Errors follow Bitbucket's {"type": "error"} body: 401 auth, 403 permission, 404 and 410 not found, 429 or a rate-limit message rate limited with the wait from Retry-After or x-ratelimit-reset, 5xx transient. Tests run on an HTTP test server with answers shaped after the OpenAPI description and anonymous answers of public repositories. --- docs/guide/providers.md | 35 +- internal/cli/distribute_deps.go | 5 +- internal/cli/drivers.go | 10 +- internal/cli/plan.go | 14 +- internal/cli/setup.go | 3 + internal/cli/setup_test.go | 3 +- internal/hubch/keys.go | 5 +- internal/platform/bitbucket/bitbucket.go | 81 +++ internal/platform/bitbucket/client.go | 392 ++++++++++++++ internal/platform/bitbucket/errors.go | 338 ++++++++++++ internal/platform/bitbucket/errors_test.go | 153 ++++++ internal/platform/bitbucket/files.go | 296 +++++++++++ internal/platform/bitbucket/files_test.go | 322 ++++++++++++ internal/platform/bitbucket/helpers_test.go | 347 +++++++++++++ internal/platform/bitbucket/probe.go | 138 +++++ internal/platform/bitbucket/probe_test.go | 156 ++++++ internal/platform/bitbucket/pulls.go | 542 ++++++++++++++++++++ internal/platform/bitbucket/pulls_test.go | 350 +++++++++++++ internal/platform/bitbucket/repos.go | 213 ++++++++ internal/platform/bitbucket/repos_test.go | 205 ++++++++ internal/platform/platform.go | 12 +- 21 files changed, 3602 insertions(+), 18 deletions(-) create mode 100644 internal/platform/bitbucket/bitbucket.go create mode 100644 internal/platform/bitbucket/client.go create mode 100644 internal/platform/bitbucket/errors.go create mode 100644 internal/platform/bitbucket/errors_test.go create mode 100644 internal/platform/bitbucket/files.go create mode 100644 internal/platform/bitbucket/files_test.go create mode 100644 internal/platform/bitbucket/helpers_test.go create mode 100644 internal/platform/bitbucket/probe.go create mode 100644 internal/platform/bitbucket/probe_test.go create mode 100644 internal/platform/bitbucket/pulls.go create mode 100644 internal/platform/bitbucket/pulls_test.go create mode 100644 internal/platform/bitbucket/repos.go create mode 100644 internal/platform/bitbucket/repos_test.go diff --git a/docs/guide/providers.md b/docs/guide/providers.md index 3e0c928..63ac212 100644 --- a/docs/guide/providers.md +++ b/docs/guide/providers.md @@ -2,7 +2,8 @@ One hub can deliver to GitHub, GitLab, Gitea and Forgejo at once, including self-managed instances. Each platform is a **provider** in `hub.yml`, with its own reader -and writer; `targets.yml` names repositories by provider. +and writer; `targets.yml` names repositories by provider. Bitbucket Cloud can be read +already, not written to yet: see [Bitbucket Cloud](#bitbucket-cloud-reads-only). ## One provider: the shorthand @@ -58,7 +59,8 @@ providers: - `id` is how `targets.yml` names the provider, and the `` of its variables: upper-cased, with `-` as `_`. - `api_url` is needed only when it cannot be derived: `api.github.com`, `/api/v3` - for GitHub Enterprise Server, `/api/v4` for GitLab, `/api/v1` for Gitea and Forgejo. + for GitHub Enterprise Server, `/api/v4` for GitLab, `/api/v1` for Gitea and Forgejo, + `https://api.bitbucket.org/2.0` for Bitbucket Cloud. - `known_authors` are former writers whose pull requests stay the hub's own: a replaced App, a recreated group access token's bot, the account of a multi-gitter setup. touchmark never adds anyone there itself. @@ -124,6 +126,35 @@ on GitHub or GitLab can still deliver to them: add them as providers, and keep t write token in the GitHub environment or the GitLab protected variable, where the probe covers it. +## Bitbucket Cloud (reads only) + +A provider of type `bitbucket` is Bitbucket Cloud at `https://bitbucket.org` (leave `url` +out). This release reads it: `plan` resolves its targets, reads their opt-in files and +their pull requests, and reports what a delivery would do. Delivering does not work yet: +`distribute` and `doctor` stop at a `bitbucket` provider, and `setup` has nothing for it. +Bitbucket Data Center is not supported. + +```yaml +# hub.yml +providers: + - id: bb + type: bitbucket + writer: "{3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d}" # the bot account's UUID, in braces +``` + +- **Credentials.** `TOUCHMARK_BB_READ_TOKEN` is an API token of a bot account with read + access to the repositories (app passwords are gone; repository, project and workspace + access tokens are not supported, since they cannot tell who they are). +- **Accounts are UUIDs.** Bitbucket finds no account by its nickname, so `writer`, + `known_authors` and `automation_accounts` name accounts by their UUID in braces, as + `GET https://api.bitbucket.org/2.0/user` shows it. +- **Targets.** `org` is a workspace, `repo` is `workspace/repository`, and web URLs such as + `https://bitbucket.org/acme/billing` work. Bitbucket has no topics: `check` refuses + `topics` on a Bitbucket entry; select repositories with `match` (`acme/svc-*`) or list + them with `repo`. +- **Pace.** Bitbucket allows an account about 1 000 API requests an hour, so touchmark + reads 15 times a minute by default, with two targets at once; `limits` overrides it. + ## Moving - **A target moves to another provider on the same host** (a new App, a new group): add diff --git a/internal/cli/distribute_deps.go b/internal/cli/distribute_deps.go index 7ec9c07..f2a9fbb 100644 --- a/internal/cli/distribute_deps.go +++ b/internal/cli/distribute_deps.go @@ -32,8 +32,9 @@ import ( type distributeDriver func(config.ResolvedProvider, auth.Credential, *httpx.Client) (platform.Writer, error) // The write drivers of distribute by provider type: Gitea and Forgejo, -// GitLab and GitHub (drivers.go). Tests install the fake -// platform here; tests that do must not run in parallel. +// GitLab and GitHub, and Bitbucket Cloud's, which refuses for now +// (drivers.go). Tests install the fake platform here; tests that do must +// not run in parallel. var distributeDrivers = map[string]distributeDriver{} // writeSecrets are what distribute reads from the environment for each diff --git a/internal/cli/drivers.go b/internal/cli/drivers.go index 2584f24..8a77196 100644 --- a/internal/cli/drivers.go +++ b/internal/cli/drivers.go @@ -1,15 +1,17 @@ package cli import ( + "github.com/bedrock-python/touchmark/internal/platform/bitbucket" "github.com/bedrock-python/touchmark/internal/platform/gitea" "github.com/bedrock-python/touchmark/internal/platform/github" "github.com/bedrock-python/touchmark/internal/platform/gitlab" ) // init registers the platform drivers this build has: Gitea and Forgejo, one -// driver for both, GitLab and GitHub (github.com, GHE.com and GitHub -// Enterprise Server). Tests that install their own drivers replace the maps -// whole and restore them afterwards. +// driver for both, GitLab, GitHub (github.com, GHE.com and GitHub +// Enterprise Server) and Bitbucket Cloud, whose write driver refuses with +// ClassUnsupported until its writer comes. Tests that install their own +// drivers replace the maps whole and restore them afterwards. func init() { for _, typ := range []string{"gitea", "forgejo"} { planDrivers[typ] = gitea.NewReader @@ -19,4 +21,6 @@ func init() { distributeDrivers["gitlab"] = gitlab.NewWriter planDrivers["github"] = github.NewReader distributeDrivers["github"] = github.NewWriter + planDrivers["bitbucket"] = bitbucket.NewReader + distributeDrivers["bitbucket"] = bitbucket.NewWriter } diff --git a/internal/cli/plan.go b/internal/cli/plan.go index bc6d8b4..42db517 100644 --- a/internal/cli/plan.go +++ b/internal/cli/plan.go @@ -33,10 +33,10 @@ import ( type planDriver func(config.ResolvedProvider, auth.Credential, *httpx.Client) (platform.Reader, error) // The drivers of plan by provider type, and a snapshot source that replaces -// plan's own. This build has the Gitea and Forgejo, GitLab and GitHub -// drivers (drivers.go). plan's own source is distribute's: a private git -// repository per target (sharedDeps), so that plan and distribute --dry-run -// decide alike. Tests install the fake platform here; a planSnapshots that +// plan's own. This build has the Gitea and Forgejo, GitLab, GitHub and +// Bitbucket Cloud drivers (drivers.go). plan's own source is distribute's: +// a private git repository per target (sharedDeps), so that plan and +// distribute --dry-run decide alike. Tests install the fake platform here; a planSnapshots that // is only a snapshot.Source (the fake in memory mode) keeps the in-memory // plan (distribute.Plan). Tests that install them must not run in // parallel. @@ -50,9 +50,9 @@ var ( var gitVersion = func(ctx context.Context) ([3]int, error) { return gitx.New("").Version(ctx) } // basicUsers are the users a token goes with in HTTP Basic headers (GitHub, -// GitLab and Gitea git over HTTPS, GitLab's job token): the base64 forms of -// "user:token" are masked too. -var basicUsers = []string{"x-access-token", "oauth2", hubch.GitLabUser} +// GitLab, Gitea and Bitbucket git over HTTPS, GitLab's job token): the +// base64 forms of "user:token" are masked too. +var basicUsers = []string{"x-access-token", "oauth2", hubch.GitLabUser, "x-bitbucket-api-token-auth"} // Suffixes of the credential variables after a provider's prefix. var ( diff --git a/internal/cli/setup.go b/internal/cli/setup.go index 0fd2776..50552e1 100644 --- a/internal/cli/setup.go +++ b/internal/cli/setup.go @@ -97,6 +97,9 @@ func runSetup(ctx context.Context, e *env, o *options, s *setupOptions) (err err switch platform { case "gitea", "forgejo": return giteaRefusal(platform) + case "bitbucket": + return configErrorf("setup does not set up bitbucket yet: touchmark reads Bitbucket Cloud (plan), and its writer, " + + "which delivery and setup need, comes in a later release") case "github", "gitlab": default: return usageErrorf("unknown platform %q: setup sets up github or gitlab", platform) diff --git a/internal/cli/setup_test.go b/internal/cli/setup_test.go index ea4cde3..da1a77c 100644 --- a/internal/cli/setup_test.go +++ b/internal/cli/setup_test.go @@ -45,7 +45,8 @@ func TestSetupRefusals(t *testing.T) { want string }{ {"no-platform", nil, []string{"setup"}, "missing the platform"}, - {"unknown", nil, []string{"setup", "bitbucket", "--hub", h.dir}, `unknown platform "bitbucket"`}, + {"unknown", nil, []string{"setup", "azure-devops", "--hub", h.dir}, `unknown platform "azure-devops"`}, + {"bitbucket", nil, []string{"setup", "bitbucket", "--hub", h.dir}, "setup does not set up bitbucket yet"}, {"gitea", nil, []string{"setup", "gitea", "--hub", h.dir}, "every branch the secrets"}, {"forgejo", nil, []string{"setup", "forgejo"}, "setup does not set up forgejo"}, {"flags-first", nil, []string{"setup", "--hub", h.dir, "gitea", "--dry-run"}, "setup does not set up gitea"}, diff --git a/internal/hubch/keys.go b/internal/hubch/keys.go index 7daeff4..de3484c 100644 --- a/internal/hubch/keys.go +++ b/internal/hubch/keys.go @@ -20,7 +20,8 @@ import ( // KeyStoreInput locates the hub for ReadKeyStore. type KeyStoreInput struct { - // Platform is "github", "gitlab", "gitea" or "forgejo". + // Platform is "github", "gitlab", "gitea" or "forgejo"; "bitbucket" is + // refused for now. Platform string // APIURL is the REST base of the hub's platform (…/api/v3, /api/v4, // /api/v1, https://api.github.com). @@ -148,6 +149,8 @@ func ReadKeyStore(ctx context.Context, in KeyStoreInput) (KeyStore, error) { case "gitea", "forgejo": k.auth = &httpx.Auth{Hosts: []string{api.Host}, Header: k.header("token ")} return k.gitea(ctx) + case "bitbucket": + return KeyStore{}, errors.New("hub keys: a hub on Bitbucket is not supported yet") } return KeyStore{}, fmt.Errorf("hub keys: unknown platform %q", in.Platform) } diff --git a/internal/platform/bitbucket/bitbucket.go b/internal/platform/bitbucket/bitbucket.go new file mode 100644 index 0000000..9a837d8 --- /dev/null +++ b/internal/platform/bitbucket/bitbucket.go @@ -0,0 +1,81 @@ +// Package bitbucket is the platform driver for Bitbucket Cloud +// (bitbucket.org, REST API 2.0). Bitbucket Data Center has another API and +// is not served here. +// +// This release reads only: NewReader serves plan, and NewWriter refuses +// with ClassUnsupported until the writer comes. +// +// Authentication is an API token of a bot account (app passwords are gone: +// their creation closed on 2025-09-09). The API gets +// "Authorization: Bearer "; git over HTTPS gets Basic +// "x-bitbucket-api-token-auth:" (REST intro, "API tokens"; +// support.atlassian.com/bitbucket-cloud, "Using API tokens"). A token's +// scopes do not narrow the repositories it reaches: the account's +// permissions do. Access tokens of a repository, project or workspace are +// left for later: GET /2.0/user does not accept them (BCLOUD-23528), so the +// identity cannot learn who it is. +// +// What the driver relies on, from the OpenAPI description +// (https://api.bitbucket.org/swagger.json), the REST intro +// (https://developer.atlassian.com/cloud/bitbucket/rest/intro/) and +// anonymous answers of public repositories: +// - accounts are known by their uuid, in braces, which never changes; +// nickname is neither unique nor free of spaces, and there is no +// lookup by name, so a login in hub.yml is a uuid (or an Atlassian +// account id), and Account.ID and Account.Login are the uuid; +// - a workspace is the namespace: no nested groups, no topics, no +// archiving; a repository is workspace/slug, its uuid is immutable; +// - collections are pages of values with an opaque next link, absent on +// the last page; q= filters (BBQL) and sort= orders them, and on pull +// request listings a q= makes the state parameter ignored, so the state +// goes into q; +// - GET …/src/{commit}/{path}?format=meta describes a path (commit_file +// or commit_directory, attributes link, executable, subrepository, +// binary, lfs, and size), and without format returns the raw bytes; +// Git LFS content is a redirect to another host; there is no blob id; +// - a pull request's states are OPEN, MERGED, DECLINED and SUPERSEDED, +// draft is a flag, closed_by names who merged or declined it, and its +// commit hashes are short (12 digits); there are no labels; +// - limits are per account and hour (1 000 requests to +// /2.0/repositories/*), answered with 429 and x-ratelimit-* headers +// whose reset is in seconds. +// +// The driver never retries or paces (internal/throttle does), never knows +// about packs or markers, and masks secrets in its errors. +package bitbucket + +import ( + "errors" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/config" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +// NewReader returns the read driver for provider p (type "bitbucket") with +// credential c (an auth.Token, an API token; a zero credential reads +// anonymously, public repositories only) over client, which sends the token +// only to p's API host. +func NewReader(p config.ResolvedProvider, c auth.Credential, client *httpx.Client) (platform.Reader, error) { + cl, err := newClient(p, c, client) + if err != nil { + return nil, err + } + return &reader{c: cl}, nil +} + +// errNoWriter is why NewWriter refuses. +var errNoWriter = errors.New("the Bitbucket writer comes in a later release; plan reads Bitbucket Cloud already") + +// NewWriter refuses with ClassUnsupported: this release reads Bitbucket +// Cloud only, so distribute and doctor cannot run for a bitbucket provider +// yet. +func NewWriter(config.ResolvedProvider, auth.Credential, *httpx.Client) (platform.Writer, error) { + return nil, &platform.Error{Op: "new writer", Class: platform.ClassUnsupported, Err: errNoWriter} +} + +// reader is platform.Reader over one identity. +type reader struct{ c *client } + +var _ platform.Reader = (*reader)(nil) diff --git a/internal/platform/bitbucket/client.go b/internal/platform/bitbucket/client.go new file mode 100644 index 0000000..a09bec2 --- /dev/null +++ b/internal/platform/bitbucket/client.go @@ -0,0 +1,392 @@ +package bitbucket + +import ( + "context" + "encoding/base64" + "errors" + "fmt" + "net/http" + "net/url" + "strings" + "sync" + "unicode" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/config" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" + "github.com/bedrock-python/touchmark/internal/redact" +) + +// basicUser is the user name of git's Basic credentials for an API token +// of an account (support.atlassian.com/bitbucket-cloud, "Using API tokens": +// the git user of an API token is x-bitbucket-api-token-auth, or the +// account's username, which touchmark does not know). It is one of the +// users whose Basic form the CLI registers for masking. +const basicUser = "x-bitbucket-api-token-auth" + +// Page lengths asked for: the API's global bounds are 10 and 100 (REST +// intro, "Pagination"); the pull request listings are taken to stop at 50 +// (an assumption to confirm live; a smaller page only costs pages). +const ( + repoPageLen = 100 + prPageLen = 50 +) + +// maxRedirects bounds the redirects one GET follows. +const maxRedirects = 3 + +// snippetLimit bounds the body kept from a failed raw request, in bytes, as +// httpx keeps for JSON requests. +const snippetLimit = 1 << 10 + +// client is the HTTP side of one driver: one provider under one identity. +// It is safe for concurrent use. +type client struct { + http *httpx.Client + // api is the REST base without a trailing slash + // (https://api.bitbucket.org/2.0); apiURL is api parsed: redirects + // and pagination links are followed only below it. + api string + apiURL *url.URL + // web is the provider's web URL without a trailing slash; git remotes + // are under it. host is the provider host (Repo.Host). + web, host string + token string // "" when anonymous + auth *httpx.Auth + masks *redact.Registry + + mu sync.Mutex + self *platform.Account // cached by selfAccount + // commits maps " " to the full commit id + // (fullCommit): a short id of a commit that exists never names another. + commits map[string]string +} + +// newClient checks the provider and the credential and builds the client. +func newClient(p config.ResolvedProvider, cred auth.Credential, hc *httpx.Client) (*client, error) { + if p.Type != "bitbucket" { + return nil, fmt.Errorf("bitbucket: provider %s has type %q, not bitbucket", p.ID, p.Type) + } + if hc == nil { + return nil, errors.New("bitbucket: no HTTP client") + } + api, apiURL, err := baseURL("api_url", p.APIURL) + if err != nil { + return nil, fmt.Errorf("bitbucket: provider %s: %w", p.ID, err) + } + web, webURL, err := baseURL("url", p.URL) + if err != nil { + return nil, fmt.Errorf("bitbucket: provider %s: %w", p.ID, err) + } + host := strings.ToLower(p.Host) + if host == "" { + host = strings.ToLower(webURL.Host) + } + c := &client{ + http: hc, + api: api, + apiURL: apiURL, + web: web, + host: host, + masks: redact.New(), + commits: map[string]string{}, + } + switch cred.Kind { + case 0: + if cred.Token != "" || cred.AppID != "" || len(cred.AppKey) > 0 { + return nil, errors.New("bitbucket: a credential without a kind") + } + case auth.Token: + if cred.Token == "" { + return nil, errors.New("bitbucket: an empty token") + } + c.token = cred.Token + c.masks.Add(cred.Token, basicUser) + c.auth = &httpx.Auth{Hosts: []string{strings.ToLower(apiURL.Host)}, Header: c.apiHeader} + case auth.App: + return nil, fmt.Errorf("bitbucket: provider %s: a GitHub App cannot sign in to Bitbucket; use an API token of a bot account", p.ID) + default: + return nil, fmt.Errorf("bitbucket: provider %s: unknown credential kind %d", p.ID, cred.Kind) + } + return c, nil +} + +// baseURL checks an http(s) base URL without credentials, query or +// fragment, and returns it without a trailing slash. +func baseURL(what, raw string) (string, *url.URL, error) { + s := strings.TrimRight(raw, "/") + u, err := url.Parse(s) + switch { + case s == "": + return "", nil, fmt.Errorf("%s is empty", what) + case err != nil: + return "", nil, fmt.Errorf("%s does not parse", what) + case u.Scheme != "https" && u.Scheme != "http": + return "", nil, fmt.Errorf("%s is not an http or https URL", what) + case u.Host == "" || u.User != nil || u.RawQuery != "" || u.Fragment != "": + return "", nil, fmt.Errorf("%s must be a URL with a host and without credentials, query or fragment", what) + } + return s, u, nil +} + +// apiHeader is the Authorization header of API requests: an API token is +// a bearer token of the REST API (REST intro, "API tokens"). +func (c *client) apiHeader(context.Context) (string, error) { return "Bearer " + c.token, nil } + +// gitHeader is the Authorization header of git over HTTPS. +func (c *client) gitHeader(ctx context.Context) (string, error) { + if err := ctx.Err(); err != nil { + return "", err + } + return "Basic " + base64.StdEncoding.EncodeToString([]byte(basicUser+":"+c.token)), nil +} + +// mask replaces the credential's forms in s. +func (c *client) mask(s string) string { return c.masks.Replace(s) } + +// endpoint joins the API base and path segments, each escaped. +func (c *client) endpoint(segments ...string) string { + var b strings.Builder + b.WriteString(c.api) + for _, s := range segments { + b.WriteByte('/') + b.WriteString(url.PathEscape(s)) + } + return b.String() +} + +// pathSegments escapes each segment of a slash-separated path (a file +// path, a branch name) and joins them with slashes, which the API reads as +// they are: the branch API's own links keep the slashes of a branch name +// ("…/refs/branches/issue-9.3/AUI-5343-assistive-class" in the OpenAPI +// description). +func pathSegments(p string) []string { return strings.Split(p, "/") } + +// call sends one API request of op and decodes a 2xx JSON response into +// out (nil skips decoding). Every error is a masked *platform.Error, but a +// canceled context's. +// +// A GET follows a redirect to the same origin below the API base, at most +// maxRedirects times, with the same credential (a renamed repository may +// answer with one; whether it does is to be confirmed live). httpx follows +// no redirect of an authenticated request itself. +func (c *client) call(ctx context.Context, op, method, u string, query url.Values, in, out any) (*httpx.Response, error) { + if len(query) > 0 { + u += "?" + query.Encode() + } + for hops := 0; ; hops++ { + resp, err := c.http.JSON(ctx, method, u, c.auth, in, out) + if err == nil { + return resp, nil + } + if method == http.MethodGet && hops < maxRedirects { + if next, ok := c.redirect(u, err); ok { + u = next + continue + } + } + return resp, c.apiError(op, err) + } +} + +// get is call with GET. +func (c *client) get(ctx context.Context, op, u string, query url.Values, out any) (*httpx.Response, error) { + return c.call(ctx, op, http.MethodGet, u, query, nil, out) +} + +// redirect returns where the redirect err answered to a request for from +// leads, when it is one to follow (see below). +func (c *client) redirect(from string, err error) (string, bool) { + var se *httpx.StatusError + if !errors.As(err, &se) { + return "", false + } + return c.follow(from, se.Status, se.Header) +} + +// follow returns where a response of status with header to a request for +// from leads, when it is a redirect to follow: a 301, 302, 303, 307 or 308 +// whose Location, resolved against from, is on the API's scheme, host and +// port, without credentials, and below the API base. +func (c *client) follow(from string, status int, header http.Header) (string, bool) { + switch status { + case http.StatusMovedPermanently, http.StatusFound, http.StatusSeeOther, + http.StatusTemporaryRedirect, http.StatusPermanentRedirect: + default: + return "", false + } + loc := strings.TrimSpace(header.Get("Location")) + base, perr := url.Parse(from) + if loc == "" || perr != nil { + return "", false + } + to, perr := base.Parse(loc) + if perr != nil || !c.belowAPI(to) { + return "", false + } + to.Fragment, to.RawFragment = "", "" + return to.String(), true +} + +// belowAPI reports whether u is on the API's scheme, host and port, without +// credentials, and below the API base. +func (c *client) belowAPI(u *url.URL) bool { + if u.User != nil || !sameOrigin(u, c.apiURL) { + return false + } + root := strings.TrimRight(c.apiURL.Path, "/") + "/" + return strings.HasPrefix(u.Path, root) && !strings.Contains(u.Path, "/../") && !strings.HasSuffix(u.Path, "/..") +} + +// sameOrigin reports whether a and b have the same scheme, host and port +// (the scheme's default when absent), ignoring case. +func sameOrigin(a, b *url.URL) bool { + port := func(u *url.URL) string { + if p := u.Port(); p != "" { + return p + } + if strings.EqualFold(u.Scheme, "https") { + return "443" + } + return "80" + } + return strings.EqualFold(a.Scheme, b.Scheme) && strings.EqualFold(a.Hostname(), b.Hostname()) && port(a) == port(b) +} + +// errOffsite says the API redirected a raw read away from itself: Git LFS +// content lives on Atlassian's media platform (the OpenAPI description of +// GET …/src/{commit}/{path}). +var errOffsite = errors.New("the content is served from another host") + +// raw reads the body of u, a GET outside JSON (a file's raw content). A +// redirect below the API base is followed as call follows it; one +// elsewhere is errOffsite. A non-2xx answer is classified as call +// classifies it. +func (c *client) raw(ctx context.Context, op, u string) ([]byte, error) { + for hops := 0; ; hops++ { + req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil) + if err != nil { + return nil, &platform.Error{Op: op, Class: platform.ClassInvalid, Err: c.masked(err)} + } + req.Header.Set("Accept", "*/*") + resp, err := c.http.Do(req, c.auth) + if err != nil { + return nil, c.apiError(op, err) + } + if resp.Status >= 200 && resp.Status <= 299 { + return resp.Body, nil + } + if next, ok := c.follow(u, resp.Status, resp.Header); ok && hops < maxRedirects { + u = next + continue + } + if resp.Status >= 300 && resp.Status <= 399 { + return nil, &platform.Error{Op: op, Class: platform.ClassUnknown, Status: resp.Status, + Err: fmt.Errorf("GET %s: %d: %w", c.mask(withoutQuery(u)), resp.Status, errOffsite)} + } + return nil, c.statusError(op, &httpx.StatusError{ + Method: http.MethodGet, URL: withoutQuery(u), Status: resp.Status, + Header: resp.Header, Snippet: snippet(resp.Body), + }) + } +} + +// withoutQuery returns u without its query and fragment, as httpx shows +// URLs in errors. +func withoutQuery(u string) string { + if i := strings.IndexAny(u, "?#"); i >= 0 { + return u[:i] + } + return u +} + +// snippet returns at most snippetLimit bytes of body on one line. +func snippet(body []byte) string { + if len(body) > snippetLimit { + body = body[:snippetLimit] + } + return strings.Join(strings.FieldsFunc(strings.ToValidUTF8(string(body), "?"), func(r rune) bool { + return unicode.IsSpace(r) || unicode.IsControl(r) + }), " ") +} + +// page is one page of a paginated collection (REST intro, "Pagination"): +// values, and next, the opaque link to the following page, absent on the +// last. +type page[T any] struct { + Values []T `json:"values"` + Next string `json:"next"` +} + +// listAll reads a paginated collection of T at u with query and calls each +// for every item, in order, following the next links, which must stay +// below the API base. complete is false when the listing was capped at +// maxPages. An error of a page after the first comes with the page in err +// (see pageError), so a caller can tell a failed listing from a failed +// first request. +func listAll[T any](ctx context.Context, c *client, op, u string, query url.Values, maxPages int, each func(T) error) (complete bool, err error) { + next := u + if len(query) > 0 { + next += "?" + query.Encode() + } + for n := 1; n <= maxPages; n++ { + var p page[T] + if _, err := c.get(ctx, op, next, nil, &p); err != nil { + if n > 1 { + return false, &pageError{page: n, err: err} + } + return false, err + } + for _, it := range p.Values { + if err := each(it); err != nil { + return false, err + } + } + if p.Next == "" { + return true, nil + } + to, perr := url.Parse(p.Next) + if perr != nil || !c.belowAPI(to) { + return false, shapeError(op, "a next page link outside the API: %s", c.mask(withoutQuery(p.Next))) + } + next = to.String() + } + return false, nil +} + +// pageError is the failure of a page after the first of a listing. +type pageError struct { + page int + err error +} + +func (e *pageError) Error() string { return fmt.Sprintf("page %d: %v", e.page, e.err) } +func (e *pageError) Unwrap() error { return e.err } + +// laterPage reports whether err is the failure of a page after the first. +func laterPage(err error) bool { + var pe *pageError + return errors.As(err, &pe) +} + +// splitRepoPath splits "workspace/slug". Bitbucket has no nested +// namespaces: a path of another shape cannot name a repository (ok false). +func splitRepoPath(p string) (workspace, slug string, ok bool) { + workspace, slug, found := strings.Cut(p, "/") + if !found || workspace == "" || slug == "" || strings.Contains(slug, "/") || + strings.ContainsAny(p, "\x00?#") || workspace == "." || workspace == ".." || slug == "." || slug == ".." { + return "", "", false + } + return workspace, slug, true +} + +// repoPath returns the workspace and slug of r, or a ClassInvalid error for +// op. +func repoPath(op string, r platform.Repo) (workspace, slug string, err error) { + workspace, slug, ok := splitRepoPath(r.Path) + if !ok { + return "", "", invalid(op, "%q is not a workspace/repository path", r.Path) + } + return workspace, slug, nil +} diff --git a/internal/platform/bitbucket/errors.go b/internal/platform/bitbucket/errors.go new file mode 100644 index 0000000..0382d71 --- /dev/null +++ b/internal/platform/bitbucket/errors.go @@ -0,0 +1,338 @@ +package bitbucket + +import ( + "context" + "crypto/tls" + "crypto/x509" + "encoding/json" + "errors" + "fmt" + "math" + "net/http" + "strconv" + "strings" + "time" + "unicode" + + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +// maxRetryAfter bounds a wait read from response headers: a broken or +// hostile header must not park the provider for days. +const maxRetryAfter = time.Hour + +// now is the clock of Retry-After dates and reset times (tests replace it). +var now = time.Now + +// maskedError carries a masked message over the original error, which +// errors.Is and errors.As still see. +type maskedError struct { + msg string + err error +} + +func (e *maskedError) Error() string { return e.msg } +func (e *maskedError) Unwrap() error { return e.err } + +// masked returns err with its message masked. +func (c *client) masked(err error) error { + return &maskedError{msg: c.mask(err.Error()), err: err} +} + +// apiError classifies err, the failure of an API request of op, into a +// platform.Error: a non-2xx status by its code, a timeout as transient, a +// request httpx refused as invalid, a certificate the client does not +// trust, a response over the size bound or of an unexpected shape as +// unknown (none of them gets better by retrying), any other transport +// failure as transient. A canceled context stays unclassified. Messages +// never hold the credential nor headers. +func (c *client) apiError(op string, err error) error { + var se *httpx.StatusError + if errors.As(err, &se) { + return c.statusError(op, se) + } + class := platform.ClassTransient + switch { + case errors.Is(err, context.Canceled): + return fmt.Errorf("%s: %w", op, c.masked(err)) + case errors.Is(err, context.DeadlineExceeded): + case errors.Is(err, httpx.ErrRefused): + class = platform.ClassInvalid + case errors.Is(err, httpx.ErrTooLarge), certificateError(err), decodeError(err): + class = platform.ClassUnknown + } + return &platform.Error{Op: op, Class: class, Err: c.masked(err)} +} + +// certificateError reports whether err is a TLS certificate failure. +func certificateError(err error) bool { + var ( + verify *tls.CertificateVerificationError + unknown x509.UnknownAuthorityError + invalid x509.CertificateInvalidError + hostError x509.HostnameError + ) + return errors.As(err, &verify) || errors.As(err, &unknown) || errors.As(err, &invalid) || errors.As(err, &hostError) +} + +// decodeError reports whether err is a JSON body that does not decode. +func decodeError(err error) bool { + var ( + syntax *json.SyntaxError + typ *json.UnmarshalTypeError + ) + return errors.As(err, &syntax) || errors.As(err, &typ) || errors.Is(err, errShape) +} + +// errShape is wrapped by errors about responses that decode but lack what +// the driver depends on. +var errShape = errors.New("unexpected response") + +// shapeError is an error of op about a response that lacks what the driver +// depends on: the API changed shape. +func shapeError(op, format string, args ...any) error { + return &platform.Error{Op: op, Class: platform.ClassUnknown, + Err: fmt.Errorf("%w: "+format, append([]any{errShape}, args...)...)} +} + +// statusError classifies a non-2xx response of op. +// +// Bitbucket's error body is {"type": "error", "error": {"message": …, +// "detail": …, "data": {…}}} (the error schema of the OpenAPI description; +// a missing repository answers "You may not have access to this repository +// or it no longer exists in this workspace. …", a missing file "No such +// file or directory: ", both with 404, seen anonymously). A rate +// limit is 429, or a message about a rate limit ("Rate limit for this +// resource has been exceeded"), with x-ratelimit-limit, -remaining and +// -reset (seconds) headers (support.atlassian.com/bitbucket-cloud, "API +// request limits"). +func (c *client) statusError(op string, se *httpx.StatusError) error { + m := parseMessage(se.Snippet) + detail := m.text() + if detail == "" { + detail = se.Snippet + } + msg := c.mask(describe(se, detail)) + e := &platform.Error{Op: op, Class: statusClass(se.Status), Status: se.Status} + switch { + case se.Status == http.StatusNotFound || se.Status == http.StatusGone: + e.Err = ¬FoundError{message: m.Error.Message, err: fmt.Errorf("%s: %w", msg, platform.ErrNotFound)} + return e + case se.Status == http.StatusTooManyRequests, se.Status == http.StatusServiceUnavailable: + e.RetryAfter = retryAfter(se.Header, now()) + case limitMessage(detail) || se.Status == http.StatusForbidden && rateLimited(se.Header): + // A limit answered with another status, by its message or a proxy's + // headers. + e.Class, e.RetryAfter = platform.ClassRateLimited, retryAfter(se.Header, now()) + } + e.Err = errors.New(msg) + return e +} + +// notFoundError is the error of a 404 (or 410): Bitbucket's message ("" when +// the body is no Bitbucket error), over the ErrNotFound it wraps. +type notFoundError struct { + message string + err error +} + +func (e *notFoundError) Error() string { return e.err.Error() } +func (e *notFoundError) Unwrap() error { return e.err } + +// missingMessage returns Bitbucket's message of a 404 in err; ok is false +// when err is no 404 of the API. +func missingMessage(err error) (message string, ok bool) { + var nf *notFoundError + if !errors.As(err, &nf) { + return "", false + } + return nf.message, true +} + +// statusClass maps an HTTP status to the class the core reacts to. +func statusClass(status int) platform.Class { + switch { + case status == http.StatusUnauthorized: + return platform.ClassAuth + case status == http.StatusForbidden: + return platform.ClassPermission + case status == http.StatusNotFound, status == http.StatusGone: + // 410: a deleted workspace (the 410 of GET /repositories/{workspace}). + return platform.ClassNotFound + case status == http.StatusConflict: + return platform.ClassConflict + case status == http.StatusTooManyRequests: + return platform.ClassRateLimited + case status == http.StatusRequestTimeout, status == http.StatusTooEarly: + return platform.ClassTransient + case status == http.StatusNotImplemented: + return platform.ClassUnsupported + case status >= 500: + // 555 is the API's own timeout of a deep directory listing. + return platform.ClassTransient + case status >= 400: + return platform.ClassInvalid + } + // An unexpected redirect or informational status. + return platform.ClassUnknown +} + +// apiMessage is Bitbucket's error body. +type apiMessage struct { + Type string `json:"type"` + Error struct { + Message string `json:"message"` + Detail string `json:"detail"` + } `json:"error"` +} + +// text returns the message and its detail on one line. +func (m apiMessage) text() string { + parts := make([]string, 0, 2) + if m.Error.Message != "" { + parts = append(parts, m.Error.Message) + } + if m.Error.Detail != "" && m.Error.Detail != m.Error.Message { + parts = append(parts, m.Error.Detail) + } + return strings.Join(strings.FieldsFunc(strings.Join(parts, "; "), func(r rune) bool { + return unicode.IsSpace(r) || unicode.IsControl(r) + }), " ") +} + +// parseMessage decodes an error body; a body of another shape gives the +// zero apiMessage. +func parseMessage(snippet string) apiMessage { + var m apiMessage + if json.Unmarshal([]byte(snippet), &m) != nil { + return apiMessage{} + } + return m +} + +// describe returns "METHOD URL: status text: detail". +func describe(se *httpx.StatusError, detail string) string { + var b strings.Builder + fmt.Fprintf(&b, "%s %s: %d", se.Method, se.URL, se.Status) + if text := http.StatusText(se.Status); text != "" { + b.WriteString(" " + text) + } + if detail != "" { + b.WriteString(": " + detail) + } + return b.String() +} + +// limitMessage reports whether an error message says a rate limit was hit. +func limitMessage(s string) bool { + s = strings.ToLower(s) + return strings.Contains(s, "rate limit") && strings.Contains(s, "exceeded") +} + +// rateLimited reports whether a response's headers say a rate limit was +// hit: a Retry-After, or no requests left in X-RateLimit-Remaining. +func rateLimited(h http.Header) bool { + return h.Get("Retry-After") != "" || strings.TrimSpace(h.Get("X-RateLimit-Remaining")) == "0" +} + +// retryAfter reads how long to wait from a response's headers, in order: +// Retry-After (seconds or an HTTP date), then X-RateLimit-Reset, which +// Bitbucket gives in seconds until the window resets (a Unix time when it +// is large, as other platforms send it). It is 0 when none is usable, and +// at most maxRetryAfter. +func retryAfter(h http.Header, at time.Time) time.Duration { + d, ok := time.Duration(0), false + if v := strings.TrimSpace(h.Get("Retry-After")); v != "" { + if secs, err := strconv.ParseInt(v, 10, 64); err == nil { + d, ok = seconds(secs), true + } else if t, err := http.ParseTime(v); err == nil { + d, ok = t.Sub(at), true + } + } + if !ok { + if n, err := strconv.ParseInt(strings.TrimSpace(h.Get("X-RateLimit-Reset")), 10, 64); err == nil { + // A Unix time is far above any sensible number of seconds. + if n > 1_000_000_000 { + d, ok = time.Unix(n, 0).Sub(at), true + } else { + d, ok = seconds(n), true + } + } + } + switch { + case !ok || d < 0: + return 0 + case d > maxRetryAfter: + return maxRetryAfter + } + return d +} + +// seconds converts a count of seconds, saturating instead of overflowing. +func seconds(n int64) time.Duration { + if n < 0 { + return -1 + } + if n > math.MaxInt64/int64(time.Second) { + return maxRetryAfter + } + return time.Duration(n) * time.Second +} + +// invalid is a ClassInvalid error of op: the request was wrong. +func invalid(op, format string, args ...any) error { + return &platform.Error{Op: op, Class: platform.ClassInvalid, Err: fmt.Errorf(format, args...)} +} + +// notFound is a ClassNotFound error of op that wraps ErrNotFound. +func notFound(op, format string, args ...any) error { + return &platform.Error{Op: op, Class: platform.ClassNotFound, Err: fmt.Errorf(format+": %w", append(args, platform.ErrNotFound)...)} +} + +// unknown is a ClassUnknown error of op that says what happened (format +// and args) and, when err is not nil, what err says, with err's status but +// without the ErrNotFound err may wrap: a 404 that must not count as a +// missing file. +func unknown(op string, err error, format string, args ...any) error { + msg := fmt.Sprintf(format, args...) + if err != nil { + msg += ": " + detail(err) + } + return &platform.Error{Op: op, Class: platform.ClassUnknown, Status: statusOf(err), Err: errors.New(msg)} +} + +// detail returns what err says without its class: the message of the +// *platform.Error in it, without the ErrNotFound suffix. +func detail(err error) string { + var pe *platform.Error + if errors.As(err, &pe) && pe.Err != nil { + return strings.TrimSuffix(pe.Err.Error(), ": "+platform.ErrNotFound.Error()) + } + return err.Error() +} + +// statusOf returns the HTTP status of a classified error, 0 for none. +func statusOf(err error) int { + var pe *platform.Error + if errors.As(err, &pe) { + return pe.Status + } + return 0 +} + +// stops reports whether err ends a call at once, whatever it already +// found: the context ended, the platform limits the rate or refuses the +// credential. +func stops(err error) bool { + if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) { + return true + } + c := platform.ClassOf(err) + return c == platform.ClassRateLimited || c == platform.ClassAuth +} + +// fatal reports whether err must fail a listing instead of making it +// incomplete: it stops the call, or the platform failed for now (the caller +// retries the whole listing). +func fatal(err error) bool { return stops(err) || platform.ClassOf(err) == platform.ClassTransient } diff --git a/internal/platform/bitbucket/errors_test.go b/internal/platform/bitbucket/errors_test.go new file mode 100644 index 0000000..ae5af05 --- /dev/null +++ b/internal/platform/bitbucket/errors_test.go @@ -0,0 +1,153 @@ +package bitbucket + +import ( + "encoding/base64" + "net/http" + "net/url" + "strconv" + "strings" + "testing" + "time" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +func TestStatusClasses(t *testing.T) { + f := newFixture(t) + for _, tc := range []struct { + status int + header map[string]string + msg string + class platform.Class + wait time.Duration + }{ + {http.StatusBadRequest, nil, "Field \".summary.raw\" does not support filtering", platform.ClassInvalid, 0}, + {http.StatusUnauthorized, nil, "Token is invalid, expired, or not supported for this endpoint.", platform.ClassAuth, 0}, + {http.StatusForbidden, nil, "Your credentials lack one or more required privilege scopes.", platform.ClassPermission, 0}, + {http.StatusForbidden, nil, "Rate limit for this resource has been exceeded", platform.ClassRateLimited, 0}, + {http.StatusForbidden, map[string]string{"X-RateLimit-Remaining": "0", "X-RateLimit-Reset": "60"}, "Forbidden", platform.ClassRateLimited, time.Minute}, + {http.StatusNotFound, nil, noRepoMessage, platform.ClassNotFound, 0}, + {http.StatusConflict, nil, "conflict", platform.ClassConflict, 0}, + {http.StatusGone, nil, "This workspace has been deleted", platform.ClassNotFound, 0}, + {http.StatusTooManyRequests, map[string]string{"X-RateLimit-Limit": "1000", "X-RateLimit-Remaining": "0", "X-RateLimit-Reset": "394"}, + "Rate limit for this resource has been exceeded", platform.ClassRateLimited, 394 * time.Second}, + {http.StatusTooManyRequests, map[string]string{"Retry-After": "30", "X-RateLimit-Reset": "394"}, "Too many", platform.ClassRateLimited, 30 * time.Second}, + {http.StatusTooManyRequests, map[string]string{"X-RateLimit-Reset": "999999"}, "Too many", platform.ClassRateLimited, time.Hour}, + {http.StatusInternalServerError, nil, "Something went wrong", platform.ClassTransient, 0}, + {http.StatusServiceUnavailable, map[string]string{"Retry-After": "5"}, "Unavailable", platform.ClassTransient, 5 * time.Second}, + {555, nil, "timeout", platform.ClassTransient, 0}, + {http.StatusNotImplemented, nil, "no", platform.ClassUnsupported, 0}, + } { + path := "/repositories/acme/s" + strconv.Itoa(tc.status) + strconv.Itoa(len(tc.header)) + f.handle(http.MethodGet, path, func(w http.ResponseWriter, _ *http.Request) { + for k, v := range tc.header { + w.Header().Set(k, v) + } + writeJSON(w, tc.status, errorBody(tc.msg)) + }) + _, err := f.reader.Repo(t.Context(), strings.TrimPrefix(path, "/repositories/")) + what := strconv.Itoa(tc.status) + " " + tc.msg + wantClass(t, what, err, tc.class, nil) + if err == nil { + continue + } + if !strings.Contains(err.Error(), tc.msg) { + t.Errorf("%s: the error %q lacks the API's message", what, err) + } + if d := retryAfterOf(err); d != tc.wait { + t.Errorf("%s: RetryAfter %v, want %v", what, d, tc.wait) + } + if tc.status == http.StatusNotFound && !isNotFound(err) { + t.Errorf("%s: %v is no ErrNotFound", what, err) + } + } +} + +func TestTransportErrors(t *testing.T) { + t.Run("a body that does not decode", func(t *testing.T) { + f := newFixture(t) + f.handle(http.MethodGet, "/repositories/acme/api", func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte("maintenance")) + }) + _, err := f.reader.Repo(t.Context(), "acme/api") + wantClass(t, "HTML instead of JSON", err, platform.ClassUnknown, nil) + }) + t.Run("a server that is gone", func(t *testing.T) { + f := newFixture(t) + f.srv.Close() + _, err := f.reader.Repo(t.Context(), "acme/api") + wantClass(t, "a closed server", err, platform.ClassTransient, nil) + }) +} + +// TestTokenMasked: the token never shows in an error, raw or in git's Basic +// form, even when the API echoes it. +func TestTokenMasked(t *testing.T) { + f := newFixture(t) + basic := base64.StdEncoding.EncodeToString([]byte("x-bitbucket-api-token-auth:" + f.token)) + echo := "token " + f.token + " basic " + basic + " escaped " + url.QueryEscape(f.token) + f.json("/repositories/acme/api", http.StatusUnauthorized, errorBody(echo)) + f.json("/user", http.StatusForbidden, map[string]any{"type": "error", "error": map[string]any{"message": "no", "detail": echo}}) + f.handle(http.MethodGet, "/repositories/acme/api/src/"+headCommit+"/"+optIn, func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("format") == "meta" { + writeJSON(w, http.StatusOK, map[string]any{"path": optIn, "type": "commit_file", "attributes": []string{}, "size": 3}) + return + } + w.WriteHeader(http.StatusInternalServerError) + _, _ = w.Write([]byte("upstream said " + echo)) + }) + _, err1 := f.reader.Repo(t.Context(), "acme/api") + _, err2 := f.reader.Self(t.Context()) + _, err3 := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 100) + for i, err := range []error{err1, err2, err3} { + if err == nil { + t.Fatalf("error %d: none", i) + } + msg := err.Error() + if strings.Contains(msg, f.token) || strings.Contains(msg, basic) || strings.Contains(msg, url.QueryEscape(f.token)) { + t.Errorf("error %d shows the token: %s", i, msg) + } + if !strings.Contains(msg, "***") { + t.Errorf("error %d: %s; want the token masked", i, msg) + } + } + wantClass(t, "a raw read that fails", err3, platform.ClassTransient, nil) +} + +func TestRedirects(t *testing.T) { + f := newFixture(t) + f.handle(http.MethodGet, "/repositories/acme/old-name", func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, "/2.0/repositories/acme/api", http.StatusMovedPermanently) + }) + f.json("/repositories/acme/api", http.StatusOK, repo(repoUUID, "acme/api")) + got, err := f.reader.Repo(t.Context(), "acme/old-name") + if err != nil || got.Path != "acme/api" || got.ID != repoUUID { + t.Errorf("Repo(old name) = %+v, %v; want the repository under its canonical path", got, err) + } + if calls := f.requests(http.MethodGet, "/repositories/acme/api"); len(calls) != 1 || calls[0].Auth != "Bearer "+f.token { + t.Errorf("calls %+v; want the redirect followed with the credential", calls) + } + f.handle(http.MethodGet, "/repositories/acme/away", func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, "https://evil.example.com/2.0/repositories/acme/api", http.StatusFound) + }) + f.handle(http.MethodGet, "/repositories/acme/up", func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, "/elsewhere", http.StatusFound) + }) + for _, p := range []string{"acme/away", "acme/up"} { + _, err := f.reader.Repo(t.Context(), p) + wantClass(t, p, err, platform.ClassUnknown, nil) + } +} + +func TestBlobID(t *testing.T) { + // git hash-object of "hello\n" and of an empty file. + for content, want := range map[string]string{ + "hello\n": "ce013625030ba8dba906f756967f9e9ca394464a", + "": "e69de29bb2d1d6434b8b29ae775ad8c2e48c5391", + } { + if got := blobID([]byte(content)); got != want { + t.Errorf("blobID(%q) = %s, want %s", content, got, want) + } + } +} diff --git a/internal/platform/bitbucket/files.go b/internal/platform/bitbucket/files.go new file mode 100644 index 0000000..828c476 --- /dev/null +++ b/internal/platform/bitbucket/files.go @@ -0,0 +1,296 @@ +package bitbucket + +import ( + "bytes" + "cmp" + "context" + "crypto/sha1" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "net/url" + "slices" + "strings" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// File modes of regular files. +const ( + modeFile = "100644" + modeExecutable = "100755" +) + +// missingFileMessage starts the message of a 404 of the source API for a +// path that a commit lacks ("No such file or directory: ", seen +// anonymously); a missing repository or commit says something else. +const missingFileMessage = "no such file or directory" + +// apiEntry is GET …/src/{commit}/{path}?format=meta: a commit_file or a +// commit_directory. +type apiEntry struct { + Type string `json:"type"` + Path string `json:"path"` + Size *int64 `json:"size"` + Attributes attributes `json:"attributes"` +} + +// attributes are a file's modifiers: link, executable, subrepository, +// binary, lfs. The OpenAPI description types them as one string; the API +// sends a list ([] for a plain file), so both are read. +type attributes []string + +func (a *attributes) UnmarshalJSON(b []byte) error { + b = bytes.TrimSpace(b) + switch { + case bytes.Equal(b, []byte("null")): + *a = nil + return nil + case len(b) > 0 && b[0] == '"': + var s string + if err := json.Unmarshal(b, &s); err != nil { + return err + } + *a = attributes{s} + return nil + } + var list []string + if err := json.Unmarshal(b, &list); err != nil { + return err + } + *a = list + return nil +} + +// has reports whether a holds the attribute name. +func (a attributes) has(name string) bool { return slices.Contains(a, name) } + +// ReadFile returns a regular file of r at ref ("" is the default branch +// head): its kind, size and mode from the source API's metadata, then its +// raw bytes, at the commit ref resolves to, so that both answers describe +// the same tree. The blob id is computed here (the API gives none). +// Symlinks, submodules, directories and Git LFS files are ErrNotRegular, a +// file over max bytes ErrTooLarge. +// +// Only a path that an existing commit lacks is ErrNotFound: the core takes a +// missing opt-in file for an opt-out and closes the target's pull requests. +// A missing repository, ref or commit, a 404 the source API gives for +// anything else, and a file that the metadata shows and the raw read does +// not find are ClassUnknown. An empty repository has no file: ErrNotFound. +func (d *reader) ReadFile(ctx context.Context, r platform.Repo, ref, filePath string, max int64) (platform.File, error) { + const op = "read file" + ws, slug, err := repoPath(op, r) + if err != nil { + return platform.File{}, err + } + if err := checkTreePath(filePath); err != nil { + return platform.File{}, invalid(op, "%v", err) + } + if max < 0 { + return platform.File{}, invalid(op, "negative size limit %d", max) + } + commit, err := d.c.commitOf(ctx, op, r, ws, slug, ref) + if err != nil { + return platform.File{}, err + } + e, err := d.c.meta(ctx, op, ws, slug, commit, filePath) + switch { + case err != nil && platform.ClassOf(err) == platform.ClassNotFound: + return platform.File{}, d.c.missingPath(ctx, op, r, ws, slug, commit, filePath, err) + case err != nil: + return platform.File{}, err + } + what := "" + switch { + case e.Type == "commit_directory": + what = "a directory" + case e.Type != "commit_file": + what = "a " + cmp.Or(e.Type, "thing without a type") + case e.Attributes.has("link"): + what = "a symlink" + case e.Attributes.has("subrepository"): + what = "a submodule" + case e.Attributes.has("lfs"): + what = "a Git LFS file" + } + if what != "" { + return platform.File{}, fmt.Errorf("%s: %s: %s is %s: %w", op, r.Path, filePath, what, platform.ErrNotRegular) + } + switch { + case e.Path != "" && e.Path != filePath: + return platform.File{}, shapeError(op, "%s: the metadata describes %s", filePath, e.Path) + case e.Size == nil || *e.Size < 0: + return platform.File{}, shapeError(op, "%s: the metadata has no size", filePath) + case *e.Size > max: + return platform.File{}, fmt.Errorf("%s: %s: %s has %d bytes, more than %d: %w", op, r.Path, filePath, *e.Size, max, platform.ErrTooLarge) + } + content, err := d.c.raw(ctx, op, d.c.srcURL(ws, slug, commit, filePath)) + switch { + case errors.Is(err, errOffsite): + return platform.File{}, fmt.Errorf("%s: %s: %s: the API serves its content from another host, as it serves Git LFS files: %w", + op, r.Path, filePath, platform.ErrNotRegular) + case err != nil && platform.ClassOf(err) == platform.ClassNotFound: + // Never ErrNotFound: the metadata of the same commit has just shown it. + return platform.File{}, unknown(op, err, "%s: %s at %s: the metadata shows it, its content is not found", r.Path, filePath, commit) + case err != nil: + return platform.File{}, err + case int64(len(content)) != *e.Size: + return platform.File{}, shapeError(op, "%s: %d bytes of content, the metadata says %d", filePath, len(content), *e.Size) + } + mode := modeFile + if e.Attributes.has("executable") { + mode = modeExecutable + } + return platform.File{Path: filePath, Mode: mode, OID: blobID(content), Content: content}, nil +} + +// commitOf resolves ref of r to a full commit id: a full id is taken as it +// is (missingPath checks that it exists when a path is missing), a branch +// through the branch API, and "" through the default branch. A ref the API +// does not find, or a repository it does not find, is ClassUnknown: never a +// missing file. A repository without a main branch is empty: ErrNotFound. +// When the default branch r names is gone, the repository's current one is +// read once more (the default branch was renamed in between). +func (c *client) commitOf(ctx context.Context, op string, r platform.Repo, ws, slug, ref string) (string, error) { + if isHexOID(ref) { + return strings.ToLower(ref), nil + } + if ref != "" { + hash, err := c.branchHead(ctx, op, ws, slug, ref) + if err != nil && platform.ClassOf(err) == platform.ClassNotFound { + return "", unknown(op, err, "%s: ref %s is not found", r.Path, ref) + } + return hash, err + } + branch := r.DefaultBranch + for range 2 { + if branch != "" { + hash, err := c.branchHead(ctx, op, ws, slug, branch) + if err == nil || platform.ClassOf(err) != platform.ClassNotFound { + return hash, err + } + } + // No default branch, or not where r says: the repository tells. + repo, err := c.getRepo(ctx, op, ws, slug) + switch { + case err != nil && platform.ClassOf(err) == platform.ClassNotFound: + return "", unknown(op, err, "%s: the repository is not found", r.Path) + case err != nil: + return "", err + case repo.MainBranch == nil || repo.MainBranch.Name == "": + return "", notFound(op, "%s: the repository is empty", r.Path) + case repo.MainBranch.Name == branch: + return "", unknown(op, nil, "%s: the branch API does not find the default branch %s", r.Path, branch) + } + branch = repo.MainBranch.Name + } + return "", unknown(op, nil, "%s: the default branch moved while it was read", r.Path) +} + +// branchHead returns the head commit of branch in ws/slug: GET +// …/refs/branches/{name}, the name's slashes kept. +func (c *client) branchHead(ctx context.Context, op, ws, slug, branch string) (string, error) { + var b apiBranch + if _, err := c.get(ctx, op, c.endpoint(append([]string{"repositories", ws, slug, "refs", "branches"}, pathSegments(branch)...)...), nil, &b); err != nil { + return "", err + } + if b.Target == nil || !isHexOID(b.Target.Hash) { + return "", shapeError(op, "branch %s: no head commit", branch) + } + return strings.ToLower(b.Target.Hash), nil +} + +// srcURL is the source API's URL of filePath at commit. +func (c *client) srcURL(ws, slug, commit, filePath string) string { + return c.endpoint(append([]string{"repositories", ws, slug, "src", commit}, pathSegments(filePath)...)...) +} + +// meta reads the metadata of filePath at commit. +func (c *client) meta(ctx context.Context, op, ws, slug, commit, filePath string) (*apiEntry, error) { + var e apiEntry + if _, err := c.get(ctx, op, c.srcURL(ws, slug, commit, filePath), url.Values{"format": {"meta"}}, &e); err != nil { + return nil, err + } + return &e, nil +} + +// missingPath judges err, a 404 of the metadata of filePath at commit. A +// path is missing only at a commit that exists: the 404 says so ("No such +// file or directory"), or the commit API finds the commit; else the +// repository or the commit is gone, which is ClassUnknown. A missing path +// is ErrNotFound, but one that leads through a symlink, which git never +// follows, is ErrNotRegular, as on the other platforms: the directories on +// the way are read until one is missing or no directory. +func (c *client) missingPath(ctx context.Context, op string, r platform.Repo, ws, slug, commit, filePath string, err error) error { + msg, _ := missingMessage(err) + if !strings.HasPrefix(strings.ToLower(msg), missingFileMessage) { + _, cerr := c.get(ctx, op, c.endpoint("repositories", ws, slug, "commit", commit), nil, nil) + switch { + case cerr != nil && platform.ClassOf(cerr) == platform.ClassNotFound: + return unknown(op, err, "%s: %s at %s: neither the file nor the commit is found", r.Path, filePath, commit) + case cerr != nil: + return cerr + } + } + segs := strings.Split(filePath, "/") + for i := 1; i < len(segs); i++ { + dir := strings.Join(segs[:i], "/") + e, derr := c.meta(ctx, op, ws, slug, commit, dir) + switch { + case derr != nil && platform.ClassOf(derr) == platform.ClassNotFound: + return notFound(op, "%s: %s at %s: %s is missing", r.Path, filePath, commit, dir) + case derr != nil: + return derr + case e.Type == "commit_directory": + continue + case e.Type == "commit_file" && e.Attributes.has("link"): + return fmt.Errorf("%s: %s: %s leads through a symlink at %s: %w", op, r.Path, filePath, dir, platform.ErrNotRegular) + } + return notFound(op, "%s: %s at %s: %s is no directory", r.Path, filePath, commit, dir) + } + return notFound(op, "%s: %s at %s", r.Path, filePath, commit) +} + +// checkTreePath accepts a repository-relative path: no leading slash, no +// empty, "." or ".." segment, no NUL. +func checkTreePath(p string) error { + if p == "" { + return errors.New("empty path") + } + if strings.IndexByte(p, 0) >= 0 { + return fmt.Errorf("path %q contains NUL", p) + } + for seg := range strings.SplitSeq(p, "/") { + if seg == "" || seg == "." || seg == ".." { + return fmt.Errorf("path %q has an empty, \".\" or \"..\" segment", p) + } + } + return nil +} + +// blobID is git's SHA-1 id of a blob with content: Bitbucket hosts SHA-1 +// repositories only. +func blobID(content []byte) string { + h := sha1.New() + fmt.Fprintf(h, "blob %d\x00", len(content)) + h.Write(content) + return hex.EncodeToString(h.Sum(nil)) +} + +// isHexOID reports whether s is a full SHA-1 object id. +func isHexOID(s string) bool { return len(s) == 40 && isHex(s) } + +// isHex reports whether s is a non-empty string of hexadecimal digits. +func isHex(s string) bool { + if s == "" { + return false + } + for i := 0; i < len(s); i++ { + c := s[i] + if (c < '0' || c > '9') && (c < 'a' || c > 'f') && (c < 'A' || c > 'F') { + return false + } + } + return true +} diff --git a/internal/platform/bitbucket/files_test.go b/internal/platform/bitbucket/files_test.go new file mode 100644 index 0000000..c3f96a8 --- /dev/null +++ b/internal/platform/bitbucket/files_test.go @@ -0,0 +1,322 @@ +package bitbucket + +import ( + "crypto/sha1" + "encoding/hex" + "fmt" + "net/http" + "strings" + "testing" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// headCommit is the head of the default branch of acme/api in these tests. +const headCommit = "be4cdd0b1f9566847c274d0d05b9549881e0ae4a" + +// optIn is the opt-in file the tests read. +const optIn = ".engineering-assets.yml" + +// apiRepoFixture is the repository the file tests read. +var apiRepoFixture = platform.Repo{ID: repoUUID, Path: "acme/api", DefaultBranch: "main", ObjectFormat: "sha1"} + +// branch declares GET …/refs/branches/{name} of acme/api with its head. +func (s *apiServer) branch(name, hash string) { + s.json("/repositories/acme/api/refs/branches/"+name, http.StatusOK, map[string]any{ + "name": name, "type": "branch", "default_merge_strategy": "merge_commit", + "merge_strategies": []string{"merge_commit", "squash", "fast_forward"}, + "target": map[string]any{"hash": hash, "type": "commit", "date": "2026-09-30T10:00:00+00:00", + "message": "chore: sync", "parents": []any{}}, + "links": map[string]any{"html": map[string]any{"href": "https://bitbucket.org/acme/api/branch/" + name}}, + }) +} + +// srcLinks are the links of an entry of the source API. +func srcLinks(commit, path string) map[string]any { + self := "https://api.bitbucket.org/2.0/repositories/acme/api/src/" + commit + "/" + path + return map[string]any{ + "self": map[string]any{"href": self}, + "meta": map[string]any{"href": self + "?format=meta"}, + } +} + +// file declares the metadata and the raw content of path at commit of +// acme/api: attributes as the API lists them, content as the raw read +// returns it (the metadata's size is its length unless size is set). +func (s *apiServer) file(commit, path string, attrs any, content string, size ...int) { + n := len(content) + if len(size) > 0 { + n = size[0] + } + s.handle(http.MethodGet, "/repositories/acme/api/src/"+commit+"/"+path, func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("format") == "meta" { + writeJSON(w, http.StatusOK, map[string]any{ + "path": path, "type": "commit_file", "attributes": attrs, "escaped_path": path, "size": n, + "mimetype": "text/plain", "has_malware_warning": false, "links": srcLinks(commit, path), + "commit": map[string]any{"hash": commit, "type": "commit"}, + }) + return + } + w.Header().Set("Content-Type", "text/plain") + w.Header().Set("Content-Disposition", "attachment") + _, _ = w.Write([]byte(content)) + }) +} + +// dir declares the metadata of a directory at commit of acme/api. +func (s *apiServer) dir(commit, path string) { + s.json("/repositories/acme/api/src/"+commit+"/"+path, http.StatusOK, map[string]any{ + "path": path, "type": "commit_directory", "links": srcLinks(commit, path+"/"), + "commit": map[string]any{"hash": commit, "type": "commit"}, + }) +} + +// missing declares a path that commit of acme/api lacks. +func (s *apiServer) missing(commit, path string) { + s.json("/repositories/acme/api/src/"+commit+"/"+path, http.StatusNotFound, errorBody(noFileMessage+path)) +} + +// gitBlob is git's id of a blob with content. +func gitBlob(content string) string { + h := sha1.New() + fmt.Fprintf(h, "blob %d\x00%s", len(content), content) + return hex.EncodeToString(h.Sum(nil)) +} + +func TestReadFile(t *testing.T) { + f := newFixture(t) + f.branch("main", headCommit) + content := "version: 1\npacks: [agents]\n" + f.file(headCommit, optIn, []string{}, content) + got, err := f.reader.ReadFile(t.Context(), apiRepoFixture, "", optIn, 1<<20) + if err != nil { + t.Fatal(err) + } + want := platform.File{Path: optIn, Mode: "100644", OID: gitBlob(content), Content: []byte(content)} + if got.Path != want.Path || got.Mode != want.Mode || got.OID != want.OID || string(got.Content) != content { + t.Errorf("ReadFile = %+v\nwant %+v", got, want) + } + calls := f.requests(http.MethodGet, "/repositories/acme/api/src/"+headCommit+"/"+optIn) + if len(calls) != 2 || calls[0].Query.Get("format") != "meta" || calls[1].Query.Get("format") != "" { + t.Errorf("calls %+v; want the metadata, then the raw content, at the branch's head commit", calls) + } +} + +func TestReadFileModes(t *testing.T) { + f := newFixture(t) + c := headCommit + f.file(c, "bin/run.sh", []string{"executable"}, "#!/bin/sh\n") + f.file(c, "string-attr", "executable", "x") + f.file(c, "binary", []string{"binary"}, "\x00\x01") + for path, mode := range map[string]string{"bin/run.sh": "100755", "string-attr": "100755", "binary": "100644"} { + got, err := f.reader.ReadFile(t.Context(), apiRepoFixture, c, path, 100) + if err != nil || got.Mode != mode { + t.Errorf("%s: %+v, %v; want mode %s", path, got, err, mode) + } + } + if calls := f.requests(http.MethodGet, "/repositories/acme/api/refs/branches/main"); len(calls) != 0 { + t.Error("a full commit id was resolved through the branch API") + } +} + +func TestReadFileNotRegular(t *testing.T) { + f := newFixture(t) + c := headCommit + f.file(c, "link", []string{"link"}, "target") + f.file(c, "sub", []string{"subrepository"}, "0123456789abcdef0123456789abcdef01234567") + f.file(c, "big.bin", []string{"lfs", "binary"}, "") + f.dir(c, "docs") + f.json("/repositories/acme/api/src/"+c+"/odd", http.StatusOK, map[string]any{"path": "odd", "type": "commit_symlink"}) + for _, path := range []string{"link", "sub", "big.bin", "docs", "odd"} { + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, c, path, 100) + wantClass(t, path, err, platform.ClassUnknown, platform.ErrNotRegular) + } + if calls := f.requests("", ""); len(calls) != 5 { + t.Errorf("%d requests; want the metadata only, never the content", len(calls)) + } +} + +func TestReadFileLFSRedirect(t *testing.T) { + f := newFixture(t) + f.handle(http.MethodGet, "/repositories/acme/api/src/"+headCommit+"/model.bin", func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("format") == "meta" { + writeJSON(w, http.StatusOK, map[string]any{"path": "model.bin", "type": "commit_file", "attributes": []string{}, "size": 3}) + return + } + http.Redirect(w, r, "https://api.media.atlassian.com/file/0123/binary", http.StatusFound) + }) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, "model.bin", 100) + wantClass(t, "a redirect elsewhere", err, platform.ClassUnknown, platform.ErrNotRegular) +} + +func TestReadFileTooLarge(t *testing.T) { + f := newFixture(t) + f.file(headCommit, optIn, []string{}, strings.Repeat("x", 11)) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 10) + wantClass(t, "over the limit", err, platform.ClassUnknown, platform.ErrTooLarge) + if calls := f.requests("", ""); len(calls) != 1 { + t.Errorf("%d requests; want the metadata only", len(calls)) + } + if _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 11); err != nil { + t.Errorf("at the limit: %v", err) + } +} + +func TestReadFileMissing(t *testing.T) { + t.Run("a missing file", func(t *testing.T) { + f := newFixture(t) + f.branch("main", headCommit) + f.missing(headCommit, optIn) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, "", optIn, 100) + wantClass(t, "a missing file", err, platform.ClassNotFound, platform.ErrNotFound) + if calls := f.requests("", ""); len(calls) != 2 { + t.Errorf("%d requests; want the branch and the metadata", len(calls)) + } + }) + t.Run("a missing nested file", func(t *testing.T) { + f := newFixture(t) + f.missing(headCommit, "a/b/c.yml") + f.dir(headCommit, "a") + f.missing(headCommit, "a/b") + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, "a/b/c.yml", 100) + wantClass(t, "a missing directory", err, platform.ClassNotFound, platform.ErrNotFound) + }) + t.Run("through a symlink", func(t *testing.T) { + f := newFixture(t) + f.missing(headCommit, "conf/touchmark.yml") + f.file(headCommit, "conf", []string{"link"}, "../elsewhere") + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, "conf/touchmark.yml", 100) + wantClass(t, "through a symlink", err, platform.ClassUnknown, platform.ErrNotRegular) + }) + t.Run("through a file", func(t *testing.T) { + f := newFixture(t) + f.missing(headCommit, "README.md/x") + f.file(headCommit, "README.md", []string{}, "# api\n") + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, "README.md/x", 100) + wantClass(t, "through a file", err, platform.ClassNotFound, platform.ErrNotFound) + }) + t.Run("another 404 at a commit that exists", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/src/"+headCommit+"/"+optIn, http.StatusNotFound, errorBody("Not found")) + f.json("/repositories/acme/api/commit/"+headCommit, http.StatusOK, map[string]any{"hash": headCommit, "type": "commit"}) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 100) + wantClass(t, "a commit that exists", err, platform.ClassNotFound, platform.ErrNotFound) + }) + t.Run("another 404 at a commit that is gone", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/src/"+headCommit+"/"+optIn, http.StatusNotFound, errorBody(noRepoMessage)) + f.json("/repositories/acme/api/commit/"+headCommit, http.StatusNotFound, errorBody(noRepoMessage)) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 100) + wantClass(t, "a commit that is gone", err, platform.ClassUnknown, nil) + if platform.ClassOf(err) == platform.ClassNotFound || isNotFound(err) { + t.Errorf("%v: a missing commit must never read as a missing file", err) + } + }) + t.Run("a missing repository", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/refs/branches/main", http.StatusNotFound, errorBody(noRepoMessage)) + f.json("/repositories/acme/api", http.StatusNotFound, errorBody(noRepoMessage)) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, "", optIn, 100) + wantClass(t, "a missing repository", err, platform.ClassUnknown, nil) + if isNotFound(err) { + t.Errorf("%v: a missing repository must never read as a missing file", err) + } + }) + t.Run("a missing ref", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/refs/branches/feature/x", http.StatusNotFound, errorBody("Branch \"feature/x\" not found")) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, "feature/x", optIn, 100) + wantClass(t, "a missing ref", err, platform.ClassUnknown, nil) + if isNotFound(err) { + t.Errorf("%v: a missing ref must never read as a missing file", err) + } + }) + t.Run("the default branch is gone", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/refs/branches/main", http.StatusNotFound, errorBody("Branch \"main\" not found")) + f.json("/repositories/acme/api", http.StatusOK, repo(repoUUID, "acme/api")) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, "", optIn, 100) + wantClass(t, "a default branch without a head", err, platform.ClassUnknown, nil) + if isNotFound(err) { + t.Errorf("%v: must never read as a missing file", err) + } + }) + t.Run("the default branch was renamed", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/refs/branches/master", http.StatusNotFound, errorBody("Branch \"master\" not found")) + f.json("/repositories/acme/api", http.StatusOK, repo(repoUUID, "acme/api")) + f.branch("main", headCommit) + f.file(headCommit, optIn, []string{}, "version: 1\n") + r := apiRepoFixture + r.DefaultBranch = "master" + if _, err := f.reader.ReadFile(t.Context(), r, "", optIn, 100); err != nil { + t.Errorf("ReadFile: %v; want the file at the current main branch", err) + } + }) + t.Run("an empty repository", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api", http.StatusOK, repo(repoUUID, "acme/api", with("mainbranch", nil))) + r := apiRepoFixture + r.DefaultBranch, r.Empty = "", true + _, err := f.reader.ReadFile(t.Context(), r, "", optIn, 100) + wantClass(t, "an empty repository", err, platform.ClassNotFound, platform.ErrNotFound) + }) +} + +// isNotFound reports whether err says the file is missing, as the core +// asks. +func isNotFound(err error) bool { + return platform.ClassOf(err) == platform.ClassNotFound +} + +func TestReadFileInconsistent(t *testing.T) { + t.Run("the content is gone", func(t *testing.T) { + f := newFixture(t) + f.handle(http.MethodGet, "/repositories/acme/api/src/"+headCommit+"/"+optIn, func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("format") == "meta" { + writeJSON(w, http.StatusOK, map[string]any{"path": optIn, "type": "commit_file", "attributes": []string{}, "size": 3}) + return + } + writeJSON(w, http.StatusNotFound, errorBody(noRepoMessage)) + }) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 100) + wantClass(t, "content gone after the metadata", err, platform.ClassUnknown, nil) + if isNotFound(err) { + t.Errorf("%v: must never read as a missing file", err) + } + }) + t.Run("the size differs", func(t *testing.T) { + f := newFixture(t) + f.file(headCommit, optIn, []string{}, "abc", 4) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 100) + wantClass(t, "a size that differs", err, platform.ClassUnknown, nil) + }) + t.Run("another path", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/src/"+headCommit+"/"+optIn, http.StatusOK, + map[string]any{"path": "elsewhere.yml", "type": "commit_file", "attributes": []string{}, "size": 3}) + _, err := f.reader.ReadFile(t.Context(), apiRepoFixture, headCommit, optIn, 100) + wantClass(t, "metadata of another path", err, platform.ClassUnknown, nil) + }) +} + +func TestReadFileInvalid(t *testing.T) { + f := newFixture(t) + for _, tc := range []struct { + r platform.Repo + path string + max int64 + }{ + {platform.Repo{Path: "acme"}, optIn, 1}, + {apiRepoFixture, "", 1}, + {apiRepoFixture, "a//b", 1}, + {apiRepoFixture, "../x", 1}, + {apiRepoFixture, optIn, -1}, + } { + _, err := f.reader.ReadFile(t.Context(), tc.r, "", tc.path, tc.max) + wantClass(t, fmt.Sprintf("%s %q %d", tc.r.Path, tc.path, tc.max), err, platform.ClassInvalid, nil) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("%d requests for invalid reads", len(calls)) + } +} diff --git a/internal/platform/bitbucket/helpers_test.go b/internal/platform/bitbucket/helpers_test.go new file mode 100644 index 0000000..10a7661 --- /dev/null +++ b/internal/platform/bitbucket/helpers_test.go @@ -0,0 +1,347 @@ +package bitbucket + +import ( + "crypto/rand" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "io" + "net/http" + "net/http/httptest" + "net/url" + "strconv" + "strings" + "sync" + "testing" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/config" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +// The fixtures below follow the OpenAPI description of the REST API 2.0 +// (https://api.bitbucket.org/swagger.json) and answers of public +// repositories read anonymously (atlassian/atlaskit-mk-2, 2026-10-08): +// every field the driver reads, and a few it ignores, as the API sends +// them. + +// apiServer is an httptest server that answers the routes a test declares +// and records every request. An undeclared route fails the test. Routes +// are paths under /2.0 as sent, escaped ("/users/%7B…%7D"). +type apiServer struct { + t *testing.T + srv *httptest.Server + mu sync.Mutex + routes map[string]http.HandlerFunc + calls []apiCall +} + +// apiCall is one request the server received. +type apiCall struct { + Method, Path string // Path escaped, under /2.0 + Query url.Values + Auth string // Authorization +} + +func newAPIServer(t *testing.T) *apiServer { + t.Helper() + s := &apiServer{t: t, routes: map[string]http.HandlerFunc{}} + s.srv = httptest.NewServer(http.HandlerFunc(s.serve)) + t.Cleanup(s.srv.Close) + // The read driver never writes. + t.Cleanup(func() { + for _, c := range s.writes() { + t.Errorf("the reader wrote: %s %s", c.Method, c.Path) + } + }) + return s +} + +// base is the server's web URL; the API is under base + "/2.0". +func (s *apiServer) base() string { return s.srv.URL } + +func (s *apiServer) serve(w http.ResponseWriter, r *http.Request) { + _, _ = io.Copy(io.Discard, r.Body) + p := r.URL.EscapedPath() + s.mu.Lock() + s.calls = append(s.calls, apiCall{Method: r.Method, Path: p, Query: r.URL.Query(), Auth: r.Header.Get("Authorization")}) + h := s.routes[r.Method+" "+p] + s.mu.Unlock() + if h == nil { + s.t.Errorf("unexpected request %s %s?%s", r.Method, p, r.URL.RawQuery) + writeJSON(w, http.StatusNotFound, errorBody("Resource not found")) + return + } + h(w, r) +} + +// apiPath returns path under /2.0. +func apiPath(path string) string { return "/2.0" + path } + +// handle declares a route: method and an escaped path under /2.0. +func (s *apiServer) handle(method, path string, h http.HandlerFunc) { + s.mu.Lock() + defer s.mu.Unlock() + s.routes[method+" "+apiPath(path)] = h +} + +// json declares a GET route that answers status with v as JSON. +func (s *apiServer) json(path string, status int, v any) { + s.handle(http.MethodGet, path, func(w http.ResponseWriter, _ *http.Request) { writeJSON(w, status, v) }) +} + +// pages declares a GET route that answers items per a page, with the +// page's next link (absolute, the query kept, page=n+1) on every page but +// the last, as the API paginates. +func (s *apiServer) pages(path string, per int, items []any) { + s.handle(http.MethodGet, path, func(w http.ResponseWriter, r *http.Request) { + servePage(w, r, per, items) + }) +} + +// servePage writes the page of items r asks for. +func servePage(w http.ResponseWriter, r *http.Request, per int, items []any) { + q := r.URL.Query() + n, _ := strconv.Atoi(q.Get("page")) + if n <= 0 { + n = 1 + } + from := min((n-1)*per, len(items)) + to := min(from+per, len(items)) + body := map[string]any{"values": items[from:to], "pagelen": per, "page": n, "size": len(items)} + if to < len(items) { + q.Set("page", strconv.Itoa(n+1)) + body["next"] = fmt.Sprintf("http://%s%s?%s", r.Host, r.URL.EscapedPath(), q.Encode()) + } + writeJSON(w, http.StatusOK, body) +} + +// requests returns the recorded calls of method to path under /2.0 ("" +// matches all). +func (s *apiServer) requests(method, path string) []apiCall { + s.mu.Lock() + defer s.mu.Unlock() + var out []apiCall + for _, c := range s.calls { + if (method == "" || c.Method == method) && (path == "" || c.Path == apiPath(path)) { + out = append(out, c) + } + } + return out +} + +// writes returns the recorded requests that are not GETs. +func (s *apiServer) writes() []apiCall { + var out []apiCall + for _, c := range s.requests("", "") { + if c.Method != http.MethodGet { + out = append(out, c) + } + } + return out +} + +// reset forgets the recorded calls. +func (s *apiServer) reset() { + s.mu.Lock() + s.calls = nil + s.mu.Unlock() +} + +func writeJSON(w http.ResponseWriter, status int, v any) { + w.Header().Set("Content-Type", "application/json; charset=utf-8") + w.WriteHeader(status) + _ = json.NewEncoder(w).Encode(v) +} + +// errorBody is Bitbucket's error body. +func errorBody(msg string) map[string]any { + return map[string]any{"type": "error", "error": map[string]any{"message": msg}} +} + +// Messages of 404s, as the API sends them anonymously. +const ( + noRepoMessage = "You may not have access to this repository or it no longer exists in this workspace. " + + "If you think this repository exists and you have access, make sure you are authenticated." + noFileMessage = "No such file or directory: " +) + +// testToken returns a token made at run time, as long as a real one. +func testToken(t *testing.T) string { + t.Helper() + b := make([]byte, 16) + if _, err := rand.Read(b); err != nil { + t.Fatal(err) + } + return "ATATT3x-" + hex.EncodeToString(b) +} + +// provider returns the resolved Bitbucket provider at the server. +func (s *apiServer) provider() config.ResolvedProvider { + u, _ := url.Parse(s.base()) + return config.ResolvedProvider{ + Provider: config.Provider{ID: "bb", Type: "bitbucket", URL: s.base(), APIURL: s.base() + "/2.0"}, + Host: strings.ToLower(u.Host), + APIURL: s.base() + "/2.0", + EnvPrefix: "TOUCHMARK_BB_", + } +} + +// fixture is a reader with a token over one test server. +type fixture struct { + *apiServer + token string + reader *reader +} + +func newFixture(t *testing.T) *fixture { + t.Helper() + s := newAPIServer(t) + tok := testToken(t) + r, err := NewReader(s.provider(), auth.Credential{Kind: auth.Token, Token: tok}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + return &fixture{apiServer: s, token: tok, reader: r.(*reader)} +} + +// Fixed ids. +const ( + repoUUID = "{6380b4e9-6ac5-4dd4-a8e0-65f09cabe4c8}" + forkUUID = "{0d3c2b1a-9f8e-4d7c-b6a5-948372615041}" + botUUID = "{583f7ec5-ed93-49a9-b449-cfc4556cd7f8}" + personUUID = "{bd1ac3ae-5a4e-416a-beab-37f06dd4cc93}" + otherUUID = "{b2457ec9-dd07-49b5-817e-9c9168905a3d}" + syncBranch = "touchmark/acme-eng" +) + +// uuidPath is a uuid as a path segment: braces escaped. +func uuidPath(uuid string) string { return url.PathEscape(uuid) } + +// account is a user as pull requests and GET /users/{uuid} send it. +func account(uuid, nickname string) map[string]any { + return map[string]any{ + "display_name": nickname, + "links": map[string]any{ + "self": map[string]any{"href": "https://api.bitbucket.org/2.0/users/" + uuidPath(uuid)}, + "avatar": map[string]any{"href": "https://avatar-management--avatars.us-west-2.prod.public.atl-paas.net/initials/H-5.png"}, + "html": map[string]any{"href": "https://bitbucket.org/" + uuidPath(uuid) + "/"}, + }, + "type": "user", "uuid": uuid, "account_id": "5a8383aa89ef572e36a586ac", "nickname": nickname, + } +} + +// appUser is an app user (an access token's bot). +func appUser(uuid, name string) map[string]any { + return map[string]any{"display_name": name, "type": "app_user", "uuid": uuid, + "account_id": "712020:1c2d3e4f-5a6b-4c7d-8e9f-0a1b2c3d4e5f", "kind": "access_token", "links": map[string]any{}} +} + +// repoOpt changes a repository fixture. +type repoOpt func(map[string]any) + +func with(k string, v any) repoOpt { return func(m map[string]any) { m[k] = v } } + +// repo is a repository as GET /repositories/{workspace}/{slug} sends it. +func repo(uuid, full string, opts ...repoOpt) map[string]any { + ws, slug, _ := strings.Cut(full, "/") + m := map[string]any{ + "type": "repository", "full_name": full, "name": slug, "slug": slug, "description": "", "scm": "git", + "website": "", "is_private": true, "fork_policy": "no_public_forks", "uuid": uuid, + "owner": map[string]any{"display_name": ws, "type": "team", "uuid": "{02b941e3-cfaa-40f9-9a58-cec53e20bdc3}", "username": ws}, + "workspace": map[string]any{"type": "workspace", "uuid": "{02b941e3-cfaa-40f9-9a58-cec53e20bdc3}", "name": ws, "slug": ws}, + "project": map[string]any{"type": "project", "key": "PROJ", "uuid": "{8b56daff-dbc7-4cae-a7a3-1228c526906b}", "name": "Project"}, + "created_on": "2026-09-01T10:00:00.000000+00:00", "updated_on": "2026-09-02T10:00:00.000000+00:00", + "size": 191411, "language": "python", "mainbranch": map[string]any{"name": "main", "type": "branch"}, + "parent": nil, "enforced_signed_commits": nil, "has_issues": false, "has_wiki": false, + "override_settings": map[string]any{"default_merge_strategy": false, "branching_model": false}, + "links": map[string]any{ + "self": map[string]any{"href": "https://api.bitbucket.org/2.0/repositories/" + full}, + "html": map[string]any{"href": "https://bitbucket.org/" + full}, + }, + } + for _, o := range opts { + o(m) + } + return m +} + +// repoRef is the short repository of a pull request's endpoint. +func repoRef(uuid, full string) map[string]any { + return map[string]any{"type": "repository", "full_name": full, "name": full[strings.IndexByte(full, '/')+1:], "uuid": uuid, + "links": map[string]any{"html": map[string]any{"href": "https://bitbucket.org/" + full}}} +} + +// prSpec describes a pull request fixture. +type prSpec struct { + id int64 + state string // OPEN (default), MERGED, DECLINED, SUPERSEDED + author map[string]any + closedBy map[string]any + source string + sourceRepo map[string]any // default the destination repository + target string // default "main" + hash string // the source commit, 12 digits by default + title string + body string + draft bool + noSummary bool // a listing without summary and description +} + +// pr is a pull request as the pull request APIs send it. +func pr(p prSpec) map[string]any { + state := cmpStr(p.state, "OPEN") + dest := repoRef(repoUUID, "acme/api") + src := p.sourceRepo + if src == nil { + src = dest + } + hash := cmpStr(p.hash, fmt.Sprintf("%012x", 0xa6dcc7402000+p.id)) + m := map[string]any{ + "comment_count": 0, "task_count": 0, "type": "pullrequest", "id": p.id, "title": cmpStr(p.title, "sync"), + "state": state, "draft": p.draft, "merge_commit": nil, "close_source_branch": false, "reason": "", + "author": p.author, "closed_by": nil, + "created_on": "2026-09-05T08:00:00.000000+00:00", "updated_on": "2026-09-10T12:00:00.123456+00:00", + "destination": map[string]any{ + "branch": map[string]any{"name": cmpStr(p.target, "main")}, + "commit": map[string]any{"hash": "d2391895ff0b", "type": "commit"}, + "repository": dest, + }, + "source": map[string]any{ + "branch": map[string]any{"name": p.source, "links": map[string]any{}, "sync_strategies": []string{"merge_commit", "rebase"}}, + "commit": map[string]any{"hash": hash, "type": "commit"}, + "repository": src, + }, + "links": map[string]any{"html": map[string]any{"href": fmt.Sprintf("https://bitbucket.org/acme/api/pull-requests/%d", p.id)}}, + } + if state != "OPEN" && p.closedBy != nil { + m["closed_by"] = p.closedBy + } + if !p.noSummary { + m["description"] = p.body + m["summary"] = map[string]any{"type": "rendered", "raw": p.body, "markup": "markdown", "html": "

" + p.body + "

"} + } + return m +} + +func cmpStr(a, b string) string { + if a != "" { + return a + } + return b +} + +// wantClass checks the class of err (and, with sentinel, that it wraps it). +func wantClass(t *testing.T, what string, err error, class platform.Class, sentinel error) { + t.Helper() + switch { + case err == nil: + t.Errorf("%s: no error, want class %v", what, class) + case platform.ClassOf(err) != class: + t.Errorf("%s: class %v of %v, want %v", what, platform.ClassOf(err), err, class) + case sentinel != nil && !errors.Is(err, sentinel): + t.Errorf("%s: %v does not wrap %v", what, err, sentinel) + } +} diff --git a/internal/platform/bitbucket/probe.go b/internal/platform/bitbucket/probe.go new file mode 100644 index 0000000..54fbae2 --- /dev/null +++ b/internal/platform/bitbucket/probe.go @@ -0,0 +1,138 @@ +package bitbucket + +import ( + "context" + "errors" + "regexp" + "time" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// Capabilities of Bitbucket Cloud. +const ( + // maxBody is the body budget. The API documents no limit on a pull + // request's description (Renovate assumes 250 000 characters); 60 000 + // bytes keeps parity with the other platforms until a live check + // finds the real bound. + maxBody = 60000 +) + +// limits are the default pacing. The repository API allows 1 000 requests +// an hour per account (about 16 a minute), and a run reads several times +// per target: 15 reads a minute, two targets and two git fetches at once, +// and at least a second between two writes, keep a run under the limit. +var limits = platform.Limits{Reads: 2, GitReads: 2, ReadsPerMinute: 15, MinInterval: time.Second} + +// branchRestrictions is the check the writer cannot make upfront: branch +// restrictions are readable with repository:admin only, so a push meets +// them ("Permission denied to update branch …"). +const branchRestrictions = "branch-restrictions" + +// Account ids: a uuid in braces, and an Atlassian account id (24 hex +// digits, or ":"). +var ( + uuidRe = regexp.MustCompile(`^\{[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}\}$`) + accountIDRe = regexp.MustCompile(`^(?:[0-9a-f]{24}|[0-9]{1,10}:[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})$`) +) + +// apiAccount is an account as the API reports it: type "user" for a +// person or a bot account with API tokens, "app_user" for an app or an +// access token's bot, "team" for a workspace. +type apiAccount struct { + Type string `json:"type"` + UUID string `json:"uuid"` + AccountID string `json:"account_id"` + Nickname string `json:"nickname"` + DisplayName string `json:"display_name"` + // Kind is the kind of an app_user (the OpenAPI description names it + // without listing values). + Kind string `json:"kind"` +} + +// Probe reports the capabilities of Bitbucket Cloud, which has one version: +// it asks the API nothing. +func (d *reader) Probe(context.Context) (platform.Caps, error) { + return platform.Caps{ + Flavor: "bitbucket", + MaxBody: maxBody, + Draft: platform.DraftNative, + // closed_by names who declined or merged a pull request. + CloserKnown: true, + // The API escapes HTML in descriptions: an HTML comment would show. + Marker: platform.MarkerInRefDef, + RuntimeOnly: []string{branchRestrictions}, + Limits: limits, + }, nil +} + +// Self is the account the credential acts as: GET /2.0/user, which takes +// the API token of an account. +func (d *reader) Self(ctx context.Context) (platform.Account, error) { + return d.c.selfAccount(ctx) +} + +// selfAccount returns the credential's account, looked up once. +func (c *client) selfAccount(ctx context.Context) (platform.Account, error) { + const op = "get account" + c.mu.Lock() + if c.self != nil { + a := *c.self + c.mu.Unlock() + return a, nil + } + c.mu.Unlock() + if c.token == "" { + return platform.Account{}, &platform.Error{Op: op, Class: platform.ClassAuth, Err: errors.New("no credential: the reader is anonymous")} + } + var u apiAccount + if _, err := c.get(ctx, op, c.endpoint("user"), nil, &u); err != nil { + return platform.Account{}, err + } + if !uuidRe.MatchString(u.UUID) { + return platform.Account{}, shapeError(op, "GET /user: no uuid") + } + a := toAccount(&u) + c.mu.Lock() + c.self = &a + c.mu.Unlock() + return a, nil +} + +// Lookup resolves a login to its account: a uuid in braces or an Atlassian +// account id (GET /2.0/users/{selected_user} takes both). Bitbucket finds no +// account by its nickname, so anything else is ClassInvalid; an id the API +// does not know is ErrNotFound. +func (d *reader) Lookup(ctx context.Context, login string) (platform.Account, error) { + const op = "look up account" + if !uuidRe.MatchString(login) && !accountIDRe.MatchString(login) { + return platform.Account{}, invalid(op, "%q: Bitbucket logins are account UUIDs in braces, such as {3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d}, "+ + "as GET /2.0/user shows them (a nickname names no account)", login) + } + var u apiAccount + if _, err := d.c.get(ctx, op, d.c.endpoint("users", login), nil, &u); err != nil { + return platform.Account{}, err + } + if !uuidRe.MatchString(u.UUID) { + return platform.Account{}, shapeError(op, "GET /users/%s: no uuid", login) + } + return toAccount(&u), nil +} + +// toAccount converts an API account: its uuid is both its id and its login. +// An app user is a bot; a user is a person or a bot account with API tokens, +// which the API does not tell apart; a workspace (team) or an account of +// another type is unknown. +func toAccount(u *apiAccount) platform.Account { + if u == nil || u.UUID == "" { + return platform.Account{} + } + a := platform.Account{ID: u.UUID, Login: u.UUID, Kind: platform.KindUnknown} + switch u.Type { + case "user": + a.Kind = platform.KindUser + case "app_user": + a.Kind = platform.KindBot + } + return a +} diff --git a/internal/platform/bitbucket/probe_test.go b/internal/platform/bitbucket/probe_test.go new file mode 100644 index 0000000..f386d03 --- /dev/null +++ b/internal/platform/bitbucket/probe_test.go @@ -0,0 +1,156 @@ +package bitbucket + +import ( + "errors" + "net/http" + "slices" + "strings" + "testing" + "time" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/config" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +func TestProbe(t *testing.T) { + f := newFixture(t) + caps, err := f.reader.Probe(t.Context()) + if err != nil { + t.Fatal(err) + } + want := platform.Caps{ + Flavor: "bitbucket", MaxBody: 60000, Draft: platform.DraftNative, CloserKnown: true, + Marker: platform.MarkerInRefDef, RuntimeOnly: []string{"branch-restrictions"}, + Limits: platform.Limits{Reads: 2, GitReads: 2, ReadsPerMinute: 15, MinInterval: time.Second}, + } + if caps.Flavor != want.Flavor || caps.MaxBody != want.MaxBody || caps.Draft != want.Draft || !caps.CloserKnown || + caps.Marker != want.Marker || !slices.Equal(caps.RuntimeOnly, want.RuntimeOnly) || caps.Limits != want.Limits || + caps.LabelsByID || caps.QuickActions || caps.WorkflowPerm || caps.Commit.API { + t.Errorf("Probe = %+v\nwant %+v", caps, want) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("Probe sent %d requests; Bitbucket Cloud has one version", len(calls)) + } +} + +func TestSelf(t *testing.T) { + f := newFixture(t) + me := account(botUUID, "touchmark bot") + me["account_status"] = "active" + me["has_2fa_enabled"] = nil + me["is_staff"] = false + me["created_on"] = "2026-09-01T10:00:00.000000+00:00" + f.json("/user", http.StatusOK, me) + for range 2 { + a, err := f.reader.Self(t.Context()) + if err != nil { + t.Fatal(err) + } + if a != (platform.Account{ID: botUUID, Login: botUUID, Kind: platform.KindUser}) { + t.Errorf("Self = %+v; want the uuid as id and login, a user", a) + } + } + calls := f.requests(http.MethodGet, "/user") + if len(calls) != 1 { + t.Fatalf("%d requests of /user; want one, then the cache", len(calls)) + } + if calls[0].Auth != "Bearer "+f.token { + t.Errorf("Authorization %q; want the token as a bearer token", calls[0].Auth) + } +} + +func TestSelfAppUser(t *testing.T) { + f := newFixture(t) + f.json("/user", http.StatusOK, appUser(botUUID, "touchmark")) + a, err := f.reader.Self(t.Context()) + if err != nil || a.Kind != platform.KindBot || a.ID != botUUID { + t.Errorf("Self = %+v, %v; want a bot", a, err) + } +} + +func TestSelfErrors(t *testing.T) { + t.Run("anonymous", func(t *testing.T) { + s := newAPIServer(t) + r, err := NewReader(s.provider(), auth.Credential{}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + _, err = r.Self(t.Context()) + wantClass(t, "Self", err, platform.ClassAuth, nil) + }) + t.Run("refused token", func(t *testing.T) { + f := newFixture(t) + f.json("/user", http.StatusUnauthorized, errorBody("Token is invalid, expired, or not supported for this endpoint.")) + _, err := f.reader.Self(t.Context()) + wantClass(t, "Self", err, platform.ClassAuth, nil) + if err != nil && !strings.Contains(err.Error(), "Token is invalid") { + t.Errorf("the error %q lacks the API's message", err) + } + }) + t.Run("no uuid", func(t *testing.T) { + f := newFixture(t) + f.json("/user", http.StatusOK, map[string]any{"type": "user", "nickname": "x"}) + _, err := f.reader.Self(t.Context()) + wantClass(t, "Self", err, platform.ClassUnknown, nil) + }) +} + +func TestLookup(t *testing.T) { + f := newFixture(t) + f.json("/users/"+uuidPath(personUUID), http.StatusOK, account(personUUID, "Wilson Mendes Neto")) + f.json("/users/5b57c56fdfe79e2c947cd85f", http.StatusOK, account(personUUID, "Wilson Mendes Neto")) + f.json("/users/"+uuidPath(otherUUID), http.StatusNotFound, errorBody(otherUUID+" not found")) + for _, login := range []string{personUUID, "5b57c56fdfe79e2c947cd85f"} { + a, err := f.reader.Lookup(t.Context(), login) + if err != nil || a.ID != personUUID || a.Login != personUUID || a.Kind != platform.KindUser { + t.Errorf("Lookup(%s) = %+v, %v; want the account by its uuid", login, a, err) + } + } + _, err := f.reader.Lookup(t.Context(), otherUUID) + wantClass(t, "an unknown uuid", err, platform.ClassNotFound, platform.ErrNotFound) + f.reset() + for _, login := range []string{"touchmark-bot", "Wilson Mendes Neto", "", "{not-a-uuid}", "../user"} { + _, err := f.reader.Lookup(t.Context(), login) + wantClass(t, "Lookup("+login+")", err, platform.ClassInvalid, nil) + if err != nil && !strings.Contains(err.Error(), "Bitbucket logins are account UUIDs") { + t.Errorf("Lookup(%q): %v; want the message to say logins are UUIDs", login, err) + } + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("a login that is no uuid sent %d requests", len(calls)) + } +} + +func TestNewReaderAndWriter(t *testing.T) { + s := newAPIServer(t) + client := httpx.New(httpx.Options{}) + tok := auth.Credential{Kind: auth.Token, Token: testToken(t)} + other := s.provider() + other.Type = "gitea" + noAPI := s.provider() + noAPI.APIURL = "" + for name, tc := range map[string]struct { + p config.ResolvedProvider + c auth.Credential + want string + }{ + "another type": {other, tok, `type "gitea", not bitbucket`}, + "an app": {s.provider(), auth.Credential{Kind: auth.App, AppID: "1", AppKey: []byte("k")}, "a GitHub App cannot sign in to Bitbucket"}, + "empty token": {s.provider(), auth.Credential{Kind: auth.Token}, "an empty token"}, + "no api url": {noAPI, tok, "api_url is empty"}, + } { + if _, err := NewReader(tc.p, tc.c, client); err == nil || !strings.Contains(err.Error(), tc.want) { + t.Errorf("%s: NewReader: %v; want %q", name, err, tc.want) + } + } + if _, err := NewReader(s.provider(), tok, nil); err == nil { + t.Error("NewReader without an HTTP client: no error") + } + w, err := NewWriter(s.provider(), tok, client) + wantClass(t, "NewWriter", err, platform.ClassUnsupported, nil) + if w != nil || !errors.Is(err, errNoWriter) || !strings.Contains(err.Error(), "the Bitbucket writer comes in a later release") { + t.Errorf("NewWriter = %v, %v; want no writer and the reason", w, err) + } +} diff --git a/internal/platform/bitbucket/pulls.go b/internal/platform/bitbucket/pulls.go new file mode 100644 index 0000000..f8bde52 --- /dev/null +++ b/internal/platform/bitbucket/pulls.go @@ -0,0 +1,542 @@ +package bitbucket + +import ( + "cmp" + "context" + "errors" + "fmt" + "net/url" + "slices" + "strconv" + "strings" + "time" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// Bounds of listings. +const ( + // maxPRPages bounds a listing of one repository's pull requests. PRs + // has no way to say it saw only part: past the bound it fails. + maxPRPages = 200 + // maxSweepPages bounds the open pull requests of one author in one + // workspace. + maxSweepPages = 200 + // maxWorkspacePages bounds the workspaces of the identity. + maxWorkspacePages = 50 +) + +// Pull request states (the state enum of the OpenAPI description). A +// declined pull request cannot be reopened; a superseded one was replaced +// by another and is closed too. +const ( + stateOpen = "OPEN" + stateMerged = "MERGED" + stateDeclined = "DECLINED" + stateSuperseded = "SUPERSEDED" +) + +// allStates are every state, which a listing with q= must name itself: the +// state parameter is ignored next to q. +var allStates = []string{stateOpen, stateMerged, stateDeclined, stateSuperseded} + +// apiPR is a pull request as the API reports it, in lists and alone. +// summary.raw is the description as typed; description is the same text +// (sent by the API, not in the OpenAPI description). +type apiPR struct { + ID int64 `json:"id"` + Title string `json:"title"` + State string `json:"state"` + Draft bool `json:"draft"` + Summary *apiText `json:"summary"` + Description *string `json:"description"` + Author *apiAccount `json:"author"` + ClosedBy *apiAccount `json:"closed_by"` + Source apiPREndpoint `json:"source"` + Destination apiPREndpoint `json:"destination"` + CreatedOn *time.Time `json:"created_on"` + UpdatedOn *time.Time `json:"updated_on"` + Links apiLinks `json:"links"` +} + +// apiText is rendered markup: raw is what was typed. +type apiText struct { + Raw *string `json:"raw"` +} + +// apiPREndpoint is the source or destination of a pull request: its branch, +// its commit (a short hash) and its repository (gone for the source of a +// deleted fork). +type apiPREndpoint struct { + Branch *apiBranch `json:"branch"` + Commit *apiCommit `json:"commit"` + Repository *apiRepoRef `json:"repository"` +} + +// apiWorkspaceAccess is an entry of GET /2.0/user/workspaces. +type apiWorkspaceAccess struct { + Workspace *struct { + Slug string `json:"slug"` + } `json:"workspace"` +} + +// check reports what the driver depends on that a pull request lacks. +func (p *apiPR) check(op string) error { + switch { + case p.ID <= 0: + return shapeError(op, "a pull request without an id") + case !slices.Contains(allStates, p.State): + return shapeError(op, "pull request #%d has state %q", p.ID, p.State) + case p.Destination.Repository == nil || !uuidRe.MatchString(p.Destination.Repository.UUID): + return shapeError(op, "pull request #%d lacks its destination repository", p.ID) + case p.Destination.Branch == nil || p.Destination.Branch.Name == "": + return shapeError(op, "pull request #%d lacks its destination branch", p.ID) + case p.CreatedOn == nil: + return shapeError(op, "pull request #%d has no created_on", p.ID) + } + return nil +} + +// head returns the source branch of p, "" when the API names none. +func (p *apiPR) head() string { + if p.Source.Branch == nil { + return "" + } + return p.Source.Branch.Name +} + +// authorID returns the uuid of p's author, "" when there is none (a +// deleted account). +func (p *apiPR) authorID() string { + if p.Author == nil { + return "" + } + return p.Author.UUID +} + +// body returns the description and whether the API sent it. +func (p *apiPR) body() (string, bool) { + switch { + case p.Summary != nil && p.Summary.Raw != nil: + return *p.Summary.Raw, true + case p.Description != nil: + return *p.Description, true + } + return "", false +} + +// toPR converts an API pull request; BaseExists and HeadSHA are left to the +// caller (fill). Declined and superseded pull requests are closed, merged +// ones merged; closed_by names who did it. The API has no time of closing: +// ClosedAt is updated_on, which a later change of a closed pull request +// (a comment) moves on, so it is never earlier than the close. +func (c *client) toPR(p *apiPR) platform.PR { + body, _ := p.body() + pr := platform.PR{ + Number: p.ID, + URL: p.Links.HTML.Href, + State: platform.Open, + Draft: p.Draft, + Head: p.head(), + Base: p.Destination.Branch.Name, + RepoID: p.Destination.Repository.UUID, + Title: p.Title, + Body: body, + Labels: []string{}, + Author: toAccount(p.Author), + CreatedAt: p.CreatedOn.UTC(), + } + if pr.URL == "" && p.Destination.Repository.FullName != "" { + pr.URL = c.web + "/" + p.Destination.Repository.FullName + "/pull-requests/" + strconv.FormatInt(p.ID, 10) + } + if p.Source.Repository != nil && uuidRe.MatchString(p.Source.Repository.UUID) && pr.Head != "" { + pr.HeadRepoID = p.Source.Repository.UUID + } + switch p.State { + case stateMerged: + pr.State = platform.Merged + case stateDeclined, stateSuperseded: + pr.State = platform.Closed + } + if pr.State != platform.Open { + if by := toAccount(p.ClosedBy); by.ID != "" { + pr.ClosedBy = &by + } + if p.UpdatedOn != nil { + pr.ClosedAt = p.UpdatedOn.UTC() + } + } + return pr +} + +// prQuery returns the BBQL filter of pull requests from heads in states: +// `source.branch.name IN ("a", "b") AND state IN ("OPEN")`. The escaping +// of BBQL strings is not documented: a head with a quote, a backslash or a +// control character drops the branch filter, and the caller, which +// filters every listing anyway, reads them all. +func prQuery(heads, states []string) string { + quote := func(list []string) string { + parts := make([]string, len(list)) + for i, s := range list { + parts[i] = `"` + s + `"` + } + return "(" + strings.Join(parts, ", ") + ")" + } + q := "state IN " + quote(states) + for _, h := range heads { + if strings.ContainsFunc(h, func(r rune) bool { return r == '"' || r == '\\' || r < 0x20 || r == 0x7f }) { + return q + } + } + return "source.branch.name IN " + quote(heads) + " AND " + q +} + +// PRs returns, newest first, the pull requests of r from heads by authors +// in every state and the open ones by anyone: one listing of r's pull +// requests filtered by source branch and every state in q=, sorted by +// descending id, filtered again here. A pull request's head is its source +// branch in whatever repository: those from forks are included with their +// own HeadRepoID. The listing is complete or the call fails. +// +// BaseExists reads the branch API for a base other than r's default branch +// (once per base). HeadSHA is the full id of the source commit of an open +// pull request from r itself, from the commit API (the pull request shows +// 12 digits; a short id once expanded is remembered); "" for the others, +// which the core reads no head of. +func (d *reader) PRs(ctx context.Context, r platform.Repo, heads []string, authors []platform.Account) ([]platform.PR, error) { + const op = "list pull requests" + ws, slug, err := repoPath(op, r) + if err != nil { + return nil, err + } + wanted := setOf(heads) + if len(wanted) == 0 { + return []platform.PR{}, nil + } + ids := accountIDs(authors) + q := url.Values{ + "q": {prQuery(sortedKeys(wanted), allStates)}, + "sort": {"-id"}, + "pagelen": {strconv.Itoa(prPageLen)}, + } + var found []apiPR + seen := map[int64]bool{} + complete, err := listAll(ctx, d.c, op, d.c.endpoint("repositories", ws, slug, "pullrequests"), q, maxPRPages, func(p apiPR) error { + if err := p.check(op); err != nil { + return err + } + if !seen[p.ID] && wanted[p.head()] && (p.State == stateOpen || ids[p.authorID()]) { + seen[p.ID] = true + found = append(found, p) + } + return nil + }) + if err != nil { + return nil, err + } + if !complete { + return nil, &platform.Error{Op: op, Class: platform.ClassUnknown, + Err: fmt.Errorf("%s has more pull requests than %d pages; touchmark does not act on a partial list", r.Path, maxPRPages)} + } + out := make([]platform.PR, 0, len(found)) + bases := map[string]bool{} + for i := range found { + p := &found[i] + if err := d.c.fillBody(ctx, op, ws, slug, p); err != nil { + return nil, err + } + pr := d.c.toPR(p) + if pr.BaseExists, err = d.c.baseExists(ctx, op, r, ws, slug, pr.Base, bases); err != nil { + return nil, err + } + if pr.State == platform.Open && pr.HeadRepoID != "" && pr.HeadRepoID == pr.RepoID && p.Source.Commit != nil { + if pr.HeadSHA, err = d.c.fullCommit(ctx, op, ws, slug, pr.RepoID, p.Source.Commit.Hash); err != nil { + return nil, err + } + } + out = append(out, pr) + } + slices.SortFunc(out, func(a, b platform.PR) int { return cmp.Compare(b.Number, a.Number) }) + return out, nil +} + +// fillBody reads a pull request alone when its listing left its +// description out (a listing is taken to carry summary; to be confirmed +// live), so that the marker in it is never missed. +func (c *client) fillBody(ctx context.Context, op, ws, slug string, p *apiPR) error { + if _, ok := p.body(); ok { + return nil + } + var full apiPR + if _, err := c.get(ctx, op, c.endpoint("repositories", ws, slug, "pullrequests", strconv.FormatInt(p.ID, 10)), nil, &full); err != nil { + return err + } + if err := full.check(op); err != nil { + return err + } + if _, ok := full.body(); !ok || full.ID != p.ID { + return shapeError(op, "pull request #%d has no description", p.ID) + } + *p = full + return nil +} + +// baseExists reports whether branch, a pull request's base in r, exists: +// r's default branch does; another is looked up once per call (known holds +// the answers). A pull request whose base was deleted stays with its +// branch's name. +func (c *client) baseExists(ctx context.Context, op string, r platform.Repo, ws, slug, branch string, known map[string]bool) (bool, error) { + if branch == r.DefaultBranch { + return true, nil + } + if v, ok := known[branch]; ok { + return v, nil + } + _, err := c.get(ctx, op, c.endpoint(append([]string{"repositories", ws, slug, "refs", "branches"}, pathSegments(branch)...)...), nil, nil) + switch { + case err == nil: + known[branch] = true + case platform.ClassOf(err) == platform.ClassNotFound: + known[branch] = false + default: + return false, err + } + return known[branch], nil +} + +// fullCommit returns the full id of the commit whose id starts with short +// in ws/slug (the repository with uuid repo): GET …/commit/{short}. A +// commit the API does not find, or an answer that is no full id with that +// prefix, gives "" (the head stays unknown); a failure that stops the call +// or is transient is returned. +func (c *client) fullCommit(ctx context.Context, op, ws, slug, repo, short string) (string, error) { + short = strings.ToLower(strings.TrimSpace(short)) + switch { + case !isHex(short) || len(short) < 7: + return "", nil + case isHexOID(short): + return short, nil + } + key := repo + " " + short + c.mu.Lock() + full, ok := c.commits[key] + c.mu.Unlock() + if ok { + return full, nil + } + var cm apiCommit + _, err := c.get(ctx, op, c.endpoint("repositories", ws, slug, "commit", short), nil, &cm) + switch { + case err != nil && fatal(err): + return "", err + case err != nil: + return "", nil + } + full = strings.ToLower(cm.Hash) + if !isHexOID(full) || !strings.HasPrefix(full, short) { + return "", nil + } + c.mu.Lock() + c.commits[key] = full + c.mu.Unlock() + return full, nil +} + +// OpenPRsBy lists the open pull requests by authors from heads in every +// repository this identity sees, for the stale sweep: for each workspace of +// the identity (GET /2.0/user/workspaces) and each author, the open pull +// requests the author made there (GET +// /2.0/workspaces/{workspace}/pullrequests/{uuid}, filtered by source +// branch and state in q=), filtered again here; each repository is read +// once, for the report and the default branch. HeadSHA stays "": the sweep +// closes pull requests and reads no heads. +// +// A listing that fails, on any page, with a rate limit, a refused +// credential, a transient failure or the end of ctx fails the call, so the +// core can pause or retry; any other failure of a listing (an author the +// workspace does not know, a missing scope) makes the result incomplete, as +// does a listing capped at its bound. A repository or pull request gone +// since the listing is left out; one that cannot be read otherwise makes +// the result incomplete. +func (d *reader) OpenPRsBy(ctx context.Context, authors []platform.Account, heads []string) (platform.Swept, error) { + const op = "list open pull requests" + ids := accountIDs(authors) + wanted := setOf(heads) + out := platform.Swept{PRs: []platform.RepoPR{}, Complete: true} + if len(ids) == 0 || len(wanted) == 0 { + return out, nil + } + if d.c.token == "" { + return platform.Swept{}, &platform.Error{Op: op, Class: platform.ClassAuth, Err: errAnonymousSweep} + } + workspaces, complete, err := d.c.workspaces(ctx, op) + switch { + case err != nil && fatal(err): + return platform.Swept{}, err + case err != nil: + out.Complete = false + } + out.Complete = out.Complete && complete + type hit struct { + ws string + pr apiPR + } + hits := map[string]hit{} + q := url.Values{ + "q": {prQuery(sortedKeys(wanted), []string{stateOpen})}, + "sort": {"-id"}, + "pagelen": {strconv.Itoa(prPageLen)}, + } + for _, ws := range workspaces { + for _, id := range sortedKeys(ids) { + if !uuidRe.MatchString(id) { + out.Complete = false + continue + } + complete, err := listAll(ctx, d.c, op, d.c.endpoint("workspaces", ws, "pullrequests", id), q, maxSweepPages, func(p apiPR) error { + if err := p.check(op); err != nil { + return err + } + if p.State == stateOpen && wanted[p.head()] && ids[p.authorID()] { + hits[p.Destination.Repository.UUID+"#"+strconv.FormatInt(p.ID, 10)] = hit{ws: ws, pr: p} + } + return nil + }) + switch { + case err == nil: + out.Complete = out.Complete && complete + case fatal(err): + return platform.Swept{}, err + default: + out.Complete = false + } + } + } + keys := make([]string, 0, len(hits)) + for k := range hits { + keys = append(keys, k) + } + slices.Sort(keys) + repos := map[string]*apiRepo{} + bases := map[string]map[string]bool{} + for _, k := range keys { + h := hits[k] + p := h.pr + uuid := p.Destination.Repository.UUID + ws := h.ws + if w, _, ok := splitRepoPath(p.Destination.Repository.FullName); ok { + ws = w + } + repo, ok := repos[uuid] + if !ok { + repo, err = d.c.getRepo(ctx, op, ws, uuid) + if err == nil && repo.UUID != uuid { + err = shapeError(op, "repository %s answers with uuid %s", uuid, repo.UUID) + } + switch { + case err == nil: + case platform.ClassOf(err) == platform.ClassNotFound: + repo = nil + case stops(err): + return platform.Swept{}, err + default: + out.Complete = false + repo = nil + } + repos[uuid] = repo + } + if repo == nil { + continue + } + rws, rslug, _ := splitRepoPath(repo.FullName) + r := d.c.toRepo(repo) + if bases[uuid] == nil { + bases[uuid] = map[string]bool{} + } + err := d.c.fillBody(ctx, op, rws, rslug, &p) + var pr platform.PR + if err == nil { + pr = d.c.toPR(&p) + pr.BaseExists, err = d.c.baseExists(ctx, op, r, rws, rslug, pr.Base, bases[uuid]) + } + switch { + case err == nil: + case platform.ClassOf(err) == platform.ClassNotFound: + continue // gone since the listing + case stops(err): + return platform.Swept{}, err + default: + out.Complete = false + continue + } + out.PRs = append(out.PRs, platform.RepoPR{Repo: r, PR: pr}) + } + slices.SortFunc(out.PRs, func(a, b platform.RepoPR) int { + if c := strings.Compare(strings.ToLower(a.Repo.Path), strings.ToLower(b.Repo.Path)); c != 0 { + return c + } + if c := strings.Compare(a.Repo.ID, b.Repo.ID); c != 0 { + return c + } + return cmp.Compare(b.PR.Number, a.PR.Number) + }) + return out, nil +} + +// errAnonymousSweep is why an anonymous reader cannot sweep: it has no +// workspaces and no pull requests of its own. +var errAnonymousSweep = errors.New("no credential: the reader is anonymous, and only an account has workspaces to sweep") + +// workspaces returns the slugs of the workspaces the identity reaches, +// sorted; complete is false when the listing was capped or a later page +// failed. +func (c *client) workspaces(ctx context.Context, op string) ([]string, bool, error) { + set := map[string]bool{} + q := url.Values{"pagelen": {strconv.Itoa(repoPageLen)}} + complete, err := listAll(ctx, c, op, c.endpoint("user", "workspaces"), q, maxWorkspacePages, func(a apiWorkspaceAccess) error { + if a.Workspace == nil || a.Workspace.Slug == "" || strings.ContainsAny(a.Workspace.Slug, "/\x00?#") { + return shapeError(op, "a workspace without a slug") + } + set[a.Workspace.Slug] = true + return nil + }) + if err != nil && (!laterPage(err) || fatal(err)) { + return nil, false, err + } + if err != nil { + complete = false + } + return sortedKeys(set), complete, nil +} + +// accountIDs returns the set of the accounts' ids. +func accountIDs(accounts []platform.Account) map[string]bool { + ids := map[string]bool{} + for _, a := range accounts { + if a.ID != "" { + ids[a.ID] = true + } + } + return ids +} + +// setOf returns the set of the non-empty strings of list. +func setOf(list []string) map[string]bool { + out := map[string]bool{} + for _, s := range list { + if s != "" { + out[s] = true + } + } + return out +} + +// sortedKeys returns the keys of m in order. +func sortedKeys(m map[string]bool) []string { + out := make([]string, 0, len(m)) + for k := range m { + out = append(out, k) + } + slices.Sort(out) + return out +} diff --git a/internal/platform/bitbucket/pulls_test.go b/internal/platform/bitbucket/pulls_test.go new file mode 100644 index 0000000..af137e1 --- /dev/null +++ b/internal/platform/bitbucket/pulls_test.go @@ -0,0 +1,350 @@ +package bitbucket + +import ( + "errors" + "net/http" + "strings" + "testing" + "time" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +// The accounts of the pull request tests: the writer (a bot account with an +// API token, type user) and a person. +var ( + botAccount = platform.Account{ID: botUUID, Login: botUUID, Kind: platform.KindUser} + personAccount = platform.Account{ID: personUUID, Login: personUUID, Kind: platform.KindUser} +) + +// fullHash is the full id of the source commit of pull request id in pr's +// fixtures, whose short form pr sends. +func fullHash(short string) string { return short + strings.Repeat("0", 40-len(short)) } + +// commitRoute declares GET …/commit/{short} of acme/api with its full id. +func (s *apiServer) commitRoute(short string) { + s.json("/repositories/acme/api/commit/"+short, http.StatusOK, map[string]any{ + "hash": fullHash(short), "type": "commit", "date": "2026-09-30T10:00:00+00:00", "message": "chore: sync", + }) +} + +func TestPRs(t *testing.T) { + f := newFixture(t) + bot, person := account(botUUID, "touchmark bot"), account(personUUID, "Wilson Mendes Neto") + fork := repoRef(forkUUID, "someone/api") + items := []any{ + pr(prSpec{id: 9, author: bot, source: syncBranch, body: "sync\n\n[touchmark]: # \"touchmark:v1 …\"", draft: true}), + pr(prSpec{id: 8, author: person, source: syncBranch, sourceRepo: fork, body: "from a fork"}), + pr(prSpec{id: 7, state: "DECLINED", author: bot, closedBy: person, source: syncBranch, body: "declined"}), + pr(prSpec{id: 6, state: "MERGED", author: bot, closedBy: person, source: syncBranch, target: "release"}), + pr(prSpec{id: 5, state: "SUPERSEDED", author: bot, closedBy: bot, source: syncBranch, target: "gone"}), + pr(prSpec{id: 4, state: "DECLINED", author: person, closedBy: person, source: syncBranch}), + pr(prSpec{id: 3, author: bot, source: "feature/other"}), + } + f.pages("/repositories/acme/api/pullrequests", 3, items) + f.commitRoute("a6dcc7402009") + f.json("/repositories/acme/api/refs/branches/release", http.StatusOK, map[string]any{"name": "release", "type": "branch", + "target": map[string]any{"hash": headCommit, "type": "commit"}}) + f.json("/repositories/acme/api/refs/branches/gone", http.StatusNotFound, errorBody("Branch \"gone\" not found")) + + got, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch, ""}, []platform.Account{botAccount}) + if err != nil { + t.Fatal(err) + } + var numbers []int64 + byNumber := map[int64]platform.PR{} + for _, p := range got { + numbers = append(numbers, p.Number) + byNumber[p.Number] = p + } + if len(numbers) != 5 || numbers[0] != 9 || numbers[1] != 8 || numbers[2] != 7 || numbers[3] != 6 || numbers[4] != 5 { + t.Fatalf("PRs = %v; want 9, 8, 7, 6, 5: the bot's in every state and anyone's open ones, newest first", numbers) + } + open := byNumber[9] + if open.State != platform.Open || !open.Draft || open.Head != syncBranch || open.Base != "main" || !open.BaseExists || + open.RepoID != repoUUID || open.HeadRepoID != repoUUID || open.HeadSHA != fullHash("a6dcc7402009") || + open.Author != botAccount || open.ClosedBy != nil || !open.ClosedAt.IsZero() || + open.Body != "sync\n\n[touchmark]: # \"touchmark:v1 …\"" || open.URL != "https://bitbucket.org/acme/api/pull-requests/9" || + len(open.Labels) != 0 || open.Labels == nil || !open.CreatedAt.Equal(time.Date(2026, 9, 5, 8, 0, 0, 0, time.UTC)) { + t.Errorf("open = %+v", open) + } + if fromFork := byNumber[8]; fromFork.HeadRepoID != forkUUID || fromFork.HeadSHA != "" || fromFork.State != platform.Open { + t.Errorf("from a fork = %+v; want its own head repository and no head commit read", fromFork) + } + declined := byNumber[7] + if declined.State != platform.Closed || declined.ClosedBy == nil || *declined.ClosedBy != personAccount || + !declined.ClosedAt.Equal(time.Date(2026, 9, 10, 12, 0, 0, 123456000, time.UTC)) || declined.HeadSHA != "" { + t.Errorf("declined = %+v; want closed, by the person, at updated_on", declined) + } + if merged := byNumber[6]; merged.State != platform.Merged || !merged.BaseExists || merged.ClosedBy == nil { + t.Errorf("merged = %+v", merged) + } + if superseded := byNumber[5]; superseded.State != platform.Closed || superseded.BaseExists || superseded.ClosedBy == nil || + *superseded.ClosedBy != botAccount { + t.Errorf("superseded = %+v; want closed, its deleted base reported", superseded) + } + + calls := f.requests(http.MethodGet, "/repositories/acme/api/pullrequests") + if len(calls) != 3 { + t.Fatalf("%d pages read; want 3", len(calls)) + } + q := calls[0].Query + wantQ := `source.branch.name IN ("touchmark/acme-eng") AND state IN ("OPEN", "MERGED", "DECLINED", "SUPERSEDED")` + if q.Get("q") != wantQ || q.Get("sort") != "-id" || q.Get("pagelen") != "50" || q.Has("state") { + t.Errorf("query %v; want q %s, sorted by descending id, the state in q", q, wantQ) + } + if n := len(f.requests(http.MethodGet, "/repositories/acme/api/refs/branches/main")); n != 0 { + t.Errorf("the default branch was looked up %d times; it exists", n) + } + + // A second listing reads the full id of the head from memory. + f.reset() + if _, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch}, []platform.Account{botAccount}); err != nil { + t.Fatal(err) + } + if n := len(f.requests(http.MethodGet, "/repositories/acme/api/commit/a6dcc7402009")); n != 0 { + t.Errorf("the short id was expanded again (%d requests)", n) + } +} + +func TestPRsNoHeads(t *testing.T) { + f := newFixture(t) + got, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{""}, []platform.Account{botAccount}) + if err != nil || got == nil || len(got) != 0 { + t.Errorf("PRs without heads = %v, %v; want an empty list", got, err) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("%d requests without heads", len(calls)) + } +} + +func TestPRsQuoting(t *testing.T) { + f := newFixture(t) + f.pages("/repositories/acme/api/pullrequests", 50, []any{ + pr(prSpec{id: 2, author: account(botUUID, "bot"), source: `odd"name`, hash: strings.Repeat("c", 40)}), + pr(prSpec{id: 1, author: account(botUUID, "bot"), source: "other"}), + }) + got, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{`odd"name`}, []platform.Account{botAccount}) + if err != nil || len(got) != 1 || got[0].Number != 2 || got[0].HeadSHA != strings.Repeat("c", 40) { + t.Errorf("PRs = %+v, %v; want #2, its full head taken as it is", got, err) + } + q := f.requests(http.MethodGet, "/repositories/acme/api/pullrequests")[0].Query.Get("q") + if q != `state IN ("OPEN", "MERGED", "DECLINED", "SUPERSEDED")` { + t.Errorf("q %s; want no branch filter for a name BBQL cannot hold safely", q) + } +} + +func TestPRsBody(t *testing.T) { + f := newFixture(t) + f.pages("/repositories/acme/api/pullrequests", 50, []any{ + pr(prSpec{id: 4, state: "MERGED", author: account(botUUID, "bot"), source: syncBranch, noSummary: true}), + }) + f.json("/repositories/acme/api/pullrequests/4", http.StatusOK, + pr(prSpec{id: 4, state: "MERGED", author: account(botUUID, "bot"), source: syncBranch, body: "the marker"})) + got, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch}, []platform.Account{botAccount}) + if err != nil || len(got) != 1 || got[0].Body != "the marker" { + t.Errorf("PRs = %+v, %v; want the description of the pull request read alone", got, err) + } +} + +func TestPRsErrors(t *testing.T) { + t.Run("a missing repository", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api/pullrequests", http.StatusNotFound, errorBody(noRepoMessage)) + _, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch}, nil) + wantClass(t, "PRs", err, platform.ClassNotFound, nil) + }) + t.Run("a later page fails", func(t *testing.T) { + f := newFixture(t) + items := []any{ + pr(prSpec{id: 2, author: account(botUUID, "bot"), source: syncBranch}), + pr(prSpec{id: 1, author: account(botUUID, "bot"), source: syncBranch}), + } + f.handle(http.MethodGet, "/repositories/acme/api/pullrequests", func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("page") == "2" { + writeJSON(w, http.StatusBadGateway, errorBody("Bad gateway")) + return + } + servePage(w, r, 1, items) + }) + _, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch}, nil) + wantClass(t, "PRs", err, platform.ClassTransient, nil) + }) + t.Run("an unknown state", func(t *testing.T) { + f := newFixture(t) + p := pr(prSpec{id: 1, author: account(botUUID, "bot"), source: syncBranch}) + p["state"] = "QUEUED" + f.pages("/repositories/acme/api/pullrequests", 50, []any{p}) + _, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch}, nil) + wantClass(t, "PRs", err, platform.ClassUnknown, nil) + }) + t.Run("a head commit the API does not find", func(t *testing.T) { + f := newFixture(t) + f.pages("/repositories/acme/api/pullrequests", 50, []any{pr(prSpec{id: 1, author: account(botUUID, "bot"), source: syncBranch})}) + f.json("/repositories/acme/api/commit/a6dcc7402001", http.StatusNotFound, errorBody("Commit not found")) + got, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch}, nil) + if err != nil || len(got) != 1 || got[0].HeadSHA != "" { + t.Errorf("PRs = %+v, %v; want the pull request with its head unknown", got, err) + } + }) + t.Run("the commit API limits the rate", func(t *testing.T) { + f := newFixture(t) + f.pages("/repositories/acme/api/pullrequests", 50, []any{pr(prSpec{id: 1, author: account(botUUID, "bot"), source: syncBranch})}) + f.handle(http.MethodGet, "/repositories/acme/api/commit/a6dcc7402001", func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Retry-After", "30") + writeJSON(w, http.StatusTooManyRequests, errorBody("Rate limit for this resource has been exceeded")) + }) + _, err := f.reader.PRs(t.Context(), apiRepoFixture, []string{syncBranch}, nil) + wantClass(t, "PRs", err, platform.ClassRateLimited, nil) + }) +} + +// sweepFixture declares two workspaces, acme and labs, where the bot has +// open pull requests. +func sweepFixture(t *testing.T) *fixture { + f := newFixture(t) + bot := account(botUUID, "touchmark bot") + f.pages("/user/workspaces", 1, []any{ + map[string]any{"administrator": false, "type": "workspace_access", + "workspace": map[string]any{"type": "workspace_base", "slug": "labs", "uuid": "{470c176d-3574-44ea-bb41-89e8638bcca4}"}}, + map[string]any{"administrator": false, "type": "workspace_access", + "workspace": map[string]any{"type": "workspace_base", "slug": "acme", "uuid": "{02b941e3-cfaa-40f9-9a58-cec53e20bdc3}"}}, + }) + labsUUID := "{9c8b7a6d-5e4f-4a3b-9c2d-1e0f9a8b7c6d}" + labsPR := pr(prSpec{id: 3, author: bot, source: syncBranch, body: "labs", target: "develop"}) + labsPR["destination"].(map[string]any)["repository"] = repoRef(labsUUID, "labs/tool") + labsPR["source"].(map[string]any)["repository"] = repoRef(labsUUID, "labs/tool") + f.pages("/workspaces/acme/pullrequests/"+uuidPath(botUUID), 50, []any{ + pr(prSpec{id: 12, author: bot, source: syncBranch, body: "twelve"}), + pr(prSpec{id: 11, author: bot, source: "feature/x"}), + pr(prSpec{id: 10, author: bot, source: syncBranch, noSummary: true}), + }) + f.pages("/workspaces/labs/pullrequests/"+uuidPath(botUUID), 50, []any{labsPR}) + f.json("/repositories/acme/"+uuidPath(repoUUID), http.StatusOK, repo(repoUUID, "acme/api")) + f.json("/repositories/labs/"+uuidPath(labsUUID), http.StatusOK, repo(labsUUID, "labs/tool", with("is_private", false))) + f.json("/repositories/acme/api/pullrequests/10", http.StatusOK, pr(prSpec{id: 10, author: bot, source: syncBranch, body: "ten"})) + f.json("/repositories/labs/tool/refs/branches/develop", http.StatusNotFound, errorBody("Branch \"develop\" not found")) + return f +} + +func TestOpenPRsBy(t *testing.T) { + f := sweepFixture(t) + got, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{botAccount}, []string{syncBranch}) + if err != nil { + t.Fatal(err) + } + if !got.Complete || len(got.PRs) != 3 { + t.Fatalf("OpenPRsBy = %+v; want three pull requests, complete", got) + } + want := []struct { + path string + number int64 + body string + base bool + }{{"acme/api", 12, "twelve", true}, {"acme/api", 10, "ten", true}, {"labs/tool", 3, "labs", false}} + for i, w := range want { + rp := got.PRs[i] + if rp.Repo.Path != w.path || rp.PR.Number != w.number || rp.PR.Body != w.body || rp.PR.BaseExists != w.base || + rp.PR.State != platform.Open || rp.PR.RepoID != rp.Repo.ID || rp.PR.HeadRepoID != rp.Repo.ID || rp.Repo.DefaultBranch != "main" { + t.Errorf("PRs[%d] = %s #%d %+v; want %+v", i, rp.Repo.Path, rp.PR.Number, rp.PR, w) + } + } + if got.PRs[2].Repo.Visibility != "public" || got.PRs[0].Repo.Visibility != "private" { + t.Errorf("visibility %s, %s", got.PRs[0].Repo.Visibility, got.PRs[2].Repo.Visibility) + } + if n := len(f.requests(http.MethodGet, "/repositories/acme/"+uuidPath(repoUUID))); n != 1 { + t.Errorf("acme/api read %d times; want once", n) + } + calls := f.requests(http.MethodGet, "/workspaces/acme/pullrequests/"+uuidPath(botUUID)) + if len(calls) != 1 || calls[0].Query.Get("q") != `source.branch.name IN ("touchmark/acme-eng") AND state IN ("OPEN")` { + t.Errorf("calls %+v; want the open pull requests from the sync branch", calls) + } +} + +func TestOpenPRsByIncomplete(t *testing.T) { + t.Run("an author a workspace does not know", func(t *testing.T) { + f := sweepFixture(t) + f.json("/workspaces/labs/pullrequests/"+uuidPath(botUUID), http.StatusNotFound, errorBody("No such user")) + got, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{botAccount}, []string{syncBranch}) + if err != nil || got.Complete || len(got.PRs) != 2 { + t.Errorf("OpenPRsBy = %+v, %v; want acme's pull requests, incomplete", got, err) + } + }) + t.Run("a repository that cannot be read", func(t *testing.T) { + f := sweepFixture(t) + f.json("/repositories/acme/"+uuidPath(repoUUID), http.StatusForbidden, errorBody("Forbidden")) + got, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{botAccount}, []string{syncBranch}) + if err != nil || got.Complete || len(got.PRs) != 1 { + t.Errorf("OpenPRsBy = %+v, %v; want labs' pull request, incomplete", got, err) + } + }) + t.Run("a repository gone since the listing", func(t *testing.T) { + f := sweepFixture(t) + f.json("/repositories/acme/"+uuidPath(repoUUID), http.StatusNotFound, errorBody(noRepoMessage)) + got, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{botAccount}, []string{syncBranch}) + if err != nil || !got.Complete || len(got.PRs) != 1 { + t.Errorf("OpenPRsBy = %+v, %v; want labs' pull request, complete", got, err) + } + }) + t.Run("an author id that is no uuid", func(t *testing.T) { + f := sweepFixture(t) + got, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{botAccount, {ID: "5b57c56fdfe79e2c947cd85f"}}, []string{syncBranch}) + if err != nil || got.Complete || len(got.PRs) != 3 { + t.Errorf("OpenPRsBy = %+v, %v; want the bot's, incomplete", got, err) + } + }) +} + +func TestOpenPRsByErrors(t *testing.T) { + t.Run("rate limited", func(t *testing.T) { + f := sweepFixture(t) + f.handle(http.MethodGet, "/workspaces/labs/pullrequests/"+uuidPath(botUUID), func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("X-RateLimit-Remaining", "0") + w.Header().Set("X-RateLimit-Reset", "120") + writeJSON(w, http.StatusTooManyRequests, errorBody("Rate limit for this resource has been exceeded")) + }) + _, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{botAccount}, []string{syncBranch}) + wantClass(t, "OpenPRsBy", err, platform.ClassRateLimited, nil) + if d := retryAfterOf(err); d != 120*time.Second { + t.Errorf("RetryAfter %v; want x-ratelimit-reset's 120 s", d) + } + }) + t.Run("no workspaces", func(t *testing.T) { + f := newFixture(t) + f.json("/user/workspaces", http.StatusForbidden, errorBody("Your credentials lack one or more required privilege scopes.")) + got, err := f.reader.OpenPRsBy(t.Context(), []platform.Account{botAccount}, []string{syncBranch}) + if err != nil || got.Complete || len(got.PRs) != 0 { + t.Errorf("OpenPRsBy = %+v, %v; want nothing, incomplete", got, err) + } + }) + t.Run("anonymous", func(t *testing.T) { + s := newAPIServer(t) + r, err := NewReader(s.provider(), auth.Credential{}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + _, err = r.OpenPRsBy(t.Context(), []platform.Account{botAccount}, []string{syncBranch}) + wantClass(t, "anonymous", err, platform.ClassAuth, nil) + }) + t.Run("nothing to look for", func(t *testing.T) { + f := newFixture(t) + got, err := f.reader.OpenPRsBy(t.Context(), nil, []string{syncBranch}) + if err != nil || !got.Complete || got.PRs == nil { + t.Errorf("OpenPRsBy without authors = %+v, %v", got, err) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("%d requests", len(calls)) + } + }) +} + +// retryAfterOf returns the RetryAfter of err's *platform.Error. +func retryAfterOf(err error) time.Duration { + var pe *platform.Error + if errors.As(err, &pe) { + return pe.RetryAfter + } + return 0 +} diff --git a/internal/platform/bitbucket/repos.go b/internal/platform/bitbucket/repos.go new file mode 100644 index 0000000..f2fd703 --- /dev/null +++ b/internal/platform/bitbucket/repos.go @@ -0,0 +1,213 @@ +package bitbucket + +import ( + "context" + "errors" + "net/url" + "slices" + "strconv" + "strings" + + "github.com/bedrock-python/touchmark/internal/platform" +) + +// maxRepoPages bounds a workspace listing: 20 000 repositories at 100 a +// page. A larger one resolves incompletely. +const maxRepoPages = 200 + +// apiRepo is a repository as the API reports it. +type apiRepo struct { + UUID string `json:"uuid"` + FullName string `json:"full_name"` + IsPrivate bool `json:"is_private"` + MainBranch *apiBranch `json:"mainbranch"` + Parent *apiRepoRef `json:"parent"` + Links apiLinks `json:"links"` +} + +// apiRepoRef is the short repository of a fork's parent or of a pull +// request's source or destination. +type apiRepoRef struct { + UUID string `json:"uuid"` + FullName string `json:"full_name"` +} + +// apiBranch is a branch: a repository's main branch, a pull request's +// source or destination branch, or GET …/refs/branches/{name}, where target +// is its head commit. +type apiBranch struct { + Name string `json:"name"` + Target *apiCommit `json:"target"` +} + +// apiCommit is a commit; hash is full in the commit and branch APIs and +// short (12 digits) in pull requests. +type apiCommit struct { + Hash string `json:"hash"` +} + +// apiLinks are the links of a resource that the driver reads. +type apiLinks struct { + HTML struct { + Href string `json:"href"` + } `json:"html"` +} + +// check reports what the driver depends on that a repository lacks. +func (r *apiRepo) check(op string) error { + if !uuidRe.MatchString(r.UUID) || r.FullName == "" { + return shapeError(op, "a repository without uuid or full_name") + } + if _, _, ok := splitRepoPath(r.FullName); !ok { + return shapeError(op, "repository full_name %q is not workspace/slug", r.FullName) + } + return nil +} + +// toRepo converts an API repository. A repository without a main branch is +// taken for empty (nothing was pushed yet; to be confirmed live). Bitbucket +// archives nothing and always has pull requests; a fork has a parent. +func (c *client) toRepo(r *apiRepo) platform.Repo { + visibility := "public" + if r.IsPrivate { + visibility = "private" + } + web := r.Links.HTML.Href + if web == "" { + web = c.web + "/" + r.FullName + } + out := platform.Repo{ + Host: c.host, + ID: r.UUID, + Path: r.FullName, + WebURL: web, + Visibility: visibility, + ObjectFormat: "sha1", + Fork: r.Parent != nil, + Empty: r.MainBranch == nil || r.MainBranch.Name == "", + } + if r.MainBranch != nil { + out.DefaultBranch = r.MainBranch.Name + } + return out +} + +// Repo returns one repository by its workspace/slug path. The API takes +// paths ignoring case (slugs are lowercase); the result carries the +// canonical full_name. +func (d *reader) Repo(ctx context.Context, path string) (platform.Repo, error) { + const op = "get repository" + ws, slug, ok := splitRepoPath(path) + if !ok { + return platform.Repo{}, notFound(op, "repository %q: Bitbucket paths are workspace/repository", path) + } + r, err := d.c.getRepo(ctx, op, ws, slug) + if err != nil { + return platform.Repo{}, err + } + return d.c.toRepo(r), nil +} + +// getRepo reads GET /2.0/repositories/{workspace}/{slug}; slug may be the +// repository's uuid in braces. +func (c *client) getRepo(ctx context.Context, op, ws, slug string) (*apiRepo, error) { + var r apiRepo + if _, err := c.get(ctx, op, c.endpoint("repositories", ws, slug), nil, &r); err != nil { + return nil, err + } + if err := r.check(op); err != nil { + return nil, err + } + return &r, nil +} + +// Resolve lists the repositories sel selects, sorted by path ignoring case. +// A namespace is a workspace (GET /2.0/repositories/{workspace}, every +// repository the identity sees there); Bitbucket has no nested namespaces, +// so Subgroups changes nothing, and no topics, so a selector with topics is +// ClassInvalid (check refuses such an entry before). Forks only with +// sel.Forks. +// +// The listing is incomplete when it is capped or a later page fails; a +// failure of the first page, a rate limit, an auth error or a transient +// failure is returned instead, so the caller can retry or pause. +func (d *reader) Resolve(ctx context.Context, sel platform.Selector) (platform.Resolved, error) { + const op = "resolve" + if sel.Repo != "" { + r, err := d.Repo(ctx, sel.Repo) + if err != nil { + return platform.Resolved{}, err + } + return platform.Resolved{Repos: []platform.Repo{r}, Complete: true}, nil + } + switch { + case sel.Namespace == "": + return platform.Resolved{}, invalid(op, "the selector names no repository and no namespace") + case len(sel.Topics) > 0: + return platform.Resolved{}, &platform.Error{Op: op, Class: platform.ClassInvalid, Err: errTopics} + case strings.ContainsAny(sel.Namespace, "/\x00?#") || sel.Namespace == "." || sel.Namespace == "..": + return platform.Resolved{}, notFound(op, "workspace %q: Bitbucket has no nested namespaces", sel.Namespace) + } + var found []apiRepo + q := url.Values{"pagelen": {strconv.Itoa(repoPageLen)}} + complete, err := listAll(ctx, d.c, op, d.c.endpoint("repositories", sel.Namespace), q, maxRepoPages, func(r apiRepo) error { + if err := r.check(op); err != nil { + return err + } + found = append(found, r) + return nil + }) + if err != nil { + if !laterPage(err) || fatal(err) { + return platform.Resolved{}, err + } + complete = false + } + out := platform.Resolved{Repos: []platform.Repo{}, Complete: complete} + seen := map[string]bool{} + for i := range found { + r := &found[i] + if seen[r.UUID] || r.Parent != nil && !sel.Forks { + continue + } + seen[r.UUID] = true + out.Repos = append(out.Repos, d.c.toRepo(r)) + } + sortRepos(out.Repos) + return out, nil +} + +// errTopics is why a selector with topics is refused. +var errTopics = errors.New("repositories on Bitbucket have no topics; select them with match (workspace/svc-*) or list them by repo") + +// sortRepos orders repositories by path ignoring case, then path, then id. +func sortRepos(repos []platform.Repo) { + slices.SortFunc(repos, func(a, b platform.Repo) int { + if c := strings.Compare(strings.ToLower(a.Path), strings.ToLower(b.Path)); c != 0 { + return c + } + if c := strings.Compare(a.Path, b.Path); c != 0 { + return c + } + return strings.Compare(a.ID, b.ID) + }) +} + +// Remote is the repository's git URL under the provider's web URL +// (https://bitbucket.org/{full_name}.git), never a host the API names, with +// the credential's Basic header (none when anonymous). +func (d *reader) Remote(_ context.Context, r platform.Repo) (platform.Remote, error) { + const op = "get remote" + if r.Host != "" && !strings.EqualFold(r.Host, d.c.host) { + return platform.Remote{}, invalid(op, "repository %s is on %s, not %s", r.Path, r.Host, d.c.host) + } + ws, slug, err := repoPath(op, r) + if err != nil { + return platform.Remote{}, err + } + rem := platform.Remote{URL: d.c.web + "/" + url.PathEscape(ws) + "/" + url.PathEscape(slug) + ".git"} + if d.c.token != "" { + rem.Header = d.c.gitHeader + } + return rem, nil +} diff --git a/internal/platform/bitbucket/repos_test.go b/internal/platform/bitbucket/repos_test.go new file mode 100644 index 0000000..39c9b6a --- /dev/null +++ b/internal/platform/bitbucket/repos_test.go @@ -0,0 +1,205 @@ +package bitbucket + +import ( + "encoding/base64" + "fmt" + "net/http" + "strings" + "testing" + + "github.com/bedrock-python/touchmark/internal/auth" + "github.com/bedrock-python/touchmark/internal/httpx" + "github.com/bedrock-python/touchmark/internal/platform" +) + +func TestRepo(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api", http.StatusOK, repo(repoUUID, "acme/api")) + f.json("/repositories/acme/fork", http.StatusOK, repo(forkUUID, "acme/fork", with("is_private", false), + with("parent", repoRef(repoUUID, "acme/api")), with("mainbranch", map[string]any{"name": "develop", "type": "branch"}))) + f.json("/repositories/acme/empty", http.StatusOK, repo("{11111111-2222-4333-8444-555555555555}", "acme/empty", with("mainbranch", nil))) + f.json("/repositories/acme/gone", http.StatusNotFound, errorBody(noRepoMessage)) + + got, err := f.reader.Repo(t.Context(), "acme/api") + if err != nil { + t.Fatal(err) + } + want := platform.Repo{Host: f.provider().Host, ID: repoUUID, Path: "acme/api", DefaultBranch: "main", + WebURL: "https://bitbucket.org/acme/api", Visibility: "private", ObjectFormat: "sha1"} + if fmt.Sprint(got) != fmt.Sprint(want) { + t.Errorf("Repo = %+v\nwant %+v", got, want) + } + fork, err := f.reader.Repo(t.Context(), "acme/fork") + if err != nil || !fork.Fork || fork.Visibility != "public" || fork.DefaultBranch != "develop" || fork.Empty { + t.Errorf("fork = %+v, %v", fork, err) + } + empty, err := f.reader.Repo(t.Context(), "acme/empty") + if err != nil || !empty.Empty || empty.DefaultBranch != "" { + t.Errorf("a repository without a main branch = %+v, %v; want empty", empty, err) + } + _, err = f.reader.Repo(t.Context(), "acme/gone") + wantClass(t, "a missing repository", err, platform.ClassNotFound, platform.ErrNotFound) + if err != nil && !strings.Contains(err.Error(), "no longer exists") { + t.Errorf("the error %q lacks the API's message", err) + } + f.reset() + for _, p := range []string{"acme", "acme/a/b", "/api", "acme/..", ""} { + _, err := f.reader.Repo(t.Context(), p) + wantClass(t, "Repo("+p+")", err, platform.ClassNotFound, platform.ErrNotFound) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("paths that name no repository sent %d requests", len(calls)) + } +} + +func TestRepoShape(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api", http.StatusOK, repo("", "acme/api")) + _, err := f.reader.Repo(t.Context(), "acme/api") + wantClass(t, "a repository without uuid", err, platform.ClassUnknown, nil) +} + +// workspaceRepos are the repositories of workspace acme, out of order, with +// a fork. +func workspaceRepos() []any { + return []any{ + repo("{00000000-0000-4000-8000-000000000003}", "acme/web"), + repo("{00000000-0000-4000-8000-000000000001}", "acme/API"), + repo(forkUUID, "acme/api-fork", with("parent", repoRef(repoUUID, "other/api"))), + repo("{00000000-0000-4000-8000-000000000002}", "acme/billing"), + repo("{00000000-0000-4000-8000-000000000004}", "acme/zeta"), + } +} + +func TestResolve(t *testing.T) { + f := newFixture(t) + f.pages("/repositories/acme", 2, workspaceRepos()) + got, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme", Subgroups: true}) + if err != nil { + t.Fatal(err) + } + var paths []string + for _, r := range got.Repos { + paths = append(paths, r.Path) + } + if !got.Complete || strings.Join(paths, " ") != "acme/API acme/billing acme/web acme/zeta" { + t.Errorf("Resolve = %v, complete %v; want the four repositories that are no fork, sorted ignoring case", paths, got.Complete) + } + calls := f.requests(http.MethodGet, "/repositories/acme") + if len(calls) != 3 || calls[0].Query.Get("pagelen") != "100" || calls[2].Query.Get("page") != "3" { + t.Errorf("calls %+v; want three pages, the next links followed", calls) + } + withForks, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme", Forks: true}) + if err != nil || len(withForks.Repos) != 5 { + t.Errorf("with forks: %d repositories, %v; want five", len(withForks.Repos), err) + } +} + +func TestResolveRepo(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme/api", http.StatusOK, repo(repoUUID, "acme/api", with("parent", repoRef(forkUUID, "other/api")))) + got, err := f.reader.Resolve(t.Context(), platform.Selector{Repo: "acme/api", Topics: []string{"ignored"}}) + if err != nil || !got.Complete || len(got.Repos) != 1 || got.Repos[0].ID != repoUUID { + t.Errorf("Resolve(repo) = %+v, %v; want the repository, a fork too", got, err) + } +} + +func TestResolveErrors(t *testing.T) { + t.Run("topics", func(t *testing.T) { + f := newFixture(t) + _, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme", Topics: []string{"python"}}) + wantClass(t, "topics", err, platform.ClassInvalid, nil) + if err != nil && !strings.Contains(err.Error(), "no topics") { + t.Errorf("error %q; want it to say Bitbucket has no topics", err) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("%d requests for a selector with topics", len(calls)) + } + }) + t.Run("no namespace", func(t *testing.T) { + f := newFixture(t) + _, err := f.reader.Resolve(t.Context(), platform.Selector{}) + wantClass(t, "empty", err, platform.ClassInvalid, nil) + _, err = f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme/sub"}) + wantClass(t, "nested", err, platform.ClassNotFound, platform.ErrNotFound) + }) + t.Run("missing workspace", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/gone", http.StatusNotFound, errorBody("No workspace with identifier 'gone'.")) + _, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "gone"}) + wantClass(t, "a missing workspace", err, platform.ClassNotFound, platform.ErrNotFound) + }) + t.Run("a later page fails", func(t *testing.T) { + f := newFixture(t) + f.handle(http.MethodGet, "/repositories/acme", func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("page") == "2" { + writeJSON(w, http.StatusForbidden, errorBody("Forbidden")) + return + } + servePage(w, r, 2, workspaceRepos()) + }) + got, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme"}) + if err != nil || got.Complete || len(got.Repos) != 2 { + t.Errorf("Resolve = %+v, %v; want the first page, incomplete", got, err) + } + }) + t.Run("a later page is rate limited", func(t *testing.T) { + f := newFixture(t) + f.handle(http.MethodGet, "/repositories/acme", func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("page") == "2" { + w.Header().Set("X-RateLimit-Limit", "1000") + w.Header().Set("X-RateLimit-Remaining", "0") + w.Header().Set("X-RateLimit-Reset", "394") + writeJSON(w, http.StatusTooManyRequests, errorBody("Rate limit for this resource has been exceeded")) + return + } + servePage(w, r, 2, workspaceRepos()) + }) + _, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme"}) + wantClass(t, "rate limited", err, platform.ClassRateLimited, nil) + }) + t.Run("a next link elsewhere", func(t *testing.T) { + f := newFixture(t) + f.json("/repositories/acme", http.StatusOK, map[string]any{ + "values": workspaceRepos()[:1], "pagelen": 1, "next": "https://evil.example.com/2.0/repositories/acme?page=2", + }) + _, err := f.reader.Resolve(t.Context(), platform.Selector{Namespace: "acme"}) + wantClass(t, "a next link to another host", err, platform.ClassUnknown, nil) + if len(f.requests("", "")) != 1 { + t.Error("the driver followed a next link to another host") + } + }) +} + +func TestRemote(t *testing.T) { + f := newFixture(t) + r := platform.Repo{Host: f.provider().Host, ID: repoUUID, Path: "acme/api"} + rem, err := f.reader.Remote(t.Context(), r) + if err != nil { + t.Fatal(err) + } + if rem.URL != f.base()+"/acme/api.git" || rem.Header == nil { + t.Fatalf("Remote = %+v; want the web URL with .git and a header", rem) + } + h, err := rem.Header(t.Context()) + want := "Basic " + base64.StdEncoding.EncodeToString([]byte("x-bitbucket-api-token-auth:"+f.token)) + if err != nil || h != want { + t.Errorf("header %q, %v; want git's Basic form of an API token", h, err) + } + if _, err := f.reader.Remote(t.Context(), platform.Repo{Host: "bitbucket.example.com", Path: "acme/api"}); err == nil { + t.Error("a repository of another host got a remote") + } + if _, err := f.reader.Remote(t.Context(), platform.Repo{Path: "acme"}); err == nil { + t.Error("a path that is no workspace/repository got a remote") + } + anon, err := NewReader(f.provider(), auth.Credential{}, httpx.New(httpx.Options{})) + if err != nil { + t.Fatal(err) + } + if rem, err := anon.Remote(t.Context(), r); err != nil || rem.Header != nil { + t.Errorf("anonymous Remote = %+v, %v; want no header", rem, err) + } + if calls := f.requests("", ""); len(calls) != 0 { + t.Errorf("Remote sent %d requests", len(calls)) + } +} diff --git a/internal/platform/platform.go b/internal/platform/platform.go index b7c0b05..dea292a 100644 --- a/internal/platform/platform.go +++ b/internal/platform/platform.go @@ -1,5 +1,5 @@ // Package platform defines what touchmark needs from a code hosting platform -// (GitHub, GitLab, Gitea, Forgejo; later Bitbucket and Azure DevOps): the +// (GitHub, GitLab, Gitea, Forgejo, Bitbucket Cloud; later Azure DevOps): the // Reader and Writer interfaces drivers implement, their capabilities, and one // error model. // @@ -257,6 +257,13 @@ type MarkerStore uint8 const ( MarkerInBody MarkerStore = iota // last line of the description MarkerInProperties // PR properties (Azure DevOps, later) + // MarkerInRefDef is the last line of the description too, with the + // payload of MarkerInBody wrapped in a Markdown link reference + // definition, `[touchmark]: # "touchmark:v1 …"`, which renderers do not + // show: Bitbucket Cloud escapes HTML in descriptions, so an HTML comment + // would show as text. The codec of this wrapper comes with the + // Bitbucket writer. + MarkerInRefDef ) // Limits are a provider's default pacing (internal/throttle applies them, @@ -273,7 +280,8 @@ type Limits struct { // Caps describes a platform instance, found once per run by Reader.Probe. type Caps struct { - // Flavor is "github", "ghe.com", "ghes", "gitlab", "gitea" or "forgejo". + // Flavor is "github", "ghe.com", "ghes", "gitlab", "gitea", "forgejo" + // or "bitbucket" (Bitbucket Cloud). Flavor string Version string // MaxBody is the body budget in bytes, marker included. From 8c6320c18ac2a3e77b8e188d9b91a144e67917d3 Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Thu, 8 Oct 2026 23:40:37 +0300 Subject: [PATCH 03/20] test(report): an unknown provider type is azure-devops now that bitbucket is one --- internal/report/doctor_test.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/internal/report/doctor_test.go b/internal/report/doctor_test.go index fc5df1e..13a84dc 100644 --- a/internal/report/doctor_test.go +++ b/internal/report/doctor_test.go @@ -99,7 +99,7 @@ func TestDoctorSchema(t *testing.T) { bad := map[string]func(*Doctor){ "status": func(d *Doctor) { d.HubChecks[0].Status = "red" }, "check": func(d *Doctor) { d.HubChecks[0].Name = "Write Isolation" }, - "provider": func(d *Doctor) { d.Providers[0].Type = "bitbucket" }, + "provider": func(d *Doctor) { d.Providers[0].Type = "azure-devops" }, "summary": func(d *Doctor) { delete(d.Summary, StatusUnknown) }, "schema": func(d *Doctor) { d.Schema = "doctor/v2" }, } From 6605bf53a874889a6294d01310cba357a60de982 Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Thu, 8 Oct 2026 23:41:14 +0300 Subject: [PATCH 04/20] docs: say that Bitbucket Cloud is read only for now --- docs/agents.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/agents.md b/docs/agents.md index bf0b73b..e6e99a6 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -47,7 +47,8 @@ no lock file, no header, no marker in their files. It never checks a target out and never runs a script, hook, filter or LFS from the hub or a target. It delivers nothing to a repository without the opt-in file, unless `targets.yml` subscribes it (`opt_in: assumed`). There is no pull mode (targets do not -fetch from the hub), no remote pack sources, and no Bitbucket or Azure DevOps driver. +fetch from the hub), no remote pack sources, no Azure DevOps driver, and Bitbucket +Cloud is read only: `plan` reads it, delivering to it comes later. ## Mental model From b247dab5a95de416aa2d6e0438ab64fc8884806e Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Fri, 9 Oct 2026 00:01:20 +0300 Subject: [PATCH 05/20] feat(platform): add Caps.NoLabels for platforms without pull request labels --- internal/platform/platform.go | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/internal/platform/platform.go b/internal/platform/platform.go index dea292a..368bfb9 100644 --- a/internal/platform/platform.go +++ b/internal/platform/platform.go @@ -261,8 +261,8 @@ const ( // payload of MarkerInBody wrapped in a Markdown link reference // definition, `[touchmark]: # "touchmark:v1 …"`, which renderers do not // show: Bitbucket Cloud escapes HTML in descriptions, so an HTML comment - // would show as text. The codec of this wrapper comes with the - // Bitbucket writer. + // would show as text (marker.FrameRefDef; the core reads both frames on + // every platform). MarkerInRefDef ) @@ -296,6 +296,11 @@ type Caps struct { // are refused or dropped silently (Gitea and Forgejo answer 422 to // names when a pull request is created, and drop unknown ids). LabelsByID bool + // NoLabels is set when the platform has no pull request labels + // (Bitbucket Cloud): the core asks for none (NewPR.Labels, + // PREdit.AddLabels and EnsureLabels stay empty, the marker records + // none set) and reports none, and the driver ignores any it is given. + NoLabels bool // CloserKnown is set when PR.ClosedBy is reliable. CloserKnown bool Marker MarkerStore From 874006675baa809103e2e50d5ebdb9d9de5ba026 Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Fri, 9 Oct 2026 00:01:21 +0300 Subject: [PATCH 06/20] feat(marker): read and write the marker as a Markdown reference definition Bitbucket Cloud escapes HTML in descriptions, so the marker's HTML comment would show as text. EncodeFrame writes the same payload in a link reference definition, [touchmark]: # "touchmark:v1 ...", which renderers hide. Find, Strip and Parse read both frames, with CRLF line endings and the backslash escapes Bitbucket adds after a web edit undone. prbody accepts the new frame as Render's marker, keeps it as a marker line and reserves it in the intro. --- internal/marker/find.go | 19 +- internal/marker/frame.go | 103 +++++++ internal/marker/frame_test.go | 274 ++++++++++++++++++ internal/marker/fuzz_test.go | 26 +- internal/marker/marker.go | 45 ++- .../marker/testdata/encode/sync-open.refdef | 1 + internal/prbody/controls.go | 4 +- internal/prbody/helpers_test.go | 56 +++- internal/prbody/intro.go | 10 + internal/prbody/prbody.go | 19 +- internal/prbody/render_test.go | 8 +- internal/prbody/testdata/assumed/bitbucket.md | 19 ++ .../prbody/testdata/declined/bitbucket.md | 21 ++ internal/prbody/testdata/huge/bitbucket.md | 235 +++++++++++++++ internal/prbody/testdata/local/bitbucket.md | 27 ++ internal/prbody/testdata/new/bitbucket.md | 30 ++ .../prbody/testdata/nothing-more/bitbucket.md | 20 ++ internal/prbody/testdata/paused/bitbucket.md | 34 +++ .../testdata/sensitive-only/bitbucket.md | 32 ++ 19 files changed, 938 insertions(+), 45 deletions(-) create mode 100644 internal/marker/frame.go create mode 100644 internal/marker/frame_test.go create mode 100644 internal/marker/testdata/encode/sync-open.refdef create mode 100644 internal/prbody/testdata/assumed/bitbucket.md create mode 100644 internal/prbody/testdata/declined/bitbucket.md create mode 100644 internal/prbody/testdata/huge/bitbucket.md create mode 100644 internal/prbody/testdata/local/bitbucket.md create mode 100644 internal/prbody/testdata/new/bitbucket.md create mode 100644 internal/prbody/testdata/nothing-more/bitbucket.md create mode 100644 internal/prbody/testdata/paused/bitbucket.md create mode 100644 internal/prbody/testdata/sensitive-only/bitbucket.md diff --git a/internal/marker/find.go b/internal/marker/find.go index c850a69..5c4e2ab 100644 --- a/internal/marker/find.go +++ b/internal/marker/find.go @@ -12,7 +12,7 @@ const scanLimit = 1 << 20 type Status uint8 const ( - // None: no " + FrameComment Frame = iota + // FrameRefDef is a Markdown link reference definition whose title is + // the payload, which renderers hide too, for platforms that escape HTML + // in descriptions (Bitbucket Cloud), where a comment would show as + // text: + // + // [touchmark]: # "touchmark:v1 hub=… fp=… stream=… key=… data=…" + FrameRefDef +) + +// Parts of the frames. +const ( + // commentOpen is what a comment frame puts before the payload. + commentOpen = "") are not marker lines. +func IsLine(line string) bool { + _, ok := asComment(line) + return ok +} + +// asComment returns line in the comment frame, which Parse and Find read: +// line itself when it starts with "" +// (its payload left open when the closing quote is missing, so that Parse +// refuses it). ok is false for any other line. Nothing is trimmed: the +// callers trim the trailing blanks and carriage return they accept. +// +// A reference definition is read after unescape: Bitbucket Cloud returns an +// edited description with backslashes before punctuation ("\[touchmark\]", +// "\_"), which Markdown drops when it renders. +func asComment(line string) (string, bool) { + if strings.HasPrefix(line, commentPrefix) { + return line, true + } + if !strings.HasPrefix(line, "[") && !strings.HasPrefix(line, `\[`) { + return "", false + } + rest, ok := strings.CutPrefix(unescape(line), refDefOpen) + if !ok || !strings.HasPrefix(rest, payloadPrefix) { + return "", false + } + if payload, ok := strings.CutSuffix(rest, refDefClose); ok { + return commentOpen + payload + commentEnd, true + } + return commentOpen + rest, true +} + +// unescape undoes Markdown's backslash escapes in s: a backslash before +// an ASCII punctuation character is dropped (CommonMark, "Backslash +// escapes"); every other byte stays, a backslash before anything else +// included. +func unescape(s string) string { + if !strings.Contains(s, `\`) { + return s + } + var b strings.Builder + b.Grow(len(s)) + for i := 0; i < len(s); i++ { + if s[i] == '\\' && i+1 < len(s) && isASCIIPunct(s[i+1]) { + i++ + } + b.WriteByte(s[i]) + } + return b.String() +} + +// isASCIIPunct reports whether c is one of CommonMark's ASCII punctuation +// characters: !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~. +func isASCIIPunct(c byte) bool { + return '!' <= c && c <= '/' || ':' <= c && c <= '@' || '[' <= c && c <= '`' || '{' <= c && c <= '~' +} diff --git a/internal/marker/frame_test.go b/internal/marker/frame_test.go new file mode 100644 index 0000000..5261268 --- /dev/null +++ b/internal/marker/frame_test.go @@ -0,0 +1,274 @@ +package marker + +import ( + "fmt" + "math/rand/v2" + "strings" + "testing" +) + +// refDef returns line, a marker in the comment frame, as a reference +// definition, as frame writes it. +func refDef(line string) string { return frame(line, FrameRefDef) } + +// escapeAll puts a backslash before every ASCII punctuation character of +// s: the most an editor can add (Bitbucket adds some, "\[", "\_", "\{"). +func escapeAll(s string) string { + var b strings.Builder + for i := 0; i < len(s); i++ { + if isASCIIPunct(s[i]) { + b.WriteByte('\\') + } + b.WriteByte(s[i]) + } + return b.String() +} + +func TestEncodeFrameRefDef(t *testing.T) { + for _, v := range goldenVectors() { + t.Run(v.name, func(t *testing.T) { + comment := mustEncode(t, v.m) + line, err := EncodeFrame(v.m, FrameRefDef) + if err != nil { + t.Fatal(err) + } + payload := strings.TrimSuffix(strings.TrimPrefix(comment, "") + if want := `[touchmark]: # "` + payload + `"`; line != want { + t.Fatalf("EncodeFrame = %.120q…, want %.120q…", line, want) + } + if strings.ContainsAny(payload, "\"\r\n") { + t.Fatal("the payload holds a quote or a line break, which would end the title") + } + got, err := Parse(line) + if err != nil { + t.Fatalf("Parse(refdef): %v", err) + } + equalMarkers(t, got, written(v.m)) + if again, _ := EncodeFrame(v.m, FrameComment); again != comment { + t.Error("EncodeFrame(FrameComment) differs from Encode") + } + }) + } +} + +// TestEncodeFrameGolden pins the reference definition frame around the +// payload of a golden vector. +func TestEncodeFrameGolden(t *testing.T) { + v := goldenVectors()[0] + line, err := EncodeFrame(v.m, FrameRefDef) + if err != nil { + t.Fatal(err) + } + if *update { + golden(t, "encode/"+v.name+".refdef", line+"\n") + return + } + pinned := strings.TrimSuffix(readGolden(t, "encode/"+v.name+".refdef"), "\n") + if !strings.HasPrefix(pinned, `[touchmark]: # "touchmark:v1 hub=`) || !strings.HasSuffix(pinned, `"`) { + t.Fatalf("golden line %.60q… is not a reference definition", pinned) + } + got, err := Parse(pinned) + if err != nil { + t.Fatalf("Parse(golden line): %v", err) + } + equalMarkers(t, got, written(v.m)) + comment := strings.TrimSuffix(readGolden(t, "encode/"+v.name+".marker"), "\n") + if refDef(comment) != pinned { + t.Error("the golden reference definition does not frame the golden comment's payload") + } +} + +func TestParseRefDef(t *testing.T) { + m := sampleMarker() + comment := mustEncode(t, m) + line := refDef(comment) + want := written(m) + accepted := []struct{ name, line string }{ + {"as written", line}, + {"brackets escaped", strings.Replace(strings.Replace(line, "[", `\[`, 1), "]", `\]`, 1)}, + {"every punctuation escaped", escapeAll(line)}, + {"hash and quotes escaped", strings.NewReplacer("#", `\#`, `"`, `\"`).Replace(line)}, + } + for _, c := range accepted { + t.Run(c.name, func(t *testing.T) { + got, err := Parse(c.line) + if err != nil { + t.Fatalf("Parse: %v", err) + } + equalMarkers(t, got, want) + }) + } + refused := []struct{ name, line string }{ + {"no closing quote", strings.TrimSuffix(line, `"`)}, + {"trailing blank", line + " "}, + {"CR", line + "\r"}, + {"single quotes", strings.Replace(strings.Replace(line, `# "`, `# '`, 1), `"`, `'`, 1)}, + {"another label", strings.Replace(line, "[touchmark]", "[touchmark2]", 1)}, + {"another destination", strings.Replace(line, `]: # "`, `]: #x "`, 1)}, + {"indented", " " + line}, + {"escaped backslash before the label", `\\` + line}, + {"letter escaped", strings.Replace(line, "hub=", `h\ub=`, 1)}, + {"comment inside", strings.Replace(line, `"touchmark:v1 `, `"", true}, + {"- [ ] Rebuild this branch", false}, + {`[touchmark]: # "something else"`, false}, + {`[touchmark]: https://example.com "touchmark:v1"`, false}, + {"[touchmark]", false}, + {" " + line, false}, + {"see " + line, false}, + {"", false}, + {`\`, false}, + {"[", false}, + } + for _, c := range cases { + if got := IsLine(c.line); got != c.want { + t.Errorf("IsLine(%.60q) = %v, want %v", c.line, got, c.want) + } + } +} + +func TestUnescape(t *testing.T) { + cases := []struct{ in, want string }{ + {"", ""}, + {"plain", "plain"}, + {`\[a\]`, "[a]"}, + {`a\_b\*c\#d\(e\)f`, "a_b*c#d(e)f"}, + {`\\`, `\`}, + {`\\\[`, `\[`}, + {`\a\1\ \é`, `\a\1\ \é`}, + {`trailing\`, `trailing\`}, + {`\{\}\~\!\/\:\@\` + "`", "{}~!/:@`"}, + } + for _, c := range cases { + if got := unescape(c.in); got != c.want { + t.Errorf("unescape(%q) = %q, want %q", c.in, got, c.want) + } + } + for c := 0; c < 128; c++ { + want := strings.ContainsRune("!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~", rune(c)) + if got := isASCIIPunct(byte(c)); got != want { + t.Errorf("isASCIIPunct(%q) = %v, want %v", rune(c), got, want) + } + } +} + +func TestFindRefDef(t *testing.T) { + l := newFindLines(t) + fps := []string{ourFP, prevFP} + ours, ours2 := parsed(t, l.ours), parsed(t, l.ours2) + rd := refDef(l.ours) + cases := []struct { + name string + body string + want Status + marker Marker + }{ + {"ours last", "text\n\n" + rd, Found, ours}, + {"CRLF", "text\r\n\r\n" + rd + "\r\n", Found, ours}, + {"escaped by a web edit, CRLF", "text\\_with\\_escapes\r\n\r\n" + escapeAll(rd) + " \r\n", Found, ours}, + {"comment after refdef", rd + "\n" + l.ours2, Found, ours2}, + {"refdef after comment", l.ours2 + "\n" + rd, Found, ours}, + {"foreign refdef", "text\n" + refDef(l.foreign), Foreign, Marker{}}, + {"v2 refdef", refDef(l.v2), Foreign, Marker{}}, + {"broken refdef", "text\n" + strings.TrimSuffix(rd, `"`), Invalid, Marker{}}, + {"tampered refdef", refDef(l.tampered), Invalid, Marker{}}, + {"indented refdef", " " + rd, None, Marker{}}, + {"refdef in a quote", "> " + rd, None, Marker{}}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + m, status := Find(c.body, fps) + if status != c.want { + t.Fatalf("status = %v, want %v", status, c.want) + } + equalMarkers(t, m, c.marker) + }) + } +} + +func TestStripRefDef(t *testing.T) { + l := newFindLines(t) + rd := refDef(l.ours) + cases := []struct{ name, body, want string }{ + {"refdef last", "text\n\n" + rd, "text"}, + {"CRLF", "a\r\nb\r\n\r\n" + rd + "\r\n", "a\r\nb"}, + {"escaped", "a\\_b\n\n" + escapeAll(rd), "a\\_b"}, + {"both frames", "a\n" + l.ours + "\n" + rd + "\nb", "a\nb"}, + {"another reference stays", "a\n\n[docs]: https://example.com \"Docs\"\n" + rd, "a\n\n[docs]: https://example.com \"Docs\""}, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + if got := Strip(c.body); got != c.want { + t.Errorf("Strip = %q, want %q", got, c.want) + } + }) + } +} + +// TestEncodeFrameMeasuresTheFramedLine: a marker whose comment fits +// MaxLine but whose reference definition, 8 bytes longer, does not, loses +// its changes in the reference definition frame only. +func TestEncodeFrameMeasuresTheFramedLine(t *testing.T) { + extra := len(refDefOpen) + len(refDefClose) - len(commentOpen) - len(commentEnd) + r := rand.New(rand.NewPCG(5, 6)) + const letters = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789+/" + random := func(n int) string { + b := make([]byte, n) + for i := range b { + b[i] = letters[r.IntN(len(letters))] + } + return string(b) + } + base := sampleMarker() + base.Data.Changes = []Change{{Path: random(40), Mode: "100644", To: fmt.Sprintf("%016x", r.Uint64())}} + fixed := []string{random(MaxString), random(MaxString), random(MaxString)} + tail := random(MaxString) + for n := 1000; n <= MaxString; n++ { + m := base + m.Data.Packs = append(append([]string{}, fixed...), tail[:n]) + comment, err := Encode(m) + if err != nil { + t.Fatal(err) + } + if len(comment) <= MaxLine-extra || len(comment) > MaxLine { + continue + } + if got := parsed(t, comment); len(got.Data.Changes) != 1 { + t.Fatalf("the comment lost its changes at %d bytes", len(comment)) + } + line, err := EncodeFrame(m, FrameRefDef) + if err != nil { + t.Fatal(err) + } + if len(line) > MaxLine { + t.Fatalf("a reference definition of %d bytes, more than %d", len(line), MaxLine) + } + equalMarkers(t, parsed(t, line), dropped(m)) + return + } + t.Fatal("fixture: no pack length put the comment within the last bytes of MaxLine") +} diff --git a/internal/marker/fuzz_test.go b/internal/marker/fuzz_test.go index 3bf049a..9b4be79 100644 --- a/internal/marker/fuzz_test.go +++ b/internal/marker/fuzz_test.go @@ -15,6 +15,8 @@ func seedLines(t testing.TB) []string { for _, v := range goldenVectors() { line := mustEncode(t, v.m) lines = append(lines, line, line[:len(line)/2], strings.Replace(line, "v1", "v2", 1), line+"\r") + rd := refDef(line) + lines = append(lines, rd, escapeAll(rd), rd[:len(rd)/2], rd+"\r") } lines = append(lines, lineWithData(gz(t, bytes.Repeat([]byte{' '}, 8<<20))), @@ -22,6 +24,8 @@ func seedLines(t testing.TB) []string { lineWithJSON(t, "{}"), "", "", + `[touchmark]: # "touchmark:v1 hub= fp= stream= key= data="`, + `\[touchmark\]: \# "touchmark:v1\"`, ) return lines } @@ -89,6 +93,22 @@ func FuzzEncode(f *testing.F) { if len(line) > MaxLine { t.Fatalf("line of %d bytes", len(line)) } + rd, err := EncodeFrame(m, FrameRefDef) + switch { + case err != nil && !errors.Is(err, errTooLarge): + t.Fatalf("EncodeFrame(FrameRefDef) of an encodable marker: %v", err) + case err != nil: + case len(rd) > MaxLine: + t.Fatalf("reference definition of %d bytes", len(rd)) + default: + back, err := Parse(rd) + if err != nil { + t.Fatalf("Parse(EncodeFrame(m, FrameRefDef)): %v", err) + } + if !reflect.DeepEqual(back, written(m)) && !reflect.DeepEqual(back, dropped(m)) { + t.Fatalf("reference definition round trip differs\n got %s", describeMarker(back)) + } + } got, err := Parse(line) if err != nil { t.Fatalf("Parse(Encode(m)): %v\nline %.300q", err, line) @@ -137,7 +157,7 @@ func FuzzFind(f *testing.F) { t.Fatal("Strip is not idempotent") } for line := range strings.SplitSeq(stripped, "\n") { - if strings.HasPrefix(line, commentPrefix) { + if IsLine(line) { t.Fatalf("Strip left a marker line: %.80q", line) } } @@ -155,5 +175,9 @@ func FuzzFind(f *testing.F) { if Strip(appended) != stripped { t.Fatalf("Strip(body + marker) = %q, want %q", Strip(appended), stripped) } + appended = body + "\r\n" + escapeAll(refDef(last)) + " \r\n" + if got, status := Find(appended, fps); status != Found || !reflect.DeepEqual(got, want) { + t.Fatalf("appended reference definition: status %v", status) + } }) } diff --git a/internal/marker/marker.go b/internal/marker/marker.go index 7fbeafe..8b3ad6d 100644 --- a/internal/marker/marker.go +++ b/internal/marker/marker.go @@ -3,6 +3,11 @@ // // // +// or, where a platform escapes HTML in descriptions (Bitbucket Cloud), the +// same payload in a Markdown link reference definition (FrameRefDef): +// +// [touchmark]: # "touchmark:v1 hub= fp=<16 hex> stream= key=sha256:<64 hex> data=" +// // The marker is the only memory touchmark has: what the PR carried, who // the hub is, and what the engine wrote last. Everything in a PR body is // untrusted input, so parsing is strict and bounded. @@ -28,7 +33,10 @@ import ( const ( // Version is the marker format version ("touchmark:v1"). Version = 1 - // MaxLine is the largest marker comment accepted or written. + // MaxLine is the largest marker line written, in either frame, and the + // largest accepted in the comment frame. A reference definition is + // accepted up to twice as long before its backslash escapes are undone + // (an escape per byte at most), and up to MaxLine after. MaxLine = 16 << 10 // MaxData is the largest decompressed JSON accepted. MaxData = 64 << 10 @@ -139,18 +147,23 @@ func FP16(fingerprint string) string { return hex.EncodeToString(sum[:8]) } -// Encode returns the marker comment line (no trailing newline). +// Encode returns the marker line in the comment frame (no trailing +// newline): EncodeFrame(m, FrameComment). +func Encode(m Marker) (string, error) { return EncodeFrame(m, FrameComment) } + +// EncodeFrame returns the marker line in frame f (no trailing newline). // // It fills Hub, FP16 and Stream from Data, validates every field with the // rules of Parse (and every string as valid UTF-8, which JSON needs to // round-trip), and enforces the limits: when there are more than MaxChanges // changes, the JSON exceeds MaxJSON or the line exceeds MaxLine, it drops // Changes and sets ChangesComplete to false; if the JSON still exceeds -// MaxJSON or the line MaxLine it fails. Output is deterministic: JSON with -// empty arrays for nil slices, gzip at the best compression level without a -// timestamp or name, standard padded base64. Parse accepts every line -// Encode returns and yields the marker it encoded. -func Encode(m Marker) (string, error) { +// MaxJSON or the line MaxLine it fails; the line is measured in frame f. +// Output is deterministic: JSON with empty arrays for nil slices, gzip at +// the best compression level without a timestamp or name, standard padded +// base64. Parse accepts every line EncodeFrame returns and yields the +// marker it encoded. +func EncodeFrame(m Marker, f Frame) (string, error) { m.Hub, m.Stream, m.FP16 = m.Data.Hub, m.Data.Stream, FP16(m.Data.FP) if err := checkHeader(m); err != nil { return "", fmt.Errorf("encode marker: %w", err) @@ -166,11 +179,13 @@ func Encode(m Marker) (string, error) { if err != nil { return "", fmt.Errorf("encode marker: %w", err) } + line = frame(line, f) if (size > MaxJSON || len(line) > MaxLine) && len(d.Changes) > 0 { d.Changes, d.ChangesComplete = nil, false if line, size, err = encodeLine(m, d); err != nil { return "", fmt.Errorf("encode marker: %w", err) } + line = frame(line, f) } switch { case size > MaxJSON: @@ -231,9 +246,11 @@ func wire(d Data) Data { return d } -// Parse parses one marker comment strictly: exact prefix and attribute -// order separated by single spaces, base64 that decodes (standard alphabet, -// padded, canonical), one gzip member without a name, comment, extra field +// Parse parses one marker line strictly, in either frame: a reference +// definition is read as the comment it frames (see IsLine), its backslash +// escapes undone, and the comment is parsed. Strictly means: exact prefix +// and attribute order separated by single spaces, base64 that decodes +// (standard alphabet, padded, canonical), one gzip member without a name, comment, extra field // or time and with nothing after it, that decompresses to at most MaxData // bytes (read through a limited reader), UTF-8 JSON with exactly the keys of // Data (compared case-sensitively, none repeated, no unknown fields, none @@ -257,6 +274,14 @@ func Parse(line string) (Marker, error) { } func parse(line string) (Marker, error) { + if !strings.HasPrefix(line, commentPrefix) { + if len(line) > 2*MaxLine { + return Marker{}, fmt.Errorf("the line is %d bytes, more than %d", len(line), 2*MaxLine) + } + if c, ok := asComment(line); ok { + line = c + } + } if len(line) > MaxLine { return Marker{}, fmt.Errorf("the line is %d bytes, more than %d", len(line), MaxLine) } diff --git a/internal/marker/testdata/encode/sync-open.refdef b/internal/marker/testdata/encode/sync-open.refdef new file mode 100644 index 0000000..c2b56d8 --- /dev/null +++ b/internal/marker/testdata/encode/sync-open.refdef @@ -0,0 +1 @@ +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5xSzYrcPBB8laXP2ln9y/btO3zklktyG4ah1WqNzfgP2zNhCXn3oDWbncCeAgYjuqpLqqqfcIdGCVi3hXGABtbXkUBAe4vQANLAzzxeQECeoYFLt7W3eKBpeAlKG+t8qHbweeF5+htRyC88XrqReenGyzOuK28rCEhMXeJ0xg0aMFmTwlinikP26KIlkzSrLLGOFYXk2WWLBgTQNG48bmeahqErXKxV1GSSZZe9DKrStUEbHfkUuMq1xH0OAiKuDA3USbLKGk205JLnkCtZK9TRkE2OfQ6yUrWWIGCat24sjrSonW8cB5RRkU6GbXbSq6ArU1t00VNIFddZSqW0NsZa57wPoarqGjFGolSusHsBDciDPhSFGem6QnMEvPD45gz1eEsMJwHU4njhMj3Cf1/+//r922EoS0LAKEklzSYXbRCgpPTWgoAqG1Ioo+aQLHoFJ3GEeZmGeVtfpj7tG2qUKpI2ia3Le4Dle8QufO/4xw5/FCgePFr1Rlpp6QqJWqbrYW1BwLvv77nsO4Jzn81OH68tyc49bwzNttxYwNZtPZ9XLmFTOy3cPJWCPj3U6ulPreKUXj/yIpkzs5T/+gcBPUbu113+CJ9U+SQA6QpNxn5lAQvfpyunhzMtjBuf87RAM976/tfvAAAA//+h3bfxcAMAAA==" diff --git a/internal/prbody/controls.go b/internal/prbody/controls.go index a7ab3f5..e3699fd 100644 --- a/internal/prbody/controls.go +++ b/internal/prbody/controls.go @@ -28,7 +28,7 @@ func ControlLine(control string) string { // a closed PR's body). A body that already has a line of control, ticked or // not, and an unknown control leave body as it is. // -// Marker lines are the lines that start with "" + end := " -->" + if !strings.HasPrefix(line, "" line -// are errors too. +// URL, or a Marker that is neither "" nor one marker line in a frame of +// marker.EncodeFrame are errors too. func Render(in Input) (string, error) { r, err := newRenderer(in) if err != nil { @@ -810,11 +811,19 @@ func moreLine(shown, more int, one, many string) string { } // checkMarker accepts one "" line whose comment ends -// only at the end of the line. +// only at the end of the line, or one `[touchmark]: # "touchmark:…"` +// reference definition whose title ends only at the end of the line (the +// two frames of marker.EncodeFrame). func checkMarker(m string) error { + const refDef = `[touchmark]: # "touchmark:` switch { case strings.ContainsAny(m, "\r\n"): return errors.New("the marker is not one line") + case strings.HasPrefix(m, refDef): + if !strings.HasSuffix(m, `"`) || strings.IndexByte(m[len(refDef):], '"') != len(m)-len(refDef)-1 || + strings.Contains(m, `\`) { + return fmt.Errorf("%.40q… is not a marker reference definition", m) + } case !strings.HasPrefix(m, "") || strings.Index(m, "-->") != len(m)-len("-->"): return fmt.Errorf("%.40q… is not a marker comment", m) } diff --git a/internal/prbody/render_test.go b/internal/prbody/render_test.go index f1d5c05..1be9605 100644 --- a/internal/prbody/render_test.go +++ b/internal/prbody/render_test.go @@ -25,7 +25,7 @@ func base(t testing.TB, flavor string) Input { Packs: []string{"agents", "claude", "base"}, OptInFile: optInName, Caps: capsOf(flavor), - Marker: testMarker(t), + Marker: markerFor(t, flavor), } } @@ -154,7 +154,7 @@ func checkBody(t *testing.T, in Input, body string) { if in.Marker == "" { full = body + "\n\n" + longestMarker for _, line := range strings.Split(body, "\n") { - if strings.HasPrefix(line, "" }, nil, "not a marker"}, {"marker ends early", func(in *Input) { in.Marker = " x -->" }, nil, "not a marker"}, + {"reference definition ends early", func(in *Input) { in.Marker = `[touchmark]: # "touchmark:v1" x "` }, nil, "not a marker"}, + {"reference definition unclosed", func(in *Input) { in.Marker = `[touchmark]: # "touchmark:v1 x` }, nil, "not a marker"}, + {"reference definition escaped", func(in *Input) { in.Marker = `[touchmark]: # "touchmark:v1 hub\=x"` }, nil, "not a marker"}, + {"intro reference definition", func(in *Input) { in.Intro = "Hello\n\n[touchmark]: # \"touchmark:v1 x\"" }, ErrIntro, "line 3"}, {"unknown action", func(in *Input) { in.Changes = []Change{{Path: "a", Action: "rename"}} }, nil, "unknown action"}, {"unknown pending action", func(in *Input) { in.Paused = true diff --git a/internal/prbody/testdata/assumed/bitbucket.md b/internal/prbody/testdata/assumed/bitbucket.md new file mode 100644 index 0000000..2730c1e --- /dev/null +++ b/internal/prbody/testdata/assumed/bitbucket.md @@ -0,0 +1,19 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` ([`github.com/acme/engineering-assets`](https://redirect.github.com/acme/engineering-assets)) at commit [`3f2c1ab9d8e7`](https://redirect.github.com/acme/engineering-assets/commit/3f2c1ab9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3). + +The hub subscribed this repository, which has no `.engineering-assets.yml`. To add packs or keep files out of the sync, add `.engineering-assets.yml` with `packs` or `ignore`; to stop these pull requests, add it with `enabled: false`. + +### Changes + +| Change | File | Pack | +|---|---|---| +| create | `AGENTS.md` | `agents` | +| create | `prompts/review.md` | `agents` | + +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/declined/bitbucket.md b/internal/prbody/testdata/declined/bitbucket.md new file mode 100644 index 0000000..93f0bfb --- /dev/null +++ b/internal/prbody/testdata/declined/bitbucket.md @@ -0,0 +1,21 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` ([`github.com/acme/engineering-assets`](https://redirect.github.com/acme/engineering-assets)) at commit [`3f2c1ab9d8e7`](https://redirect.github.com/acme/engineering-assets/commit/3f2c1ab9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3). + +### Changes + +| Change | File | Pack | +|---|---|---| +| update | `AGENTS.md` | `agents` | +| create | `prompts/review.md` | `agents` | + +### Previously declined + +Some of these changes were in #7 and #9, which were closed without merging. This pull request brings the whole current set anyway, because the files of a pack go together. To keep a file as it is here, add it to `ignore` in `.engineering-assets.yml`. + +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/huge/bitbucket.md b/internal/prbody/testdata/huge/bitbucket.md new file mode 100644 index 0000000..8548dce --- /dev/null +++ b/internal/prbody/testdata/huge/bitbucket.md @@ -0,0 +1,235 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` ([`github.com/acme/engineering-assets`](https://redirect.github.com/acme/engineering-assets)) at commit [`3f2c1ab9d8e7`](https://redirect.github.com/acme/engineering-assets/commit/3f2c1ab9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3). + +### ⚠ Sensitive paths + +These files can run code in CI or change how tools and agents work in this repository. Review them first. + +- `.github/workflows/docs.yml`: update, pack `ci` +- `.pre-commit-config.yaml`: create, pack `base` +- `bin/setup`: create, executable, pack `base` + +### Changes + +| Change | File | Pack | +|---|---|---| +| ⚠ update | `.github/workflows/docs.yml` | `ci` | +| ⚠ create | `.pre-commit-config.yaml` | `base` | +| ⚠ create, executable | `bin/setup` | `base` | +| delete | `docs/handbook/chapter-02/page-002.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-014.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-026.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-038.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-050.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-062.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-074.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-086.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-098.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-110.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-122.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-134.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-146.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-158.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-170.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-182.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-194.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-206.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-218.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-230.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-242.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-254.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-266.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-278.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-290.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-302.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-314.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-326.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-338.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-350.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-362.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-374.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-386.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-398.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-410.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-422.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-434.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-446.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-458.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-470.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-482.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-494.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-506.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-518.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-530.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-542.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-554.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-566.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-578.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-590.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-602.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-614.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-626.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-638.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-650.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-662.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-674.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-686.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-698.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-710.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-722.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-734.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-746.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-758.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-770.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-782.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-794.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-806.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-818.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-830.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-842.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-854.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-866.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-878.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-890.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-902.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-914.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-926.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-938.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-950.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-962.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-974.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-986.md` | `handbook` | +| delete | `docs/handbook/chapter-02/page-998.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-006.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-018.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-030.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-042.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-054.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-066.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-078.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-090.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-102.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-114.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-126.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-138.md` | `handbook` | +| delete | `docs/handbook/chapter-06/page-150.md` | `handbook` | + +…and 903 more. + +
+150 files here differ from the hub and stay as they are + +touchmark does not update files changed in this repository. To take the hub's version of one, run `touchmark apply --adopt `. To stop seeing it here, add it to `ignore` in `.engineering-assets.yml`. + +- `docs/handbook/chapter-00/local-000.md` +- `docs/handbook/chapter-00/local-012.md` +- `docs/handbook/chapter-00/local-024.md` +- `docs/handbook/chapter-00/local-036.md` +- `docs/handbook/chapter-00/local-048.md` +- `docs/handbook/chapter-00/local-060.md` +- `docs/handbook/chapter-00/local-072.md` +- `docs/handbook/chapter-00/local-084.md` +- `docs/handbook/chapter-00/local-096.md` +- `docs/handbook/chapter-00/local-108.md` +- `docs/handbook/chapter-00/local-120.md` +- `docs/handbook/chapter-00/local-132.md` +- `docs/handbook/chapter-00/local-144.md` +- `docs/handbook/chapter-01/local-001.md` +- `docs/handbook/chapter-01/local-013.md` +- `docs/handbook/chapter-01/local-025.md` +- `docs/handbook/chapter-01/local-037.md` +- `docs/handbook/chapter-01/local-049.md` +- `docs/handbook/chapter-01/local-061.md` +- `docs/handbook/chapter-01/local-073.md` +- `docs/handbook/chapter-01/local-085.md` +- `docs/handbook/chapter-01/local-097.md` +- `docs/handbook/chapter-01/local-109.md` +- `docs/handbook/chapter-01/local-121.md` +- `docs/handbook/chapter-01/local-133.md` +- `docs/handbook/chapter-01/local-145.md` +- `docs/handbook/chapter-02/local-002.md` +- `docs/handbook/chapter-02/local-014.md` +- `docs/handbook/chapter-02/local-026.md` +- `docs/handbook/chapter-02/local-038.md` +- `docs/handbook/chapter-02/local-050.md` +- `docs/handbook/chapter-02/local-062.md` +- `docs/handbook/chapter-02/local-074.md` +- `docs/handbook/chapter-02/local-086.md` +- `docs/handbook/chapter-02/local-098.md` +- `docs/handbook/chapter-02/local-110.md` +- `docs/handbook/chapter-02/local-122.md` +- `docs/handbook/chapter-02/local-134.md` +- `docs/handbook/chapter-02/local-146.md` +- `docs/handbook/chapter-03/local-003.md` +- `docs/handbook/chapter-03/local-015.md` +- `docs/handbook/chapter-03/local-027.md` +- `docs/handbook/chapter-03/local-039.md` +- `docs/handbook/chapter-03/local-051.md` +- `docs/handbook/chapter-03/local-063.md` +- `docs/handbook/chapter-03/local-075.md` +- `docs/handbook/chapter-03/local-087.md` +- `docs/handbook/chapter-03/local-099.md` +- `docs/handbook/chapter-03/local-111.md` +- `docs/handbook/chapter-03/local-123.md` +- `docs/handbook/chapter-03/local-135.md` +- `docs/handbook/chapter-03/local-147.md` +- `docs/handbook/chapter-04/local-004.md` +- `docs/handbook/chapter-04/local-016.md` +- `docs/handbook/chapter-04/local-028.md` +- `docs/handbook/chapter-04/local-040.md` +- `docs/handbook/chapter-04/local-052.md` +- `docs/handbook/chapter-04/local-064.md` +- `docs/handbook/chapter-04/local-076.md` +- `docs/handbook/chapter-04/local-088.md` +- `docs/handbook/chapter-04/local-100.md` +- `docs/handbook/chapter-04/local-112.md` +- `docs/handbook/chapter-04/local-124.md` +- `docs/handbook/chapter-04/local-136.md` +- `docs/handbook/chapter-04/local-148.md` +- `docs/handbook/chapter-05/local-005.md` +- `docs/handbook/chapter-05/local-017.md` +- `docs/handbook/chapter-05/local-029.md` +- `docs/handbook/chapter-05/local-041.md` +- `docs/handbook/chapter-05/local-053.md` +- `docs/handbook/chapter-05/local-065.md` +- `docs/handbook/chapter-05/local-077.md` +- `docs/handbook/chapter-05/local-089.md` +- `docs/handbook/chapter-05/local-101.md` +- `docs/handbook/chapter-05/local-113.md` +- `docs/handbook/chapter-05/local-125.md` +- `docs/handbook/chapter-05/local-137.md` +- `docs/handbook/chapter-05/local-149.md` +- `docs/handbook/chapter-06/local-006.md` +- `docs/handbook/chapter-06/local-018.md` +- `docs/handbook/chapter-06/local-030.md` +- `docs/handbook/chapter-06/local-042.md` +- `docs/handbook/chapter-06/local-054.md` +- `docs/handbook/chapter-06/local-066.md` +- `docs/handbook/chapter-06/local-078.md` +- `docs/handbook/chapter-06/local-090.md` +- `docs/handbook/chapter-06/local-102.md` +- `docs/handbook/chapter-06/local-114.md` +- `docs/handbook/chapter-06/local-126.md` +- `docs/handbook/chapter-06/local-138.md` +- `docs/handbook/chapter-07/local-007.md` +- `docs/handbook/chapter-07/local-019.md` +- `docs/handbook/chapter-07/local-031.md` +- `docs/handbook/chapter-07/local-043.md` +- `docs/handbook/chapter-07/local-055.md` +- `docs/handbook/chapter-07/local-067.md` +- `docs/handbook/chapter-07/local-079.md` +- `docs/handbook/chapter-07/local-091.md` +- `docs/handbook/chapter-07/local-103.md` +- `docs/handbook/chapter-07/local-115.md` + +…and 50 more. + +
+ +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/local/bitbucket.md b/internal/prbody/testdata/local/bitbucket.md new file mode 100644 index 0000000..98165b0 --- /dev/null +++ b/internal/prbody/testdata/local/bitbucket.md @@ -0,0 +1,27 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` at commit `3f2c1ab9d8e7`. + +### Changes + +| Change | File | Pack | +|---|---|---| +| update | `AGENTS.md` | `agents` | + +
+3 files here differ from the hub and stay as they are + +touchmark does not update files changed in this repository. To take the hub's version of one, run `touchmark apply --adopt `. To stop seeing it here, add it to `ignore` in `.engineering-assets.yml`. + +- `.github/CODEOWNERS` +- `CONTRIBUTING.md` +- `docs/guide.md` + +
+ +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/new/bitbucket.md b/internal/prbody/testdata/new/bitbucket.md new file mode 100644 index 0000000..95f6ea1 --- /dev/null +++ b/internal/prbody/testdata/new/bitbucket.md @@ -0,0 +1,30 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` ([`github.com/acme/engineering-assets`](https://redirect.github.com/acme/engineering-assets)) at commit [`3f2c1ab9d8e7`](https://redirect.github.com/acme/engineering-assets/commit/3f2c1ab9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3). + +### ⚠ Sensitive paths + +These files can run code in CI or change how tools and agents work in this repository. Review them first. + +- `.claude/settings.json`: update, pack `claude` +- `scripts/check.sh`: create, executable, pack `base` +- `tools/lint.sh`: chmod +x, pack `base` + +### Changes + +| Change | File | Pack | +|---|---|---| +| ⚠ update | `.claude/settings.json` | `claude` | +| ⚠ chmod +x | `tools/lint.sh` | `base` | +| ⚠ create, executable | `scripts/check.sh` | `base` | +| delete | `prompts/old.md` | `agents` | +| update | `AGENTS.md` | `agents` | +| chmod -x | `docs/guide.md` | `base` | +| create | `prompts/review.md` | `agents` | + +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/nothing-more/bitbucket.md b/internal/prbody/testdata/nothing-more/bitbucket.md new file mode 100644 index 0000000..97166c3 --- /dev/null +++ b/internal/prbody/testdata/nothing-more/bitbucket.md @@ -0,0 +1,20 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` ([`github.com/acme/engineering-assets`](https://redirect.github.com/acme/engineering-assets)) at commit [`3f2c1ab9d8e7`](https://redirect.github.com/acme/engineering-assets/commit/3f2c1ab9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3). + +### Changes + +| Change | File | Pack | +|---|---|---| +| create | `prompts/review.md` | `agents` | + +### Nothing more to sync + +The hub proposes nothing more for this repository: the default branch has the hub's versions of these files, or they are ignored or changed here. This branch has commits touchmark did not make, so touchmark leaves this pull request open. Close it, or keep your commits and merge it. + +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/paused/bitbucket.md b/internal/prbody/testdata/paused/bitbucket.md new file mode 100644 index 0000000..d632886 --- /dev/null +++ b/internal/prbody/testdata/paused/bitbucket.md @@ -0,0 +1,34 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` ([`github.com/acme/engineering-assets`](https://redirect.github.com/acme/engineering-assets)) at commit [`3f2c1ab9d8e7`](https://redirect.github.com/acme/engineering-assets/commit/3f2c1ab9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3). + +### ⚠ Sensitive paths + +These files can run code in CI or change how tools and agents work in this repository. Review them first. + +- `.github/workflows/lint.yml`: create, pack `ci` (after a rebuild) + +### Changes + +| Change | File | Pack | +|---|---|---| +| update | `AGENTS.md` | `agents` | +| create | `prompts/review.md` | `agents` | + +### touchmark paused + +This branch has commits touchmark did not make, so touchmark stopped updating it to keep them. The changes above are what the branch holds now. A rebuild from the default branch would bring: + +- ⚠ create `.github/workflows/lint.yml` (pack `ci`) +- update `AGENTS.md` (pack `agents`) + +To rebuild the branch and drop the other commits, tick **Rebuild this branch** below. + +- [ ] Rebuild this branch (drops commits added by others) + +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/sensitive-only/bitbucket.md b/internal/prbody/testdata/sensitive-only/bitbucket.md new file mode 100644 index 0000000..d057fd0 --- /dev/null +++ b/internal/prbody/testdata/sensitive-only/bitbucket.md @@ -0,0 +1,32 @@ +Acme keeps its shared engineering files (agent instructions, editor and CI settings) in one hub. +Review this like any other change; questions go to the platform team's channel. + +touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` ([`github.com/acme/engineering-assets`](https://redirect.github.com/acme/engineering-assets)) at commit [`3f2c1ab9d8e7`](https://redirect.github.com/acme/engineering-assets/commit/3f2c1ab9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3). + +### ⚠ Sensitive paths + +These files can run code in CI or change how tools and agents work in this repository. Review them first. + +- `.gitea/workflows/ci.yml`: create, pack `ci` +- `.github/CODEOWNERS`: delete, pack `base` +- `.github/workflows/ci.yml`: update, pack `ci` +- `deploy/prod.yaml`: update, pack `ops` +- `scripts/release.sh`: chmod +x, pack `base` + +This pull request adds `.gitea/workflows` or `.forgejo/workflows`, which turns off `.github/workflows` in this repository on Gitea and Forgejo. + +### Changes + +| Change | File | Pack | +|---|---|---| +| ⚠ delete | `.github/CODEOWNERS` | `base` | +| ⚠ update | `.github/workflows/ci.yml` | `ci` | +| ⚠ update | `deploy/prod.yaml` | `ops` | +| ⚠ chmod +x | `scripts/release.sh` | `base` | +| ⚠ create | `.gitea/workflows/ci.yml` | `ci` | + +--- + +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. touchmark rewrites this description, so please comment instead of editing it. + +[touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file From 756f89d990b980a0e7a52f49e8be9c478815b683 Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Fri, 9 Oct 2026 00:09:18 +0300 Subject: [PATCH 07/20] feat(distribute): respect Caps.NoLabels and Caps.Marker on Bitbucket On a platform without labels the core asks for none: no labels on new pull requests, none recorded as set in the marker, no AddLabels edits when hub.yml gains one, no label writes in the estimates, and no advice about labels in the auto-decline comment. The marker is written in the frame of Caps.Marker (a reference definition on Bitbucket), and doctor reads markers in both frames. A push Bitbucket refuses with "Permission denied to update branch" is blocked:rules:protected-branch. A Bitbucket commit without the account's email is authored by @touchmark.invalid. The fake refuses labels where its caps say there are none. --- internal/distribute/body.go | 24 ++- internal/distribute/commit.go | 12 ++ internal/distribute/doctor.go | 4 +- internal/distribute/estimate.go | 2 +- internal/distribute/execute.go | 5 +- internal/distribute/execute_nolabels_test.go | 159 +++++++++++++++++++ internal/distribute/execworld_test.go | 7 + internal/distribute/gate.go | 5 +- internal/distribute/legacy.go | 5 +- internal/distribute/steps.go | 14 +- internal/distribute/target_exec.go | 2 +- internal/distribute/units_test.go | 32 ++++ internal/distribute/write.go | 9 +- internal/platform/fake/writer.go | 13 +- 14 files changed, 267 insertions(+), 26 deletions(-) create mode 100644 internal/distribute/execute_nolabels_test.go diff --git a/internal/distribute/body.go b/internal/distribute/body.go index b0a1be9..2dc38fd 100644 --- a/internal/distribute/body.go +++ b/internal/distribute/body.go @@ -253,6 +253,28 @@ func draftPrefix(c platform.Caps) string { return c.DraftPrefix } +// prLabels returns the labels p's pull requests get: pr.labels of hub.yml, +// none on a platform without labels (Caps.NoLabels), so that the core never +// asks for them there, nor records them in the marker as set. +func (p *provider) prLabels(hub *config.Hub) []string { + if p.caps.NoLabels { + return nil + } + return hub.PR.Labels +} + +// createsLabels reports whether a first pull request of p may create a +// label: hub.yml sets labels and the platform has them. +func (p *provider) createsLabels(hub *config.Hub) bool { return len(p.prLabels(hub)) > 0 } + +// markerFrame is the frame of the marker line on p (Caps.Marker). +func (p *provider) markerFrame() marker.Frame { + if p.caps.Marker == platform.MarkerInRefDef { + return marker.FrameRefDef + } + return marker.FrameComment +} + // reportDecision writes w's decision into the target's report line: the // outcome, reason and pull request, and for a branch touchmark leaves alone // (blocked:edited, blocked:branch-taken) why (decide.Branch.Detail). It @@ -290,7 +312,7 @@ func (r *run) reportDecision(w *Work, prs prSet, reads branchReads) error { if !ok { return fmt.Errorf("the decision edits #%d, which is not an open pull request of touchmark", s.PR) } - desired := decide.DesiredPR{Title: r.hub.PR.Title, Body: human, Labels: r.hub.PR.Labels, Base: s.Base, + desired := decide.DesiredPR{Title: r.hub.PR.Title, Body: human, Labels: w.t.prov.prLabels(r.hub), Base: s.Base, DraftPrefix: draftPrefix(w.t.prov.caps)} edit, _, writes := decide.PlanPREdit(o.PR, o.Marker, desired) if !writes && !s.Content { diff --git a/internal/distribute/commit.go b/internal/distribute/commit.go index 8dc1729..0cacce9 100644 --- a/internal/distribute/commit.go +++ b/internal/distribute/commit.go @@ -1,6 +1,7 @@ package distribute import ( + "cmp" "context" "errors" "fmt" @@ -9,6 +10,7 @@ import ( "net" "path" "slices" + "strings" "time" "github.com/bedrock-python/touchmark/internal/config" @@ -415,11 +417,21 @@ func (r *run) commitPerson(p *provider) gitx.Person { // email private: GitHub "+@users.noreply.", GitLab // "-@users.noreply.", Gitea and Forgejo // "@noreply.". host loses its port. +// +// Bitbucket Cloud has no such address: it links a commit to the account one +// of whose confirmed addresses is the author's, and the driver gives the +// account's primary confirmed address when GET /2.0/user/emails shows one. +// Without it, the author is "@touchmark.invalid", the account's UUID +// without braces under a domain that can never exist (RFC 6761, .invalid): +// the commit then names the bot by its UUID, links to no account, and +// can never be taken for someone else's. func noReplyEmail(typ, host, id, login string) string { if h, _, err := net.SplitHostPort(host); err == nil { host = h } switch typ { + case "bitbucket": + return strings.Trim(cmp.Or(id, login), "{}") + "@touchmark.invalid" case "gitlab": if id != "" { return id + "-" + login + "@users.noreply." + host diff --git a/internal/distribute/doctor.go b/internal/distribute/doctor.go index f5b3f49..883b925 100644 --- a/internal/distribute/doctor.go +++ b/internal/distribute/doctor.go @@ -519,12 +519,12 @@ func listMarkers(items []string) string { } // markerFingerprints returns the fingerprints of the valid v1 markers in -// body whose hub id is id, in the order they appear. +// body, in either frame, whose hub id is id, in the order they appear. func markerFingerprints(body, id string) []string { var out []string for line := range strings.SplitSeq(body, "\n") { line = strings.TrimRight(line, " \t\r") - if !strings.HasPrefix(line, "`. It records the -fingerprint and what the pull request carries. +fingerprint and what the pull request carries. On Bitbucket Cloud, which shows HTML +comments as text, the same payload is a Markdown link reference definition: +`[touchmark]: # "touchmark:v1 hub=… fp=… stream=sync key=sha256:… data=…"`. **opt-in file** — `.engineering-assets.yml` at a target's root. Its presence is consent; it can add packs and `ignore` paths, and `enabled: false` in it opts the target out. diff --git a/docs/reference/hub.md b/docs/reference/hub.md index 70313e2..c6f7a02 100644 --- a/docs/reference/hub.md +++ b/docs/reference/hub.md @@ -104,7 +104,7 @@ In a hub pull request, `providers` and `ca_file` are read from the default branc | `security.private_targets_in_public_hub` | one of `skip`, `deliver` | What a public hub does with non-public targets: skip them, or deliver and let their names reach public CI logs. Default: skip. | | `providers` | list of objects | Platforms this hub delivers to. Targets refer to them by id. | | `providers[].id` | string, required | Name targets.yml uses for this provider, and the <ID> of its TOUCHMARK_<ID>_\* variables. | -| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, required | github: github.com, GHE.com or GitHub Enterprise Server; gitlab; gitea; forgejo; bitbucket: Bitbucket Cloud, read only for now (plan reads it; distribute, doctor and setup do not support it yet). | +| `providers[].type` | one of `github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, required | github: github.com, GHE.com or GitHub Enterprise Server; gitlab; gitea; forgejo; bitbucket: Bitbucket Cloud (plan, distribute and doctor; setup does not set up a hub on it yet). | | `providers[].url` | string | Web URL of the instance. Default for github: https://github.com; for gitlab: https://gitlab.com; for bitbucket: https://bitbucket.org, the only url it takes without api_url. Required for gitea and forgejo. A target targets.yml names by web URL belongs to the provider whose url it lies under. | | `providers[].api_url` | string | API URL, when it cannot be derived from url. | | `providers[].ca_file` | string | Hub file with extra CA certificates for this instance. | diff --git a/schemas/hub.schema.json b/schemas/hub.schema.json index d0dc2dd..1d3549d 100644 --- a/schemas/hub.schema.json +++ b/schemas/hub.schema.json @@ -246,7 +246,7 @@ "pattern": "^[a-z][a-z0-9-]{0,31}$" }, "providerType": { - "description": "github: github.com, GHE.com or GitHub Enterprise Server; gitlab; gitea; forgejo; bitbucket: Bitbucket Cloud, read only for now (plan reads it; distribute, doctor and setup do not support it yet).", + "description": "github: github.com, GHE.com or GitHub Enterprise Server; gitlab; gitea; forgejo; bitbucket: Bitbucket Cloud (plan, distribute and doctor; setup does not set up a hub on it yet).", "enum": ["github", "gitlab", "gitea", "forgejo", "bitbucket"] }, "sign": { From 4ecae105c3ddf4fc38ff23c9c1f5af9d90b9fb73 Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Fri, 9 Oct 2026 01:28:30 +0300 Subject: [PATCH 11/20] feat(platform): add Caps.ClosedImmutable and Caps.BodyControls Bitbucket Cloud's declined pull requests can never be edited, and its descriptions show the HTML comments the tick boxes rest on. ClosedImmutable says the first (Bitbucket's Probe sets it); BodyControls derives the second from the marker store. --- internal/platform/bitbucket/probe.go | 2 ++ internal/platform/bitbucket/probe_test.go | 5 +++-- internal/platform/platform.go | 23 +++++++++++++++++++++-- internal/platform/platform_test.go | 20 ++++++++++++++++++++ 4 files changed, 46 insertions(+), 4 deletions(-) diff --git a/internal/platform/bitbucket/probe.go b/internal/platform/bitbucket/probe.go index 885eef4..3054e87 100644 --- a/internal/platform/bitbucket/probe.go +++ b/internal/platform/bitbucket/probe.go @@ -64,6 +64,8 @@ func (d *reader) Probe(context.Context) (platform.Caps, error) { NoLabels: true, // closed_by names who declined or merged a pull request. CloserKnown: true, + // A declined pull request can never be changed again. + ClosedImmutable: true, // The API escapes HTML in descriptions: an HTML comment would show. Marker: platform.MarkerInRefDef, RuntimeOnly: []string{branchRestrictions}, diff --git a/internal/platform/bitbucket/probe_test.go b/internal/platform/bitbucket/probe_test.go index 3ec1c1b..d2b13f9 100644 --- a/internal/platform/bitbucket/probe_test.go +++ b/internal/platform/bitbucket/probe_test.go @@ -20,13 +20,14 @@ func TestProbe(t *testing.T) { t.Fatal(err) } want := platform.Caps{ - Flavor: "bitbucket", MaxBody: 60000, Draft: platform.DraftNative, CloserKnown: true, NoLabels: true, + Flavor: "bitbucket", MaxBody: 60000, Draft: platform.DraftNative, CloserKnown: true, NoLabels: true, ClosedImmutable: true, Marker: platform.MarkerInRefDef, RuntimeOnly: []string{"branch-restrictions"}, Limits: platform.Limits{Reads: 2, GitReads: 2, ReadsPerMinute: 15, MinInterval: time.Second}, } if caps.Flavor != want.Flavor || caps.MaxBody != want.MaxBody || caps.Draft != want.Draft || !caps.CloserKnown || caps.Marker != want.Marker || !slices.Equal(caps.RuntimeOnly, want.RuntimeOnly) || caps.Limits != want.Limits || - caps.LabelsByID || caps.QuickActions || caps.WorkflowPerm || caps.Commit.API || !caps.NoLabels { + caps.LabelsByID || caps.QuickActions || caps.WorkflowPerm || caps.Commit.API || !caps.NoLabels || !caps.ClosedImmutable || + caps.BodyControls() { t.Errorf("Probe = %+v\nwant %+v", caps, want) } if calls := f.requests("", ""); len(calls) != 0 { diff --git a/internal/platform/platform.go b/internal/platform/platform.go index 368bfb9..6eb0074 100644 --- a/internal/platform/platform.go +++ b/internal/platform/platform.go @@ -303,14 +303,33 @@ type Caps struct { NoLabels bool // CloserKnown is set when PR.ClosedBy is reliable. CloserKnown bool - Marker MarkerStore - Commit struct{ API, SignedByPlatform, CAS bool } + // ClosedImmutable is set when a pull request closed without merging can + // never be edited again, by anyone (Bitbucket Cloud's declined pull + // requests): the core never plans a write to the body of a closed pull + // request there. Memory then reads a decline from what the marker held + // while the pull request was open (its optin), and a forget_declines + // entry acts while it is present instead of once (see + // decide.MemoryConfig.ClosedImmutable). + ClosedImmutable bool + Marker MarkerStore + Commit struct{ API, SignedByPlatform, CAS bool } // RuntimeOnly lists checks the writer cannot read upfront, e.g. // "push_rules" on GitLab. RuntimeOnly []string Limits Limits } +// BodyControls reports whether pull request descriptions on the platform +// can carry touchmark's tick boxes ("Rebuild this branch", "Propose this +// content again"). Each is a task list item tagged with an HTML comment, +// which renderers hide; a platform that escapes HTML in descriptions would +// show the tag as text, and it is the same platform whose marker is a +// Markdown reference definition (MarkerInRefDef). There the description +// offers no tick box, says to ask for a rebuild or a new proposal through +// .touchmark/operations.yml instead, and the core reads no tick box from +// it. +func (c Caps) BodyControls() bool { return c.Marker != MarkerInRefDef } + // Reader is one provider from hub.yml under one identity. Implementations // are safe for concurrent use: the core inspects targets in parallel. type Reader interface { diff --git a/internal/platform/platform_test.go b/internal/platform/platform_test.go index 366822f..ce0b9db 100644 --- a/internal/platform/platform_test.go +++ b/internal/platform/platform_test.go @@ -220,3 +220,23 @@ func TestFileErrors(t *testing.T) { t.Errorf("empty: %q", msg) } } + +// TestBodyControls: descriptions carry tick boxes unless the platform's +// marker is a reference definition (HTML shows as text there), whatever +// else it says. +func TestBodyControls(t *testing.T) { + for _, c := range []struct { + caps Caps + want bool + }{ + {Caps{Marker: MarkerInBody}, true}, + {Caps{Marker: MarkerInBody, ClosedImmutable: true, NoLabels: true}, true}, + {Caps{Marker: MarkerInProperties}, true}, + {Caps{Marker: MarkerInRefDef}, false}, + {Caps{Marker: MarkerInRefDef, ClosedImmutable: true}, false}, + } { + if got := c.caps.BodyControls(); got != c.want { + t.Errorf("%+v: BodyControls = %v, want %v", c.caps, got, c.want) + } + } +} From 84a8bfcebd007b5d7fdf73203d458f59123f9bd9 Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Fri, 9 Oct 2026 01:28:30 +0300 Subject: [PATCH 12/20] feat(decide): remember declines without writes where closed pull requests are immutable With MemoryConfig.ClosedImmutable the optin an open pull request's marker held stands for the ack: a decline holds while the opt-in file hashes the same and lapses when it changes. One without optin holds and is listed in Memory.Unanchored. A forget_declines entry lifts a decline while present (Memory.Forgotten), ticked repropose controls are ignored, and ToAck and ToRevoke stay empty, so no write to a closed pull request is planned. --- internal/decide/memory.go | 69 +++++++-- internal/decide/memory_immutable_test.go | 175 +++++++++++++++++++++++ internal/decide/memory_test.go | 73 ++++++++-- internal/decide/target.go | 5 +- internal/marker/marker.go | 21 +-- 5 files changed, 314 insertions(+), 29 deletions(-) create mode 100644 internal/decide/memory_immutable_test.go diff --git a/internal/decide/memory.go b/internal/decide/memory.go index 560f87d..5ac5649 100644 --- a/internal/decide/memory.go +++ b/internal/decide/memory.go @@ -76,6 +76,29 @@ type MemoryConfig struct { Automation map[string]bool // CloserKnown is Caps.CloserKnown. CloserKnown bool + // ClosedImmutable is Caps.ClosedImmutable: a closed pull request can + // never be edited (Bitbucket Cloud's declined pull requests), so + // memory asks for no write to one. Declines are then read as follows + // (see BuildMemory): + // - a decline whose marker holds optin, the hash of the opt-in file + // touchmark wrote into the marker while the pull request was open + // (on every create and content edit, and on any edit that finds + // it stale), counts as acked with that optin: in force while the + // opt-in file hashes the same, lapsed once it changes, as after an + // ack written elsewhere; + // - a decline whose marker holds no optin (a marker touchmark did + // not write in full) is in force whatever the opt-in file says, + // and is listed in Memory.Unanchored so that the report says how to + // lift it: no state of the file was recorded to compare with, and a + // decline that lapsed silently would propose the content again; + // - a pull request a forget_declines entry names is no memory while + // the entry is present (Memory.Forgotten): its revocation cannot be + // written into the pull request, so the entry does not act once; + // - ticked repropose controls are ignored: such a platform shows no + // tick box (Caps.BodyControls), and no ack adds one. + // + // ToAck and ToRevoke are always empty then. + ClosedImmutable bool // Window is how many closed own PRs are considered, newest first (50; // a value that is not positive means 50). Window int @@ -129,8 +152,12 @@ type Decline struct { Key string Changes []marker.Change Complete bool // marker.Data.ChangesComplete + // Acked is set when the opt-in state the decline holds under is known: + // an ack was written into its marker, or, on a platform whose closed + // pull requests are immutable (MemoryConfig.ClosedImmutable), the + // marker recorded optin while the pull request was open. Acked bool - OptIn string // marker.Data.OptIn at ack time ("" before ack) + OptIn string // marker.Data.OptIn of an acked decline ("" otherwise) ClosedAt time.Time } @@ -157,7 +184,7 @@ type MemoryInput struct { // Forget lists PR numbers operations.yml forget_declines names. Forget []int64 // Repropose lists declined PRs whose "Propose this content again" - // checkbox is ticked. + // checkbox is ticked (ignored with Config.ClosedImmutable). Repropose map[int64]bool // Now is the run's clock. What memory holds does not depend on it: // Cooldown takes the time it compares with. @@ -193,8 +220,18 @@ type Memory struct { // repropose ticked). Auto-closes named there are revoked too, which // ends their cooldown. A PR in ToRevoke is already out of every other // list, so a run that writes the revocation decides as the next run - // will. + // will. Always empty with Config.ClosedImmutable. ToRevoke []int64 + // Forgotten are, with Config.ClosedImmutable, the closes a + // forget_declines entry names: no memory while the entry is present, + // and out of every other list (an auto-close among them ends its + // cooldown). Nothing is written to them. + Forgotten []int64 + // Unanchored are, with Config.ClosedImmutable, the declines in force + // whose marker records no opt-in state (see MemoryConfig): they hold + // until a forget_declines entry names them or a path of theirs becomes + // local or ignored, and the report says so. + Unanchored []int64 // Auto are auto-closes among the window, newest first. Revoked ones // (and those in ToRevoke) are left out: a revoked PR is not memory. Auto []AutoClose @@ -211,11 +248,16 @@ type Memory struct { // - named by Forget, or its repropose checkbox ticked: ToRevoke, and no // memory from this run on (revoking a lapsed decline makes the revocation // permanent, even if the opt-in file later returns to the acked state); +// with Config.ClosedImmutable, named by Forget: Forgotten, and no memory +// while the entry is present (a ticked checkbox counts for nothing +// there); // - CloseAuto: an entry of Auto; it is a decline as well when its marker // is acked or it is the third or later of a run (see Memory.Declines); // - declines are then in force (Declines, and ToAck while unacked) or // Lapsed: acked with an optin other than OptIn, or a path of their -// Changes LocalOrIgnored. +// Changes LocalOrIgnored. With Config.ClosedImmutable a decline whose +// marker holds optin counts as acked with it, and one in force without +// it is Unanchored instead of ToAck. // // A run of an auto-close is itself and the consecutive older entries of // Auto with the same key: an auto-close with another key ends it, while a @@ -233,6 +275,7 @@ func BuildMemory(in MemoryInput) Memory { for _, n := range in.Forget { forget[n] = true } + immutable := in.Config.ClosedImmutable var m Memory // kept are the closes that carry memory, in window order, with whether // the class was CloseAuto. @@ -255,7 +298,10 @@ func BuildMemory(in MemoryInput) Memory { switch { case class == CloseSelf, o.Marker.Data.Revoked: continue - case forget[n] || in.Repropose[n]: + case immutable && forget[n]: + m.Forgotten = append(m.Forgotten, n) + continue + case !immutable && (forget[n] || in.Repropose[n]): m.ToRevoke = append(m.ToRevoke, n) continue } @@ -282,10 +328,13 @@ func BuildMemory(in MemoryInput) Memory { Key: c.o.Marker.Key, Changes: slices.Clone(data.Changes), Complete: data.ChangesComplete, - Acked: data.Ack, + // On a platform whose closed pull requests are immutable, the + // optin the open pull request's marker held stands for the ack + // no one can write. + Acked: data.Ack || (immutable && data.OptIn != ""), ClosedAt: c.o.PR.ClosedAt, } - if data.Ack { + if d.Acked { d.OptIn = data.OptIn } if memLapsed(d, in) { @@ -293,7 +342,11 @@ func BuildMemory(in MemoryInput) Memory { continue } m.Declines = append(m.Declines, d) - if !d.Acked { + switch { + case d.Acked: + case immutable: + m.Unanchored = append(m.Unanchored, n) + default: m.ToAck = append(m.ToAck, n) } } diff --git a/internal/decide/memory_immutable_test.go b/internal/decide/memory_immutable_test.go new file mode 100644 index 0000000..7139dd8 --- /dev/null +++ b/internal/decide/memory_immutable_test.go @@ -0,0 +1,175 @@ +package decide + +import ( + "slices" + "testing" +) + +// memImmutable is memInput on a platform whose closed pull requests are +// immutable (Bitbucket Cloud), with the current opt-in hash optIn. +func memImmutable(own []OwnPR, optIn string) MemoryInput { + in := memInput(own) + in.OptIn = optIn + in.Config.ClosedImmutable = true + return in +} + +// TestMemoryClosedImmutable: where a declined pull request can never be +// edited, the optin its marker held while open stands for the ack: the +// decline holds while the opt-in file hashes the same and lapses when it +// changes; one without optin holds whatever the file says and is +// unanchored; a forget_declines entry lifts a decline while it is present; +// a ticked repropose control counts for nothing; and memory never asks to +// write to a closed pull request. +func TestMemoryClosedImmutable(t *testing.T) { + a := []Pair{memCreate(memA, memV1)} + declined := memPR(t, 7, a, &memPerson, 0) // optin memH1, written while open + bare := memPR(t, 7, a, &memPerson, 0, memNoOptIn) + cases := []struct { + name string + in MemoryInput + // want is memNumbers of the memory; verdict is memVerdict's for a. + want, verdict string + acked bool + }{ + { + name: "in force while the opt-in file is the same", + in: memImmutable([]OwnPR{declined}, memH1), + want: "Declines [7] Lapsed [] ToAck [] ToRevoke [] Auto []", verdict: "declined", acked: true, + }, + { + name: "lapsed once the opt-in file changes", + in: memImmutable([]OwnPR{declined}, memH2), + want: "Declines [] Lapsed [7] ToAck [] ToRevoke [] Auto []", verdict: "new PR", + }, + { + name: "in force when the current state is unknown", + in: memImmutable([]OwnPR{declined}, ""), + want: "Declines [7] Lapsed [] ToAck [] ToRevoke [] Auto []", verdict: "declined", acked: true, + }, + { + name: "without optin: in force whatever the opt-in file says", + in: memImmutable([]OwnPR{bare}, memH2), + want: "Declines [7] Lapsed [] ToAck [] ToRevoke [] Auto [] Forgotten [] Unanchored [7]", verdict: "declined", + }, + { + name: "without optin: lapsed when a path becomes local", + in: func() MemoryInput { + in := memImmutable([]OwnPR{bare}, memH1) + in.LocalOrIgnored = func(p string) bool { return p == memA } + return in + }(), + want: "Declines [] Lapsed [7] ToAck [] ToRevoke [] Auto []", verdict: "new PR", + }, + { + name: "a forget_declines entry lifts it while present", + in: func() MemoryInput { + in := memImmutable([]OwnPR{bare}, memH1) + in.Forget = []int64{7} + return in + }(), + want: "Declines [] Lapsed [] ToAck [] ToRevoke [] Auto [] Forgotten [7] Unanchored []", verdict: "new PR", + }, + { + name: "a ticked repropose control counts for nothing", + in: func() MemoryInput { + in := memImmutable([]OwnPR{declined}, memH1) + in.Repropose = map[int64]bool{7: true} + return in + }(), + want: "Declines [7] Lapsed [] ToAck [] ToRevoke [] Auto []", verdict: "declined", acked: true, + }, + { + name: "the writer's own close is no memory", + in: memImmutable([]OwnPR{memPR(t, 7, a, &memWriter, 0)}, memH1), + want: "Declines [] Lapsed [] ToAck [] ToRevoke [] Auto []", verdict: "new PR", + }, + { + name: "touchmark's close is no memory", + in: memImmutable([]OwnPR{memPR(t, 7, a, &memWriter, 0, memSelfClosed)}, memH1), + want: "Declines [] Lapsed [] ToAck [] ToRevoke [] Auto []", verdict: "new PR", + }, + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + m := BuildMemory(c.in) + if got := memNumbers(m); got != c.want { + t.Errorf("memory %s\nwant %s", got, c.want) + } + if got, _ := memVerdict(m, a, c.in.Now); got != c.verdict { + t.Errorf("verdict %q, want %q", got, c.verdict) + } + if len(m.Declines) == 1 && m.Declines[0].Acked != c.acked { + t.Errorf("decline %+v, want acked %v", m.Declines[0], c.acked) + } + d := DecideTarget(TargetInput{Stream: StreamSync, D: a, Key: Key(StreamSync, a), B: memX, DefaultBranch: "main", + Branch: Branch{Name: "touchmark/acme-eng"}, Own: c.in.Own, Memory: m, Now: c.in.Now}) + for _, s := range d.Steps { + if s.PR != 0 && s.Kind != StepCreatePR { + t.Errorf("a write to closed #%d: %s", s.PR, s.Kind) + } + } + }) + } +} + +// TestMemoryClosedImmutableSame: on a platform whose closed pull requests +// can be edited, the same marker of an unacked decline asks for the ack, +// whatever optin it holds, and a forget_declines entry revokes it: the +// platform decides, not the marker. +func TestMemoryClosedImmutableSame(t *testing.T) { + a := []Pair{memCreate(memA, memV1)} + in := memInput([]OwnPR{memPR(t, 7, a, &memPerson, 0)}) + in.OptIn = memH2 + if got, want := memNumbers(BuildMemory(in)), "Declines [7] Lapsed [] ToAck [7] ToRevoke [] Auto []"; got != want { + t.Errorf("memory %s\nwant %s", got, want) + } + in.Forget = []int64{7} + if got, want := memNumbers(BuildMemory(in)), "Declines [] Lapsed [] ToAck [] ToRevoke [7] Auto []"; got != want { + t.Errorf("forgotten: memory %s\nwant %s", got, want) + } +} + +// TestMemoryClosedImmutableEscalation: auto-closes count as before from +// the closer and the marker: the first two defer the content, the third +// counts as a decline, which holds with the optin of its marker and +// lapses when the opt-in file changes; nothing is written to any of them. +func TestMemoryClosedImmutableEscalation(t *testing.T) { + a := []Pair{memCreate(memA, memV1)} + key := Key(StreamSync, a) + var own []OwnPR + for i, n := range []int64{9, 8, 7} { + own = append(own, memPR(t, n, a, &memStale, 20-i)) + } + m := BuildMemory(memImmutable(own[2:], memH1)) + if until, declined := m.Cooldown(key, memT0.Add(19*memDay), 0); until.IsZero() || declined { + t.Errorf("one auto-close: until %v, declined %v", until, declined) + } + m = BuildMemory(memImmutable(own, memH1)) + if want := "Declines [9] Lapsed [] ToAck [] ToRevoke [] Auto [9 8 7]"; memNumbers(m) != want { + t.Errorf("three auto-closes: %s\nwant %s", memNumbers(m), want) + } + if _, declined := m.Cooldown(key, memT0.Add(100*memDay), 0); !declined { + t.Error("the third auto-close does not count as a decline") + } + d := DecideTarget(TargetInput{Stream: StreamSync, D: a, Key: key, B: memX, DefaultBranch: "main", + Branch: Branch{Name: "touchmark/acme-eng"}, Own: own, Memory: m, CooldownDeclined: true, Now: memT0.Add(100 * memDay)}) + if d.Outcome != OutcomeDeclined || d.PR != 9 || len(d.Steps) != 0 { + t.Errorf("decision %s #%d, steps %v", d.Outcome, d.PR, d.Steps) + } + m = BuildMemory(memImmutable(own, memH2)) + if want := "Declines [] Lapsed [9] ToAck [] ToRevoke [] Auto [9 8 7]"; memNumbers(m) != want { + t.Errorf("after an opt-in change: %s\nwant %s", memNumbers(m), want) + } + if until, declined := m.Cooldown(key, memT0.Add(100*memDay), 0); !until.IsZero() || declined { + t.Errorf("after an opt-in change: until %v, declined %v", until, declined) + } + // A forget_declines entry ends the cooldown of an auto-close it names, + // while present. + in := memImmutable(own[2:], memH1) + in.Forget = []int64{7} + m = BuildMemory(in) + if len(m.Auto) != 0 || !slices.Equal(m.Forgotten, []int64{7}) { + t.Errorf("a forgotten auto-close: %s", memNumbers(m)) + } +} diff --git a/internal/decide/memory_test.go b/internal/decide/memory_test.go index 4c454f5..11b618a 100644 --- a/internal/decide/memory_test.go +++ b/internal/decide/memory_test.go @@ -109,6 +109,7 @@ func memAcked(optIn string) func(*marker.Data) { func memRevoked(d *marker.Data) { d.Revoked = true } func memIncomplete(d *marker.Data) { d.Changes, d.ChangesComplete = nil, false } func memSelfClosed(d *marker.Data) { d.Closed = &marker.Closed{By: "touchmark", Reason: ReasonNoDiff} } +func memNoOptIn(d *marker.Data) { d.OptIn = "" } // memState returns o in another state: open or merged. func memState(o OwnPR, s platform.PRState) OwnPR { @@ -1033,6 +1034,10 @@ func TestMemoryProperties(t *testing.T) { blobs := []string{memV1, memV2, memX} closers := []*platform.Account{nil, &memPerson, &memStale, &memWriter, &memKnown, &memRunner} r := rand.New(rand.NewPCG(3, 5)) + // r2 draws what only the platforms with immutable closed pull requests + // read (markers without optin), so that r draws the inputs it drew + // before they existed. + r2 := rand.New(rand.NewPCG(7, 11)) for run := range runs { var own []OwnPR pairs := map[int64][]Pair{} @@ -1062,6 +1067,9 @@ func TestMemoryProperties(t *testing.T) { if r.IntN(8) == 0 { edits = append(edits, memIncomplete) } + if r2.IntN(6) == 0 { + edits = append(edits, memNoOptIn) + } o := memPR(t, n, d, closers[r.IntN(len(closers))], 100-i, edits...) o.PR.BaseExists = r.IntN(10) > 0 switch r.IntN(8) { @@ -1095,9 +1103,14 @@ func TestMemoryProperties(t *testing.T) { } } slices.Sort(in.Forget) - m := BuildMemory(in) - if err := memCheck(in, m, pairs); err != nil { - t.Fatalf("run %d: %v\ninput: %+v\nmemory: %s", run, err, in, memNumbers(m)) + // The same input on a platform whose closed pull requests are + // immutable, and on the others. + for _, immutable := range []bool{false, true} { + in.Config.ClosedImmutable = immutable + m := BuildMemory(in) + if err := memCheck(in, m, pairs); err != nil { + t.Fatalf("run %d, closed immutable %v: %v\ninput: %+v\nmemory: %s", run, immutable, err, in, memNumbers(m)) + } } } } @@ -1116,7 +1129,7 @@ func memCheck(in MemoryInput, m Memory, pairs map[int64][]Pair) error { closed = append(closed, o) } } - type where struct{ decline, lapsed, revoke, auto, ack bool } + type where struct{ decline, lapsed, revoke, auto, ack, forgotten, unanchored bool } got := map[int64]*where{} at := func(n int64) *where { if got[n] == nil { @@ -1139,6 +1152,19 @@ func memCheck(in MemoryInput, m Memory, pairs map[int64][]Pair) error { for _, n := range m.ToAck { at(n).ack = true } + for _, n := range m.Forgotten { + at(n).forgotten = true + } + for _, n := range m.Unanchored { + at(n).unanchored = true + } + immutable := in.Config.ClosedImmutable + if immutable && len(m.ToAck)+len(m.ToRevoke) > 0 { + return fmt.Errorf("closed pull requests are immutable, yet memory asks to write ack %v, revoke %v", m.ToAck, m.ToRevoke) + } + if !immutable && len(m.Forgotten)+len(m.Unanchored) > 0 { + return fmt.Errorf("closed pull requests can be edited, yet memory lists forgotten %v, unanchored %v", m.Forgotten, m.Unanchored) + } inWindow := map[int64]bool{} var order []int64 for _, o := range closed { @@ -1153,14 +1179,19 @@ func memCheck(in MemoryInput, m Memory, pairs map[int64][]Pair) error { return fmt.Errorf("#%d holds no memory, yet it is in %+v", n, *w) } continue - case slices.Contains(in.Forget, n) || in.Repropose[n]: + case immutable && slices.Contains(in.Forget, n): + if *w != (where{forgotten: true}) { + return fmt.Errorf("#%d is forgotten, yet it is in %+v", n, *w) + } + continue + case !immutable && (slices.Contains(in.Forget, n) || in.Repropose[n]): if *w != (where{revoke: true}) { return fmt.Errorf("#%d is to revoke, yet it is in %+v", n, *w) } continue } - if w.revoke { - return fmt.Errorf("#%d is in ToRevoke unasked", n) + if w.revoke || w.forgotten { + return fmt.Errorf("#%d is in ToRevoke or Forgotten unasked", n) } if (class == CloseAuto) != w.auto { return fmt.Errorf("#%d: class %v, in Auto %v", n, class, w.auto) @@ -1174,10 +1205,22 @@ func memCheck(in MemoryInput, m Memory, pairs map[int64][]Pair) error { if (candidate && w.decline == w.lapsed) || (!candidate && (w.decline || w.lapsed)) { return fmt.Errorf("#%d: a decline %v, in Declines %v, in Lapsed %v", n, candidate, w.decline, w.lapsed) } - if w.ack != (w.decline && !data.Ack) { - return fmt.Errorf("#%d: in ToAck %v, in force %v, acked %v", n, w.ack, w.decline, data.Ack) + // Where no one can write an ack, the optin the marker held while + // open stands for one. + acked := data.Ack || (immutable && data.OptIn != "") + if w.ack != (w.decline && !acked && !immutable) || w.unanchored != (w.decline && !acked && immutable) { + return fmt.Errorf("#%d: in ToAck %v, unanchored %v, in force %v, acked %v", n, w.ack, w.unanchored, w.decline, acked) + } + if i := slices.IndexFunc(m.Declines, func(d Decline) bool { return d.PR == n }); i >= 0 { + want := "" + if acked { + want = data.OptIn + } + if d := m.Declines[i]; d.Acked != acked || d.OptIn != want { + return fmt.Errorf("#%d: decline %+v, want acked %v with %q", n, d, acked, want) + } } - lapse := data.Ack && in.OptIn != "" && data.OptIn != in.OptIn + lapse := acked && in.OptIn != "" && data.OptIn != in.OptIn for _, c := range data.Changes { lapse = lapse || in.LocalOrIgnored(c.Path) } @@ -1209,7 +1252,7 @@ func memCheck(in MemoryInput, m Memory, pairs map[int64][]Pair) error { for _, a := range m.Auto { auto = append(auto, a.PR) } - for _, list := range [][]int64{declines, m.Lapsed, m.ToAck, m.ToRevoke, auto} { + for _, list := range [][]int64{declines, m.Lapsed, m.ToAck, m.ToRevoke, auto, m.Forgotten, m.Unanchored} { if !inOrder(list) { return fmt.Errorf("a list is out of window order: %v", list) } @@ -1243,7 +1286,11 @@ func memNumbers(m Memory) string { for _, a := range m.Auto { auto = append(auto, a.PR) } - return fmt.Sprintf("Declines %v Lapsed %v ToAck %v ToRevoke %v Auto %v", declines, m.Lapsed, m.ToAck, m.ToRevoke, auto) + s := fmt.Sprintf("Declines %v Lapsed %v ToAck %v ToRevoke %v Auto %v", declines, m.Lapsed, m.ToAck, m.ToRevoke, auto) + if len(m.Forgotten)+len(m.Unanchored) > 0 { + s += fmt.Sprintf(" Forgotten %v Unanchored %v", m.Forgotten, m.Unanchored) + } + return s } // memSameNumbers compares the PR numbers of every list of two memories. @@ -1256,5 +1303,7 @@ func memClone(m Memory) Memory { m.ToAck = slices.Clone(m.ToAck) m.ToRevoke = slices.Clone(m.ToRevoke) m.Auto = slices.Clone(m.Auto) + m.Forgotten = slices.Clone(m.Forgotten) + m.Unanchored = slices.Clone(m.Unanchored) return m } diff --git a/internal/decide/target.go b/internal/decide/target.go index 4dd5042..b3a6b61 100644 --- a/internal/decide/target.go +++ b/internal/decide/target.go @@ -236,7 +236,10 @@ type TargetDecision struct { // Memory upkeep, always first: StepRevoke for each Memory.ToRevoke, then // StepAck and StepComment for each Memory.ToAck (Reason // CommentAutoDeclined for an auto-close in Memory.Auto, OutcomeDeclined -// otherwise). +// otherwise). These are the only steps that write to a closed PR; on a +// platform whose closed PRs are immutable memory lists none of them +// (MemoryConfig.ClosedImmutable), so no write to a closed PR is planned +// there. // // 1. An open PR in MarkerInvalid → blocked:marker-invalid (no other write), // unless operations.yml lets touchmark adopt it (see below): an adopted diff --git a/internal/marker/marker.go b/internal/marker/marker.go index 8b3ad6d..13beee3 100644 --- a/internal/marker/marker.go +++ b/internal/marker/marker.go @@ -76,14 +76,19 @@ type Data struct { // header carries only FP16(FP). FP string `json:"fp"` // HubRepo is omitted when the hub is private and the target public. - HubRepo string `json:"hub_repo,omitempty"` - DecidedAt string `json:"decided_at"` - ContentCommit string `json:"content_commit"` - Base string `json:"base"` - OptIn string `json:"optin"` - Engine string `json:"engine"` - Packs []string `json:"packs"` - Changes []Change `json:"changes"` + HubRepo string `json:"hub_repo,omitempty"` + DecidedAt string `json:"decided_at"` + ContentCommit string `json:"content_commit"` + Base string `json:"base"` + // OptIn is the hash of the opt-in file (config.OptIn.Hash): the one the + // content was proposed under, written with the content, and the one an + // ack records. Where closed pull requests are immutable + // (platform.Caps.ClosedImmutable) touchmark keeps it current while the + // pull request is open, and memory reads it as the ack of a decline. + OptIn string `json:"optin"` + Engine string `json:"engine"` + Packs []string `json:"packs"` + Changes []Change `json:"changes"` // ChangesComplete is false when Changes was dropped to fit the limits; // memory then matches only by the exact key. ChangesComplete bool `json:"changes_complete"` From 7df8af2c2956b4ff6ebe3653228e71be0b117616 Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Fri, 9 Oct 2026 01:28:31 +0300 Subject: [PATCH 13/20] feat(prbody): offer no tick boxes where descriptions cannot carry them Without Caps.BodyControls the body shows no control, the paused block asks for a recreate entry in .touchmark/operations.yml, and the footnote names forget_declines (also where closed pull requests are immutable). Only the Bitbucket goldens change. --- internal/prbody/controls_test.go | 47 +++++++++++++++++++ internal/prbody/helpers_test.go | 1 + internal/prbody/prbody.go | 34 +++++++++++++- internal/prbody/testdata/assumed/bitbucket.md | 2 +- .../prbody/testdata/declined/bitbucket.md | 2 +- internal/prbody/testdata/huge/bitbucket.md | 2 +- internal/prbody/testdata/local/bitbucket.md | 2 +- internal/prbody/testdata/new/bitbucket.md | 2 +- .../prbody/testdata/nothing-more/bitbucket.md | 2 +- internal/prbody/testdata/paused/bitbucket.md | 6 +-- .../testdata/sensitive-only/bitbucket.md | 2 +- 11 files changed, 89 insertions(+), 13 deletions(-) diff --git a/internal/prbody/controls_test.go b/internal/prbody/controls_test.go index 3abf671..56b68b2 100644 --- a/internal/prbody/controls_test.go +++ b/internal/prbody/controls_test.go @@ -5,6 +5,7 @@ import ( "testing" "github.com/bedrock-python/touchmark/internal/marker" + "github.com/bedrock-python/touchmark/internal/platform" ) // TestControlLine: the exported lines are Render's, and Ticked reads them. @@ -161,3 +162,49 @@ func TestUntick(t *testing.T) { } } } + +// TestNoBodyControls: where descriptions cannot carry tick boxes +// (Caps.BodyControls false: the marker is a reference definition), Render +// writes no control whatever ShowRecreate and ShowRepropose ask, names the +// recreate entry of operations.yml in the paused block and the +// forget_declines entry in the footnote. Where only closed pull requests +// are immutable, the controls stay and the footnote names the entry too; +// elsewhere it does not. +func TestNoBodyControls(t *testing.T) { + const recreateHint = "add a `recreate` entry with this branch's head to `.touchmark/operations.yml`" + const forgetHint = "add a `forget_declines` entry with the number of this pull request to `.touchmark/operations.yml`" + for _, c := range []struct { + name string + marker platform.MarkerStore + immutable bool + controls, recreate, forget bool + }{ + {"reference definition, immutable", platform.MarkerInRefDef, true, false, true, true}, + {"reference definition", platform.MarkerInRefDef, false, false, true, true}, + {"comment, immutable", platform.MarkerInBody, true, true, false, true}, + {"comment", platform.MarkerInBody, false, true, false, false}, + } { + in := scenarioInput(t, "paused", "github") + in.Caps.Marker, in.Caps.ClosedImmutable = c.marker, c.immutable + in.ShowRepropose = true + in.Marker = "" + body, err := Render(in) + if err != nil { + t.Fatalf("%s: %v", c.name, err) + } + for _, control := range []string{ControlRecreate, ControlRepropose} { + if got := strings.Contains(body, ControlLine(control)); got != c.controls { + t.Errorf("%s: control %s shown %v, want %v", c.name, control, got, c.controls) + } + } + if strings.Contains(body, " Rebuild this branch (drops commits added by others) +To rebuild the branch and drop the other commits, ask the hub's maintainers to add a `recreate` entry with this branch's head to `.touchmark/operations.yml`. --- -If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. touchmark rewrites this description, so please comment instead of editing it. +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. To have the same changes proposed again sooner, the hub's maintainers can add a `forget_declines` entry with the number of this pull request to `.touchmark/operations.yml`. touchmark rewrites this description, so please comment instead of editing it. [touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file diff --git a/internal/prbody/testdata/sensitive-only/bitbucket.md b/internal/prbody/testdata/sensitive-only/bitbucket.md index d057fd0..3dde12f 100644 --- a/internal/prbody/testdata/sensitive-only/bitbucket.md +++ b/internal/prbody/testdata/sensitive-only/bitbucket.md @@ -27,6 +27,6 @@ This pull request adds `.gitea/workflows` or `.forgejo/workflows`, which turns o --- -If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. touchmark rewrites this description, so please comment instead of editing it. +If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. To have the same changes proposed again sooner, the hub's maintainers can add a `forget_declines` entry with the number of this pull request to `.touchmark/operations.yml`. touchmark rewrites this description, so please comment instead of editing it. [touchmark]: # "touchmark:v1 hub=acme-eng fp=b7461799636b11ec stream=sync key=sha256:6b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b586b1f0c3a9e2d4b58 data=H4sIAAAAAAAC/5SQ4YrCMBCEX0Xmd/Rsq1bzKiJls9m0wTQpTRTkuHc/So87/96/nWGG+dhPPKErhVxmoREa+RUZCsPDQIN4lK3EHgpugkbvy/AwO07jR1vVzeF4as9QsMLeiu2oQKNxNVdkLvYsrTvR0Ry4sbVUbk8Xc+bWnuToDtRAgVMsEkvHaRz9P7uGskADCmkqPq6nxN7Hxd7v6t0eChPxPUNfQb3EknFT4IFiL4v5JxaCKUgRaEchi0LxJUiXZaHiIc2iN8tnNuuCzD72G8pZSl5Ykn2tAIGMhLz2rngLb3/CNwXi++/MLM90F/umeRYq0rk0Q8dHCF/fAQAA//9AxDKXogEAAA==" \ No newline at end of file From 7270bf80b7e331cfa34183dc6a538184acd82363 Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Fri, 9 Oct 2026 01:28:31 +0300 Subject: [PATCH 14/20] test(fake): add the Bitbucket flavor CapsFor(Bitbucket) matches the driver's Probe: no labels, closed pull requests immutable, the marker as a reference definition, native drafts, a known closer, no workflow permission. EditPR of a declined pull request and its reopening are ClassUnsupported, a person cannot reopen one, a close counts as an edit and a decline. --- internal/platform/fake/fake.go | 39 +++++++++++++-- internal/platform/fake/fake_test.go | 2 +- internal/platform/fake/gitserve.go | 7 +-- internal/platform/fake/meter.go | 9 ++-- internal/platform/fake/writer.go | 16 +++++++ internal/platform/fake/writer_test.go | 68 +++++++++++++++++++++++++++ 6 files changed, 129 insertions(+), 12 deletions(-) diff --git a/internal/platform/fake/fake.go b/internal/platform/fake/fake.go index 439342b..5578c66 100644 --- a/internal/platform/fake/fake.go +++ b/internal/platform/fake/fake.go @@ -14,7 +14,9 @@ // capabilities of its flavor: title-prefix drafts on // GitLab, Gitea and Forgejo; bodies and comments that would run GitLab // quick actions are refused (git mode runs them and records a violation); -// closers are not reported on Gitea and Forgejo. Branch rulesets, the +// closers are not reported on Gitea and Forgejo; on Bitbucket Cloud labels +// are refused and a pull request closed without merging can never be +// edited or reopened (Caps.ClosedImmutable). Branch rulesets, the // Preflighter of GitHub's driver (WithPreflight) and its API commits // (WithAPICommits, platform.Committer) are modeled in rules.go. // @@ -62,6 +64,8 @@ const ( GitLab Flavor = "gitlab" Gitea Flavor = "gitea" Forgejo Flavor = "forgejo" + // Bitbucket is Bitbucket Cloud. + Bitbucket Flavor = "bitbucket" ) // Tree entry modes. @@ -76,9 +80,12 @@ const ( var Epoch = time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC) // CapsFor returns the capabilities the fake reports for f, with the default -// limits of its platform. Every flavor keeps the marker in the body and has -// no API commit (WithAPICommits adds one). A flavor other than the four known -// ones gets GitHub's capabilities under its own name. +// limits of its platform. Every flavor keeps the marker in the body (as a +// Markdown reference definition on Bitbucket, MarkerInRefDef) and has no +// API commit (WithAPICommits adds one). Bitbucket's are those its driver's +// Probe reports: native drafts, no labels, a known closer, closed pull +// requests immutable, no permission of its own for CI files. A flavor other +// than the five known ones gets GitHub's capabilities under its own name. func CapsFor(f Flavor) platform.Caps { c := platform.Caps{ Flavor: string(f), @@ -102,6 +109,13 @@ func CapsFor(f Flavor) platform.Caps { c.LabelsByID = true c.CloserKnown = false c.Limits = platform.Limits{Reads: 4, GitReads: 2, MinInterval: 250 * time.Millisecond} + case Bitbucket: + c.MaxBody = 60000 + c.NoLabels = true + c.ClosedImmutable = true + c.Marker = platform.MarkerInRefDef + c.RuntimeOnly = []string{"branch-restrictions"} + c.Limits = platform.Limits{Reads: 2, GitReads: 2, ReadsPerMinute: 15, MinInterval: time.Second} default: c.WorkflowPerm = true c.Limits = platform.Limits{Reads: 8, GitReads: 4, WritesPerMinute: 60, WritesPerHour: 450, MinInterval: time.Second} @@ -696,7 +710,9 @@ func (p *Platform) AddPR(repoID string, pr platform.PR) int64 { // SetPRState moves a pull request to state as a person or bot would: // Closed and Merged record closedBy (nil: the platform does not say) and -// at (the clock when zero); Open reopens it and clears both. +// at (the clock when zero); Open reopens it and clears both. With +// Caps.ClosedImmutable no one can reopen a pull request closed without +// merging: that is a setup error. func (p *Platform) SetPRState(repoID string, number int64, state platform.PRState, closedBy *platform.Account, at time.Time) { p.mu.Lock() defer p.mu.Unlock() @@ -706,6 +722,10 @@ func (p *Platform) SetPRState(repoID string, number int64, state platform.PRStat } switch state { case platform.Open: + if p.caps.ClosedImmutable && ps.pr.State == platform.Closed { + p.setupf("SetPRState(%s, #%d): a declined pull request cannot be reopened", repoID, number) + return + } ps.pr.State, ps.pr.ClosedBy, ps.pr.ClosedAt = platform.Open, nil, time.Time{} if tip := p.headTip(ps.pr); tip != "" { ps.pr.HeadSHA = tip @@ -1152,6 +1172,8 @@ func (p *Platform) prURL(s *repoState, number int64) string { return base + "/-/merge_requests/" + n case Gitea, Forgejo: return base + "/pulls/" + n + case Bitbucket: + return base + "/pull-requests/" + n } return base + "/pull/" + n } @@ -1241,6 +1263,13 @@ func notFound(op, format string, args ...any) error { Err: fmt.Errorf(format+": %w", append(args, platform.ErrNotFound)...)} } +// unsupported is a write the platform refuses whatever the identity may +// do (a declined pull request on Bitbucket Cloud). +func unsupported(op, format string, args ...any) error { + return &platform.Error{Op: op, Class: platform.ClassUnsupported, Status: http.StatusBadRequest, + Err: fmt.Errorf(format, args...)} +} + func invalid(op, format string, args ...any) error { return &platform.Error{Op: op, Class: platform.ClassInvalid, Status: http.StatusUnprocessableEntity, Err: fmt.Errorf(format, args...)} diff --git a/internal/platform/fake/fake_test.go b/internal/platform/fake/fake_test.go index 701edde..cd6078a 100644 --- a/internal/platform/fake/fake_test.go +++ b/internal/platform/fake/fake_test.go @@ -46,7 +46,7 @@ func TestCapsFor(t *testing.T) { t.Errorf("Probe with an unknown account = %+v, %v; want an auth error", got, err) } } - if c := fake.CapsFor("bitbucket"); c.Flavor != "bitbucket" || !c.WorkflowPerm || c.Draft != platform.DraftNative { + if c := fake.CapsFor("azure-devops"); c.Flavor != "azure-devops" || !c.WorkflowPerm || c.Draft != platform.DraftNative { t.Errorf("CapsFor(unknown) = %+v, want GitHub's under its own name", c) } } diff --git a/internal/platform/fake/gitserve.go b/internal/platform/fake/gitserve.go index 175911c..aaa514c 100644 --- a/internal/platform/fake/gitserve.go +++ b/internal/platform/fake/gitserve.go @@ -83,11 +83,12 @@ func (p *Platform) gitHeader(id string) func(context.Context) (string, error) { } // appTokens reports whether the flavor is GitHub-like: identities are Apps -// whose tokens carry permissions (every flavor but GitLab, Gitea and -// Forgejo). Called with mu held. +// whose tokens carry permissions (every flavor but GitLab, Gitea, Forgejo +// and Bitbucket, whose identities are accounts with tokens of their own). +// Called with mu held. func (p *Platform) appTokens() bool { switch Flavor(p.caps.Flavor) { - case GitLab, Gitea, Forgejo: + case GitLab, Gitea, Forgejo, Bitbucket: return false } return true diff --git a/internal/platform/fake/meter.go b/internal/platform/fake/meter.go index ca337df..b45661c 100644 --- a/internal/platform/fake/meter.go +++ b/internal/platform/fake/meter.go @@ -178,15 +178,18 @@ func (t *target) createWrites(np platform.NewPR) (first, then int) { // editWrites is how many writes the flavor's driver sends for e, before // the edit is known to apply (first) and after (then): the edit, on Gitea -// and Forgejo after a state or base change in a request of its own; then -// labels added in a call of their own but on GitLab, after each missing one -// is created. +// and Forgejo after a state or base change in a request of its own, on +// Bitbucket followed by the decline of a close; then labels added in a call +// of their own but on GitLab, after each missing one is created. func (t *target) editWrites(e platform.PREdit) (first, then int) { first = 1 flavor := Flavor(t.p.caps.Flavor) if (flavor == Gitea || flavor == Forgejo) && (e.State != nil || e.Base != nil) && (e.Title != nil || e.Body != nil) { first++ } + if flavor == Bitbucket && e.State != nil && (e.Title != nil || e.Body != nil || e.Base != nil) { + first++ + } if len(e.AddLabels) > 0 && flavor != GitLab { then = 1 + t.missingLabels(e.AddLabels) } diff --git a/internal/platform/fake/writer.go b/internal/platform/fake/writer.go index 9fb0189..adb7420 100644 --- a/internal/platform/fake/writer.go +++ b/internal/platform/fake/writer.go @@ -266,6 +266,12 @@ func (t *target) createPR(ctx context.Context, np platform.NewPR) (platform.PR, // writer as the closer), a new base is taken to exist, labels are only // added. Draft never changes. // +// With Caps.ClosedImmutable (Bitbucket Cloud) a PR closed without merging +// is final, as Bitbucket's driver reports it: reopening it, or changing its +// title, body or base, is ClassUnsupported and changes nothing; Closed +// alone changes nothing. An open PR's new body is written before it closes, +// in the same call. +// // In git mode a new base must exist and reopening needs the head branch // (HeadSHA follows it again); after a new base or a reopening the flavor // reacts as after a push, and GitLab runs the body's quick actions. @@ -310,6 +316,16 @@ func (t *target) editPR(ctx context.Context, number int64, e platform.PREdit) (p if e.Base != nil && (*e.Base == "" || *e.Base == pr.Head) { return platform.PR{}, invalid(op, "base %q is empty or the head", *e.Base) } + if p.caps.ClosedImmutable && pr.State == platform.Closed { + changes := e.Title != nil && p.draftTitle(*e.Title, pr.Draft) != pr.Title || + e.Body != nil && *e.Body != pr.Body || e.Base != nil && *e.Base != pr.Base + switch { + case e.State != nil && *e.State == platform.Open: + return platform.PR{}, unsupported(op, "#%d: a declined pull request cannot be reopened", number) + case changes: + return platform.PR{}, unsupported(op, "#%d is declined: only open pull requests can be changed", number) + } + } if err := t.p.checkLabels(op, e.AddLabels); err != nil { return platform.PR{}, err } diff --git a/internal/platform/fake/writer_test.go b/internal/platform/fake/writer_test.go index c93bdbb..3ae60f3 100644 --- a/internal/platform/fake/writer_test.go +++ b/internal/platform/fake/writer_test.go @@ -5,6 +5,7 @@ import ( "net/http" "strings" "testing" + "time" "github.com/bedrock-python/touchmark/internal/platform" "github.com/bedrock-python/touchmark/internal/platform/fake" @@ -308,3 +309,70 @@ func TestCloseAndRemote(t *testing.T) { _, err = tw2.EnsureLabels(ctx, []string{"x"}) wantClass(t, "after an applied Close fault", err, platform.ClassAuth) } + +// TestBitbucketFlavor: the Bitbucket flavor reports the capabilities of +// Bitbucket's driver, refuses labels, and treats a pull request closed +// without merging as final: an edit that closes writes the body first, +// and afterwards no title, body or base changes and no one reopens it. +func TestBitbucketFlavor(t *testing.T) { + t.Parallel() + ctx := t.Context() + c := fake.CapsFor(fake.Bitbucket) + if c.Flavor != "bitbucket" || !c.NoLabels || !c.ClosedImmutable || c.Marker != platform.MarkerInRefDef || c.BodyControls() || + c.Draft != platform.DraftNative || !c.CloserKnown || c.WorkflowPerm || c.LabelsByID || c.QuickActions || c.MaxBody != 60000 { + t.Errorf("CapsFor(Bitbucket) = %+v", c) + } + for _, f := range []fake.Flavor{fake.GitHub, fake.GitLab, fake.Gitea, fake.Forgejo} { + if c := fake.CapsFor(f); c.ClosedImmutable || c.NoLabels || !c.BodyControls() { + t.Errorf("CapsFor(%s) = %+v", f, c) + } + } + e := newEnv(t, fake.WithFlavor(fake.Bitbucket)) + r := e.repo("acme/api", "README.md", "x") + tw := e.target(r) + _, err := tw.CreatePR(ctx, platform.NewPR{Head: "touchmark/hub", Base: "main", Title: "sync", Labels: []string{"engineering-assets"}}) + wantClass(t, "a pull request with labels", err, platform.ClassInvalid) + pr, err := tw.CreatePR(ctx, platform.NewPR{Head: "touchmark/hub", Base: "main", Title: "sync", Body: "first", Draft: true}) + if err != nil || pr.URL != "https://github.com/acme/api/pull-requests/1" || pr.Title != "sync" || !pr.Draft { + t.Fatalf("CreatePR = %+v, %v", pr, err) + } + _, err = tw.EditPR(ctx, pr.Number, platform.PREdit{AddLabels: []string{"x"}}) + wantClass(t, "a label added", err, platform.ClassInvalid) + _, err = tw.EnsureLabels(ctx, []string{"x"}) + wantClass(t, "EnsureLabels", err, platform.ClassInvalid) + + // The close writes its body first, and the writer is the closer. + closed := platform.Closed + got, err := tw.EditPR(ctx, pr.Number, platform.PREdit{Body: ptr("closed by touchmark"), State: &closed}) + if err != nil || got.State != platform.Closed || got.Body != "closed by touchmark" || got.ClosedBy == nil || got.ClosedBy.ID != e.writer.ID { + t.Fatalf("close = %+v, %v", got, err) + } + open := platform.Open + for name, edit := range map[string]platform.PREdit{ + "reopen": {State: &open}, + "body": {Body: ptr("again")}, + "title": {Title: ptr("renamed")}, + "base": {Base: ptr("develop")}, + "body and open": {Body: ptr("again"), State: &open}, + } { + _, err := tw.EditPR(ctx, pr.Number, edit) + wantRule(t, name, err, platform.ClassUnsupported, http.StatusBadRequest, "") + } + // What the declined pull request already holds, and closing it again, + // change nothing and succeed. + for name, edit := range map[string]platform.PREdit{ + "close again": {State: &closed}, + "the same body": {Body: ptr("closed by touchmark"), Title: ptr("sync"), Base: ptr("main")}, + } { + if _, err := tw.EditPR(ctx, pr.Number, edit); err != nil { + t.Errorf("%s: %v", name, err) + } + } + if after := e.p.PR(r.ID, pr.Number); after.State != platform.Closed || after.Body != "closed by touchmark" || after.Title != "sync" || + after.Base != "main" || len(after.Labels) != 0 { + t.Errorf("after the refused edits: %+v", after) + } + // A person cannot reopen it either. + e.p.SetPRState(r.ID, pr.Number, platform.Open, nil, time.Time{}) + wantSetupErr(t, e.p, "cannot be reopened") +} From d4bea94898441fce370218e865f3c2cc565ad6e5 Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Fri, 9 Oct 2026 01:28:32 +0300 Subject: [PATCH 15/20] feat(distribute): deliver to platforms whose closed pull requests are immutable Memory gets Caps.ClosedImmutable and reads tick boxes only where descriptions carry them; the open pull request's marker keeps the current optin there (one edit when it is stale); an unanchored decline gets a target warning; a forget_declines entry that lifts a decline while present is reported as applying. --- internal/distribute/body.go | 12 ++--- internal/distribute/execute_nolabels_test.go | 24 ++++------ internal/distribute/execworld_test.go | 8 ++-- internal/distribute/inspect.go | 2 +- internal/distribute/memory.go | 47 +++++++++++++++++--- internal/distribute/operations.go | 12 +++-- internal/distribute/run.go | 4 +- internal/distribute/steps.go | 11 +++-- 8 files changed, 82 insertions(+), 38 deletions(-) diff --git a/internal/distribute/body.go b/internal/distribute/body.go index 2dc38fd..585a357 100644 --- a/internal/distribute/body.go +++ b/internal/distribute/body.go @@ -280,9 +280,10 @@ func (p *provider) markerFrame() marker.Frame { // (blocked:edited, blocked:branch-taken) why (decide.Branch.Detail). It // renders the description when the decision writes one, and applies field // ownership (decide.PlanPREdit) to every StepEditPR: an edit that changes -// nothing and rewrites no content is skipped (Work.idle), and an unchanged -// target whose edit writes is reported as updated:title when only the title -// changes, else updated:body. +// nothing, rewrites no content and finds the marker's optin current +// (Work.staleOptIn) is skipped (Work.idle), and an unchanged target whose +// edit writes is reported as updated:title when only the title changes, +// else updated:body. func (r *run) reportDecision(w *Work, prs prSet, reads branchReads) error { res := &w.t.res d := w.Decision @@ -315,13 +316,14 @@ func (r *run) reportDecision(w *Work, prs prSet, reads branchReads) error { desired := decide.DesiredPR{Title: r.hub.PR.Title, Body: human, Labels: w.t.prov.prLabels(r.hub), Base: s.Base, DraftPrefix: draftPrefix(w.t.prov.caps)} edit, _, writes := decide.PlanPREdit(o.PR, o.Marker, desired) - if !writes && !s.Content { + stale := w.staleOptIn(o.Marker) + if !writes && !s.Content && !stale { w.idle[i] = true continue } if changed != "body" { changed = "body" - if titleOnly(edit, o.Marker, human) && !s.Content { + if titleOnly(edit, o.Marker, human) && !s.Content && !stale { changed = "title" } } diff --git a/internal/distribute/execute_nolabels_test.go b/internal/distribute/execute_nolabels_test.go index 715ad07..ed16dba 100644 --- a/internal/distribute/execute_nolabels_test.go +++ b/internal/distribute/execute_nolabels_test.go @@ -9,21 +9,15 @@ import ( "github.com/bedrock-python/touchmark/internal/decide" "github.com/bedrock-python/touchmark/internal/marker" "github.com/bedrock-python/touchmark/internal/platform" + "github.com/bedrock-python/touchmark/internal/platform/fake" "github.com/bedrock-python/touchmark/internal/report" ) -// bitbucketLike gives the fake what the core reads of Bitbucket Cloud's -// capabilities (the driver's Probe): no labels, the marker as a Markdown -// reference definition, native drafts, a known closer and no permission of -// its own for CI files. The fake refuses any label on such a platform. -func bitbucketLike(c *platform.Caps) { - c.NoLabels = true - c.Marker = platform.MarkerInRefDef - c.Draft = platform.DraftNative - c.CloserKnown = true - c.WorkflowPerm = false - c.LabelsByID = false -} +// noLabels gives the fake a platform without labels whose closed pull +// requests can still be edited (the core's handling of the two +// capabilities is independent; Bitbucket Cloud, fake.Bitbucket, has both). +// The fake refuses any label on such a platform. +func noLabels(c *platform.Caps) { c.NoLabels = true } // lastLine returns the last line of body. func lastLine(body string) string { return body[strings.LastIndexByte(body, '\n')+1:] } @@ -50,7 +44,7 @@ func noLabelCalls(t *testing.T, w *exWorld) { // label, write nothing; a content change and a close keep the frame and // still ask for no label. func TestExecuteNoLabelsLifecycle(t *testing.T) { - w := newExWorld(t, exConfig{git: true, caps: bitbucketLike}) + w := newExWorld(t, exConfig{git: true, flavor: fake.Bitbucket}) if len(w.hub.PR.Labels) == 0 { t.Fatal("fixture: hub.yml sets no labels") } @@ -111,7 +105,7 @@ func TestExecuteNoLabelsLifecycle(t *testing.T) { // and the pull request's marker in the comment frame (one written before) // is still found and replaced. func TestExecuteRefDefSweepClose(t *testing.T) { - w := newExWorld(t, exConfig{caps: bitbucketLike}) + w := newExWorld(t, exConfig{flavor: fake.Bitbucket}) tg := w.target("acme/gone", exOptIn, "version: 1\n") n := w.addPR(tg, platform.PR{Head: exBranch, Author: w.writer, Title: exTitle, Body: w.ownBody(w.missing(), nil)}) pr := w.pr(tg, n) @@ -137,7 +131,7 @@ func TestExecuteRefDefSweepClose(t *testing.T) { // The comment after a third auto-close does not advise exempting labels a // platform without labels never set. func TestExecuteNoLabelsAutoDeclinedComment(t *testing.T) { - w := newExWorld(t, exConfig{caps: bitbucketLike}) + w := newExWorld(t, exConfig{caps: noLabels}) stale := w.p.AddAccount("stale[bot]", platform.KindBot) tg := w.target("acme/docs", exOptIn, "version: 1\n") var last int64 diff --git a/internal/distribute/execworld_test.go b/internal/distribute/execworld_test.go index a92cfdc..528c5df 100644 --- a/internal/distribute/execworld_test.go +++ b/internal/distribute/execworld_test.go @@ -361,9 +361,10 @@ func (w *exWorld) work(t *target, edit func(*exState)) *Work { in.ForeignOpen = append(in.ForeignOpen, pr) } } + caps := w.prov.caps repropose := map[int64]bool{} for _, o := range in.Own { - if o.PR.State == platform.Closed && prbody.Ticked(o.PR.Body, prbody.ControlRepropose) { + if caps.BodyControls() && o.PR.State == platform.Closed && prbody.Ticked(o.PR.Body, prbody.ControlRepropose) { repropose[o.PR.Number] = true } } @@ -372,13 +373,14 @@ func (w *exWorld) work(t *target, edit func(*exState)) *Work { OptIn: wk.OptInHash, Repropose: repropose, Now: exEpoch, - Config: decide.MemoryConfig{Writers: map[string]bool{w.writer.ID: true}, CloserKnown: w.prov.caps.CloserKnown}, + Config: decide.MemoryConfig{Writers: map[string]bool{w.writer.ID: true}, CloserKnown: caps.CloserKnown, + ClosedImmutable: caps.ClosedImmutable}, }) in.CooldownUntil, in.CooldownDeclined = in.Memory.Cooldown(wk.Key, exEpoch, 0) in.PlatformWorkflowPerm, in.CanWorkflows = w.prov.caps.WorkflowPerm, true in.Now = exEpoch for _, o := range in.Own { - if o.PR.State == platform.Open && prbody.Ticked(o.PR.Body, prbody.ControlRecreate) { + if caps.BodyControls() && o.PR.State == platform.Open && prbody.Ticked(o.PR.Body, prbody.ControlRecreate) { in.RecreateTicked = true } } diff --git a/internal/distribute/inspect.go b/internal/distribute/inspect.go index b9711c2..0055308 100644 --- a/internal/distribute/inspect.go +++ b/internal/distribute/inspect.go @@ -658,7 +658,7 @@ func (r *run) decideTarget(w *Work, prs prSet, reads branchReads, ops decide.Tar CooldownUntil: until, CooldownDeclined: declined, Ops: ops, - RecreateTicked: hasKept && prbody.Ticked(kept.PR.Body, prbody.ControlRecreate), + RecreateTicked: hasKept && p.caps.BodyControls() && prbody.Ticked(kept.PR.Body, prbody.ControlRecreate), PlatformWorkflowPerm: p.caps.WorkflowPerm, CanWorkflows: can, WorkflowsDiffer: reads.differ, diff --git a/internal/distribute/memory.go b/internal/distribute/memory.go index cfac0ef..abad02f 100644 --- a/internal/distribute/memory.go +++ b/internal/distribute/memory.go @@ -1,10 +1,12 @@ package distribute import ( + "fmt" "slices" "github.com/bedrock-python/touchmark/internal/config" "github.com/bedrock-python/touchmark/internal/decide" + "github.com/bedrock-python/touchmark/internal/marker" "github.com/bedrock-python/touchmark/internal/platform" "github.com/bedrock-python/touchmark/internal/prbody" ) @@ -12,11 +14,17 @@ import ( // memory runs step 7: what the target's closed own pull requests say // (decide.BuildMemory), with the opt-in file's hash, the paths the per-path // plan finds local or ignored, the forget_declines of operations and the -// ticked repropose controls of closed bodies. +// ticked repropose controls of closed bodies (none where descriptions carry +// no control, Caps.BodyControls). +// +// On a platform whose closed pull requests are immutable +// (Caps.ClosedImmutable), each decline in force whose marker records no +// opt-in state gets a target warning that says how to lift it: memory +// cannot tell when the team changed its choice (Memory.Unanchored). func (r *run) memory(t *target, w *Work, ops decide.TargetOps) decide.Memory { repropose := map[int64]bool{} for _, o := range w.Own { - if o.PR.State == platform.Closed && prbody.Ticked(o.PR.Body, prbody.ControlRepropose) { + if t.prov.caps.BodyControls() && o.PR.State == platform.Closed && prbody.Ticked(o.PR.Body, prbody.ControlRepropose) { repropose[o.PR.Number] = true } } @@ -31,7 +39,7 @@ func (r *run) memory(t *target, w *Work, ops decide.TargetOps) decide.Memory { for _, id := range t.prov.ids { writers[id] = true } - return decide.BuildMemory(decide.MemoryInput{ + m := decide.BuildMemory(decide.MemoryInput{ Own: w.Own, OptIn: w.OptInHash, LocalOrIgnored: func(path string) bool { return local[path] }, @@ -39,12 +47,37 @@ func (r *run) memory(t *target, w *Work, ops decide.TargetOps) decide.Memory { Repropose: repropose, Now: r.now, Config: decide.MemoryConfig{ - Cooldown: r.cooldown, - Writers: writers, - Automation: t.prov.automation, - CloserKnown: t.prov.caps.CloserKnown, + Cooldown: r.cooldown, + Writers: writers, + Automation: t.prov.automation, + CloserKnown: t.prov.caps.CloserKnown, + ClosedImmutable: t.prov.caps.ClosedImmutable, }, }) + for _, n := range m.Unanchored { + r.targetWarning(t, unanchoredWarning(r.prName(t, n))) + } + return m +} + +// unanchoredWarning is the target warning of a decline in force whose +// marker records no opt-in state, on a platform whose closed pull requests +// cannot be edited (decide.Memory.Unanchored): pr is "#7". +func unanchoredWarning(pr string) string { + return fmt.Sprintf("the decline of %s holds until a forget_declines entry in .touchmark/operations.yml names it: "+ + "its marker records no state of the opt-in file, so a change of packs or ignore does not lift it, "+ + "and a declined pull request cannot be edited on this platform", pr) +} + +// staleOptIn reports whether the marker m of an open pull request of w must +// be written again to record the current opt-in state: on a platform whose +// closed pull requests are immutable, the optin an open pull request's +// marker holds is what memory reads once the pull request is declined +// (decide.MemoryConfig.ClosedImmutable), so it follows the opt-in file +// while the pull request is open. Never on the other platforms, whose +// bodies an opt-in change alone leaves as they are. +func (w *Work) staleOptIn(m marker.Marker) bool { + return w.t.prov.caps.ClosedImmutable && m.Key != "" && m.Data.OptIn != w.OptInHash } // operations are the operations of the run: operations.yml of the default diff --git a/internal/distribute/operations.go b/internal/distribute/operations.go index d4401c1..f9f39c8 100644 --- a/internal/distribute/operations.go +++ b/internal/distribute/operations.go @@ -33,9 +33,11 @@ const opShort = 7 // spent), when touchmark's commit is on top anyway, or when the // decision does not rebuild it (the detail names the outcome). // - forget_declines: applies when the decision revokes the pull request's -// decline (StepRevoke); none when it is no decline in force (already -// revoked, open, no pull request of touchmark's) or the target writes -// nothing in this run. +// decline (StepRevoke), or, on a platform whose closed pull requests are +// immutable, when memory leaves the close out because of the entry +// (decide.Memory.Forgotten: the entry acts while it is present); none +// when it is no decline in force (already revoked, open, no pull +// request of touchmark's) or the target writes nothing in this run. // - allow_mass_close: expired past its date; else applies when the run // closes more pull requests than the guard allows without the entry and // no more than its max; none when the closes fit anyway or exceed even @@ -247,6 +249,10 @@ func (r *run) opForget(op *report.Operation, t *target) { case slices.Contains(w.Memory.ToRevoke, n): op.Detail = fmt.Sprintf("would revoke the decline of %s, but the target writes nothing in this run: %s", r.prName(t, n), outcomeText(report.Outcome(w.Decision.Outcome), w.Decision.Reason)) + case slices.Contains(w.Memory.Forgotten, n): + op.Effect, op.Detail = report.EffectApplies, fmt.Sprintf("lifts the decline of %s while the entry is present: "+ + "a declined pull request cannot be edited on this platform, so touchmark cannot record it there; "+ + "keep it until the pull request it brings is merged or closed", r.prName(t, n)) case i < 0: op.Detail = fmt.Sprintf("%s is no pull request of touchmark's on the target's sync branches: it does nothing; remove it", r.prName(t, n)) case w.Own[i].PR.State == platform.Open: diff --git a/internal/distribute/run.go b/internal/distribute/run.go index 5610015..d41bd62 100644 --- a/internal/distribute/run.go +++ b/internal/distribute/run.go @@ -140,7 +140,9 @@ type WriteDeps struct { // 7. memory: decide.BuildMemory over own closed PRs with the opt-in hash // (config.OptIn.Hash), paths local, retired-local or ignored in the // per-path plan, operations forget_declines and ticked repropose -// controls (prbody.Ticked on closed bodies); Cooldown for the key. +// controls (prbody.Ticked on closed bodies, where descriptions carry +// controls: Caps.BodyControls), and Caps.ClosedImmutable; Cooldown for +// the key. // 8. the commit, when D is not empty: BuildCommit of D on B with the hub's // blobs, the writer as author and committer (with the platform's // no-reply address when it gives no email), dated diff --git a/internal/distribute/steps.go b/internal/distribute/steps.go index ebb6449..43b753c 100644 --- a/internal/distribute/steps.go +++ b/internal/distribute/steps.go @@ -233,7 +233,8 @@ func (p *preparer) create(ctx context.Context, s decide.Step) ([]writeAct, error } // editOpen edits the open pull request's fields touchmark owns: nothing -// when PlanPREdit changes nothing and Content is not set. +// when PlanPREdit changes nothing, Content is not set and the marker's +// optin is current (Work.staleOptIn; a stale one is brought up to date). func (p *preparer) editOpen(ctx context.Context, s decide.Step) ([]writeAct, error) { x := p.x pr, m, err := x.known(s.PR) @@ -253,13 +254,17 @@ func (p *preparer) editOpen(ctx context.Context, s decide.Step) ([]writeAct, err DraftPrefix: draftPrefix(x.t.prov.caps), }) empty := m.Key == "" - if !changed && !s.Content && !empty { + stale := x.w.staleOptIn(m) + if !changed && !s.Content && !empty && !stale { return nil, nil } key := m.Key if s.Content || empty { next, key = x.withContent(ctx, next, p.pushed, x.branchOf(s.Branch)) } + if stale { + next.OptIn = x.w.OptInHash + } // A consumed recreate is done once the branch is rebuilt: by this run's // push, or by the push of a run that stopped before this edit. Any other // edit finds it stale (people moved the branch again, and the block asks @@ -279,7 +284,7 @@ func (p *preparer) editOpen(ctx context.Context, s decide.Step) ([]writeAct, err } body := human + "\n\n" + line field := "body" - if titleOnly(edit, m, human) && !s.Content && !empty { + if titleOnly(edit, m, human) && !s.Content && !empty && !stale { field = "title" } edit.Body = &body From 5bb133748713bd42223c9d8ec71e6343a53ebd36 Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Fri, 9 Oct 2026 01:28:32 +0300 Subject: [PATCH 16/20] test(distribute): delivery scenarios on Bitbucket Decline, its lapse on an opt-in change, forget_declines while present, an unanchored decline, touchmark's own closes (no-diff, target-dropped), no labels and no tick boxes; bodies elsewhere stay byte for byte on an opt-in change. The property test runs 30 Bitbucket scenarios, the crash matrix three Bitbucket cases. --- internal/distribute/run_bitbucket_test.go | 323 ++++++++++++++++++++++ internal/distribute/sim_check_test.go | 6 +- internal/distribute/sim_crash_test.go | 11 + internal/distribute/sim_property_test.go | 43 +++ internal/distribute/sim_world_test.go | 7 +- 5 files changed, 386 insertions(+), 4 deletions(-) create mode 100644 internal/distribute/run_bitbucket_test.go diff --git a/internal/distribute/run_bitbucket_test.go b/internal/distribute/run_bitbucket_test.go new file mode 100644 index 0000000..39c0c6b --- /dev/null +++ b/internal/distribute/run_bitbucket_test.go @@ -0,0 +1,323 @@ +package distribute + +import ( + "slices" + "strings" + "testing" + "time" + + "github.com/bedrock-python/touchmark/internal/config" + "github.com/bedrock-python/touchmark/internal/marker" + "github.com/bedrock-python/touchmark/internal/platform" + "github.com/bedrock-python/touchmark/internal/platform/fake" + "github.com/bedrock-python/touchmark/internal/provenance" + "github.com/bedrock-python/touchmark/internal/report" +) + +// The delivery scenarios on Bitbucket Cloud's flavor of the fake: no +// labels, the marker as a Markdown reference definition, no tick boxes in +// descriptions, and declined pull requests that no one can edit or reopen +// (Caps.ClosedImmutable). Every run goes through Run's full phase C in git +// mode. + +// bbWorld is a git world of the Bitbucket flavor. +func bbWorld(t *testing.T) *gitWorld { + t.Helper() + g := newGitWorld(t, fake.WithFlavor(fake.Bitbucket)) + if c := g.p.Caps(); !c.ClosedImmutable || !c.NoLabels || c.BodyControls() { + t.Fatalf("fixture: caps %+v", c) + } + return g +} + +// bbForget is a deps edit that adds the operations.yml entry forgetting +// pull request n of acme/api (nil ops when n is 0). +func bbForget(n int64) func(*Deps) { + return func(d *Deps) { + if n > 0 { + d.Write.Operations = &config.Operations{Version: 1, ForgetDeclines: []config.ForgetOp{{Target: "acme/api", PR: n}}} + } + } +} + +// bbCheckBody checks what touchmark writes into a description on +// Bitbucket: the marker as a reference definition on the last line, no HTML +// comment and no tick box anywhere, the forget_declines hint in the +// footnote, no labels; it returns the marker. +func (g *gitWorld) bbCheckBody(pr platform.PR) marker.Marker { + g.t.Helper() + last := pr.Body[strings.LastIndexByte(pr.Body, '\n')+1:] + if !strings.HasPrefix(last, `[touchmark]: # "touchmark:v1 `) || strings.Contains(pr.Body, "" -// (its payload left open when the closing quote is missing, so that Parse -// refuses it). ok is false for any other line. Nothing is trimmed: the -// callers trim the trailing blanks and carriage return they accept. +// definition, `[touchmark]: # ""` becomes "". +// A reference definition without its closing quote is still a marker line +// (ok is true), but its comment form never ends in " -->", so Parse refuses +// it even when the payload itself ends in " -->". ok is false for any other +// line. Nothing is trimmed: the callers trim the trailing blanks and +// carriage return they accept. // // A reference definition is read after unescape: Bitbucket Cloud returns an // edited description with backslashes before punctuation ("\[touchmark\]", @@ -74,7 +76,10 @@ func asComment(line string) (string, bool) { if payload, ok := strings.CutSuffix(rest, refDefClose); ok { return commentOpen + payload + commentEnd, true } - return commentOpen + rest, true + // No closing quote: the comment form ends in a quote, never in " -->", + // whatever the payload ends in, so Parse refuses it while Find and + // Strip still see a marker line. + return commentOpen + rest + refDefClose, true } // unescape undoes Markdown's backslash escapes in s: a backslash before diff --git a/internal/marker/frame_test.go b/internal/marker/frame_test.go index 5261268..b6a607b 100644 --- a/internal/marker/frame_test.go +++ b/internal/marker/frame_test.go @@ -100,6 +100,8 @@ func TestParseRefDef(t *testing.T) { } refused := []struct{ name, line string }{ {"no closing quote", strings.TrimSuffix(line, `"`)}, + {"no closing quote, comment end", strings.TrimSuffix(line, `"`) + commentEnd}, + {"no closing quote, comment end escaped", escapeAll(strings.TrimSuffix(line, `"`) + commentEnd)}, {"trailing blank", line + " "}, {"CR", line + "\r"}, {"single quotes", strings.Replace(strings.Replace(line, `# "`, `# '`, 1), `"`, `'`, 1)}, @@ -132,6 +134,7 @@ func TestIsLine(t *testing.T) { {escapeAll(line), true}, {line + "\r", true}, {strings.TrimSuffix(line, `"`), true}, + {strings.TrimSuffix(line, `"`) + commentEnd, true}, {`[touchmark]: # "touchmark:v9 x"`, true}, {"", true}, {"- [ ] Rebuild this branch", false}, diff --git a/internal/marker/fuzz_test.go b/internal/marker/fuzz_test.go index 9b4be79..9789dfe 100644 --- a/internal/marker/fuzz_test.go +++ b/internal/marker/fuzz_test.go @@ -16,7 +16,7 @@ func seedLines(t testing.TB) []string { line := mustEncode(t, v.m) lines = append(lines, line, line[:len(line)/2], strings.Replace(line, "v1", "v2", 1), line+"\r") rd := refDef(line) - lines = append(lines, rd, escapeAll(rd), rd[:len(rd)/2], rd+"\r") + lines = append(lines, rd, escapeAll(rd), rd[:len(rd)/2], rd+"\r", strings.TrimSuffix(rd, refDefClose)+commentEnd) } lines = append(lines, lineWithData(gz(t, bytes.Repeat([]byte{' '}, 8<<20))), @@ -26,6 +26,7 @@ func seedLines(t testing.TB) []string { "", `[touchmark]: # "touchmark:v1 hub= fp= stream= key= data="`, `\[touchmark\]: \# "touchmark:v1\"`, + `[touchmark]: # "touchmark:v1 hub= fp= stream= key= data= -->`, ) return lines } @@ -43,6 +44,9 @@ func FuzzParse(f *testing.F) { if err != nil { return } + if !strings.HasPrefix(line, commentPrefix) && !strings.HasSuffix(unescape(line), refDefClose) { + t.Fatalf("accepted a reference definition without its closing quote: %.80q", line) + } if m.FP16 != FP16(m.Data.FP) || m.Hub != m.Data.Hub || m.Stream != m.Data.Stream { t.Fatalf("header does not match the data: %s", describeMarker(m)) } From 5e05d63cb3901aa14a4ae2cbbe8039a225700c86 Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Fri, 9 Oct 2026 02:02:28 +0300 Subject: [PATCH 19/20] fix(distribute): record the opt-in state in a blocked open pull request on Bitbucket --- docs/concepts/memory.md | 14 ++- internal/decide/target.go | 9 ++ internal/decide/target_test.go | 2 +- internal/distribute/execute.go | 11 +- internal/distribute/inspect.go | 1 + internal/distribute/memory.go | 39 +++++++ internal/distribute/run.go | 7 +- internal/distribute/run_bitbucket_test.go | 135 ++++++++++++++++++++++ internal/distribute/steps.go | 92 ++++++++++++++- internal/distribute/target_exec.go | 3 + internal/distribute/write.go | 43 +++++++ 11 files changed, 346 insertions(+), 10 deletions(-) diff --git a/docs/concepts/memory.md b/docs/concepts/memory.md index 0d451f0..2e302dd 100644 --- a/docs/concepts/memory.md +++ b/docs/concepts/memory.md @@ -83,7 +83,10 @@ request was open: in its marker the state of the opt-in file it is proposed under, which the acknowledgment records elsewhere. When the file changes while the pull request is open, the next run records the new state, with one edit of the description even when - the proposed changes stay the same. + the proposed changes stay the same, and even when the target is blocked + (`branch-in-use`, `rules:*`, …): when the run writes nothing else to the pull + request, that edit changes only the marker. A push a branch restriction refuses at + run time is followed by that edit too. - **A decline** holds while the opt-in file is parsed the same as that recorded state, and lapses when the team edits `packs` or `ignore`, as an acknowledged decline does elsewhere. A declined path becoming `local` or `ignored` lifts it too. @@ -118,6 +121,9 @@ request was open: - Closed pull requests of the multi-gitter era, without a marker, are never declines: their content is unknown. - On Bitbucket Cloud the opt-in state of a decline is the one recorded by the last run - before the decline. When the team changes `packs` or `ignore` and declines the pull - request before touchmark runs again, the decline counts from the earlier state and has - lapsed already: the content is proposed once more, under the new choice. + before the decline, whatever that run decided for the target. When the team changes + `packs` or `ignore` and declines the pull request before touchmark runs again, the + decline counts from the earlier state and has lapsed already: the content is proposed + once more, under the new choice. A blocked target does not widen that window: its run + records the state as well. Only a run that writes nothing to the target at all (a + failed or deferred target, `blocked:mass-close`) leaves it open until the next run. diff --git a/internal/decide/target.go b/internal/decide/target.go index b3a6b61..bb8ec9a 100644 --- a/internal/decide/target.go +++ b/internal/decide/target.go @@ -82,6 +82,13 @@ const ( // StepConsumeRecreate unticks the recreate control and writes // recreate_for = Expect into the open PR's marker, before the push. StepConsumeRecreate + // StepRefreshMarker writes the current opt-in state into the marker of + // an open own PR no other step writes to: one body-only edit, the rest + // of the body and of the marker kept. DecideTarget never asks for it: + // the caller adds it where closed PRs are immutable, whose memory reads + // a decline's opt-in state from the marker the PR had while open + // (MemoryConfig.ClosedImmutable). + StepRefreshMarker ) // String returns the step name for plans, messages and tests. @@ -107,6 +114,8 @@ func (k StepKind) String() string { return "revoke" case StepConsumeRecreate: return "consume-recreate" + case StepRefreshMarker: + return "refresh-marker" } return "unknown" } diff --git a/internal/decide/target_test.go b/internal/decide/target_test.go index 0e853eb..d477b2f 100644 --- a/internal/decide/target_test.go +++ b/internal/decide/target_test.go @@ -768,7 +768,7 @@ func TestStepKindString(t *testing.T) { want := map[StepKind]string{ StepPush: "push", StepRecreateBranch: "recreate-branch", StepDeleteBranch: "delete-branch", StepCreatePR: "create-pr", StepEditPR: "edit-pr", StepClosePR: "close-pr", StepComment: "comment", StepAck: "ack", StepRevoke: "revoke", - StepConsumeRecreate: "consume-recreate", 0: "unknown", 99: "unknown", + StepConsumeRecreate: "consume-recreate", StepRefreshMarker: "refresh-marker", 0: "unknown", 99: "unknown", } for k, w := range want { if got := k.String(); got != w { diff --git a/internal/distribute/execute.go b/internal/distribute/execute.go index 062622d..690cc6d 100644 --- a/internal/distribute/execute.go +++ b/internal/distribute/execute.go @@ -67,7 +67,10 @@ import ( // AutoDeclinedComment by Step.Reason; // - StepAck: EditPR of the declined PR: marker ack and optin, body with // prbody.AddControl(repropose); -// - StepRevoke: EditPR of the declined PR: marker revoked. +// - StepRevoke: EditPR of the declined PR: marker revoked; +// - StepRefreshMarker: EditPR of an open own PR, the body as last read +// with the marker's optin set to the opt-in file's hash (nothing when +// it is current already). // Before writing any text (body, comment, commit message, branch // name), Write.Redact.Contains → failed:secret-exposure, nothing // written. @@ -148,6 +151,12 @@ import ( // in a warning) and no target starts; a target's block of writes has 10 // minutes from its start (then failed:transient), and never lasts past // Write.Deadline by more than the grace (then deferred:deadline). +// - Where closed pull requests are immutable, a push refused for good +// that ends the target blocked is followed by the edits that record +// the current opt-in state in the open own pull requests the writes +// after it would have edited, when their marker's optin is stale +// (fallbacks, recordOptIn): secondary writes, rendered and checked with +// the others before the first write. // - An edit step phase C found idle (Work.idle) is skipped; every other // edit renders the body as phase C did (Work.humanBody), so that the // run writes what its plan showed. diff --git a/internal/distribute/inspect.go b/internal/distribute/inspect.go index 0055308..36012c5 100644 --- a/internal/distribute/inspect.go +++ b/internal/distribute/inspect.go @@ -603,6 +603,7 @@ func (r *run) inspectFull(ctx context.Context, t *target, optIn *config.OptIn, s r.blockProtected(w, protected) } r.checkSigning(w) + w.refreshOptIn() w.NeedPerms = needPerms(w.Decision.Steps) r.fillBody(w, sel, reads) if err := r.reportDecision(w, prs, reads); err != nil { diff --git a/internal/distribute/memory.go b/internal/distribute/memory.go index abad02f..a719780 100644 --- a/internal/distribute/memory.go +++ b/internal/distribute/memory.go @@ -80,6 +80,45 @@ func (w *Work) staleOptIn(m marker.Marker) bool { return w.t.prov.caps.ClosedImmutable && m.Key != "" && m.Data.OptIn != w.OptInHash } +// refreshOptIn adds a decide.StepRefreshMarker for each open own pull +// request of w whose marker's optin is stale (Work.staleOptIn) and that no +// step of the final decision writes to, after every rule that rewrites the +// decision in phase C (blockProtected, checkSigning): one body-only edit +// each, which the estimates count. Without it, a decline of such a pull +// request would be read from the opt-in state it was opened under, and +// would lapse at once. Only where closed pull requests are immutable +// (staleOptIn is false elsewhere). +// +// Which pull requests it writes to, and why: +// - Only w.Own: pull requests of touchmark's authors that carry a valid +// marker of this hub. A foreign pull request (the one that blocks +// branch-in-use, a fork's, another hub's) and a marker-invalid one are +// never there; an adopted one has no marker key, so it is never stale. +// - A pull request a step writes to already gets the current optin: an +// edit (Work.staleOptIn makes it write), a close (a close of touchmark's +// is no decline), a consumed recreate (an edit follows it). +// - blocked:branch-in-use, blocked:rules:*, blocked:permission:workflows +// and blocked:cannot-sign keep our open pull request without writing to +// it: it gets the edit. The edit moves no branch, so the foreign pull +// request or the rule that blocks the push plays no part. +// - blocked:edited edits the kept pull request already (its Paused or +// NothingMore block); a duplicate left open on an edited or foreign +// branch gets the edit, which touches no commit of people's. +// - blocked:branch-taken has no own open pull request (DecideTarget's +// rule 3): there is nothing to refresh. +// - blocked:marker-invalid writes memory upkeep only (DecideTarget's rule +// 1); the edit is upkeep of the same kind, on a pull request that is +// ours by its marker, and leaves the marker-invalid one alone. +func (w *Work) refreshOptIn() { + d := &w.Decision + for _, o := range w.Own { + if o.PR.State != platform.Open || !w.staleOptIn(o.Marker) || touches(*d, o.PR.Number) { + continue + } + d.Steps = append(d.Steps, decide.Step{Kind: decide.StepRefreshMarker, PR: o.PR.Number, Branch: o.PR.Head}) + } +} + // operations are the operations of the run: operations.yml of the default // branch, then those given as flags in a local run; both apply. func (r *run) operations() []*config.Operations { diff --git a/internal/distribute/run.go b/internal/distribute/run.go index d41bd62..c3ebbc5 100644 --- a/internal/distribute/run.go +++ b/internal/distribute/run.go @@ -176,8 +176,11 @@ type WriteDeps struct { // else the platform's API commit (Caps.Commit.API: phase F commits // through the stage ref), else, outside ModePlan, which never holds // the key and plans as if it could sign, a push becomes -// blocked:cannot-sign (memory upkeep stays). The report gets the -// outcome, reason and PR; an unchanged target whose StepEditPR writes +// blocked:cannot-sign (memory upkeep stays). Where closed pull +// requests are immutable, each open own pull request whose marker's +// optin is stale and that no step writes to gets a body-only edit that +// records the current opt-in state (Work.refreshOptIn). The report gets +// the outcome, reason and PR; an unchanged target whose StepEditPR writes // (decide.PlanPREdit on the rendered description, or content to // rewrite) is updated:title or updated:body, and one that writes // nothing marks the step idle. diff --git a/internal/distribute/run_bitbucket_test.go b/internal/distribute/run_bitbucket_test.go index 39c0c6b..012d6eb 100644 --- a/internal/distribute/run_bitbucket_test.go +++ b/internal/distribute/run_bitbucket_test.go @@ -1,6 +1,7 @@ package distribute import ( + "reflect" "slices" "strings" "testing" @@ -321,3 +322,137 @@ func TestRunOptInChangeElsewhere(t *testing.T) { }) } } + +// TestRunBitbucketBlockedOptIn: an opt-in change while the open pull +// request is blocked reaches its marker in one body-only edit, even though +// the decision writes nothing else to it, so that a decline of it then +// holds under the new opt-in state instead of lapsing at once. The blocks: +// someone else's pull request on the sync branch (branch-in-use), and a +// rule against force pushes the plan reads ahead (rules:non-fast-forward). +func TestRunBitbucketBlockedOptIn(t *testing.T) { + t.Parallel() + t0 := time.Date(2026, 9, 2, 12, 0, 0, 0, time.UTC) + optIn2 := "version: 1\nignore: [notes/**]\n" + + t.Run("branch-in-use", func(t *testing.T) { + t.Parallel() + g := bbWorld(t) + api := g.optedIn("acme/api", nil) + g.bbStep("the first run", nil, report.OutcomeOpened, "", 1, 2) + foreign := g.p.AddPR(api.ID, platform.PR{Head: branch, Author: g.person, Title: "wip", Body: "My changes."}) + g.ok() + g.bbStep("someone else's pull request on the sync branch", nil, report.OutcomeBlocked, "branch-in-use", foreign, 0) + before := g.p.PR(api.ID, 1) + + g.push(api, "main", g.person, optInName, optIn2) + g.bbStep("the opt-in file changed while blocked", nil, report.OutcomeBlocked, "branch-in-use", foreign, 1) + g.bbCheckRefreshed(api, before, optIn2) + + g.p.SetPRState(api.ID, foreign, platform.Closed, &g.person, t0) + g.p.SetPRState(api.ID, 1, platform.Closed, &g.person, t0.Add(time.Minute)) + g.ok() + g.bbStep("#1 declined", nil, report.OutcomeDeclined, "", 1, 0) + }) + + t.Run("rules:non-fast-forward", func(t *testing.T) { + t.Parallel() + g := newGitWorld(t, fake.WithFlavor(fake.Bitbucket), fake.WithPreflight()) + api := g.optedIn("acme/api", nil) + g.bbStep("the first run", nil, report.OutcomeOpened, "", 1, 2) + g.p.AddRuleset(api.ID, fake.Ruleset{Branches: []string{"touchmark/*"}, NonFastForward: true}) + g.ok() + g.edit = bbHubChange(g) + g.bbStep("the hub changed, force pushes refused", nil, report.OutcomeBlocked, "rules:non-fast-forward", 1, 0) + before := g.p.PR(api.ID, 1) + + g.push(api, "main", g.person, optInName, optIn2) + g.bbStep("the opt-in file changed while blocked", nil, report.OutcomeBlocked, "rules:non-fast-forward", 1, 1) + g.bbCheckRefreshed(api, before, optIn2) + + // Declined, and the hub proposes what #1 carries again: the decline + // holds. + g.p.SetPRState(api.ID, 1, platform.Closed, &g.person, t0) + g.ok() + g.edit = nil + g.bbStep("#1 declined", nil, report.OutcomeDeclined, "", 1, 0) + }) + + // Bitbucket Cloud's driver cannot read branch restrictions ahead: the + // push meets the rule at run time, and the edit after it does not + // happen. The marker is refreshed then instead, once. + t.Run("rules at run time", func(t *testing.T) { + t.Parallel() + g := bbWorld(t) + api := g.optedIn("acme/api", nil) + g.bbStep("the first run", nil, report.OutcomeOpened, "", 1, 2) + g.p.AddRuleset(api.ID, fake.Ruleset{Branches: []string{"touchmark/*"}, NonFastForward: true}) + g.ok() + g.edit = bbHubChange(g) + edits := func(what string, want int) { + t.Helper() + rep := g.run(g.deps(ModeDistribute), ModeDistribute) + tg := wantAPI(t, rep, report.OutcomeBlocked, "rules:non-fast-forward", 1) + n := 0 + for _, w := range g.p.Writes() { + if strings.HasPrefix(w, "EditPR") { + n++ + } + } + if n != want || tg.Writes != want { + t.Errorf("%s: %d edits (report %d writes), want %d: %q", what, n, tg.Writes, want, g.p.Writes()) + } + } + edits("the hub changed, force pushes refused", 0) + before := g.p.PR(api.ID, 1) + + g.push(api, "main", g.person, optInName, optIn2) + edits("the opt-in file changed while blocked", 1) + g.bbCheckRefreshed(api, before, optIn2) + edits("again", 0) + + g.p.SetPRState(api.ID, 1, platform.Closed, &g.person, t0) + g.ok() + g.edit = nil + g.bbStep("#1 declined", nil, report.OutcomeDeclined, "", 1, 0) + }) +} + +// wantAPI is want for the target acme/api. +func wantAPI(t *testing.T, rep *report.Delivery, outcome report.Outcome, reason string, pr int64) report.DeliveryTarget { + t.Helper() + return want(t, rep, "gh:acme/api", outcome, reason, pr) +} + +// bbHubChange is a deps edit under which the hub proposes a new AGENTS.md. +func bbHubChange(g *gitWorld) func(*Deps) { + agents3 := version("AGENTS.md", 3) + man, cur := manifestAndCurrent() + v := provenance.Version{OID: oid(agents3), Size: int64(len(agents3))} + man.Add("AGENTS.md", "base", v) + cur["base"]["AGENTS.md"] = provenance.File{Pack: "base", Path: "AGENTS.md", OID: v.OID, Size: v.Size, Mode: "100644"} + g.blobs[v.OID] = []byte(agents3) + return func(d *Deps) { d.Manifest, d.Current = man, cur } +} + +// bbCheckRefreshed checks that open pull request #1 of r, before as it +// was, now records the opt-in file optIn in its marker, and that nothing +// else of its description or marker changed. +func (g *gitWorld) bbCheckRefreshed(r platform.Repo, before platform.PR, optIn string) { + g.t.Helper() + after := g.p.PR(r.ID, 1) + m := g.bbCheckBody(after) + if after.State != platform.Open || m.Data.OptIn != optInHash(g.t, optIn) { + g.t.Errorf("#1: %s, optin %q", after.State, m.Data.OptIn) + } + if a, b := marker.Strip(after.Body), marker.Strip(before.Body); a != b { + g.t.Errorf("#1: the description changed beyond its marker:\n%s\nwas\n%s", a, b) + } + was, _ := marker.Find(before.Body, []string{hubFP}) + was.Data.OptIn = m.Data.OptIn + if !reflect.DeepEqual(was, m) { + g.t.Errorf("#1: the marker changed beyond its optin:\n%+v\nwas\n%+v", m, was) + } + if after.Title != before.Title { + g.t.Errorf("#1: title %q, was %q", after.Title, before.Title) + } +} diff --git a/internal/distribute/steps.go b/internal/distribute/steps.go index 43b753c..788c68c 100644 --- a/internal/distribute/steps.go +++ b/internal/distribute/steps.go @@ -84,7 +84,15 @@ func (x *targetExec) prepare(ctx context.Context) ([]writeAct, bool) { } acts = append(acts, a...) } - what, err := x.leak(ctx, acts) + fallback, err := x.fallbacks(acts) + if err != nil { + x.end(report.OutcomeFailed, "internal", fmt.Sprintf("prepare the record of the opt-in state: %v", err)) + x.progress() + return nil, false + } + x.fallback = fallback + checked := append(slices.Clip(acts), fallback...) + what, err := x.leak(ctx, checked) switch { case err != nil: x.fail("read the commit to push", gitFailure(err)) @@ -94,7 +102,7 @@ func (x *targetExec) prepare(ctx context.Context) ([]writeAct, bool) { x.progress() return nil, false } - if err := unsafeText(acts); err != nil { + if err := unsafeText(checked); err != nil { x.end(report.OutcomeFailed, "internal", err.Error()+"; nothing is written") x.progress() return nil, false @@ -163,6 +171,8 @@ func (p *preparer) step(ctx context.Context, s decide.Step) ([]writeAct, error) return p.comment(s) case decide.StepAck, decide.StepRevoke: return p.remember(s) + case decide.StepRefreshMarker: + return p.refresh(s) } return nil, fmt.Errorf("unknown step %d", s.Kind) } @@ -360,6 +370,84 @@ func (p *preparer) remember(s decide.Step) ([]writeAct, error) { return []writeAct{{kind: actEdit, pr: s.PR, edit: platform.PREdit{Body: &body}, desc: desc}}, nil } +// refresh writes the current opt-in state into the marker of an open pull +// request of touchmark's that no other step writes to (Work.refreshOptIn): +// the body as last read, people's part of it made inert (keptBody), with +// the marker's optin replaced and nothing else of it. Nothing when that +// marker's optin is current already. +func (p *preparer) refresh(s decide.Step) ([]writeAct, error) { + x := p.x + pr, m, err := x.known(s.PR) + if err != nil { + return nil, err + } + if !x.w.staleOptIn(m) { + return nil, nil + } + a, err := x.refreshAct(pr, m) + if err != nil { + return nil, err + } + return []writeAct{a}, nil +} + +// refreshAct is the edit that writes the current opt-in state into the +// marker m of the open pull request pr: one body-only edit. +func (x *targetExec) refreshAct(pr platform.PR, m marker.Marker) (writeAct, error) { + d := dataCopy(m.Data) + d.OptIn = x.w.OptInHash + x.stamp(&d, false) + line, err := x.markerLine(m.Key, d) + if err != nil { + return writeAct{}, err + } + body := keptBody(pr.Body, line) + return writeAct{kind: actEdit, pr: pr.Number, edit: platform.PREdit{Body: &body}, + desc: fmt.Sprintf("recorded the opt-in state in #%d", pr.Number)}, nil +} + +// fallbacks returns, where closed pull requests are immutable, the edits +// that record the current opt-in state in each open pull request of +// touchmark's whose marker's optin is stale (Work.staleOptIn) and which +// acts write to only after their first push: should that push be refused +// (a branch restriction, which Bitbucket Cloud's driver cannot read ahead +// of the push), those writes do not happen, and perform makes these +// instead (recordOptIn). They are rendered with the other writes, so that +// their texts are checked before the first write. +func (x *targetExec) fallbacks(acts []writeAct) ([]writeAct, error) { + if !x.t.prov.caps.ClosedImmutable { + return nil, nil + } + first := slices.IndexFunc(acts, func(a writeAct) bool { return a.kind == actPush && !a.soft }) + if first < 0 { + return nil, nil + } + seen := map[int64]bool{} + for _, a := range acts[:first] { + if a.kind == actEdit { + seen[a.pr] = true + } + } + var out []writeAct + for _, a := range acts[first+1:] { + if a.kind != actEdit || seen[a.pr] { + continue + } + seen[a.pr] = true + pr, m, err := x.known(a.pr) + if err != nil || pr.State != platform.Open || !x.w.staleOptIn(m) { + continue + } + f, err := x.refreshAct(pr, m) + if err != nil { + return nil, err + } + f.soft = true + out = append(out, f) + } + return out, nil +} + // render renders the human part of the pull request body once per work, // exactly as phase C rendered it to decide the edits (Work.humanBody: // Work.Body with the decision's blocks, and the recreate control while the diff --git a/internal/distribute/target_exec.go b/internal/distribute/target_exec.go index 3ea16ef..e3c9975 100644 --- a/internal/distribute/target_exec.go +++ b/internal/distribute/target_exec.go @@ -67,6 +67,9 @@ type targetExec struct { // edited is "title" or "body" when an edit of the open pull request // wrote, for unchanged decisions (updated:title, updated:body). edited string + // fallback are the edits that record the opt-in state should a push be + // refused (fallbacks, recordOptIn). + fallback []writeAct } func newTargetExec(q *provQueue, w *Work) *targetExec { diff --git a/internal/distribute/write.go b/internal/distribute/write.go index 42bdc1e..a82eb57 100644 --- a/internal/distribute/write.go +++ b/internal/distribute/write.go @@ -127,6 +127,9 @@ func (x *targetExec) perform(ctx context.Context, acts []writeAct) (conflict str x.end(report.OutcomeFailed, "internal", fmt.Sprintf("unknown write %d", a.kind)) return "", false } + if !ok && a.kind == actPush && !a.soft { + x.recordOptIn(ctx) + } if !ok || conflict != "" { return conflict, ok } @@ -134,6 +137,46 @@ func (x *targetExec) perform(ctx context.Context, acts []writeAct) (conflict str return "", true } +// recordOptIn makes the fallback edits (targetExec.fallbacks) once a push +// was refused and the target ended blocked: the edits after the push will +// not happen, and the open pull request's marker must record the current +// opt-in state before a person declines it, on a platform whose declined +// pull requests no one can edit. They are secondary writes: a failure is a +// warning, and the outcome stays. Nothing once the run is over, or when +// the target ended otherwise (failed, deferred: the provider is out of +// budget or its credential refused). +func (x *targetExec) recordOptIn(ctx context.Context) { + if x.res.Outcome != report.OutcomeBlocked { + return + } + for _, a := range x.fallback { + if x.ex.stoppedAt(x.runCtx, x.now()) != "" { + return + } + x.mark = x.block.Writes() + var pr platform.PR + err := x.retryWrite(ctx, func() error { + var err error + pr, err = x.tw.EditPR(ctx, a.pr, a.edit) + return err + }, func() (bool, error) { + got, ok, err := x.editApplied(ctx, a) + if ok { + pr = got + } + return ok, err + }) + if err != nil { + x.warn(fmt.Sprintf("the opt-in state was not recorded in #%d: %v", a.pr, err)) + continue + } + if pr.Number == a.pr { + x.remember(pr) + } + x.op("edit-pr", a.pr, "", "", a.desc) + } +} + // doPush pushes a commit to a branch, or deletes it, with a lease, and // classifies a refusal. A push whose outcome is unknown (a transient // failure) reads the branch before it is tried again, at most three attempts From 0d092da4770a31269b1c41e63ebf37d5e2fc80c2 Mon Sep 17 00:00:00 2001 From: AlexeyShalaev <75322386+AlexeyShalaev@users.noreply.github.com> Date: Fri, 9 Oct 2026 02:18:17 +0300 Subject: [PATCH 20/20] fix(prbody): write no HTML into descriptions on Bitbucket Cloud --- docs/concepts/delivery.md | 3 ++- docs/guide/providers.md | 10 ++++--- internal/prbody/prbody.go | 29 +++++++++++++++++---- internal/prbody/render_test.go | 13 +++++++++ internal/prbody/testdata/huge/bitbucket.md | 7 ++--- internal/prbody/testdata/local/bitbucket.md | 7 ++--- 6 files changed, 49 insertions(+), 20 deletions(-) diff --git a/docs/concepts/delivery.md b/docs/concepts/delivery.md index 69a2c38..6cf71d0 100644 --- a/docs/concepts/delivery.md +++ b/docs/concepts/delivery.md @@ -102,7 +102,8 @@ The body, in English: `sensitive_paths` from `hub.yml`). Never cut; 3. a table of the changes, by file and pack: sensitive first, then deletions, updates, additions, up to 100 rows; -4. a folded list of the files the repository made its own, with how to take them back; +4. a folded list of the files the repository made its own, with how to take them back + (a plain section on Bitbucket Cloud, which shows HTML as text); 5. blocks for the situation: *previously declined in #N*, *touchmark paused*, *the hub proposes nothing more*; 6. the tick boxes; diff --git a/docs/guide/providers.md b/docs/guide/providers.md index 9a44564..1afc874 100644 --- a/docs/guide/providers.md +++ b/docs/guide/providers.md @@ -177,10 +177,12 @@ their scopes: `doctor` reports both as unknown, so keep a reminder to rotate the says `declined`. To have declined content proposed again, edit `packs` or `ignore`, or add a `forget_declines` entry, which acts while it is present on Bitbucket (see [Memory of declined pull requests](../concepts/memory.md#on-bitbucket-cloud)). -- **No tick boxes.** *Rebuild this branch* and *Propose this content again* rest on HTML - comments, which Bitbucket shows as text: descriptions there carry neither. A paused - pull request asks for a `recreate` entry in `.touchmark/operations.yml` instead, and the - footnote names `forget_declines`. +- **No tick boxes, no HTML.** *Rebuild this branch* and *Propose this content again* rest + on HTML comments, which Bitbucket shows as text: descriptions there carry neither. A + paused pull request asks for a `recreate` entry in `.touchmark/operations.yml` instead, + and the footnote names `forget_declines`. The files the repository made its own are a + plain section, not a folded one. Keep `pr.intro_file` free of HTML too: Bitbucket shows + it as text. - **Branch restrictions** are readable with admin rights only, which the writer should not have: `doctor` shows `rules` as unknown, and a push a restriction refuses ("Permission denied to update branch") is `blocked:rules:protected-branch`. diff --git a/internal/prbody/prbody.go b/internal/prbody/prbody.go index 125ce51..628aadc 100644 --- a/internal/prbody/prbody.go +++ b/internal/prbody/prbody.go @@ -129,7 +129,8 @@ type Input struct { // D. BranchUnknown bool // Local are paths in state local (a collapsed list with the adopt and - // ignore hints). + // ignore hints; a plain section where the description cannot carry + // HTML). Local []string // Sensitive are extra sensitive patterns from hub.yml. Sensitive []string @@ -221,6 +222,10 @@ const ( // closed pull request can never be edited (Caps.ClosedImmutable), the // footnote names the forget_declines entry that has declined changes // proposed again, since no control in a declined pull request can. +// Where descriptions cannot carry tick boxes, the platform escapes HTML, +// and the body carries none (the intro aside, which is the hub's): the +// local files are a section under a heading instead of a details +// element. // // Errors wrap ErrIntro (the intro breaks CheckIntro), ErrTooLarge or // ErrUnsafe; an unknown Change.Action, a HubURL that is not a plain http(s) @@ -658,7 +663,9 @@ func (r *renderer) changesSection() *cuttable { return newCuttable("### Changes", "\n\n| Change | File | Pack |\n|---|---|---|", "", items, len(r.rows), "change", "changes") } -// localSection is the collapsed list of local files. +// localSection is the list of local files: collapsed in a details element, +// or, where the description cannot carry HTML (renderer.html), plain +// Markdown under a heading. func (r *renderer) localSection() *cuttable { total := len(r.local) if total == 0 { @@ -668,17 +675,29 @@ func (r *renderer) localSection() *cuttable { if total == 1 { summary = "1 file here differs from the hub and stays as it is" } - prefix := "
\n" + summary + "\n\n" + - "touchmark does not update files changed in this repository. " + + hints := "touchmark does not update files changed in this repository. " + "To take the hub's version of one, run `touchmark apply --adopt `. " + "To stop seeing it here, add it to `ignore` in " + r.optIn + "." + prefix := "
\n" + summary + "\n\n" + hints + suffix := "\n\n
" + if !r.html() { + prefix, suffix = "### Local files\n\n"+summary+". "+hints, "" + } items := make([]string, 0, min(total, maxRows)) for _, p := range r.local[:min(total, maxRows)] { items = append(items, "\n- "+Code(p)) } - return newCuttable(prefix, "\n", "\n\n
", items, total, "file", "files") + return newCuttable(prefix, "\n", suffix, items, total, "file", "files") } +// html reports whether the description may carry touchmark's HTML: the +// details element of the local files, and the comments that tag the +// controls. A platform that escapes HTML in descriptions shows it as text; +// it is the platform whose marker is a reference definition and whose +// descriptions carry no controls (Caps.BodyControls false: Bitbucket +// Cloud). +func (r *renderer) html() bool { return r.in.Caps.BodyControls() } + // previouslyDeclined is the block for declines that overlap D. func (r *renderer) previouslyDeclined() string { if len(r.declined) == 0 { diff --git a/internal/prbody/render_test.go b/internal/prbody/render_test.go index 1be9605..97534fb 100644 --- a/internal/prbody/render_test.go +++ b/internal/prbody/render_test.go @@ -194,6 +194,19 @@ func checkBody(t *testing.T, in Input, body string) { } } checkSafe(t, gen, 1) + if !in.Caps.BodyControls() { + // The platform escapes HTML: touchmark writes none there. The + // marker line is the caller's, in the frame it chose. + human := gen + if in.Marker != "" { + human = strings.TrimSuffix(gen, "\n\n"+in.Marker) + } + for i, line := range strings.Split(human, "\n") { + if strings.Contains(outsideCode(line), "<") { + t.Errorf("HTML where the platform shows it as text, line %d of the generated part: %q", i+1, line) + } + } + } sensitive := in.GiteaWorkflows var paths []string for _, c := range in.Changes { diff --git a/internal/prbody/testdata/huge/bitbucket.md b/internal/prbody/testdata/huge/bitbucket.md index 9a5cbbb..3aae35c 100644 --- a/internal/prbody/testdata/huge/bitbucket.md +++ b/internal/prbody/testdata/huge/bitbucket.md @@ -118,10 +118,9 @@ These files can run code in CI or change how tools and agents work in this repos …and 903 more. -
-150 files here differ from the hub and stay as they are +### Local files -touchmark does not update files changed in this repository. To take the hub's version of one, run `touchmark apply --adopt `. To stop seeing it here, add it to `ignore` in `.engineering-assets.yml`. +150 files here differ from the hub and stay as they are. touchmark does not update files changed in this repository. To take the hub's version of one, run `touchmark apply --adopt `. To stop seeing it here, add it to `ignore` in `.engineering-assets.yml`. - `docs/handbook/chapter-00/local-000.md` - `docs/handbook/chapter-00/local-012.md` @@ -226,8 +225,6 @@ touchmark does not update files changed in this repository. To take the hub's ve …and 50 more. -
- --- If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. To have the same changes proposed again sooner, the hub's maintainers can add a `forget_declines` entry with the number of this pull request to `.touchmark/operations.yml`. touchmark rewrites this description, so please comment instead of editing it. diff --git a/internal/prbody/testdata/local/bitbucket.md b/internal/prbody/testdata/local/bitbucket.md index a0de663..1dc4da3 100644 --- a/internal/prbody/testdata/local/bitbucket.md +++ b/internal/prbody/testdata/local/bitbucket.md @@ -9,17 +9,14 @@ touchmark syncs packs `agents`, `claude` and `base` from the hub `acme-eng` at c |---|---|---| | update | `AGENTS.md` | `agents` | -
-3 files here differ from the hub and stay as they are +### Local files -touchmark does not update files changed in this repository. To take the hub's version of one, run `touchmark apply --adopt `. To stop seeing it here, add it to `ignore` in `.engineering-assets.yml`. +3 files here differ from the hub and stay as they are. touchmark does not update files changed in this repository. To take the hub's version of one, run `touchmark apply --adopt `. To stop seeing it here, add it to `ignore` in `.engineering-assets.yml`. - `.github/CODEOWNERS` - `CONTRIBUTING.md` - `docs/guide.md` -
- --- If this pull request is closed without merging, touchmark remembers it and does not propose the same changes again; a new one comes when the hub changes these files. Editing `packs` or `ignore` in `.engineering-assets.yml` resets that memory, and `ignore` opts a file out for good. To have the same changes proposed again sooner, the hub's maintainers can add a `forget_declines` entry with the number of this pull request to `.touchmark/operations.yml`. touchmark rewrites this description, so please comment instead of editing it.