Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions docs/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
| Program | `touchmark`, one static Go binary; the same binary in the image `ghcr.io/bedrock-python/touchmark` and the GitHub Action `bedrock-python/touchmark` |
| Requires | git 2.31+ for `check`, `status`, `apply`, `manifest`; git 2.45+ for `plan` and `distribute`, which read the targets (the image brings its own) |
| Install | archives on [GitHub releases](https://github.com/bedrock-python/touchmark/releases) · `go install github.com/bedrock-python/touchmark/cmd/touchmark@latest` (Go 1.26+) · `docker run ghcr.io/bedrock-python/touchmark:<version>` |
| Platforms | GitHub (github.com, GHE.com, GHES 3.19+), GitLab 17.0+, Gitea 1.26+, Forgejo 15+ |
| Platforms | GitHub (github.com, GHE.com, GHES 3.19+), GitLab 17.0+, Gitea 1.26+, Forgejo 15+; as providers a hub delivers to: Bitbucket Cloud, Azure DevOps Services |
| Hub template | <https://github.com/bedrock-python/engineering-assets-template> |
| Source | <https://github.com/bedrock-python/touchmark> |

Expand Down Expand Up @@ -47,8 +47,9 @@ no lock file, no header, no marker in their files. It never checks a target out
and never runs a script, hook, filter or LFS from the hub or a target. It delivers
nothing to a repository without the opt-in file, unless `targets.yml` subscribes it
(`opt_in: assumed`). There is no pull mode (targets do not
fetch from the hub), no remote pack sources, no Azure DevOps driver, and no `setup` for
Bitbucket: a hub on Bitbucket Cloud runs in Bitbucket Pipelines, set up by hand.
fetch from the hub), no remote pack sources, no `setup` for Bitbucket (a hub on Bitbucket Cloud
runs in Bitbucket Pipelines, set up by hand), and no hub on Azure Pipelines: Azure DevOps is a
provider a hub elsewhere delivers to.

## Mental model

Expand Down Expand Up @@ -251,7 +252,7 @@ ten `::error` and ten `::warning` annotations.
| `version` | `1` | format version |
| `id` | — | slug, 3–40 characters; names the branch `touchmark/<id>`; `change-me` fails `check` |
| `writer`, `sign` | — , `auto` | single-provider shorthand: the writer account, commit signing (`auto`, `always`) |
| `providers[]` | from CI | `id`, `type` (`github`, `gitlab`, `gitea`, `forgejo`), `url` (required for gitea and forgejo), `api_url`, `ca_file`, `writer`, `known_authors`, `automation_accounts`, `sign`, `limits`; replaces the shorthand |
| `providers[]` | from CI | `id`, `type` (`github`, `gitlab`, `gitea`, `forgejo`, `bitbucket`, `azure-devops`), `url` (required for gitea, forgejo and azure-devops: `https://dev.azure.com/<organization>`), `api_url`, `ca_file`, `writer`, `known_authors`, `automation_accounts`, `sign`, `limits`; replaces the shorthand |
| `branch`, `branch_aliases` | `touchmark/<id>`, none | the sync branch, and former names whose pull requests stay the hub's |
| `previous_fingerprints` | none | the hub's fingerprints before it moved |
| `opt_in_file` | `.engineering-assets.yml` | the opt-in file's path in targets |
Expand All @@ -275,7 +276,9 @@ the one provider whose `url` it lies under (`github` defaults to `https://github
`gitlab` to `https://gitlab.com`); none fails `check`, and so do several unless the
entry's `provider` or `defaults.provider` picks one of them, credentials, a query, a
fragment, and any scheme but `https` (`http` for loopback only). On GitHub, Gitea and
Forgejo a repository URL, and an `exclude` URL without `**`, names `owner/name`.
Forgejo a repository URL, and an `exclude` URL without `**`, names `owner/name`; on Azure
DevOps it is `https://dev.azure.com/<organization>/<project>/_git/<repository>`, naming
`<project>/<repository>`, and an `org` URL is the organization itself.

`.engineering-assets.yml` (in each target; `touchmark schema opt-in`): `version`,
`enabled` (default `true`; `false` opts out whatever the hub says), `packs` (added to
Expand All @@ -284,7 +287,7 @@ covers everything under it). At most 64 KiB.

`.touchmark/operations.yml` (in the hub; `touchmark schema operations`):
`recreate[]` (`target`, `head`: acts while the branch head is that commit),
`forget_declines[]` (`target`, `pr`: acts once; on Bitbucket Cloud while present), `allow_mass_close` (`max`, `until`),
`forget_declines[]` (`target`, `pr`: acts once; on Bitbucket Cloud and Azure DevOps while present), `allow_mass_close` (`max`, `until`),
`adopt_unmarked` (`until`). Dates are `YYYY-MM-DD`, UTC, inclusive.

Environment, one set per provider; `<ID>` is the provider id upper-cased with `-` as
Expand All @@ -307,7 +310,8 @@ from its environment once read:
every managed file would look `local`. touchmark refuses a shallow clone and a partial
clone (`--filter`). In CI: `fetch-depth: 0` (GitHub, Gitea), `GIT_DEPTH: "0"` (GitLab).
2. **Reader and writer are two different accounts** on every platform: two GitHub Apps,
two GitLab service accounts (or group access tokens), two Gitea or Forgejo bot users.
two GitLab service accounts (or group access tokens), two Gitea or Forgejo bot users,
two Azure DevOps users with personal access tokens (Code read; Code read & write).
The read key reaches any branch of the hub; the write key must not.
3. **The write key is visible to the default branch only.** GitHub: secrets in the
environment `touchmark-distribute` with deployment branches *Selected* = the default
Expand Down
11 changes: 11 additions & 0 deletions docs/concepts/memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,17 @@ request was open:
declines it; bots' closes wait for the cooldown, and the third in a row counts as a
decline.

## On Azure DevOps

touchmark never writes to an abandoned pull request on Azure DevOps either: whether its
properties, which hold the marker there, can still be changed is not known, and the
memory of Bitbucket Cloud needs no such write. Everything [above](#on-bitbucket-cloud)
applies, with three differences: the marker lives in the property `touchmark.marker`
rather than the description, so the edit that records a new opt-in state changes only
that property; descriptions do carry the tick box *Rebuild this branch* while a branch is
paused; and touchmark never reactivates an abandoned pull request, though Azure DevOps
would let it.

## Limits

- Only the last 50 closed pull requests on the sync branches count.
Expand Down
6 changes: 5 additions & 1 deletion docs/guide/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ cleanup to stay safe:
| Entry | Acts | Then |
|---|---|---|
| `recreate` | while the sync branch's head is exactly `head` | a new push to the branch makes it inert |
| `forget_declines` | once: touchmark marks pull request `pr` revoked; on Bitbucket Cloud, while the entry is present (see below) | inert |
| `forget_declines` | once: touchmark marks pull request `pr` revoked; on Bitbucket Cloud and Azure DevOps, while the entry is present (see below) | inert |
| `allow_mass_close` | up to `max` closes in a run, until the end of `until` (UTC) | inert after the date |
| `adopt_unmarked` | until the end of `until` (UTC) | inert after the date |

Expand Down Expand Up @@ -72,6 +72,10 @@ closed. Removed earlier, the decline is in force again; an open pull request sta
memory never closes one, and its description then names the decline under *Previously
declined*.

On Azure DevOps a `forget_declines` entry acts while it is present too: touchmark never
writes to an abandoned pull request there, so it cannot mark one revoked. Descriptions
there do carry *Rebuild this branch*.

## On your machine

A maintainer who holds the write key can run the same operations as flags of
Expand Down
99 changes: 99 additions & 0 deletions docs/guide/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,105 @@ their scopes: `doctor` reports both as unknown, so keep a reminder to rotate the
reads 15 times a minute by default, with two targets at once, and writes at most once a
second; `limits` overrides it.

## Azure DevOps

A provider of type `azure-devops` is one organization of Azure DevOps Services, and its
`url` names it: `https://dev.azure.com/<organization>` (one provider per organization;
the old `https://<organization>.visualstudio.com` form and Azure DevOps Server are not
supported). `plan`, `distribute` and `doctor` work on it from a hub on GitHub or GitLab,
or wherever you run touchmark; a hub whose own CI is Azure Pipelines is not supported
yet, and `setup` has nothing for it.

```yaml
# hub.yml
providers:
- id: ado
type: azure-devops
url: https://dev.azure.com/acme
writer: 3f2a8d4e-1b6c-4f0a-9e7d-5c2b1a0f9e8d # the writer user's identity id
```

**Accounts.** Use two users that only touchmark uses, each with a personal access token
(PAT) for this organization only: service principals of Microsoft Entra ID are not
supported yet. Global PATs (for all organizations) stop working on 2026-12-01.

| | Reader | Writer |
|---|---|---|
| Variable | `TOUCHMARK_ADO_READ_TOKEN` | `TOUCHMARK_ADO_WRITE_TOKEN` |
| Scopes | Code (read) | Code (read & write) |
| Repositories | read access to the targets | Contribute, Create branch and Contribute to pull requests on the targets only; no administration, and no access to the hub |

A PAT cannot read its own expiry or scopes: `doctor` reports both as unknown, so keep a
reminder to renew the tokens before they expire.

- **Accounts are identity ids.** Display names are neither unique nor stable, so
`writer`, `known_authors` and `automation_accounts` name users by their identity id, a
GUID: `GET https://dev.azure.com/<organization>/_apis/connectionData` with the user's
token shows it as `authenticatedUser.id`.
- **Targets.** A repository is `<project>/<repository>`; web URLs such as
`https://dev.azure.com/acme/Billing/_git/api` work. `org` names the whole organization
(`org: acme`, or its URL `https://dev.azure.com/acme`); select a project's repositories
with `match` (`Billing/*`). Azure DevOps has no topics: `check` refuses `topics` on its
entries. Disabled repositories are skipped. Project and repository names with spaces
can be reached through `org` only: `targets.yml` paths take letters, digits, `.`, `-`
and `_`.
- **The marker lives in a pull request property** (`touchmark.marker`), not in the
description, which Azure DevOps limits to 4 000 characters: the description is short,
its lists are cut to fit, and the marker still records every change. Editing the
description in the web UI leaves the marker as it is, and a marker pasted into the
description is ignored. The property is not protected, though: anyone who may
contribute to pull requests in the project, project Readers included by default, may
be able to write it through the API (see the
[threat model](../project/threat-model.md)). touchmark writes the property right after
it opens a pull request, trying a few times; should it still fail, it abandons the new
pull request and opens another, so that none stays without its marker.
- **People's text in the description** is theirs: a close or a refresh of the marker
writes the property alone and sends no description, so a description people filled up
to the limit never stops touchmark from closing its pull request.
- **Labels** are added by name, as `pr.labels` says. **Drafts** are Azure DevOps' own
draft flag (`pr.draft`).
- **Closing.** touchmark closes its own pull request by abandoning it, after it stored the
closing marker. It never reactivates an abandoned pull request, nor changes it: a pull
request touchmark closed stays closed, and the next proposal is a new one.
- **Declines** are remembered without writing to the abandoned pull request, as on
Bitbucket Cloud: while a pull request is open its marker records the state of the
opt-in file, and a person's decline holds while that file is parsed the same. touchmark
leaves no comment on it; the report says `declined`. To have declined content proposed
again, edit `packs` or `ignore`, or add a `forget_declines` entry, which acts while it
is present (see [Memory of declined pull requests](../concepts/memory.md#on-bitbucket-cloud)).
- **Comments** are threads touchmark closes at once, so that they never wait for a
resolution.
- **Branch policies** refuse direct pushes to the branches they protect, and the writer
reads none upfront: `doctor` shows `rules` as unknown; a push a policy refuses
(TF402455) is `blocked:rules:policy`, one the writer lacks a permission for (TF401027)
`blocked:permission:push`. The writer's permissions are read with the Has Permissions
API when the token may use it; when it may not, `doctor` shows `access` as unknown and
the first push or pull request meets a missing permission.
- **Commit author.** touchmark authors the writer's commits with its sign-in address when
Azure DevOps shows one, else with `<identity id>@touchmark.invalid`.
- **Pace.** Azure DevOps meters a user's load over five minutes and announces it in its
`X-RateLimit-*` headers, which pause the provider before it delays requests, and in
`Retry-After` on requests it delayed, which pauses the provider for that long; touchmark
reads 120 times a minute with four targets at once and writes at most 30 times a minute
by default; `limits` overrides it.
- **The reader needs a token.** Azure DevOps sends anonymous reads of files (the Trees
API) and of pull request properties to its sign-in page even in public projects, so a
plan without the reader's token fails on Azure DevOps targets.
- **Cost.** Listing a target's pull requests reads the marker of every open one and of the
60 newest abandoned and completed ones (touchmark remembers the 50 newest closed pull
requests); older closed pull requests count as without a marker.

What only a live organization can confirm, and the driver assumes from the REST
reference: the size limit of pull request properties and whether they survive an
abandon; whether labels in a create request are applied (touchmark adds missing ones
after it); the error of a duplicate pull request (TF401179); whether `closedBy` is set
on abandoned pull requests; the order of pull request listings and where they cut
descriptions; the answer to a refused PAT (401, a sign-in redirect or a 203 page); the
exact texts of push refusals; the `mode` strings of the Trees API; how disabled and
renamed repositories answer; who can write pull request properties (whether project
Readers, through "Contribute to pull requests", can set or change `touchmark.marker`);
and whether `Retry-After` comes on answers that went through.

## Moving

- **A target moves to another provider on the same host** (a new App, a new group): add
Expand Down
1 change: 1 addition & 0 deletions docs/project/threat-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ How to read the tests column:
|---|---|---|---|
| Opt-in file: at most 64 KiB, a bounded number of YAML anchors and aliases, a strict schema | `config` (`decodeStrict`, `ParseOptIn`), the drivers' `ReadFile` with a size limit | `TestDecodeLimits`, `TestParseHubAnchors`, `TestParseOptIn`, `TestOptInIgnoreLimit`, `FuzzParse`, `TestPlanAssumeOptInTooLarge` | — |
| Marker: at most 16 KiB, at most 64 KiB inflated, strict JSON | `marker` | `TestParseGzipBomb`, `TestParseRejectsHeader`, `TestParseRejectsData`, `TestParseRejectsPayload`, `TestParseMaxChanges`, `TestFindLargeBody`, `FuzzParse`, `FuzzFind` | — |
| Azure DevOps: the marker lives in a pull request property (`touchmark.marker`), apart from the description; the author, the head, the head's repository (a fork) and the closer come from the API, never from the property; a new pull request whose property cannot be stored is abandoned, or marked through an edit when the abandon fails | `azuredevops` (`markerOf`, `toPR`, `storeNewMarker`), `decide.Identity`, `distribute` (`markCreated`) | `TestPRs`, `TestCreatePRMarkerAbandons`, `TestRunAzureDevOpsMarkerLost` *(git 2.45)* | The property is not protected. Anyone who may contribute to pull requests in the project may be able to write it through the API: project Readers have "Contribute to pull requests" by default, and the REST reference names only the scope `vso.code_write` (to confirm live, [providers.md](../guide/providers.md#azure-devops)). Such a person can block a target (`blocked:marker-invalid`, by breaking the marker of touchmark's open pull request, until an `adopt` entry takes it over) or, before the team abandons a pull request, write `closed.by=touchmark` into its marker so that the decline does not count and the content is proposed again. They cannot make a pull request of their own touchmark's: its author, head and fork come from the API. |
| API answers: at most 32 MiB | `httpx` (`DefaultMaxBody`) | `TestMaxBody`, `TestOnlyHTTPXSpeaksHTTP` | — |
| Objects of a target: a commit at most 1 MiB, a blob at most 8 MiB; more is `failed:git` | `gitx` (`MaxCommitObject`, `MaxReadBlob`), `distribute.failCommit` | `TestObjectLimits` | — |
| Only the `recreate` and `repropose` checkboxes act; every other text of a target is data | `prbody.Ticked`, `prbody.Inert` | `TestTicked`, `TestUntick`, `FuzzTicked`, `TestInert`, `TestMemoryReproposeBypassesOnce`, `TestRunKeptBodyIsInert` *(git 2.45)* | — |
Expand Down
3 changes: 2 additions & 1 deletion docs/reference/compatibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ changelog before you merge it.
and reject fields they do not know.
- **What touchmark leaves in repositories:** the marker in pull request bodies
(`touchmark:v1`, an HTML comment, or a Markdown reference definition on Bitbucket
Cloud), the commit trailers (`Touchmark-Hub`, `Touchmark-Stream`,
Cloud; on Azure DevOps the comment lives in the pull request property
`touchmark.marker`), the commit trailers (`Touchmark-Hub`, `Touchmark-Stream`,
`Touchmark-Content`, `Touchmark-Hub-Commit`), the control lines of a pull request body,
the marker of the plan comment, and the sync branch's name. Every release reads what an
earlier one wrote. The marker's payload is frozen: a change to it is a `touchmark:v2`
Expand Down
4 changes: 3 additions & 1 deletion docs/reference/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,9 @@ history. `touchmark manifest` prints it.
`<!-- touchmark:v1 hub=… fp=… stream=sync key=sha256:… data=… -->`. It records the
fingerprint and what the pull request carries. On Bitbucket Cloud, which shows HTML
comments as text, the same payload is a Markdown link reference definition:
`[touchmark]: # "touchmark:v1 hub=… fp=… stream=sync key=sha256:… data=…"`.
`[touchmark]: # "touchmark:v1 hub=… fp=… stream=sync key=sha256:… data=…"`. On Azure
DevOps, whose descriptions hold 4 000 characters, the comment lives in the pull request
property `touchmark.marker` instead of the description.

**opt-in file** — `.engineering-assets.yml` at a target's root. Its presence is consent;
it can add packs and `ignore` paths, and `enabled: false` in it opts the target out.
Expand Down
Loading
Loading