Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -526,7 +526,7 @@ jobs:
# the platform conformance suite on the gitea driver, plan and distribute
# end to end, and the platform facts the driver relies on. The latest
# Gitea and Forgejo on every pull request and push, every supported
# version (Gitea 1.26 and 1.27, Forgejo 15 LTS and 16) nightly and by
# version (Gitea 1.26, 1.27 and 28, Forgejo 15 LTS and 16) nightly and by
# hand. docs/project/e2e.md explains how to run them locally.
name: e2e (${{ matrix.image }})
runs-on: ubuntu-latest
Expand All @@ -542,7 +542,7 @@ jobs:
strategy:
fail-fast: false
matrix:
image: ${{ fromJSON((github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') && '["docker.gitea.com/gitea:1.26.4","docker.gitea.com/gitea:1.27.3","codeberg.org/forgejo/forgejo:15.0.9","codeberg.org/forgejo/forgejo:16.0.5"]' || '["docker.gitea.com/gitea:1.27.3","codeberg.org/forgejo/forgejo:16.0.5"]') }}
image: ${{ fromJSON((github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') && '["docker.gitea.com/gitea:1.26.4","docker.gitea.com/gitea:1.27.3","docker.gitea.com/gitea:28.1.0","codeberg.org/forgejo/forgejo:15.0.9","codeberg.org/forgejo/forgejo:16.0.5"]' || '["docker.gitea.com/gitea:28.1.0","codeberg.org/forgejo/forgejo:16.0.5"]') }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
Expand Down
10 changes: 5 additions & 5 deletions docs/project/e2e.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@
You need Docker and bash (Git Bash on Windows). Nothing else: the tests run in `golang:1.27`.

```sh
bash scripts/e2e/gitea.sh docker.gitea.com/gitea:1.27.3
bash scripts/e2e/gitea.sh docker.gitea.com/gitea:28.1.0
bash scripts/e2e/gitea.sh codeberg.org/forgejo/forgejo:16.0.5
bash scripts/e2e/gitea.sh all # Gitea 1.26.4, 1.27.3, Forgejo 15.0.9 (LTS), 16.0.5
bash scripts/e2e/gitea.sh all # Gitea 1.26.4, 1.27.3, 28.1.0, Forgejo 15.0.9 (LTS), 16.0.5
bash scripts/e2e/gitea.sh --run TestFacts all # one test, every image
bash scripts/e2e/gitea.sh --keep docker.gitea.com/gitea:1.26.4
bash scripts/e2e/gitea.sh --require-signin --run TestVisibility all
Expand Down Expand Up @@ -109,7 +109,7 @@ Each test prints what it learned as `FINDING` lines. The run ends with a summary

## CI

The job `e2e-forge` in `.github/workflows/ci.yml` runs the script once per image: the latest Gitea and Forgejo (1.27.3 and 16.0.5) on every pull request and push, all four images nightly and on `workflow_dispatch`. The lint job of every CI run vets the module with the tag `e2e` (`go vet -tags e2e ./...`) for Linux, macOS and Windows, so the tests always compile. `e2e-forge` itself is outside the required check `All checks passed`: it pulls the forges' images from their own registries, whose outage should not block a merge; read its result before merging.
The job `e2e-forge` in `.github/workflows/ci.yml` runs the script once per image: the latest Gitea and Forgejo (28.1.0 and 16.0.5) on every pull request and push, all five images nightly and on `workflow_dispatch`. The lint job of every CI run vets the module with the tag `e2e` (`go vet -tags e2e ./...`) for Linux, macOS and Windows, so the tests always compile. `e2e-forge` itself is outside the required check `All checks passed`: it pulls the forges' images from their own registries, whose outage should not block a merge; read its result before merging.

## Without the script

Expand Down Expand Up @@ -143,7 +143,7 @@ bash scripts/e2e/gitlab.sh --only-seed gitlab/gitlab-ce:17.11.7-ce.0 # boot, s
bash scripts/e2e/gitlab.sh --only-seed --keep gitlab/gitlab-ce:18.11.12-ce.0
```

A cold GitLab boots in 3 to 6 minutes on a Linux host with 4 CPUs (expected; not yet measured on a hosted runner). Docker Desktop on Windows took 10 to 12 minutes for 17.11.7 and 19.4.1 on 2026-09-27, and 16 for 18.11.12 with other containers busy: raise the 15-minute wait with `TOUCHMARK_E2E_READY_TIMEOUT` (seconds) on a slow machine. The runner image matches the GitLab minor (`gitlab/gitlab-runner:v17.11.4`, `v18.11.4`, `v19.4.1`); the script pulls it when it is missing.
A cold GitLab boots in 3 to 6 minutes on a Linux host with 4 CPUs (expected; not yet measured on a hosted runner). Docker Desktop on Windows took 10 to 12 minutes for 17.11.7 and 19.4.1 on 2026-09-27, and 16 for 18.11.12 with other containers busy: raise the 15-minute wait with `TOUCHMARK_E2E_READY_TIMEOUT` (seconds) on a slow machine. The runner image matches the GitLab minor (`gitlab/gitlab-runner:alpine-v17.11.4`, `alpine-v18.11.4`, `alpine-v19.4.1`): the Alpine images have git 2.47 to 2.54, and the jobs that run `plan` need 2.45, which the Ubuntu images (git 2.43) lack. The script pulls the image when it is missing.

`--keep` leaves GitLab, the runner, the network, the volume with the binary and the env file with the tokens in place, publishes GitLab on a loopback port and prints how to rerun the tests and how to remove everything. `--only-seed` stops after the seeding and its checks; with `--keep` it gives a seeded instance to work against. Without `--keep`, the script removes all it created on exit, also after a failure or Ctrl-C: the containers with their anonymous volumes (the GitLab image declares `/etc/gitlab`, `/var/log/gitlab` and `/var/opt/gitlab`, about 0.5 GB per run; the runner image `/etc/gitlab-runner` and `/home/gitlab-runner`), the network, the volume with the binary and the env file. `gitea.sh` removes the forge's `/data` volume the same way.

Expand All @@ -162,7 +162,7 @@ A cold GitLab boots in 3 to 6 minutes on a Linux host with 4 CPUs (expected; not
| `jdoe` | a person, with a personal access token | Owner | `api, read_user, read_repository, write_repository` |

Which path an image takes, checked on 2026-09-27: CE 18.11.12 answers 404 to `GET /service_accounts` (there the API is EE code, `ee/lib/api/service_accounts.rb`, which the `gitlab-ce` image lacks), so its reader and writer are group access token bots (`group_<id>_bot_<hex>`); CE 19.4.1 has instance service accounts (`touchmark-reader`, `touchmark-writer`). The script logs the path, and the tests get it in `TOUCHMARK_E2E_GITLAB_ACCOUNTS`. The checks: every token acts as its account, only root is an administrator, the scopes by `GET /personal_access_tokens/self`, the roles by the reader's `GET /groups/:id/members/all/:user_id` in `acme` and `acme/sub`, and the reader cannot create a project.
5. Creates an instance runner with `POST /user/runners` (a `glrt-` token), starts `gitlab/gitlab-runner` of the same minor in GitLab's network namespace with the binary at `/opt/touchmark/touchmark`, registers it with the shell executor (the token reaches `gitlab-runner register` through the environment) and waits until GitLab reports it `online`.
5. Creates an instance runner with `POST /user/runners` (a `glrt-` token), starts `gitlab/gitlab-runner` (Alpine) of the same minor in GitLab's network namespace with the binary at `/opt/touchmark/touchmark`, registers it with the shell executor (the token reaches `gitlab-runner register` through the environment) and waits until GitLab reports it `online`.
6. Runs `go test -tags e2e -count=1 -race -v ./internal/e2e/gitlab/...` in `golang:1.27`, in GitLab's network namespace.

touchmark sends a credential over plain http only to loopback ([Tokens and git](threat-model.md#tokens-and-git)). So GitLab's `external_url` is `http://localhost`, and the tests and the runner share its network namespace: clone URLs in API answers and `CI_SERVER_URL` in jobs are the URL touchmark uses. GitLab's nginx listens on IPv4 only; Go, git and curl fall back from `::1` to `127.0.0.1`, busybox `wget` does not.
Expand Down
10 changes: 8 additions & 2 deletions internal/e2e/doctor_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -175,9 +175,15 @@ func TestDoctor(t *testing.T) {
want("provider", pc, "token-expiry", report.StatusOK, "no expiry date")
want("provider", pc, "2fa", report.StatusUnknown, "do not show")
want("provider", pc, "hub-hidden", report.StatusWarn, "sees the private hub")
// Gitea shows a token's scopes from 1.27 on; older Gitea and Forgejo
// leave them unknown.
scopes := check("provider", pc, "scopes")
if e.Flavor == "gitea" && scopes.Status != report.StatusOK {
t.Errorf("scopes on Gitea: %s %q", scopes.Status, scopes.Detail)
if v := forgeVersion(t, e); !v.Forgejo && v.atLeast(1, 27) {
if scopes.Status != report.StatusOK {
t.Errorf("scopes on %s: %s %q", v, scopes.Status, scopes.Detail)
}
} else if !v.Forgejo && (scopes.Status != report.StatusUnknown || !strings.Contains(scopes.Detail, "1.27 and later")) {
t.Errorf("scopes on %s: %s %q, want unknown", v, scopes.Status, scopes.Detail)
}
finding(t, "doctor-scopes", "the writer's scopes: %s %q", scopes.Status, scopes.Detail)

Expand Down
5 changes: 3 additions & 2 deletions internal/e2e/fixture_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -159,15 +159,16 @@ func (fx *fixture) createOrg(t testing.TB, name string) {
}

// team creates a team of org with access to the code, issues and pull
// requests of its repositories (all of them with all) and the members.
// requests of its repositories (all of them with all) and the members:
// units_map only, since Gitea 28 refuses a team with both units and
// units_map.
func (fx *fixture) team(t testing.TB, org, name, access string, all bool, members ...account) int64 {
t.Helper()
var team struct {
ID int64 `json:"id"`
}
fx.admin().ok(t, http.MethodPost, "/orgs/"+org+"/teams", map[string]any{
"name": name, "permission": access, "includes_all_repositories": all, "can_create_org_repo": false,
"units": []string{"repo.code", "repo.issues", "repo.pulls"},
"units_map": map[string]string{"repo.code": access, "repo.issues": access, "repo.pulls": access},
}, &team)
for _, m := range members {
Expand Down
2 changes: 2 additions & 0 deletions internal/platform/github/ghfake/doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,8 @@
// name 'o/n'." with HTTP 200 and the other aliases' data
// validation errors: path from "query [Name]", extensions observed (undefinedField)
// code, no data
// Int is a signed 32-bit integer: a literal or variable docs (GraphQL spec, Int)
// out of range fails validation or coercion
// a declared variable not used: variableNotUsed, "Variable observed
// $v is declared by anonymous query but not used"
// object(expression): Blob for files and symlinks (the link observed (a public repository)
Expand Down
14 changes: 10 additions & 4 deletions internal/platform/github/ghfake/gqlexec.go
Original file line number Diff line number Diff line change
Expand Up @@ -511,7 +511,7 @@ func (e *executor) literalFits(v gqlValue, t *gqlTypeRef) bool {
}
switch t.name {
case "Int":
return v.kind == valInt
return v.kind == valInt && isInt32(v.s)
case "Float":
return v.kind == valInt || v.kind == valFloat
case "Boolean":
Expand Down Expand Up @@ -702,10 +702,10 @@ func (e *executor) coerce(v any, t *gqlTypeRef) (any, string) {
if !ok {
return nil, "Could not coerce value to Int"
}
i, err := n.Int64()
if err != nil {
if !isInt32(n.String()) {
return nil, "Could not coerce value to Int"
}
i, _ := strconv.ParseInt(n.String(), 10, 32)
return i, ""
case "Float":
n, ok := v.(json.Number)
Expand All @@ -732,6 +732,12 @@ func (e *executor) coerce(v any, t *gqlTypeRef) (any, string) {
return s, ""
}

// isInt32 reports whether s is a GraphQL Int: a signed 32-bit integer.
func isInt32(s string) bool {
_, err := strconv.ParseInt(s, 10, 32)
return err == nil
}

// sortedKeys returns the keys of a set, sorted.
func sortedKeys(m map[string]bool) []string {
out := make([]string, 0, len(m))
Expand All @@ -757,7 +763,7 @@ func (e *executor) literal(v gqlValue, t *gqlTypeRef) any {
if t != nil && t.named() == "ID" {
return v.s
}
n, _ := strconv.ParseInt(v.s, 10, 64)
n, _ := strconv.ParseInt(v.s, 10, 32)
return n
case valFloat:
f, _ := strconv.ParseFloat(v.s, 64)
Expand Down
2 changes: 1 addition & 1 deletion internal/platform/github/ghfake/gqlparse.go
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ func (l *lexer) str() (string, error) {
if l.pos+6 > len(l.src) {
return bad()
}
n, err := strconv.ParseUint(l.src[l.pos+2:l.pos+6], 16, 32)
n, err := strconv.ParseUint(l.src[l.pos+2:l.pos+6], 16, 16)
if err != nil {
return bad()
}
Expand Down
11 changes: 11 additions & 0 deletions internal/platform/github/ghfake/graphql_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ func TestGraphQLValidation(t *testing.T) {
"Field 'defaultBranchRef' returns Ref but has no selections. Did you mean 'defaultBranchRef { ... }'?", nil},
{"enum", `{ repository(owner: "acme", name: "api") { pullRequests(first: 1, states: [BOGUS]) { totalCount } } }`,
"argumentLiteralsIncompatible", "", nil},
{"Int out of range", `{ repository(owner: "acme", name: "api") { pullRequests(first: 2147483648) { totalCount } } }`,
"argumentLiteralsIncompatible", "", nil},
{"fragment type", `{ repository(owner: "acme", name: "api") { ... on Nope { id } } }`, "undefinedType",
"No such type Nope, so it can't be a fragment condition", nil},
{"undeclared variable", `query V { repository(owner: $o, name: "api") { id } }`, "variableNotDefined",
Expand Down Expand Up @@ -120,6 +122,15 @@ func TestGraphQLValidation(t *testing.T) {
if field(got, "data", "repository", "name") != "api" {
t.Errorf("variables: %v", got)
}
// Int is a signed 32-bit integer.
q = `query P($n: Int) { repository(owner: "acme", name: "api") { pullRequests(first: $n) { totalCount } } }`
got = w.graphql(tok, q, map[string]any{"n": 2147483648})
if msg := field(got, "errors", 0, "message"); msg != "Variable $n of type Int was provided invalid value" {
t.Errorf("an Int out of range: %v", got)
}
if got = w.graphql(tok, q, map[string]any{"n": 2147483647}); field(got, "data", "repository") == nil {
t.Errorf("the largest Int: %v", got)
}
})
t.Run("anonymous", func(t *testing.T) {
r := w.call("POST", "/graphql", "", map[string]any{"query": "{ rateLimit { cost } }"})
Expand Down
11 changes: 6 additions & 5 deletions scripts/e2e/gitea.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
# [--template DIR [--touchmark-image REF]] IMAGE|all
#
# IMAGE a Gitea or Forgejo image, told apart by its name:
# docker.gitea.com/gitea:1.27.3, codeberg.org/forgejo/forgejo:16.0.5, ...
# all the four supported images, one after the other
# docker.gitea.com/gitea:28.1.0, codeberg.org/forgejo/forgejo:16.0.5, ...
# all the five supported images, one after the other
# --keep keep the forge, its network and an env file with the tokens
# for debugging, and publish the forge on a loopback port; the
# script prints how to reach and remove them
Expand Down Expand Up @@ -77,6 +77,7 @@ export MSYS_NO_PATHCONV=1
readonly SUPPORTED_IMAGES=(
docker.gitea.com/gitea:1.26.4
docker.gitea.com/gitea:1.27.3
docker.gitea.com/gitea:28.1.0
codeberg.org/forgejo/forgejo:15.0.9
codeberg.org/forgejo/forgejo:16.0.5
)
Expand Down Expand Up @@ -332,12 +333,12 @@ seed() {

api "$admin_token" POST /orgs \
"{\"username\":\"$ORG\",\"visibility\":\"private\",\"repo_admin_change_team_access\":false}" >/dev/null
local units='"units":["repo.code","repo.issues","repo.pulls"]'
# units_map only: Gitea 28 refuses a team with both units and units_map.
readers_id=$(api "$admin_token" POST "/orgs/$ORG/teams" \
"{\"name\":\"readers\",\"permission\":\"read\",\"includes_all_repositories\":true,\"can_create_org_repo\":false,$units,\"units_map\":{\"repo.code\":\"read\",\"repo.issues\":\"read\",\"repo.pulls\":\"read\"}}" |
"{\"name\":\"readers\",\"permission\":\"read\",\"includes_all_repositories\":true,\"can_create_org_repo\":false,\"units_map\":{\"repo.code\":\"read\",\"repo.issues\":\"read\",\"repo.pulls\":\"read\"}}" |
json_id)
writers_id=$(api "$admin_token" POST "/orgs/$ORG/teams" \
"{\"name\":\"writers\",\"permission\":\"write\",\"includes_all_repositories\":true,\"can_create_org_repo\":false,$units,\"units_map\":{\"repo.code\":\"write\",\"repo.issues\":\"write\",\"repo.pulls\":\"write\"}}" |
"{\"name\":\"writers\",\"permission\":\"write\",\"includes_all_repositories\":true,\"can_create_org_repo\":false,\"units_map\":{\"repo.code\":\"write\",\"repo.issues\":\"write\",\"repo.pulls\":\"write\"}}" |
json_id)
owners_id=$(api "$admin_token" GET "/orgs/$ORG/teams" | { grep -o '"id":[0-9]*,"name":"Owners"' || true; } | sed 's/[^0-9]//g' | head -n 1)
[ -n "$readers_id" ] && [ -n "$writers_id" ] && [ -n "$owners_id" ] || die "cannot read the ids of the teams of $ORG"
Expand Down
13 changes: 7 additions & 6 deletions scripts/e2e/gitlab.sh
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@
# has instance service accounts. The script logs which path it took,
# and the tests get it in TOUCHMARK_E2E_GITLAB_ACCOUNTS;
# - the person jdoe, an owner of acme, with a personal access token;
# - an instance runner (gitlab/gitlab-runner of the same minor, shell
# executor), registered with a runner authentication token from
# - an instance runner (gitlab/gitlab-runner of the same minor, Alpine,
# shell executor), registered with a runner authentication token from
# POST /user/runners, with a linux touchmark built from the working tree
# at /opt/touchmark/touchmark.
#
Expand Down Expand Up @@ -592,12 +592,13 @@ build_touchmark() {
}

# runner_image is the runner of the minor of the GitLab image; a runner
# works with GitLab of its own minor and older.
# works with GitLab of its own minor and older. The Alpine flavour: its git
# (2.47 to 2.54) runs plan, which needs 2.45; the Ubuntu one has 2.43.
runner_image() {
case $image in
*:17.11.*) echo gitlab/gitlab-runner:v17.11.4 ;;
*:18.11.*) echo gitlab/gitlab-runner:v18.11.4 ;;
*) echo gitlab/gitlab-runner:v19.4.1 ;;
*:17.11.*) echo gitlab/gitlab-runner:alpine-v17.11.4 ;;
*:18.11.*) echo gitlab/gitlab-runner:alpine-v18.11.4 ;;
*) echo gitlab/gitlab-runner:alpine-v19.4.1 ;;
esac
}

Expand Down
Loading