Repository navigation
375 lines (355 loc) · 18.7 KB
/
Copy pathnative-arch.yml
File metadata and controls
375 lines (355 loc) · 18.7 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
name: native-arch
# Real-HARDWARE validation of the exotic 64-bit targets, run NATIVELY (no QEMU).
#
# ppc64le / riscv64 -> GCC Compile Farm nodes (portal.cfarm.net)
# s390x -> IBM LinuxONE Community Cloud host (direct)
# loong64 -> NO free CI-able silicon today (see below) — covered
# only by the QEMU nightly (arch-qemu-nightly.yml).
#
# This aligns with the ecosystem-wide real-HW convention documented in
# go-asmgen's toolkit/bench-hw/README.md (the authoritative inventory of what
# hardware is actually reachable) and exercised by go-simd's per-repo
# z15-bench.yml SSH pattern. Reuse that convention here rather than reinvent
# node picks or secret names.
#
# rbgo is pure-Go (CGO=0, no hand-written assembly), so this is a courtesy
# cross-check that the interpreter behaves identically on real big-endian (s390x)
# and the other silicon — not a per-commit gate.
#
# ── AUP / etiquette ────────────────────────────────────────────────────────────
# The GCC Compile Farm is a SHARED ACADEMIC resource. Its acceptable-use policy
# FORBIDS using it as a per-PR CI backend hammered on every push. This workflow is
# therefore SCHEDULED (once per day, off-peak UTC) + manually dispatchable ONLY —
# never on pull_request/push — with concurrency capped at a single run. Please
# keep it that way. The LinuxONE host is a single dedicated VM, not shared, but
# stays on the same schedule-only cadence for consistency.
#
# ── Activation (repo secrets the maintainer must add) ──────────────────────────
# This workflow is INERT until the relevant secrets exist. The `guard` job below
# reads them into boolean outputs; the cfarm lane and the s390x lane are gated
# INDEPENDENTLY, so either can be activated without the other:
#
# CFARM_SSH_KEY (cfarm lane) Private SSH key registered on the cfarm
# account `delavennat`. PEM/OpenSSH, no
# passphrase. Covers ppc64le + riscv64 — the
# GCC Compile Farm is a single SSH account for
# every machine.
# CFARM_KNOWN_HOSTS (cfarm lane) `ssh-keyscan` output for cfarm433 + cfarm95
# (and the gateway, if used).
# CFARM_GATEWAY (optional) `user@gateway.host[:port]` bastion for the
# cfarm nodes, if your account reaches them
# via a jump host. Leave unset for direct
# access.
# LINUX1_SSH_KEY (s390x lane) Private SSH key authorized on the IBM
# LinuxONE Community Cloud host
# linux1@148.100.85.193. Same secret name
# used by go-simd's z15-bench.yml — reuse
# the existing key, do not mint a new one.
# LINUX1_KNOWN_HOSTS (s390x lane) `ssh-keyscan` output for 148.100.85.193.
#
# See docs/CI.md for copy-paste `gh secret set` commands and keyscan recipe.
#
# ── Retargeting hosts ──────────────────────────────────────────────────────────
# Node names / usernames / Go version live in the clearly-labelled `env:` and the
# `matrix.include` tables below — edit those, nothing else, to point at different
# cfarm nodes. Current node picks match go-asmgen/asmgen's
# toolkit/bench-hw/README.md (source of truth for what's actually reachable):
# cfarm433 IBM POWER10 -> ppc64le
# cfarm95 SpacemiT X60 (RVV 1.0) -> riscv64
# linux1@148.100.85.193 IBM LinuxONE (z/Architecture) -> s390x
#
# ── loong64 ──────────────────────────────────────────────────────────────────
# cfarm400/401 (the only Loongson nodes cfarm ever had) have been DOWN for
# roughly three years — there is no free CI-able loong64 silicon on cfarm, so
# there is no real-HW loong64 job here. Per bench-hw/README.md the two live
# options are the Debian LoongArch porter box (`shenzhou.debian.net`, needs
# Debian-maintainer standing) or a self-hosted runner built from the
# github.com/loong64 org's images — neither is wired up yet. Until one is,
# loong64 correctness coverage comes exclusively from the QEMU nightly
# (arch-qemu-nightly.yml).
on:
schedule:
# 04:42 UTC daily — off-peak, single courteous run per day (respects cfarm AUP).
- cron: "42 4 * * *"
workflow_dispatch:
permissions:
contents: read
concurrency:
# One real-HW run at a time; never cancel an in-flight run mid-suite.
group: native-arch
cancel-in-progress: false
env:
# Go toolchain fetched onto any host that lacks a recent-enough `go`.
GO_VERSION: "1.26.4"
jobs:
# ── Secret gate ────────────────────────────────────────────────────────────
# The `secrets` context is NOT available in job/step `if:` conditions, so we
# cannot write `if: ${{ secrets.CFARM_SSH_KEY != '' }}` directly. Instead this
# job reads each secret into a boolean OUTPUT, and the matching real-HW job
# gates on it. The cfarm lane and the s390x lane are independent: either can
# be activated without the other configured. Result: with no secrets
# configured the whole workflow is inert (skips green), never a red
# "missing secret".
guard:
name: check-secrets
runs-on: ubuntu-latest
outputs:
cfarm_enabled: ${{ steps.check.outputs.cfarm_enabled }}
linux1_enabled: ${{ steps.check.outputs.linux1_enabled }}
steps:
- id: check
env:
CFARM_KEY: ${{ secrets.CFARM_SSH_KEY }}
LINUX1_KEY: ${{ secrets.LINUX1_SSH_KEY }}
run: |
if [ -n "$CFARM_KEY" ]; then
echo "cfarm_enabled=true" >> "$GITHUB_OUTPUT"
echo "CFARM_SSH_KEY present — cfarm (ppc64le/riscv64) validation will run."
else
echo "cfarm_enabled=false" >> "$GITHUB_OUTPUT"
echo "::notice::CFARM_SSH_KEY not set — cfarm real-HW jobs skipped. See docs/CI.md to activate."
fi
if [ -n "$LINUX1_KEY" ]; then
echo "linux1_enabled=true" >> "$GITHUB_OUTPUT"
echo "LINUX1_SSH_KEY present — s390x validation will run."
else
echo "linux1_enabled=false" >> "$GITHUB_OUTPUT"
echo "::notice::LINUX1_SSH_KEY not set — s390x real-HW job skipped. See docs/CI.md to activate."
fi
# ── GCC Compile Farm suites (ppc64le, riscv64) ──────────────────────────────
native-cfarm:
name: native ${{ matrix.arch }} (${{ matrix.host }})
needs: guard
if: needs.guard.outputs.cfarm_enabled == 'true'
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
# ── EDIT THIS TABLE to retarget cfarm nodes. ──
# GCC Compile Farm nodes (portal.cfarm.net, account: delavennat).
# `cfarm: true` routes through CFARM_GATEWAY when that secret is set.
- { arch: ppc64le, goarch: ppc64le, user: delavennat, host: cfarm433.cfarm.net, cfarm: true }
- { arch: riscv64, goarch: riscv64, user: delavennat, host: cfarm95.cfarm.net, cfarm: true }
steps:
- name: Configure SSH (key, known_hosts, optional ProxyJump)
env:
SSH_KEY: ${{ secrets.CFARM_SSH_KEY }}
KNOWN_HOSTS: ${{ secrets.CFARM_KNOWN_HOSTS }}
GATEWAY: ${{ secrets.CFARM_GATEWAY }}
USE_CFARM: ${{ matrix.cfarm }}
TARGET_HOST: ${{ matrix.host }}
run: |
set -euo pipefail
mkdir -p ~/.ssh && chmod 700 ~/.ssh
printf '%s\n' "$SSH_KEY" > ~/.ssh/id_hw
chmod 600 ~/.ssh/id_hw
printf '%s\n' "$KNOWN_HOSTS" > ~/.ssh/known_hosts
chmod 644 ~/.ssh/known_hosts
# Route cfarm nodes through the optional bastion.
if [ "$USE_CFARM" = "true" ] && [ -n "${GATEWAY:-}" ]; then
{
echo "Host $TARGET_HOST"
echo " ProxyJump $GATEWAY"
} >> ~/.ssh/config
chmod 600 ~/.ssh/config
echo "cfarm node $TARGET_HOST will be reached via ProxyJump $GATEWAY"
fi
- name: Run test suite natively on real ${{ matrix.arch }} hardware
env:
TARGET: ${{ matrix.user }}@${{ matrix.host }}
GOARCH: ${{ matrix.goarch }}
GOVER: ${{ env.GO_VERSION }}
COMMIT: ${{ github.sha }}
run: |
set -euo pipefail
# Remote script: ensure a recent Go, clone the exact commit, then run
# `go vet` + `go test ./...` natively with GOWORK=off CGO_ENABLED=0.
# The repo is public, so the clone needs no credentials on the host.
ssh -o BatchMode=yes -o ConnectTimeout=30 -i ~/.ssh/id_hw "$TARGET" \
"GOARCH='$GOARCH' GOVER='$GOVER' COMMIT='$COMMIT' bash -s" <<'REMOTE'
set -euo pipefail
echo "== host: $(uname -a)"
# ---- ensure a recent-enough Go on PATH ----------------------------------
need_go=1
if command -v go >/dev/null 2>&1; then
have="$(go env GOVERSION 2>/dev/null | sed 's/^go//')"
# crude "have >= want" check via sort -V
if [ -n "$have" ] && [ "$(printf '%s\n%s\n' "$GOVER" "$have" | sort -V | head -1)" = "$GOVER" ]; then
need_go=0
echo "== using system go $have"
fi
fi
if [ "$need_go" -eq 1 ]; then
workdir="$(mktemp -d)"
tarball="go${GOVER}.linux-${GOARCH}.tar.gz"
echo "== fetching $tarball"
curl -fsSL "https://go.dev/dl/${tarball}" -o "$workdir/$tarball"
# Verify before executing anything out of it. These sums are the ones
# published at https://go.dev/dl/?mode=json for this GO_VERSION, and
# they live HERE, in the repository, rather than being fetched next to
# the tarball: a checksum served from the same origin as the file it
# describes cannot attest to that file. Bumping GO_VERSION means
# updating these, and the `*)` arm below is what says so out loud
# instead of silently skipping the check.
case "${GOVER}/${GOARCH}" in
1.26.4/ppc64le) want=53f49b8c7eace2d30389327b4a516b13321f90377fdf5929a6b63174609bc22e ;;
1.26.4/riscv64) want=2b9ba137baaa3031fd74330cb36ab54c5abe380867ca9fbab7c552f3db740555 ;;
1.26.4/s390x) want=1e12a2318f3dd4c57027c865f6cb51f5d1e36b02d10f3e6316ce7b61a27b3ca1 ;;
*)
echo "== no pinned SHA256 for go${GOVER} on ${GOARCH}"
echo "== add it from https://go.dev/dl/?mode=json; refusing to run an unverified toolchain"
exit 1
;;
esac
got="$(sha256sum "$workdir/$tarball" | cut -d' ' -f1)"
if [ "$got" != "$want" ]; then
echo "== SHA256 mismatch for $tarball"
echo "== want $want"
echo "== got $got"
exit 1
fi
echo "== $tarball sha256 verified"
( cd "$workdir" && tar -xzf "$tarball" )
export PATH="$workdir/go/bin:$PATH"
export GOROOT="$workdir/go"
fi
go version
# ---- fresh clone of the exact triggering commit -------------------------
src="$(mktemp -d)"
git clone --depth 50 https://github.com/go-embedded-ruby/ruby "$src/ruby"
cd "$src/ruby"
# Fail closed. These two lines used to end in `|| true` and
# `|| echo "... testing default branch HEAD"`, which defeated the
# `set -euo pipefail` above on purpose: an unreachable $COMMIT -- a
# --depth miss, a force-push, or a workflow_dispatch off a branch --
# left the clone on whatever the default branch pointed at, ran the
# suite against THAT, and still printed "validation OK" at the end.
# A green tick that did not test its subject is worse than a red one,
# because nothing downstream can tell the difference.
git fetch --depth 1 origin "$COMMIT"
git checkout -q --detach "$COMMIT"
head="$(git rev-parse HEAD)"
if [ "$head" != "$COMMIT" ]; then
echo "== HEAD is $head but \$COMMIT is $COMMIT -- refusing to report on the wrong tree"
exit 1
fi
echo "== testing $head"
# ---- native validation --------------------------------------------------
export GOWORK=off CGO_ENABLED=0
echo "== GOARCH=$(go env GOARCH) (expect $GOARCH)"
go vet ./...
go test -timeout=30m ./...
echo "== native $GOARCH validation OK"
REMOTE
# ── IBM LinuxONE Community Cloud (s390x) ────────────────────────────────────
# Separate lane, separate secrets (LINUX1_*), reused verbatim from go-simd's
# z15-bench.yml SSH pattern — same host, same key, so the existing LinuxONE
# access already granted to the ecosystem works here unchanged.
native-s390x:
name: native s390x (linux1@148.100.85.193)
needs: guard
if: needs.guard.outputs.linux1_enabled == 'true'
runs-on: ubuntu-latest
env:
TARGET_HOST: 148.100.85.193
TARGET_USER: linux1
steps:
- name: Configure SSH (key, known_hosts) — direct, no gateway
env:
SSH_KEY: ${{ secrets.LINUX1_SSH_KEY }}
KNOWN_HOSTS: ${{ secrets.LINUX1_KNOWN_HOSTS }}
run: |
set -euo pipefail
mkdir -p ~/.ssh && chmod 700 ~/.ssh
printf '%s\n' "$SSH_KEY" > ~/.ssh/id_hw
chmod 600 ~/.ssh/id_hw
printf '%s\n' "$KNOWN_HOSTS" > ~/.ssh/known_hosts
chmod 644 ~/.ssh/known_hosts
- name: Run test suite natively on real s390x hardware
env:
TARGET: ${{ env.TARGET_USER }}@${{ env.TARGET_HOST }}
GOARCH: s390x
GOVER: ${{ env.GO_VERSION }}
COMMIT: ${{ github.sha }}
run: |
set -euo pipefail
# Remote script: ensure a recent Go, clone the exact commit, then run
# `go vet` + `go test ./...` natively with GOWORK=off CGO_ENABLED=0.
# The repo is public, so the clone needs no credentials on the host.
ssh -o BatchMode=yes -o ConnectTimeout=30 -i ~/.ssh/id_hw "$TARGET" \
"GOARCH='$GOARCH' GOVER='$GOVER' COMMIT='$COMMIT' bash -s" <<'REMOTE'
set -euo pipefail
echo "== host: $(uname -a)"
# ---- ensure a recent-enough Go on PATH ----------------------------------
need_go=1
if command -v go >/dev/null 2>&1; then
have="$(go env GOVERSION 2>/dev/null | sed 's/^go//')"
# crude "have >= want" check via sort -V
if [ -n "$have" ] && [ "$(printf '%s\n%s\n' "$GOVER" "$have" | sort -V | head -1)" = "$GOVER" ]; then
need_go=0
echo "== using system go $have"
fi
fi
if [ "$need_go" -eq 1 ]; then
workdir="$(mktemp -d)"
tarball="go${GOVER}.linux-${GOARCH}.tar.gz"
echo "== fetching $tarball"
curl -fsSL "https://go.dev/dl/${tarball}" -o "$workdir/$tarball"
# Verify before executing anything out of it. These sums are the ones
# published at https://go.dev/dl/?mode=json for this GO_VERSION, and
# they live HERE, in the repository, rather than being fetched next to
# the tarball: a checksum served from the same origin as the file it
# describes cannot attest to that file. Bumping GO_VERSION means
# updating these, and the `*)` arm below is what says so out loud
# instead of silently skipping the check.
case "${GOVER}/${GOARCH}" in
1.26.4/ppc64le) want=53f49b8c7eace2d30389327b4a516b13321f90377fdf5929a6b63174609bc22e ;;
1.26.4/riscv64) want=2b9ba137baaa3031fd74330cb36ab54c5abe380867ca9fbab7c552f3db740555 ;;
1.26.4/s390x) want=1e12a2318f3dd4c57027c865f6cb51f5d1e36b02d10f3e6316ce7b61a27b3ca1 ;;
*)
echo "== no pinned SHA256 for go${GOVER} on ${GOARCH}"
echo "== add it from https://go.dev/dl/?mode=json; refusing to run an unverified toolchain"
exit 1
;;
esac
got="$(sha256sum "$workdir/$tarball" | cut -d' ' -f1)"
if [ "$got" != "$want" ]; then
echo "== SHA256 mismatch for $tarball"
echo "== want $want"
echo "== got $got"
exit 1
fi
echo "== $tarball sha256 verified"
( cd "$workdir" && tar -xzf "$tarball" )
export PATH="$workdir/go/bin:$PATH"
export GOROOT="$workdir/go"
fi
go version
# ---- fresh clone of the exact triggering commit -------------------------
src="$(mktemp -d)"
git clone --depth 50 https://github.com/go-embedded-ruby/ruby "$src/ruby"
cd "$src/ruby"
# Fail closed. These two lines used to end in `|| true` and
# `|| echo "... testing default branch HEAD"`, which defeated the
# `set -euo pipefail` above on purpose: an unreachable $COMMIT -- a
# --depth miss, a force-push, or a workflow_dispatch off a branch --
# left the clone on whatever the default branch pointed at, ran the
# suite against THAT, and still printed "validation OK" at the end.
# A green tick that did not test its subject is worse than a red one,
# because nothing downstream can tell the difference.
git fetch --depth 1 origin "$COMMIT"
git checkout -q --detach "$COMMIT"
head="$(git rev-parse HEAD)"
if [ "$head" != "$COMMIT" ]; then
echo "== HEAD is $head but \$COMMIT is $COMMIT -- refusing to report on the wrong tree"
exit 1
fi
echo "== testing $head"
# ---- native validation --------------------------------------------------
export GOWORK=off CGO_ENABLED=0
echo "== GOARCH=$(go env GOARCH) (expect $GOARCH)"
go vet ./...
go test -timeout=30m ./...
echo "== native $GOARCH validation OK"
REMOTE