From 5c643affda6cb010ae3d8cec69166576f248e65f Mon Sep 17 00:00:00 2001 From: trivedi-vatsal Date: Wed, 26 Aug 2026 11:07:33 +0530 Subject: [PATCH 1/4] docs: correct the install path and add operations and comparison The published run commands could not be run from a fresh machine: compose.yaml is `build: .`, so `docker compose up --build` needed a checkout that no page mentioned. Quickstart and Deployment now lead with compose.prod.yaml, which pulls the published image, and keep the build path as the contributor route. Fix the DOCKER_GID advice. The docs told macOS readers to run `stat -f %g` on the host, but /var/run/docker.sock there is a symlink into ~/.docker and reports a group that means nothing inside the container, where the socket is gid 0. Both Deployment and Troubleshooting now read the gid from inside the container and name 0 as the Docker Desktop answer. Surface max_concurrent_jobs. It defaults to 1, has no env var, and can only be raised after first boot, which is a sizing constraint people should meet before installing rather than after. Configuration leads with it. Correct max_workspace_bytes, which was documented as a checkout size cap. It is accepted, stored, and editable, and no code reads it. Add Operations: what state is, cold backup and restore, what a restart does to a running job, the upgrade procedure, and the migration policy. A restart does not preserve in-flight work; the job is requeued and re-run from the start, or occasionally lands cancelled and is not retried. Restore has to write into the volume rather than through `docker compose cp`, which copies as root and leaves the database unwritable to uid 10001. Add Comparison against Woodpecker, Drone, self-hosted actions/runner, and Jenkins, including the cases where the reader should pick one of those, and a plain list of where this is weaker. Add FAQ entries for GitHub Enterprise Server, monorepos, whether the project dogfoods itself, and what v1 commits to. The compatibility promise itself is left as a TODO because it is a decision, not a fact about the code. Move Coolify below Enable repos in the sidebar and retitle it so the nav stops implying it is a required step 2 of 3. Co-Authored-By: Claude Opus 5 (1M context) --- scripts/check-links.mjs | 2 + src/content/docs/setup/bindings.md | 2 +- src/content/docs/setup/coolify.md | 4 +- src/content/docs/start/comparison.md | 112 +++++++++++++++++ src/content/docs/start/configuration.md | 32 ++++- src/content/docs/start/faq.md | 79 +++++++++++- src/content/docs/start/quickstart.md | 37 ++++-- src/content/docs/understanding/deployment.md | 41 ++++-- src/content/docs/understanding/operations.md | 126 +++++++++++++++++++ src/content/docs/using/troubleshooting.md | 13 +- 10 files changed, 415 insertions(+), 33 deletions(-) create mode 100644 src/content/docs/start/comparison.md create mode 100644 src/content/docs/understanding/operations.md diff --git a/scripts/check-links.mjs b/scripts/check-links.mjs index e19f1eb..afcab93 100644 --- a/scripts/check-links.mjs +++ b/scripts/check-links.mjs @@ -18,6 +18,7 @@ const required = [ 'index.html', 'start/quickstart/index.html', 'start/configuration/index.html', + 'start/comparison/index.html', 'start/faq/index.html', 'setup/github-app/index.html', 'setup/coolify/index.html', @@ -29,6 +30,7 @@ const required = [ 'understanding/architecture/index.html', 'understanding/security-model/index.html', 'understanding/deployment/index.html', + 'understanding/operations/index.html', 'contributing/development/index.html', 'adr/005-check-suite-gating/index.html', 'favicon.svg', diff --git a/src/content/docs/setup/bindings.md b/src/content/docs/setup/bindings.md index 62f7f52..55f7c9a 100644 --- a/src/content/docs/setup/bindings.md +++ b/src/content/docs/setup/bindings.md @@ -2,7 +2,7 @@ title: "Enable repos" description: "Enable the private repos that should receive Check Runs; the bindings table is the allow-list." sidebar: - order: 3 + order: 2 --- **Repos** → pick the CI App, optionally pick a Coolify instance as the source of the repo list, then check the repositories to run checks for. Unchecking a repo diff --git a/src/content/docs/setup/coolify.md b/src/content/docs/setup/coolify.md index 35ad73e..d5b4e16 100644 --- a/src/content/docs/setup/coolify.md +++ b/src/content/docs/setup/coolify.md @@ -1,8 +1,8 @@ --- -title: "Coolify" +title: "Optional: Coolify" description: "Add Coolify as an optional deployment target and repository picker. Not required for checks to work." sidebar: - order: 2 + order: 3 --- Coolify is a supported deployment target and an optional repo source. **Not the product, and not required**. Add a team-scoped API token and you get server diff --git a/src/content/docs/start/comparison.md b/src/content/docs/start/comparison.md new file mode 100644 index 0000000..4254e13 --- /dev/null +++ b/src/content/docs/start/comparison.md @@ -0,0 +1,112 @@ +--- +title: "Comparison" +description: "Where openpreflight sits against Woodpecker CI, Drone, self-hosted actions/runner, and Jenkins, and when to pick one of those instead." +sidebar: + order: 3 +--- +openpreflight is a narrow tool. It reports one Check Run per commit for private +GitHub repos, from one binary and one SQLite file, and it runs `install`, +`test`, and `build`. That is the whole product. Most projects on this page do +more, and for a lot of teams doing more is the point. + +Read this before you install, not after. + +## The short version + +| You want | Pick | +|---|---| +| A check on the commit, on a box you already run, with as little to operate as possible | openpreflight | +| A real pipeline DSL — stages, matrices, caches, artifacts, fan-out | Woodpecker or Drone | +| The same workflows you already have in `.github/workflows/` | self-hosted `actions/runner` | +| Anything that is not GitHub, or plugins for everything | Jenkins | + +## Woodpecker CI + +The closest comparison, and for most people the better default. Woodpecker is a +container-native pipeline engine with a YAML DSL, matrix builds, plugins, +secrets management, multiple agents, and support for GitHub, GitLab, Gitea, +Bitbucket, and Forgejo. + +**Pick Woodpecker if** you want steps that are not install/test/build, matrix +builds, artifacts between steps, more than one machine running jobs, or a forge +that is not GitHub. + +**Pick openpreflight if** the server-plus-agent split is more than you want to +run for a handful of private repos, and a Check Run that says pass or fail with +a log behind it is the whole requirement. openpreflight is one container and +one file; there is no agent to register and no broker. + +## Drone + +The project Woodpecker forked from. Same shape: a server, one or more runners, +a container-per-step pipeline, a mature plugin ecosystem, several forges. +Licensing changed after Harness acquired it, which is worth checking against +your own constraints; Woodpecker is where much of the community went. + +The trade-off against openpreflight is the same as Woodpecker's, and if you are +choosing between the two of them rather than against this, that is a decision +openpreflight has no opinion on. + +## Self-hosted `actions/runner` + +If your workflows are already GitHub Actions, this is usually the right answer +and openpreflight is the wrong one. You keep the YAML, the marketplace, the +matrix syntax, the caches, and the artifacts, and you change where the compute +happens. openpreflight deliberately runs none of that: it does not read +`.github/workflows/`, and it will not. + +**Pick `actions/runner` if** you have Actions workflows you want to keep, need +the marketplace, or want GitHub to own the scheduling. + +**Pick openpreflight if** you do not want Actions minutes involved at all, do +not want a runner registered against your org, and would rather the whole CI +surface be a thing you can read in an afternoon. The scope note in +[what is not in v1](/start/faq/) is the honest boundary. + +## Jenkins + +Two decades of plugins, every SCM, every language, every deployment shape. +Nothing here competes with that, and nothing here tries. + +**Pick Jenkins if** you need something a plugin already solves, have build +infrastructure that is not just "run three commands in a checkout", or already +run it. + +**Pick openpreflight if** you would be installing Jenkins purely to get a +green check on a private repo's pull requests, and the JVM, the plugin +upgrades, and the configuration surface are cost you would rather not carry. + +## Where openpreflight is genuinely weaker + +Stated plainly, because these are the things that will bite: + +- **One machine.** There is no agent protocol. Jobs run in the process or in a + sibling container on the same Docker engine. Every one of the tools above + scales horizontally; this one does not. +- **One job at a time by default.** `max_concurrent_jobs` is 1 and can only be + raised after first boot. See + [Configuration](/start/configuration/). +- **Three steps.** `install`, `test`, `build`, in that order. No stages, no + `needs:`, no fan-out, no conditional steps. +- **No caches and no artifacts.** Every job is a fresh shallow clone. Nothing + is carried between runs or handed to a later step. +- **No matrices.** One pipeline per commit, not one per version combination. +- **GitHub only.** It is built on Check Runs, which no other forge has. +- **One admin user.** No teams, no roles, no SSO. +- **You register the GitHub App.** It is not created for you. + +## Where it is genuinely better + +- **Operationally small.** One container, one SQLite file, one process. No + broker, no agent registration, no database server. Backups are a file and a + key; see [Operations](/understanding/operations/). +- **Native Check Runs.** Not a status API shim. Runs gate on the check suite + and hold to one live run per commit, which is what makes required checks + behave under force-pushes and rapid pushes. See + [ADR 005](/adr/005-check-suite-gating/). +- **Configured in a UI, not env vars.** Apps and repo bindings are rows you + edit, not a block of environment per installation. +- **Secrets encrypted at rest.** PEMs, webhook secrets, and Coolify tokens are + AES-256-GCM columns. See [Security model](/understanding/security-model/). +- **Small enough to audit.** The whole implementation is `internal/`, with no + plugin surface. diff --git a/src/content/docs/start/configuration.md b/src/content/docs/start/configuration.md index a9a7328..0d946fd 100644 --- a/src/content/docs/start/configuration.md +++ b/src/content/docs/start/configuration.md @@ -9,6 +9,23 @@ else is a row in SQLite, edited in the UI or over the JSON API. For a procedural first run, see [Quickstart](/start/quickstart/). Pipeline file semantics are also covered under [Pipelines](/using/pipelines/). +## One job at a time, by default + +`max_concurrent_jobs` defaults to **1**. A fresh install runs one job at a +time; a second commit waits for the first to finish. There is no env var for +it — it is a settings row, so it can only be changed after the process is up, +under **Settings** or `PATCH /api/v1/settings`. + +Size for that before you install. One binary on one box with a serial runner +suits a handful of repos that push a few times an hour. It does not suit a +busy monorepo, and it is not a scheduler: raising the number raises how many +jobs this one process runs at once, on this one machine. Jobs never spread +across hosts. Every concurrent job also holds its own checkout under +`WORKSPACE_DIR` and its own log under `DATA_DIR`, so disk scales with the +number too. + +The UI caps the field at 32. The API does not. + ## Environment | Variable | Required | Purpose | @@ -25,6 +42,12 @@ semantics are also covered under [Pipelines](/using/pipelines/). There is no `GITHUB_APP_ID` and no `CI_ALLOWED_REPOS`. Those live in `github_apps` and `repo_bindings`. +`DOCKER_GID` is not in the table because the process never reads it. It is a +Compose variable: it puts uid 10001 in the docker socket's group so `runtime:` +jobs and fork PRs can reach the engine. The default of `998` suits a typical +Linux docker group and does **not** work on Docker Desktop, where the socket is +gid 0 inside the container. See [Deployment](/understanding/deployment/). + Generate a key with: ```bash @@ -47,9 +70,9 @@ Single row, `id = 1`. Changed from **Settings** in the UI or | `default_check_name` | `openpreflight` | Check Run name unless the App or binding overrides. New installs only. An existing database keeps the name it already has, because GitHub matches a required status check by name and renaming one strands its branch protection rule | | `default_pipeline_file` | `.ci.yml` | Path in the repo | | `default_timeout_seconds` | `900` | Per-job timeout | -| `max_concurrent_jobs` | `1` | Runner concurrency | -| `max_log_bytes` | 10 MiB | Cap on the on-disk log | -| `max_workspace_bytes` | 1 GiB | Checkout size cap | +| `max_concurrent_jobs` | `1` | Jobs this process runs at once. See above | +| `max_log_bytes` | 10 MiB | The log stops growing at this size; the run continues | +| `max_workspace_bytes` | 1 GiB | Accepted and stored, **not enforced**. No code reads it; a large checkout is not stopped | | `log_retention_days` | `14` | Prune old logs and job rows | | `default_runtime` | empty | Docker image used when a fork job's pipeline has no `runtime:` | | `skip_fork_prs` | `true` | Fork PRs are ignored. Saving `false` requires Docker plus `default_runtime`. | @@ -88,3 +111,6 @@ Resolution order, highest first: 3. Node defaults inferred from `package.json`: `npm ci` / `pnpm` / `yarn` by lockfile, then `test` and `build` **only if those scripts exist** 4. nothing to run → the check is reported as **skipped**, not failed + +Backups, upgrades, and what a restart does to a running job are in +[Operations](/understanding/operations/). diff --git a/src/content/docs/start/faq.md b/src/content/docs/start/faq.md index e56d419..4a8d432 100644 --- a/src/content/docs/start/faq.md +++ b/src/content/docs/start/faq.md @@ -1,12 +1,13 @@ --- title: "FAQ" -description: "Short answers to why a GitHub App is required, why Coolify is optional, why gating is on the check suite, and what v1 deliberately leaves out." +description: "Short answers on GitHub Apps, Coolify, check-suite gating, GitHub Enterprise, monorepos, dogfooding, and what v1 commits to." sidebar: - order: 3 + order: 4 --- -Positioning questions the homepage raises; each answer links the page or ADR -that argues it in full. +Positioning questions the homepage raises, and the ones people ask before +installing. Each answer links the page or ADR that argues it in full. For how +this sits against other CI, see [Comparison](/start/comparison/). ## Why a GitHub App and not OAuth? @@ -44,3 +45,73 @@ engine via `CI_DOCKER_HOST`, not Coolify's API as a job runner. See [Architecture](/understanding/architecture/) and the homepage's "What it isn't" list. + +## Does it work with GitHub Enterprise Server? + +The plumbing is there and it has not been tested against a real instance. + +Each App row carries an **API URL**, which defaults to +`https://api.github.com` and is the field to change for Enterprise. The git +origin is derived from it, so `https://ghe.example.com/api/v3` clones from +`https://ghe.example.com`. Nothing else in the request path assumes +github.com. + +What is untested is everything specific to a GHE deployment: its certificate +chain, its API version skew, and Check Runs behaviour on older releases. If you +try it, an issue saying which version and what broke is useful. + +## Does it work for monorepos? + +It runs, but it will run everything on every push. + +There is no path filter. A binding matches a repo and optionally a branch list +(exact names, or a `release/*` prefix), and that is the whole filter. A commit +touching one directory runs the same `install`, `test`, and `build` as a commit +touching all of them. + +A plan is also exactly three steps in a fixed order, so the usual monorepo +answer — one job per affected package, in parallel — has nothing to express +itself with. Combined with `max_concurrent_jobs` defaulting to 1, a busy +monorepo is the case this is worst at. Do the path filtering inside your own +`test` command, or use a tool built for it; see +[Comparison](/start/comparison/). + +## Does the project use itself for CI? + +No. `openpreflight/openpreflight` runs GitHub Actions — `ci.yml` for vet, test, +and a Docker build, and `release.yml` on a `v*` tag. + +Two reasons, both honest. Dogfooding needs a permanently reachable HTTPS +instance and a GitHub App registered against the org, which is infrastructure +the project does not run yet. And releases have to build multi-arch images and +attach binaries, which is not something this tool does at all — it reports a +check, it does not publish artifacts. + +Nothing stops it from checking a public repo, incidentally. The repository's +visibility is read from the webhook payload but never gates anything; "private +repos" is what it is aimed at, not a restriction it enforces. + +## What does v1 commit to? + +Concretely, as of 1.0.0: + +- **Releases are tagged `vMAJOR.MINOR.PATCH`** and published as an image + (`ghcr.io/openpreflight/openpreflight`, tagged full version, major.minor, and + `latest`) plus linux/amd64 and linux/arm64 binaries. +- **The JSON API is `/api/v1/`.** A breaking change to it would be `/api/v2/`, + not a change under the existing prefix. The surface is in + [API](/using/api/). +- **`default_check_name` does not change on an existing database.** GitHub + matches a required status check by name, so renaming a live install's check + would strand its branch protection rule. New installs get the current + default; existing ones keep what they have. +- **Migrations are append-only and there is no rollback.** Once a version has + booted against your database, downgrading is not supported. See + [Operations](/understanding/operations/). +- **`CI_SECRET_KEY` stays the key.** No release will make an existing key + unreadable without a documented rotation path. + +TODO(vatsal): decide and state the actual compatibility promise for 1.x — +whether minor releases may change settings defaults, remove settings fields, or +change the pipeline file schema, and what deprecation notice a breaking change +gets. diff --git a/src/content/docs/start/quickstart.md b/src/content/docs/start/quickstart.md index 777e598..438b7d7 100644 --- a/src/content/docs/start/quickstart.md +++ b/src/content/docs/start/quickstart.md @@ -22,18 +22,35 @@ App. Full environment and settings detail lives in ## Run it +`compose.prod.yaml` is standalone: it pulls the published image, so there is +nothing to clone. + ```bash -export CI_SECRET_KEY="$(openssl rand -base64 48)" # required, keep it forever -export CI_PUBLIC_BASE_URL="https://ci.example.com" # optional seed -docker compose up --build +curl -O https://raw.githubusercontent.com/openpreflight/openpreflight/main/compose.prod.yaml +export CI_SECRET_KEY="$(openssl rand -base64 48)" +docker compose -f compose.prod.yaml up -d ``` -`CI_SECRET_KEY` is required; the process refuses to start without it. Keep it -forever. Losing it makes stored PEMs and tokens unreadable. See +`CI_SECRET_KEY` is the only variable you must set; the process refuses to start +without it. Keep it forever. Losing it makes stored PEMs and tokens unreadable. +Everything else either has a default or is asked for in the wizard. See [Configuration](/start/configuration/) for the full env table and key rotation. -Compose, volumes, and reverse-proxy notes are in -[Deployment](/understanding/deployment/). +Pin a version with `OPENPREFLIGHT_VERSION=1.0.0` rather than editing the file. + +To build from source instead — this is the contributor path, and `compose.yaml` +is `build: .`, so it needs the checkout: + +```bash +git clone https://github.com/openpreflight/openpreflight +cd openpreflight +export CI_SECRET_KEY="$(openssl rand -base64 48)" +docker compose up --build +``` + +If you use `runtime:` or fork PRs, you also need `DOCKER_GID`. See +[Deployment](/understanding/deployment/), which covers compose, volumes, the +docker socket, and reverse-proxy notes. ## First boot @@ -47,10 +64,10 @@ wizard. It is ignored once a user exists. ## Setup order 1. [Register a GitHub App](/setup/github-app/) and paste it under **GitHub Apps**. -2. Optionally [add a Coolify instance](/setup/coolify/) as a repo-picker source - or to install this worker. -3. [Enable repo bindings](/setup/bindings/). The bindings table is the +2. [Enable repo bindings](/setup/bindings/). The bindings table is the allow-list. +3. Optionally [add a Coolify instance](/setup/coolify/) as a repo-picker source + or to install this worker. Then commit a [pipeline](/using/pipelines/) (or rely on Node defaults) and push. diff --git a/src/content/docs/understanding/deployment.md b/src/content/docs/understanding/deployment.md index c0654be..bfe9187 100644 --- a/src/content/docs/understanding/deployment.md +++ b/src/content/docs/understanding/deployment.md @@ -7,13 +7,19 @@ sidebar: The image is a static Go binary plus `git`, Node, and `docker-cli`. It runs as uid 10001. `tini` is the entrypoint so pipeline shells get reaped. +Two compose files ship. `compose.prod.yaml` pulls the published image and needs +no checkout; `compose.yaml` is `build: .` and is the contributor path. + ```bash +curl -O https://raw.githubusercontent.com/openpreflight/openpreflight/main/compose.prod.yaml export CI_SECRET_KEY="$(openssl rand -base64 48)" # required, keep it forever export CI_PUBLIC_BASE_URL="https://ci.example.com" # optional seed -docker compose up --build +docker compose -f compose.prod.yaml up -d ``` -Compose maps `8080:8080` and three mounts: +Set `OPENPREFLIGHT_VERSION` to pin a release; the default is `latest`. + +Both files map `8080:8080` and take three mounts: | Volume | Mount | Must persist | |---|---|---| @@ -21,10 +27,24 @@ Compose maps `8080:8080` and three mounts: | `ci-workspace` | `/workspace` | no, but a volume keeps checkouts off the container's writable layer | | host socket | `/var/run/docker.sock` | no; needed for `runtime:` and fork PRs | -`group_add: ${DOCKER_GID:-998}` puts uid 10001 in the socket's group. On the -host, `DOCKER_GID` is usually `stat -f %g /var/run/docker.sock` (macOS) or -`stat -c %g /var/run/docker.sock` (Linux). Job containers never receive that -socket; see [ADR 004](/adr/004-docker-executor/). +`group_add: ${DOCKER_GID:-998}` puts uid 10001 in the socket's group. The value +you need is the gid **the container sees**, which is not always what the host +reports. Read it from inside: + +```bash +docker compose exec openpreflight stat -c %g /var/run/docker.sock +``` + +On Linux with a native engine that matches the host's +`stat -c %g /var/run/docker.sock`, and the default of `998` is often already +right. On Docker Desktop it is **`0`**: the socket is `root:root` inside the +VM, and the host path is a symlink into `~/.docker`, so the host's `stat` +reports an unrelated group. Set `DOCKER_GID=0` there and recreate the +container. + +Nothing else needs this. The service boots and reports checks with an +unreachable socket; only `runtime:` jobs and fork PRs fail. Job containers +never receive that socket; see [ADR 004](/adr/004-docker-executor/). To run jobs on another Docker engine (including a Coolify server's), set `CI_DOCKER_HOST` (else `DOCKER_HOST`) to that daemon. That is Docker's remote @@ -60,11 +80,11 @@ has one webhook URL; repointing it steals Coolify's deploys. See 1. Complete setup (admin password + public base URL) if you did not bootstrap. See [Quickstart](/start/quickstart/). 2. [Register a GitHub App](/setup/github-app/) and paste it under **GitHub Apps**. -3. Optionally [add a Coolify instance](/setup/coolify/) (team token) as a - repo-picker source, or to install this worker. -4. [Enable bindings](/setup/bindings/). Only enable private repos you trust: a +3. [Enable bindings](/setup/bindings/). Only enable private repos you trust: a pipeline runs the repo's own commands in this process, or in a sibling container when `runtime:` is set. +4. Optionally [add a Coolify instance](/setup/coolify/) (team token) as a + repo-picker source, or to install this worker. ## Rotating `CI_SECRET_KEY` @@ -85,4 +105,5 @@ Copy `/data` (or the `ci-data` volume) and keep `CI_SECRET_KEY` with it. The database without the key is not enough to recover PEMs and tokens. The key without the database is not enough to recover configuration. -Redeploys interrupt in-flight jobs; on start the runner requeues them. +Procedure, restore, upgrades, and what a redeploy does to a running job are in +[Operations](/understanding/operations/). diff --git a/src/content/docs/understanding/operations.md b/src/content/docs/understanding/operations.md new file mode 100644 index 0000000..bf38f9f --- /dev/null +++ b/src/content/docs/understanding/operations.md @@ -0,0 +1,126 @@ +--- +title: "Operations" +description: "Backup and restore of ci.db and logs, what a restart does to a running job, the upgrade procedure, and the schema migration policy." +sidebar: + order: 4 +--- +Everything worth keeping is `DATA_DIR` plus `CI_SECRET_KEY`. Neither is much +use without the other. Nothing else on the box holds state. + +| Path | Holds | Keep | +|---|---|---| +| `DATA_DIR/ci.db` | Settings, users, Coolify instances, GitHub Apps, bindings, job rows | yes | +| `DATA_DIR/logs/{job-id}.log` | One file per job, the full build log | yes, if you want history | +| `WORKSPACE_DIR` | Per-job checkouts, deleted when the job ends | no | +| `CI_SECRET_KEY` | The AES-256-GCM key for the encrypted columns | yes, separately | + +The GitHub App PEM, webhook secrets, and Coolify tokens are encrypted columns +in `ci.db`. A backup without the key restores your bindings and job history but +no credentials, and the App has to be pasted in again. Store the key in +whatever holds your other secrets, not next to the backup. + +## Backup + +SQLite runs in WAL mode, so a live `ci.db` is not a complete copy on its own — +recent writes sit in `ci.db-wal`. Stopping the container checkpoints the WAL +and leaves a single consistent file, which is why the cold copy below is the +one to use. There is no `sqlite3` binary in the image. + +```bash +docker compose -f compose.prod.yaml stop +docker compose -f compose.prod.yaml cp openpreflight:/data ./openpreflight-backup +docker compose -f compose.prod.yaml start +``` + +That yields `ci.db` and `logs/`. The stop is brief; queued jobs wait and are +picked up when it comes back. + +If you must copy while it runs, take `ci.db`, `ci.db-wal` and `ci.db-shm` +together, or you will restore a database missing its most recent writes. + +## Restore + +Restore goes into the volume directly, not through `docker compose cp`. The +process runs as uid 10001 and `cp` writes as root; a database owned by root is +readable but not writable, so the service starts, answers `/health`, and then +fails every write with `attempt to write a readonly database`. + +```bash +docker compose -f compose.prod.yaml down +docker run --rm -v openpreflight_ci-data:/data -v "$PWD/openpreflight-backup:/backup:ro" alpine \ + sh -c 'rm -rf /data/* && cp -a /backup/. /data/ && chown -R 10001:10001 /data' +docker compose -f compose.prod.yaml up -d +``` + +The volume is `openpreflight_ci-data` because both compose files pin the +project name. Set `CI_SECRET_KEY` to the key that was in use when the backup +was taken. A different key leaves the secret columns unreadable, and the App +will fail its next `test` with a decryption error rather than at boot. + +## What a restart does to a running job + +**Nothing in progress survives.** There is no checkpointing: a job that was +half way through `npm test` does not resume, and there is no partial result. + +On the next boot the runner requeues anything left `in_progress` and runs it +again from the beginning — a fresh clone and every step. That is the usual +outcome of a redeploy, an upgrade, a crash, or `docker compose restart`. + +The exception is a job that notices the cancelled context before the process +exits. That one finishes as `cancelled`, and a cancelled job is **not** +retried. Which of the two happens is a race between the job unwinding and the +process exiting, and the process does not wait for the runner, so the requeue +is the common case and the cancel is the rare one. + +Practical consequences: + +- A requeued job opens a **new** Check Run rather than reusing the one the + interrupted attempt created, so the commit collects a second check with the + same name. Branch protection reads the newer one. +- A job cancelled by a restart leaves a cancelled check that nothing will retry + on its own. Re-run it from the job page (`POST /api/v1/jobs/{id}/rerun`) or + use GitHub's **Re-run**. +- Deploy when the queue is empty if you care. `GET /api/v1/jobs` shows what is + queued or running. + +## Upgrading + +```bash +docker compose -f compose.prod.yaml pull +docker compose -f compose.prod.yaml up -d +``` + +Back up first. Migrations run on boot and there is no way back down; see below. + +Pin `OPENPREFLIGHT_VERSION` if you would rather choose when that happens. +Without it the image is `:latest` and a `pull` takes whatever is newest. + +In-flight jobs are interrupted the same way any restart interrupts them. +`ci-data` and `ci-workspace` are named volumes and survive; nothing in the +upgrade path touches them. + +## Schema migrations + +Migrations are an ordered, append-only list compiled into the binary. Each one +runs in its own transaction and is recorded in `schema_migrations`, so every +boot applies exactly what is missing and nothing twice. There is no separate +migrate command and no flag to skip it: opening the database runs them. + +The policy that follows from that: + +- **Applied migrations are never edited.** A schema change is a new entry. +- **There are no down migrations.** Once a version has booted against your + database, an older binary may not understand the schema it left behind. + Rolling back a release is not supported, and the way out of a bad upgrade is + the backup you took before it. +- **A failed migration aborts the boot.** The transaction rolls back and the + process exits rather than serving against a half-applied schema. + +## What gets deleted on its own + +An hourly pass prunes expired sessions, then deletes job rows and their log +files older than `log_retention_days` (default 14). Queued and running jobs are +never pruned. Nothing else is cleaned up automatically — see +[Configuration](/start/configuration/) for the settings that govern it, and +note that `max_workspace_bytes` is stored but not enforced, so a checkout large +enough to fill the disk will fill it. diff --git a/src/content/docs/using/troubleshooting.md b/src/content/docs/using/troubleshooting.md index f565f3c..89b891f 100644 --- a/src/content/docs/using/troubleshooting.md +++ b/src/content/docs/using/troubleshooting.md @@ -77,9 +77,16 @@ See [Pipelines](/using/pipelines/). **Cause:** `DOCKER_GID` does not match the host socket's group. The image runs as uid 10001 and joins that group via Compose `group_add`. -**Fix:** On the host, set `DOCKER_GID` to -`stat -c %g /var/run/docker.sock` (Linux) or -`stat -f %g /var/run/docker.sock` (macOS), then recreate the container. +**Fix:** Read the gid the container sees, then set `DOCKER_GID` to it and +recreate the container: + +```bash +docker compose exec openpreflight stat -c %g /var/run/docker.sock +``` + +On Docker Desktop the answer is `0`. Do not use the host's `stat` on macOS: +`/var/run/docker.sock` is a symlink into `~/.docker`, so it reports a group +that means nothing inside the container. See [Deployment](/understanding/deployment/). From 51d35a0c98143a629b35c74a9c32871ec8eef7b1 Mon Sep 17 00:00:00 2001 From: trivedi-vatsal Date: Wed, 26 Aug 2026 11:16:57 +0530 Subject: [PATCH 2/4] docs: use one tagline and one description everywhere Nine variants of the product description were in circulation across the four repositories. They collapse to two strings, used verbatim: tagline A small CI provider for private repos. description A small CI provider for private repos. One Go binary, one SQLite file: register a GitHub App, enable your repos, and get one Check Run per commit. The description opens with the tagline so the two cannot drift, and fits a meta description, an OG description, and a GitHub repo description without being truncated. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 2 +- astro.config.mjs | 2 +- scripts/check-links.mjs | 2 +- src/content/docs/index.mdx | 4 ++-- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 4572dfc..ad5f5d8 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ /> openpreflight — a small CI provider for private repos. One Go binary, one SQLite file, one Check Run per commit. diff --git a/astro.config.mjs b/astro.config.mjs index c37ddc0..971a926 100644 --- a/astro.config.mjs +++ b/astro.config.mjs @@ -16,7 +16,7 @@ export default defineConfig({ starlight({ title: 'openpreflight', description: - 'Self-hosted GitHub Check Runs CI for private repos. One Go binary, one SQLite file.', + 'A small CI provider for private repos. One Go binary, one SQLite file: register a GitHub App, enable your repos, and get one Check Run per commit.', logo: { light: './src/assets/logo-light.svg', dark: './src/assets/logo-dark.svg', diff --git a/scripts/check-links.mjs b/scripts/check-links.mjs index afcab93..4c3a978 100644 --- a/scripts/check-links.mjs +++ b/scripts/check-links.mjs @@ -12,7 +12,7 @@ const root = join(dirname(fileURLToPath(import.meta.url)), '..'); const dist = join(root, 'dist'); const SITE_DEFAULT_DESCRIPTION = - 'Self-hosted GitHub Check Runs CI for private repos. One Go binary, one SQLite file.'; + 'A small CI provider for private repos. One Go binary, one SQLite file: register a GitHub App, enable your repos, and get one Check Run per commit.'; const required = [ 'index.html', diff --git a/src/content/docs/index.mdx b/src/content/docs/index.mdx index 3953a63..2ae25b9 100644 --- a/src/content/docs/index.mdx +++ b/src/content/docs/index.mdx @@ -1,9 +1,9 @@ --- title: openpreflight -description: Self-hosted GitHub Check Runs CI for private repos. One Go binary, one SQLite file. +description: "A small CI provider for private repos. One Go binary, one SQLite file: register a GitHub App, enable your repos, and get one Check Run per commit." template: splash hero: - tagline: A small CI provider for private repos. One Go binary that configures itself in a web UI and runs your install/test/build on the exact commit, reporting one Check Run with full logs. + tagline: "A small CI provider for private repos. One Go binary, one SQLite file: register a GitHub App, enable your repos, and get one Check Run per commit." actions: - text: Quickstart link: /start/quickstart/ From e4718fc34a86f47142a9d964641bead6934f1ccd Mon Sep 17 00:00:00 2001 From: trivedi-vatsal Date: Wed, 26 Aug 2026 11:33:55 +0530 Subject: [PATCH 3/4] docs: match the shutdown fix, drop max_workspace_bytes and the v1 promise Shutdown is no longer a race. Operations now describes two outcomes chosen by how the process died: a signal leaves in-flight jobs cancelled and not retried, a kill leaves them requeued and re-run from the start. Notes the stop_grace_period both compose files set. max_workspace_bytes is gone from the product, so it goes from the settings table. Operations says plainly that there is no checkout size cap and how to size WORKSPACE_DIR instead. Remove the "What does v1 commit to?" entry and its TODO. The facts it collected live in Operations and the API reference; the compatibility promise itself is not written yet and a half-answer is worse than no entry. Remove the demo link from Logs, which pointed at a website section that no longer exists. Co-Authored-By: Claude Opus 5 (1M context) --- src/content/docs/start/configuration.md | 1 - src/content/docs/start/faq.md | 27 +------------ src/content/docs/understanding/operations.md | 40 +++++++++++--------- 3 files changed, 24 insertions(+), 44 deletions(-) diff --git a/src/content/docs/start/configuration.md b/src/content/docs/start/configuration.md index 0d946fd..bc84d67 100644 --- a/src/content/docs/start/configuration.md +++ b/src/content/docs/start/configuration.md @@ -72,7 +72,6 @@ Single row, `id = 1`. Changed from **Settings** in the UI or | `default_timeout_seconds` | `900` | Per-job timeout | | `max_concurrent_jobs` | `1` | Jobs this process runs at once. See above | | `max_log_bytes` | 10 MiB | The log stops growing at this size; the run continues | -| `max_workspace_bytes` | 1 GiB | Accepted and stored, **not enforced**. No code reads it; a large checkout is not stopped | | `log_retention_days` | `14` | Prune old logs and job rows | | `default_runtime` | empty | Docker image used when a fork job's pipeline has no `runtime:` | | `skip_fork_prs` | `true` | Fork PRs are ignored. Saving `false` requires Docker plus `default_runtime`. | diff --git a/src/content/docs/start/faq.md b/src/content/docs/start/faq.md index 4a8d432..4c2b08e 100644 --- a/src/content/docs/start/faq.md +++ b/src/content/docs/start/faq.md @@ -1,6 +1,6 @@ --- title: "FAQ" -description: "Short answers on GitHub Apps, Coolify, check-suite gating, GitHub Enterprise, monorepos, dogfooding, and what v1 commits to." +description: "Short answers on GitHub Apps, Coolify, check-suite gating, GitHub Enterprise, monorepos, dogfooding, and what v1 leaves out." sidebar: order: 4 --- @@ -90,28 +90,3 @@ check, it does not publish artifacts. Nothing stops it from checking a public repo, incidentally. The repository's visibility is read from the webhook payload but never gates anything; "private repos" is what it is aimed at, not a restriction it enforces. - -## What does v1 commit to? - -Concretely, as of 1.0.0: - -- **Releases are tagged `vMAJOR.MINOR.PATCH`** and published as an image - (`ghcr.io/openpreflight/openpreflight`, tagged full version, major.minor, and - `latest`) plus linux/amd64 and linux/arm64 binaries. -- **The JSON API is `/api/v1/`.** A breaking change to it would be `/api/v2/`, - not a change under the existing prefix. The surface is in - [API](/using/api/). -- **`default_check_name` does not change on an existing database.** GitHub - matches a required status check by name, so renaming a live install's check - would strand its branch protection rule. New installs get the current - default; existing ones keep what they have. -- **Migrations are append-only and there is no rollback.** Once a version has - booted against your database, downgrading is not supported. See - [Operations](/understanding/operations/). -- **`CI_SECRET_KEY` stays the key.** No release will make an existing key - unreadable without a documented rotation path. - -TODO(vatsal): decide and state the actual compatibility promise for 1.x — -whether minor releases may change settings defaults, remove settings fields, or -change the pipeline file schema, and what deprecation notice a breaking change -gets. diff --git a/src/content/docs/understanding/operations.md b/src/content/docs/understanding/operations.md index bf38f9f..ce55a6d 100644 --- a/src/content/docs/understanding/operations.md +++ b/src/content/docs/understanding/operations.md @@ -61,25 +61,28 @@ will fail its next `test` with a decryption error rather than at boot. **Nothing in progress survives.** There is no checkpointing: a job that was half way through `npm test` does not resume, and there is no partial result. +What differs is how it is recorded, and that depends on how the process died. -On the next boot the runner requeues anything left `in_progress` and runs it -again from the beginning — a fresh clone and every step. That is the usual -outcome of a redeploy, an upgrade, a crash, or `docker compose restart`. +**On a signal** — `docker compose stop`, `restart`, a redeploy, an upgrade, +Ctrl-C. The process stops accepting requests, then waits up to 30 seconds for +every running job to notice the cancelled context, write itself `cancelled`, +and mark its Check Run cancelled. A cancelled job is **not** retried. -The exception is a job that notices the cancelled context before the process -exits. That one finishes as `cancelled`, and a cancelled job is **not** -retried. Which of the two happens is a race between the job unwinding and the -process exiting, and the process does not wait for the runner, so the requeue -is the common case and the cancel is the rare one. +**On a kill** — SIGKILL, an OOM, a crash, or a job still unwinding when the 30 +seconds run out. The row is left `in_progress`, and the next boot requeues it +and runs it again from the beginning: a fresh clone and every step. + +Compose sets `stop_grace_period: 30s` to match the wait. Shortening it means the +runtime kills the process mid-cancel and you get the second case instead. Practical consequences: -- A requeued job opens a **new** Check Run rather than reusing the one the - interrupted attempt created, so the commit collects a second check with the - same name. Branch protection reads the newer one. -- A job cancelled by a restart leaves a cancelled check that nothing will retry - on its own. Re-run it from the job page (`POST /api/v1/jobs/{id}/rerun`) or - use GitHub's **Re-run**. +- A restart normally leaves a **cancelled** check that nothing will retry on its + own. Re-run it from the job page (`POST /api/v1/jobs/{id}/rerun`) or use + GitHub's **Re-run**. +- A requeued job — the kill case — opens a **new** Check Run rather than + reusing the one the interrupted attempt created, so the commit collects a + second check with the same name. Branch protection reads the newer one. - Deploy when the queue is empty if you care. `GET /api/v1/jobs` shows what is queued or running. @@ -121,6 +124,9 @@ The policy that follows from that: An hourly pass prunes expired sessions, then deletes job rows and their log files older than `log_retention_days` (default 14). Queued and running jobs are never pruned. Nothing else is cleaned up automatically — see -[Configuration](/start/configuration/) for the settings that govern it, and -note that `max_workspace_bytes` is stored but not enforced, so a checkout large -enough to fill the disk will fill it. +[Configuration](/start/configuration/) for the settings that govern it. + +There is no cap on checkout size. A per-job workspace is removed when the job +ends, but a repository large enough to fill the disk will fill it while it +runs. Size `WORKSPACE_DIR` for your largest checkout times +`max_concurrent_jobs`. From 009305cf92b1cb8989393b9ed3c0d2e1f70f567e Mon Sep 17 00:00:00 2001 From: trivedi-vatsal Date: Sat, 29 Aug 2026 08:28:50 +0530 Subject: [PATCH 4/4] Ignore local graphify-out. The docs repo is not a graphify workspace; that output belongs at the workspace root only. Co-authored-by: Cursor --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 56fe4a4..92bf3f3 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,7 @@ node_modules/ dist/ .astro/ +graphify-out/ .env .env.production .DS_Store