Skip to content

docs(ci): describe the pipeline that actually runs #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
krlex merged 1 commit into from
Jul 30, 2026
Merged

docs(ci): describe the pipeline that actually runs #5

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 85 additions & 49 deletions docs/08-ci-cd-pipeline.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,36 +7,46 @@ Forail Platform uses **GitHub Actions** as the public CI/CD pipeline. Each repos
## Pipeline Overview

```
┌──────────┐ ┌────────┐ ┌────────┐ ┌────────┐ ┌──────────┐ ┌─────────┐
│ Checkout │──►│ Lint │──►│ Test │──►│ Build │──►│ Security │──►│ Release │
└──────────┘ └────────┘ └────────┘ └────────┘ └──────────┘ └─────────┘
GitHub ruff / pytest / docker pip-audit docker push
Actions tsc vitest build trivy ghcr.io/forail-platform
┌──────────┐ ┌────────┐ ┌────────┐ ┌───────────────────────────────┐
│ Checkout │──►│ Lint │──►│ Test │───────►│ publish │
└──────────┘ └────────┘ └────────┘ │ build → test in image → push │
GitHub ruff / pytest / └───────────────────────────────┘
Actions go vet vitest / envtest ONLY on a v* tag
```

Each repo's workflow file: **`.github/workflows/ci.yml`**

## Pipeline Stages

| Stage | What it does | Fails if... |
| -------------------- | ----------------------------------------------------------------- | --------------------------------- |
| Checkout | `actions/checkout@v4` clones the repo on the runner | Repo unreachable |
| Lint (Python) | `ruff check` in backend / assistant | Lint errors |
| Lint (Frontend) | `tsc --noEmit` + ESLint | TypeScript type errors |
| Test (Python) | `pytest` against `tests_standalone/` (no DB needed for fast path) | Any test fails |
| Test (Frontend) | `vitest run` | Any test fails |
| Build | `docker build` to produce the release image | Build error |
| Security (pip-audit) | CVE scan on Python dependencies | Critical CVE |
| Security (Trivy) | Container image scan | CRITICAL CVE |
| Release | `docker push` to `ghcr.io/forail-platform/*` | Only on `main` branch or tag push |
The jobs differ per repo. What each one actually runs:

| Repo | Always (push + PR) | `publish` (v* tags only) |
| ------------------ | ---------------------------------------------------- | --------------------------------------------------------- |
| `forail-backend` | `ruff check` , `pyproject.toml` parse, standalone tests | build image → **run the Django functional tests inside the built image** → push |
| `forail-frontend` | `npm run lint`, `npm test`, `npm run build` | build and push image |
| `forail-operator` | `go vet`, `go build`, controller tests (envtest) | build and push image |
| `forail-assistant` | `pytest -q` | build and push image |

### Stage Conditions

| Stage | When it runs |
| -------------------- | ------------------------------------------------------- |
| Checkout, Lint, Test | Every push / PR |
| Build, Security | `main` branch builds and tag builds |
| Release | `main` branch builds and tag builds (push to `ghcr.io`) |
| Stage | When it runs |
| ------------ | ------------------------------------------------------------------ |
| Lint, Test | Every push and every PR |
| `publish` | **Only** when the ref is a tag matching `v*` — `if: startsWith(github.ref, 'refs/tags/v')` |

> **A merge to `main` does not produce an image.** It runs lint and tests only.
> The image on `ghcr.io` changes when, and only when, someone pushes a `v*` tag.
> This is the single most common source of confusion: `main` can be many commits
> ahead of the newest published image, and that is by design.

### Known gaps

These are documented as they are, not as they should be:

- **`ruff check . || true`** in the backend — lint output is informational; lint
errors do not fail the build.
- **No container image scanning** (no Trivy) and **no `pip-audit`** stage in any
repo's workflow today.

---

Expand All @@ -54,17 +64,32 @@ No third-party credentials are required — everything runs in the GitHub-hosted

## Docker Images

| Image | Source | Description |
| ------------------------------------------------ | ---------------------------- | ------------------------------------- |
| `ghcr.io/forail-platform/forail-backend:latest` | `forail-backend/Dockerfile` | Django API + task engine |
| `ghcr.io/forail-platform/forail-backend:<version>` | Same | Version-tagged (CalVer) |
| `ghcr.io/forail-platform/forail-frontend:latest` | `forail-frontend/Dockerfile` | React SPA + nginx |
| `ghcr.io/forail-platform/forail-frontend:<version>` | Same | Version-tagged |
| `ghcr.io/forail-platform/forail-assistant:latest` | `forail-assistant/Dockerfile` | FastAPI + Ollama + ChromaDB (preview) |
| `ghcr.io/forail-platform/forail-operator:<version>` | `forail-operator/Dockerfile` | Kubernetes operator |
| Image | Source | Description |
| --------------------------------------------------- | ----------------------------- | ------------------------------------- |
| `ghcr.io/forail-platform/forail-backend:<version>` | `forail-backend/Dockerfile` | Django API + task engine |
| `ghcr.io/forail-platform/forail-frontend:<version>` | `forail-frontend/Dockerfile` | React SPA + nginx |
| `ghcr.io/forail-platform/forail-operator:<version>` | `forail-operator/Dockerfile` | Kubernetes operator |
| `ghcr.io/forail-platform/forail-assistant:<version>` | `forail-assistant/Dockerfile` | FastAPI + Ollama + ChromaDB (preview) |

Every image carries **exactly one tag: the version**, taken from the git tag with
the `v` stripped. There is **no `latest` tag** — nothing in the pipeline creates
one, so `docker pull …:latest` fails. Always pull an explicit version.

Release-candidate tags (`v2026.07.2-rc1`) go through the same `publish` job and
produce a normal image (`…:2026.07.2-rc1`), which is how a build is validated
against a real cluster before a release is cut.

All images are **public** — no pull secret required for `docker pull` or `helm install`.

To see what is actually published, ask the registry rather than guessing:

```bash
TOKEN=$(curl -s "https://ghcr.io/token?scope=repository:forail-platform/forail-backend:pull" \
| python3 -c "import json,sys; print(json.load(sys.stdin)['token'])")
curl -s -H "Authorization: Bearer $TOKEN" \
https://ghcr.io/v2/forail-platform/forail-backend/tags/list
```

---

## Versioning
Expand All @@ -78,12 +103,18 @@ YYYY.MM.PATCH
2026.04.0 # April release
```

The version is derived from the git tag on `forail-deploy`:
**Each repo carries its own tag.** There is no central tag that releases the
platform — `forail-backend`, `forail-frontend`, `forail-operator` and
`forail-assistant` are tagged individually, and only the repos that changed need
a new one. That is why published versions legitimately differ between components
(for example backend `2026.07.1` alongside frontend `2026.07.0`); the Helm chart
is what pins a working set together.

```bash
cd forail-backend
git tag -a v2026.05.0 -m "Forail 2026.05.0"
git push origin v2026.05.0
# GitHub Actions automatically: checkout → lint → test → build → security → push to ghcr.io
git push github v2026.05.0
# GitHub Actions then: lint → test → build image → test inside the image → push to ghcr.io
```

---
Expand All @@ -102,11 +133,8 @@ ruff check forail/
DJANGO_SETTINGS_MODULE=forail.settings.development \
python -m pytest forail/main/tests/unit/ -q

# Security
pip install pip-audit && pip-audit -r requirements/requirements.txt

# Build image
docker build -t ghcr.io/forail-platform/forail-backend:latest .
# Build image (tag it with a version — there is no :latest)
docker build -t ghcr.io/forail-platform/forail-backend:2026.05.0 .
```

### Frontend
Expand All @@ -120,27 +148,35 @@ npx tsc --noEmit
# Tests
npx vitest run

# Build image
docker build -t ghcr.io/forail-platform/forail-frontend:latest .
# Build image (tag it with a version — there is no :latest)
docker build -t ghcr.io/forail-platform/forail-frontend:2026.05.0 .
```

---

## Release Process

1. Ensure all tests pass on both backend and frontend (GitHub Actions on PR/push must be green)
2. Update docs and release notes
3. Commit to `forail-deploy`: `git commit -m "chore: prepare release v2026.05.0"`
4. Tag: `git tag -a v2026.05.0 -m "Forail 2026.05.0"`
5. Push tag: `git push origin v2026.05.0`
6. GitHub Actions automatically:
- Runs lint + tests
- Builds Docker images with version tag
- Scans for vulnerabilities
- Pushes `ghcr.io/forail-platform/forail-backend:<version>` and friends to GHCR
1. Ensure GitHub Actions is green on `main` for every repo being released
2. Bump `VERSION` in the repos that ship a version, and the Helm chart's
`version` / `appVersion` so it pins the images you are about to publish
3. Write the release notes (`forail-deploy/docs/RELEASE_NOTES_v<version>.md`) and
the docs-site release page
4. Tag each changed repo and push the tag — this is what builds and publishes:
`git tag -a v2026.05.0 -m "Forail 2026.05.0" && git push github v2026.05.0`
5. Install the published chart and images into a clean cluster and run the full
regression before announcing anything
6. Create the GitHub Release

### Validate before you release

Cut an rc tag first (`v2026.05.0-rc1`). It publishes a real image through the
same job, so the release candidate can be installed into a clean cluster and put
through the full Cypress suite. Only then cut the real tag. Rc images stay on
`ghcr.io` — harmless, but do not point a chart at one.

### Watch out

- **Never release without passing tests.**
- **Tag format must have `v` prefix:** `v2026.05.0`, not `2026.05.0`.
- **Merging to `main` publishes nothing** — only a `v*` tag does.
- **Image visibility** — when a new package is first pushed to `ghcr.io`, GitHub creates it as **private** by default. You must manually flip it to public via the Packages settings (`https://github.com/orgs/forail-platform/packages`).
48 changes: 40 additions & 8 deletions docs/10-contributing-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,28 @@ Git workflow, commit conventions, coding standards, and the PR process.
2. **Every change must be understood** — if you can't explain why, don't commit it
3. **Author of all commits is Krstan Vjestica** — never attribute tools as authors
4. **Review the diff before committing** — always
5. **Everything must pass before you commit** — not `vagrant validate`, not a syntax
check: the actual thing running. A change to the dev environment is proven by
destroying the VM and building it again; a change to the platform is proven by
a clean install plus the full Cypress regression

---

## Development environment

- **VirtualBox is the only supported provider.** libvirt/KVM must not be running
on the same host — one hypervisor owns AMD-V per boot, and the loser's guests
die with `Guru Meditation VERR_SVM_IN_USE` or wedge mid-boot. Keep `libvirtd`
masked; `forail-dev-cluster/scripts/up.sh` refuses to start if it is active.
Full root-cause writeup: `forail-dev-cluster/docs/TROUBLESHOOTING-vagrant.md`.
- **Bring the cluster up with `scripts/up.sh`**, not bare `vagrant up` — it goes
node by node in dependency order and recreates any node whose boot wedges.
- **Start from a clean state when you are testing.** `vagrant up` on an existing
VM only boots it; it does **not** re-provision. Mixing a freshly created node
with previously provisioned ones gives two different cluster CAs and a control
plane that never reaches quorum. Use `vagrant destroy -f` first.
- **The dev VMs collide on host ports.** `forail-backend` and `forail-deploy`
both forward `8013` and `8080`, so only one of them can run at a time.

---

Expand All @@ -28,10 +50,13 @@ chore/update-dependencies # Maintenance

### Standard flow

**`main` is the integration branch.** There is no `devel` branch in any Forail
repo — branch from `main` and target `main` in the PR.

```bash
# 1. Create branch from devel
git checkout devel
git pull origin devel
# 1. Create branch from main
git checkout main
git pull github main
git checkout -b feature/my-feature

# 2. Make changes, test, commit
Expand All @@ -41,17 +66,24 @@ git add forail/main/models/my_model.py
git commit -m "feat(models): add Policy model for governance"

# 3. Push and create PR
git push origin feature/my-feature
git push github feature/my-feature
gh pr create --base main
```

### Updating the branch

```bash
git checkout devel && git pull origin devel
git checkout main && git pull github main
git checkout feature/my-feature
git rebase devel
git rebase main
```

### Remotes

Most repos have two: `github` (github.com/forail-platform, the canonical one,
where CI and releases run) and `origin` (the GitLab mirror). Push branches and
tags to **both** when you are done, or the mirror silently falls behind.

---

## Commit Conventions
Expand Down Expand Up @@ -121,15 +153,15 @@ cd forail/ui_next && npx tsc --noEmit
- [ ] All tests pass
- [ ] No lint errors
- [ ] Commit messages follow conventions
- [ ] Branch is rebased on latest `devel`
- [ ] Branch is rebased on latest `main`
- [ ] Changes are minimal and focused

### PR guidelines

- **Keep PRs small** — ideally under 500 lines. Large PRs are harder to review.
- **One concern per PR** — don't mix a bug fix with refactoring.
- **Include tests** — new features need tests, bug fixes need a regression test.
- **Target `devel` branch** — all PRs merge into `devel`.
- **Target `main`** — all PRs merge into `main`.

### Review Checklist

Expand Down
Loading