diff --git a/.claude/settings.json b/.claude/settings.json new file mode 100644 index 000000000..82bca4b30 --- /dev/null +++ b/.claude/settings.json @@ -0,0 +1,5 @@ +{ + "attribution": { + "pr": "" + } +} diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..7bc5074ab --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,26 @@ +# Agent instructions + +The Docker sandbox in `local/sbx/` tells Claude to follow these rules in every +GOV.UK Forms repo, so keep them general and keep this file at this path. + +## Commits + +- Split features into well-scoped commits. Each commit should leave the tests + green. +- Commit titles: 50 characters or fewer, imperative, naming only the thing + changed ("Add Brand model", not "Add Brand model backed by database table"). +- Commit bodies: explain why the change is needed, and include only context + the diff doesn't show, such as external constraints or why an approach was + forced. +- Fold additional changes into the commit they belong to rather than adding + fixup commits. +- Don't push or update pull requests unless asked. Committing locally is fine. + +## Pull requests + +- Use `.github/PULL_REQUEST_TEMPLATE.md`. +- Keep descriptions to a few short paragraphs, with only context a reviewer + can't see in the diff. +- Don't mention AI tooling in pull request descriptions. +- Don't describe fixes as security or vulnerability issues in public pull + requests. Describe what the code now enforces. diff --git a/local/README.md b/local/README.md index c80077539..09b5768ff 100644 --- a/local/README.md +++ b/local/README.md @@ -11,6 +11,9 @@ The components which make up the service are: - https://github.com/govuk-forms/forms-admin - https://github.com/govuk-forms/forms-product-page +To work on the apps with Claude Code in an isolated Docker Sandbox instead, +see [sbx/README.md](sbx/README.md). + ## How it works Each of the components above has a Dockerfile in its repo which is used to build a docker image. The file `compose.yaml` defines the configuration for using diff --git a/local/sbx/README.md b/local/sbx/README.md new file mode 100644 index 000000000..439f6cb2f --- /dev/null +++ b/local/sbx/README.md @@ -0,0 +1,211 @@ +# GOV.UK Forms in a Docker Sandbox + +Use this to work on GOV.UK Forms with Claude Code in a [Docker Sandbox][sbx]. +A sandbox is an isolated Linux virtual machine. Claude can only see and change +what's inside it. + +The sandbox has: + +- the folder holding forms-admin, forms-runner, forms-product-page and + forms-deploy, shared with your machine so changes show on both sides + straight away. Claude starts in that folder, and can see anything else you + keep in it +- the Ruby and Node versions each app needs +- Postgres and Valkey +- Chrome and chromedriver, for the specs that run in a browser +- each app's gems, npm packages and databases, ready to use +- Claude Code, connected to the team's model gateway + +The sandbox cannot see files outside that folder, so don't keep anything in it +you wouldn't want Claude to read. It also cannot see your gateway key or +GitHub token. + +[sbx]: https://docs.docker.com/ai/sandboxes/ + +## Before you start + +You need: + +- [Docker Sandboxes][install] 0.48.0 or newer installed, and to be signed in + with `sbx login`. Check with `sbx version`, and upgrade with + `brew upgrade --cask sbx` if you installed it with Homebrew +- the GOV.UK Forms repos cloned side by side, as described in + [`../README.md`](../README.md) +- your model gateway key, which starts with `sk-` + +Your repos should look like this: + +``` +top-level/ +├── forms-admin +├── forms-deploy +├── forms-product-page +└── forms-runner +``` + +If you want Claude to push and open pull requests, you also need a +[fine-grained GitHub token][pat]. Give it read and write access to +**Contents** and **Pull requests** on the govuk-forms repos. Without a token, +Claude can commit in the sandbox, and you push from your machine as usual. + +[install]: https://docs.docker.com/ai/sandboxes/install/ +[pat]: https://github.com/settings/personal-access-tokens/new + +## Set up the sandbox + +You only need to do this once. + +1. Run the setup script: + + ```bash + cd forms-deploy/local/sbx + ./setup.sh + ``` + +2. Choose whether to add a GitHub token. +3. Paste your model gateway key when asked. +4. Choose whether to add the status line. + +The script: + +- stores your gateway key and GitHub token as sbx secrets +- installs the [commit messages skill][skill] +- adds your git name and email to `~/.sbxenv.yaml`, so your commits in the + sandbox are in your name +- if you choose to, copies [`statusline.sh`](statusline.sh) into `.claude/` in + the folder holding the repos and sets it in `.claude/settings.local.json` + there. Claude then shows the model, folder, git state, context use and cost + under the prompt, in the sandbox and when you run Claude in that folder on + your machine + +The sbx proxy adds your secrets to requests as they leave the sandbox, so the +sandbox never sees them. + +You can safely run the script again. It skips anything already done. To +answer the questions up front, use `--github` or `--no-github`, and +`--statusline` or `--no-statusline`. + +[skill]: https://gitlab.com/gitlab-org/ai/skills/-/blob/main/skills/commit-messages/SKILL.md + +## Start the sandbox + +From this directory, run: + +```bash +sbx env run +``` + +Always run `sbx env` commands from this directory, without a path. Otherwise +sbx will not use your `~/.sbxenv.yaml`, and your commits will not have your +name and email. sbx also loses track of the sandbox if you mix the two. + +The first run takes a few minutes to build and create the sandbox. After that, +it reconnects to the same sandbox and keeps your files, gems and databases. + +Each time Claude starts, the sandbox installs anything missing and prepares +the databases in the background. This takes about a minute the first time and +a few seconds after that. Claude knows to wait for it before running specs. + +## Run the apps + +When you or Claude run an app in the sandbox, open it on your machine: + +- forms-admin: http://localhost:3100 +- forms-runner: http://localhost:3101 +- forms-product-page: http://localhost:3102 + +These are not the apps' usual ports, so the sandbox can run alongside the +[Docker Compose setup](../README.md). + +## Manage the sandbox + +Run these commands from this directory. + +| To | Run | +| ---------------------------- | --------------------------- | +| Run a command in the sandbox | `sbx env exec -- ` | +| Stop the sandbox | `sbx stop govuk-forms` | +| Delete the sandbox | `sbx env rm` | + +Deleting the sandbox does not affect your repos. + +## Change the sandbox + +Changes to `env:` in `sbxenv.yaml` apply the next time you run `sbx env run`. + +Changes to kits, bindings or secrets only apply when the sandbox is created. +To pick them up, delete the sandbox with `sbx env rm`, then run `sbx env run`. + +## How it works + +- [`sbxenv.yaml`](sbxenv.yaml) defines the sandbox: its kits, repos, + environment variables and ports, and which hosts can receive each secret. +- `~/.sbxenv.yaml` holds your personal settings. sbx merges it with + `sbxenv.yaml`. +- The agent is Docker's [Claude Code kit][claude-kit]. The 2 [kits][kits] in + this directory add to it. Each has a descriptor (`.yaml`), and + forms-dev also has a Dockerfile for the files it adds. +- [`ai-gateway/`](ai-gateway) connects Claude Code to the model gateway. + Claude only sees a placeholder in `ANTHROPIC_AUTH_TOKEN`. The sbx proxy + replaces it with your key on requests to the gateway, and nowhere else. +- [`forms-dev/`](forms-dev) installs build tools, mise and Chrome, allows the + domains the setup needs, and declares the optional `github` secret. Each time the + sandbox starts, it runs: + - `forms-dev-up`, which starts Postgres and Valkey and gives each app a + separate `node_modules` in the sandbox. This keeps Linux builds of + packages like esbuild and rolldown out of your machine's `node_modules`. + - `forms-app-setup`, which installs everything each app needs. +- forms-dev also gives Claude + [notes about the sandbox](forms-dev/forms-dev-context.md), and tells it to + follow the commit and pull request rules in this repo's + [`AGENTS.md`](../../AGENTS.md) in every repo. Change the rules there. +- forms-dev turns off the Claude Code footer on pull requests and turns on + fullscreen rendering in the sandbox user's Claude settings, so both apply + in every repo. +- The sandbox cannot use SSH, so inside the sandbox git rewrites GitHub + remotes to HTTPS. The proxy adds your GitHub token to those requests. + +[claude-kit]: https://hub.docker.com/r/docker/sbx-kit-claude +[kits]: https://docs.docker.com/ai/sandboxes/customize/use-kits/ + +## Troubleshooting + +### Specs fail before they run + +The app setup may not have finished, or it may have failed. In the sandbox, +check `/tmp/forms-app-setup.status` and `/tmp/forms-app-setup.log`. If there is +no status file, the setup has not started. To run the setup again, run +`forms-app-setup`. It stops straight away if the setup is still running. + +### Claude gets a 401 error from the gateway + +Your gateway key is missing or wrong. Run `sbx secret ls` to check there's a +global `ai-gateway` secret. To replace it, run +`sbx secret rm ai-gateway -f && ./setup.sh`, then delete and recreate the +sandbox. + +### Building a kit fails with "failed to connect to the docker API" + +If the `docker` command is installed, sbx uses Docker to build the kits in +this directory. Start Docker Desktop and try again. + +### A download or request is blocked + +Run `sbx policy log govuk-forms` to see which domains were blocked. Add them +to the network policy's `runtime` list in +[`forms-dev/forms-dev.yaml`](forms-dev/forms-dev.yaml). + +### `git push` or `gh` fails in the sandbox + +The sandbox needs a GitHub token to push. Push from your machine instead, or +run `sbx secret set github` and then delete and recreate the sandbox. + +### Vite fails on your machine after using the sandbox + +Linux packages have been written to your machine's `node_modules`. Run +`npm ci` in that app on your machine. + +### Claude Code's full screen view will not scroll in tmux + +Add `set -g mouse on` to `~/.tmux.conf` to turn on mouse mode, or use the +Page Up and Page Down keys. diff --git a/local/sbx/ai-gateway/ai-gateway.yaml b/local/sbx/ai-gateway/ai-gateway.yaml new file mode 100644 index 000000000..719c77a97 --- /dev/null +++ b/local/sbx/ai-gateway/ai-gateway.yaml @@ -0,0 +1,55 @@ +# syntax=docker/sandbox-kit:3 +# +# Points Claude Code at the team's model gateway, a LiteLLM proxy, and has +# the sbx proxy supply each person's gateway key. The key stays in the host's +# secret store under the `ai-gateway` service (setup.sh stores it). The +# sandbox only sees a placeholder in ANTHROPIC_AUTH_TOKEN, and the proxy +# replaces the Authorization header on requests to the gateway. +schemaVersion: "3" +kind: mixin +displayName: AI Engineering Lab model gateway +description: >- + Routes Claude Code through the AI Engineering Lab LiteLLM gateway, with + the gateway key supplied by the sandbox proxy. +version: "1.0.0" + +# Hooks run after the kits they require, so the hook below runs after the +# claude kit writes settings.json +requires: ["claude"] + +build: | + FROM scratch + ENV ANTHROPIC_BASE_URL=https://licenseportal.aiengineeringlab.co.uk + +capabilities: + - type: com.docker.sandbox/network-policy@1 + config: + runtime: + allow: + - licenseportal.aiengineeringlab.co.uk + + - type: com.docker.sandbox/credential@1 + description: Model gateway key + config: + service: ai-gateway + phase: runtime + apiKey: + name: ANTHROPIC_AUTH_TOKEN + proxyManaged: true + inject: + - domain: licenseportal.aiengineeringlab.co.uk + header: Authorization + format: "Bearer %s" + + # The claude kit adds an apiKeyHelper when the person has an anthropic + # secret, and Claude Code warns when that and ANTHROPIC_AUTH_TOKEN are both + # set. jq comes with the claude kit's image. + - type: com.docker.sandbox/lifecycle@1 + config: + install: + - description: Remove the claude kit's apiKeyHelper + user: agent + command: > + f=~/.claude/settings.json; + [ ! -s "$f" ] || + { new=$(jq 'del(.apiKeyHelper)' "$f") && printf '%s\n' "$new" > "$f"; } diff --git a/local/sbx/forms-dev/bin/forms-app-setup b/local/sbx/forms-dev/bin/forms-app-setup new file mode 100755 index 000000000..9575ad99e --- /dev/null +++ b/local/sbx/forms-dev/bin/forms-app-setup @@ -0,0 +1,100 @@ +#!/usr/bin/env bash +# Installs Ruby, Node, gems and npm packages and prepares the development and +# test databases for each app. Idempotent: runs in the background on every +# sandbox start, and can be re-run by hand. Progress: /tmp/forms-app-setup.log +set -uo pipefail + +export PATH="$HOME/.local/share/mise/shims:$HOME/.local/bin:$PATH" +export PGPASSWORD=postgres +WS="$(cat "$HOME/.config/forms-dev/workdir")" +STATUS=/tmp/forms-app-setup.status + +# One run at a time: a second would install over the first, and the last to +# finish would overwrite the status +exec 9> /tmp/forms-app-setup.lock +if ! flock -n 9; then + echo "forms-app-setup is already running, see $STATUS" >&2 + exit 1 +fi + +# Opened after the lock, so a run that exits early leaves the log of the run +# in progress alone. Written here rather than by the startup hook, so a run +# by hand doesn't leave the last startup's log in place. +exec > >(tee /tmp/forms-app-setup.log) 2>&1 + +echo running > "$STATUS" +failed="" + +log() { echo "== [$(date +%T)] $*"; } + +db_exists() { + psql -h localhost -U postgres -tAc "SELECT 1 FROM pg_database WHERE datname = '$1'" | grep -q 1 +} + +setup_app() { + local app="$1" dir="$WS/$1" + cd "$dir" || return 1 + + # Versions from the app's .ruby-version and .nvmrc + log "$app: ruby $(cat .ruby-version 2>/dev/null), node $(cat .nvmrc 2>/dev/null)" + mise install || return 1 + + # Never run npm against the host's node_modules (see forms-dev-up) + if ! mountpoint -q node_modules; then + log "$app: node_modules is not the sandbox-only mount, skipping npm" + return 1 + fi + # Native packages are built for one Node version, so a new .nvmrc + # reinstalls them even when package-lock.json hasn't changed + local install_sum + install_sum="$({ node --version && cat package-lock.json; } | sha256sum | cut -d' ' -f1)" + if [ "$(cat node_modules/.forms-install-sum 2>/dev/null)" != "$install_sum" ]; then + log "$app: npm ci" + npm ci --no-audit --no-fund || return 1 + echo "$install_sum" > node_modules/.forms-install-sum + fi + + log "$app: bundle install" + bundle check >/dev/null || bundle install || return 1 + + [ -f config/database.yml ] || return 0 + + log "$app: development database" + bin/rails db:prepare || return 1 + + local test_db="${app//-/_}_test" + if ! db_exists "$test_db"; then + log "$app: test database" + RAILS_ENV=test bin/rails db:create db:schema:load || return 1 + fi +} + +for _ in $(seq 1 60); do + pg_isready -q -h localhost && break + sleep 1 +done + +first="" +# The same list forms-dev-up mounted node_modules for +for app in $(forms-apps); do + if setup_app "$app"; then + log "$app: ready" + first="${first:-$app}" + else + log "$app: FAILED" + failed="$failed $app" + fi +done + +# Default versions outside the app directories, from the first app set up +if [ -n "$first" ] && + ! (cd "$WS/$first" && mise use -g "ruby@$(cat .ruby-version)" "node@$(cat .nvmrc)" >/dev/null); then + log "default ruby and node: FAILED" + failed="$failed default-versions" +fi + +if [ -n "$failed" ]; then + echo "failed:$failed" > "$STATUS" +else + echo done > "$STATUS" +fi diff --git a/local/sbx/forms-dev/bin/forms-apps b/local/sbx/forms-dev/bin/forms-apps new file mode 100755 index 000000000..a4876eab8 --- /dev/null +++ b/local/sbx/forms-dev/bin/forms-apps @@ -0,0 +1,8 @@ +#!/usr/bin/env bash +# Prints the name of every app with Ruby and npm dependencies, one per line. +# setup.sh, forms-dev-up and forms-app-setup all read the list from here, so +# the apps given a sandbox-only node_modules are the ones npm runs in. Listed +# by name, so other repos in the workspace are never touched. +set -euo pipefail + +printf '%s\n' forms-admin forms-runner forms-product-page diff --git a/local/sbx/forms-dev/bin/forms-dev-up b/local/sbx/forms-dev/bin/forms-dev-up new file mode 100755 index 000000000..6b291f9c2 --- /dev/null +++ b/local/sbx/forms-dev/bin/forms-dev-up @@ -0,0 +1,30 @@ +#!/usr/bin/env bash +# Runs on every sandbox start, before the app setup. Starts Postgres and +# Valkey, and gives each app a sandbox-only node_modules so Linux packages +# never overwrite the macOS ones in the host checkout. +set -euo pipefail + +WS="$(cat "$HOME/.config/forms-dev/workdir")" + +# /tmp outlives a restart. Cleared first, so the last session's status never +# stands in for this one's, even if the app setup doesn't get to run. +rm -f /tmp/forms-app-setup.status + +for app in $(forms-apps); do + # Skipped rather than created, so a missing repo isn't left as an empty one + # on the host. forms-app-setup reports it as failed. + [ -d "$WS/$app" ] || continue + target="$WS/$app/node_modules" + store="$HOME/.cache/forms-node-modules/$app" + mkdir -p "$store" "$target" + if ! mountpoint -q "$target"; then + sudo mount --bind "$store" "$target" + fi +done + +# Last, so a Docker failure never leaves an app without its mount +for _ in $(seq 1 60); do + docker info >/dev/null 2>&1 && break + sleep 1 +done +docker compose -f "$WS/forms-deploy/local/compose.yaml" up -d postgres redis diff --git a/local/sbx/forms-dev/forms-dev-context.md b/local/sbx/forms-dev/forms-dev-context.md new file mode 100644 index 000000000..b4352e1fa --- /dev/null +++ b/local/sbx/forms-dev/forms-dev-context.md @@ -0,0 +1,43 @@ +# GOV.UK Forms sandbox + +You are in a Docker sandbox, in the folder holding the GOV.UK Forms repos +(forms-admin, forms-runner, forms-product-page, forms-deploy), mounted from +the host at its host path. Each repo is its own git repository, so run git +from inside the repo you are changing. + +## Commits and pull requests + +The rules are in `forms-deploy/AGENTS.md` and apply in every repo here, not +only forms-deploy. Read that file and follow it before you commit, push, or +open or update a pull request. + +## Sandbox + +- Ruby and Node come from mise, at the versions in each app's `.ruby-version` + and `.nvmrc`, and are already on PATH. +- Postgres (postgres/postgres) and Valkey run on localhost:5432 and :6379 via + Docker Compose. +- On each start, `forms-app-setup` runs in the background. It installs Ruby, + Node, gems and npm packages and creates the development and test databases + for each app. Check `/tmp/forms-app-setup.status` (`running`, `done` or + `failed: `, where `default-versions` means `mise use -g` failed) and + `/tmp/forms-app-setup.log` before running specs. No status file means the + setup hasn't started. Re-run it with `forms-app-setup`, which exits straight + away if a run is still going. +- Chrome for Testing and its chromedriver are installed for the feature and + integration specs, which run headless as they do in CI. There is no + display, so don't set `SETTINGS__SHOW_BROWSER_DURING_TESTS`. +- Each app's `node_modules` is a sandbox-only Linux copy mounted over the + host's macOS one. `npm ci` and `npm install` are safe here and don't affect + the host. Don't unmount it. +- Don't export RAILS_ENV, REDIS_URL or SETTINGS__* globally. They leak into + specs: `spec/rails_helper.rb` only picks `test` when RAILS_ENV is unset, and + the others override the apps' test settings. +- Rails apps listen on 0.0.0.0 (BINDING) on ports 3000 (admin), 3001 (runner) + and 3002 (product page). The host reaches them on 3100, 3101 and 3102, so + give the user those ports when you tell them where to open an app. +- GitHub remotes are rewritten to HTTPS in this sandbox, and the proxy + supplies the user's GitHub token if they set one. Check with + `gh auth status`. If there's no token, `git push` and `gh pr create` won't + work: commit locally and tell the user to push from the host. Don't change + the repos' remote URLs. diff --git a/local/sbx/forms-dev/forms-dev.dockerfile b/local/sbx/forms-dev/forms-dev.dockerfile new file mode 100644 index 000000000..4a9814369 --- /dev/null +++ b/local/sbx/forms-dev/forms-dev.dockerfile @@ -0,0 +1,9 @@ +# syntax=docker/dockerfile:1 +# The overlay for the forms-dev mixin: the scripts its hooks run. Kept out of +# /home/agent, which may be a volume on the composed sandbox. +FROM scratch +COPY --chmod=755 bin/ /usr/local/bin/ +ENV MISE_YES=1 +# Selenium would otherwise look up a chromedriver online, which the sandbox +# blocks, before falling back to the installed one +ENV SE_CHROMEDRIVER=/usr/local/bin/chromedriver diff --git a/local/sbx/forms-dev/forms-dev.yaml b/local/sbx/forms-dev/forms-dev.yaml new file mode 100644 index 000000000..a0ac581d8 --- /dev/null +++ b/local/sbx/forms-dev/forms-dev.yaml @@ -0,0 +1,222 @@ +# syntax=docker/sandbox-kit:3 +# The line above must stay first: it tells the builder this is a kit +# descriptor rather than a Dockerfile. +schemaVersion: "3" +kind: mixin +displayName: GOV.UK Forms development +# sbx copies this description into the CLAUDE.md Claude loads at the start of +# every session; forms-dev-context.md is only read on demand. The commit and +# pull request rules live in AGENTS.md at the root of this repo, which isn't +# loaded here because Claude starts in the folder above. So the last sentence +# sends Claude to that file instead of repeating the rules. +description: >- + Ruby and Node (via mise, versions from each repo's .ruby-version and .nvmrc), + Postgres and Valkey (via Docker Compose), and Chrome with chromedriver for + the GOV.UK Forms Rails apps and their specs. On each start it installs gems + and npm packages and prepares the dev and test databases. In every one of + these repos, read forms-deploy/AGENTS.md and follow it before you commit, + push, or open or update a pull request. +version: "1.0.0" + +# Hooks run after the kits they require, so the attribution hook below runs +# after the claude kit writes settings.json +requires: ["claude"] + +capabilities: + - type: com.docker.sandbox/network-policy@1 + config: + # Open only while the install hooks run + install: + allow: + # apt-get update fails if any configured source is unreachable, so + # every source in the claude workload is listed. The workload + # allows its own dhi.io source. + - deb.debian.org + # mise's installer + - mise.run + - mise.jdx.dev + - github.com + - "*.githubusercontent.com" + # Chrome for Testing downloads + - storage.googleapis.com + runtime: + allow: + # mise, Ruby and Node downloads + - mise.jdx.dev + - mise-versions.jdx.dev + - github.com + - api.github.com + - "*.githubusercontent.com" + - nodejs.org + - cache.ruby-lang.org + # gems and npm packages + - rubygems.org + - "*.rubygems.org" + - registry.npmjs.org + # Postgres and Valkey images + - registry-1.docker.io + - auth.docker.io + - production.cloudflare.docker.com + # Docs, for Claude's web fetches. Terraform provider docs are on + # the Terraform Registry. ** matches any number of subdomains, so + # it covers sites like frontend.design-system.service.gov.uk. + - docs.aws.amazon.com + - developer.hashicorp.com + - registry.terraform.io + - "**.gov.uk" + - "*.rubyonrails.org" + - docs.ruby-lang.org + - ruby-doc.org + - rspec.info + - govuk-components.x-govuk.org + - govuk-form-builder.x-govuk.org + - docs.docker.com + - developer.mozilla.org + - www.w3.org + + # The GitHub token from `sbx secret set github`, if the person stored one. + # The repos' SSH remotes are rewritten to HTTPS below so git uses it too. + - type: com.docker.sandbox/credential@1 + optional: true + description: GitHub access for git push and gh + config: + service: github + phase: runtime + apiKey: + name: GH_TOKEN + proxyManaged: true + inject: + - {domain: api.github.com, header: Authorization, format: "Bearer %s"} + - {domain: github.com, scheme: basic, username: x-access-token} + + - type: com.docker.sandbox/lifecycle@1 + config: + install: + - description: Install build dependencies for Ruby and native gems + user: "0" + env: &network + # The sandbox's proxy, and the CA bundle that trusts it. Hooks only + # see the variables they list. + - HTTP_PROXY + - HTTPS_PROXY + - NO_PROXY + - http_proxy + - https_proxy + - no_proxy + - SSL_CERT_FILE + - NODE_EXTRA_CA_CERTS + - NODE_USE_ENV_PROXY + command: > + apt-get update && + apt-get install -y --no-install-recommends + build-essential libpq-dev libyaml-dev libssl-dev zlib1g-dev libffi-dev + postgresql-client git curl ca-certificates unzip + + # The libraries Chrome for Testing links against. Its zip, unlike + # Chrome's apt package, doesn't pull them in. + - description: Install the libraries Chrome needs + user: "0" + env: *network + command: > + apt-get install -y --no-install-recommends + fonts-liberation libasound2t64 libatk-bridge2.0-0t64 libatk1.0-0t64 + libatspi2.0-0t64 libcairo2 libcups2t64 libdbus-1-3 libexpat1 libgbm1 + libglib2.0-0t64 libgtk-3-0t64 libnspr4 libnss3 libpango-1.0-0 + libvulkan1 libx11-6 libxcb1 libxcomposite1 libxdamage1 libxext6 + libxfixes3 libxkbcommon0 libxrandr2 xdg-utils + + # The feature specs drive headless Chrome through Selenium, and + # forms-admin's integration specs through Cuprite. Debian's + # chromium package and Google's apt repository only offer the latest + # version, and Google's has no arm64 build. Chrome for Testing keeps + # every version, with a chromedriver to match. To upgrade, pick a + # version from https://googlechromelabs.github.io/chrome-for-testing/ + - description: Install Chrome for Testing and chromedriver + user: "0" + env: *network + command: > + version=154.0.8037.92 && + case "$(dpkg --print-architecture)" in + arm64) platform=linux-arm64 ;; + *) platform=linux64 ;; + esac && + for name in chrome chromedriver; do + curl -fsSL -o "/tmp/$name.zip" + "https://storage.googleapis.com/chrome-for-testing-public/$version/$platform/$name-$platform.zip" && + unzip -oq "/tmp/$name.zip" -d /opt && + rm "/tmp/$name.zip" && + ln -sf "/opt/$name-$platform/$name" "/usr/local/bin/$name" || exit 1; + done + + - description: Install mise and read .ruby-version and .nvmrc files + user: agent + env: *network + command: > + curl -fsSL https://mise.run | sh && + ~/.local/bin/mise settings add idiomatic_version_file_enable_tools ruby && + ~/.local/bin/mise settings add idiomatic_version_file_enable_tools node + + # Ahead of the image's own Node. Claude's Bash tool sources + # $CLAUDE_ENV_FILE before every command, so this reaches non-login + # shells too. + - description: Put mise shims first on PATH for every shell + user: agent + env: [CLAUDE_ENV_FILE] + command: > + for rc in ~/.bashrc ~/.profile "${CLAUDE_ENV_FILE:-/etc/sandbox-persistent.sh}"; do + grep -q 'mise/shims' "$rc" 2>/dev/null || + echo 'export PATH="$HOME/.local/share/mise/shims:$HOME/.local/bin:$PATH"' >> "$rc"; + done + + # The repos' remotes are SSH, but the sandbox has no SSH key. Over + # HTTPS the proxy adds the GitHub token. This only changes the + # sandbox's ~/.gitconfig. + - description: Use HTTPS for GitHub so pushes use the github secret + user: agent + command: > + git config --global --replace-all url.https://github.com/.insteadOf git@github.com: git@github.com: && + git config --global --add url.https://github.com/.insteadOf ssh://git@github.com/ + + # AGENTS.md asks for no mention of AI tooling in pull requests. + # forms-deploy's .claude/settings.json only applies when Claude starts + # in forms-deploy, and here it starts in the folder above, so the + # setting goes in the sandbox user's settings, which apply everywhere. + # jq comes with the claude kit's image. + - description: Leave the Claude Code footer off pull requests + user: agent + command: > + mkdir -p ~/.claude && f=~/.claude/settings.json && + { [ -s "$f" ] || echo '{}' > "$f"; } && + new=$(jq '.attribution.pr = ""' "$f") && printf '%s\n' "$new" > "$f" + + # Fullscreen rendering draws Claude Code in the terminal's alternate + # screen, which stops the flicker the default renderer shows in long + # sessions. + - description: Run Claude Code in fullscreen render mode + user: agent + command: > + mkdir -p ~/.claude && f=~/.claude/settings.json && + { [ -s "$f" ] || echo '{}' > "$f"; } && + new=$(jq '.tui = "fullscreen"' "$f") && printf '%s\n' "$new" > "$f" + + # sbxenv.yaml mounts the folder holding the repos as the workspace. + # The scripts read it from here, so they also work when run by hand. + - description: Record the folder holding the repos for the forms-dev scripts + user: agent + env: [WORKSPACE_DIR] + command: > + mkdir -p ~/.config/forms-dev && + printf '%s\n' "$WORKSPACE_DIR" > ~/.config/forms-dev/workdir + + startup: + - description: Start Postgres and Valkey, mount sandbox-only node_modules + env: [DOCKER_HOST] + command: [forms-dev-up] + - description: Install gems and npm packages, prepare databases + background: true + env: *network + command: [sh, -c, "forms-app-setup > /dev/null 2>&1"] + + - type: com.docker.sandbox/agent-context@1 + config: + contentFile: ./forms-dev-context.md diff --git a/local/sbx/sbxenv.yaml b/local/sbx/sbxenv.yaml new file mode 100644 index 000000000..bc5f2d306 --- /dev/null +++ b/local/sbx/sbxenv.yaml @@ -0,0 +1,60 @@ +# Docker Sandboxes environment for GOV.UK Forms development. +# +# Mounts the GOV.UK Forms repos into one Claude sandbox, with Ruby, Node and +# the Postgres/Valkey services from ../compose.yaml. See README.md in this +# directory for setup. +# +# ./setup.sh # once per person +# sbx env run # from this directory +# +# Run sbx env commands from this directory without a path: only then does sbx +# merge in your personal ~/.sbxenv.yaml (your git identity). +# +# https://docs.docker.com/ai/sandboxes/configuration/environment-files +schemaVersion: "1" +name: govuk-forms + +# The v3 Claude Code workload. `agent:` is the handle sbx derives from the +# kit reference under kits:. Bump the tag to move to a newer Claude Code. +agent: sbx-kit-claude +kits: + - docker.io/docker/sbx-kit-claude:2.1.296 + # Claude Code through the team's model gateway (ai-gateway/) + - ./ai-gateway + # Ruby, Node, Postgres and Valkey for the apps (forms-dev/) + - ./forms-dev + +# The folder holding the repos, cloned side by side. It is mounted at its +# host path and Claude starts in it, so everything in that folder is shared +# with the sandbox. sbx always starts the agent in the primary workspace. +workspace: ../../.. + +env: + # Rails binds to localhost in development; listen on all interfaces so the + # published ports below can reach the apps. + # Don't add app config here (RAILS_ENV, REDIS_URL, SETTINGS__*): these + # variables also reach specs and override the apps' test settings. + BINDING: 0.0.0.0 + +# Where the proxy may use each stored secret. setup.sh stores the secrets; +# this only approves them for these hosts. +bindings: + ai-gateway: + apiKey: + domains: + - licenseportal.aiengineeringlab.co.uk + github: + apiKey: + domains: + - api.github.com + - github.com + +# The host ports differ from the apps' own, which ../compose.yaml publishes, +# so the sandbox and Docker Compose can run at the same time. +ports: + - sandbox: 3000 # forms-admin + host: 3100 + - sandbox: 3001 # forms-runner + host: 3101 + - sandbox: 3002 # forms-product-page + host: 3102 diff --git a/local/sbx/setup.sh b/local/sbx/setup.sh new file mode 100755 index 000000000..4c9bcd9e9 --- /dev/null +++ b/local/sbx/setup.sh @@ -0,0 +1,201 @@ +#!/usr/bin/env bash +# One-time setup for the GOV.UK Forms sandbox. Safe to re-run: each step is +# skipped if it's already done. See README.md in this directory. +# +# ./setup.sh [--github | --no-github] [--statusline | --no-statusline] +# +# --github / --no-github answer the "GitHub token" question without prompting. +# --statusline / --no-statusline do the same for the "Status line" question. +set -euo pipefail + +GATEWAY_HOST=licenseportal.aiengineeringlab.co.uk +SKILLS_REPO=https://gitlab.com/gitlab-org/ai/skills +# The oldest sbx that can read the Claude kit pinned in sbxenv.yaml. Raise it +# when a kit bump needs a newer sbx. +MIN_SBX_VERSION=0.48.0 + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +WS="$(cd "$SCRIPT_DIR/../../.." && pwd)" + +github=ask +statusline=ask +for arg in "$@"; do + case "$arg" in + --github) github=yes ;; + --no-github) github=no ;; + --statusline) statusline=yes ;; + --no-statusline) statusline=no ;; + -h | --help) sed -n '2,8p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; + *) echo "Unknown option: $arg" >&2; exit 1 ;; + esac +done + +step() { printf '\n== %s\n' "$*"; } +ok() { echo " ✓ $*"; } +note() { echo " - $*"; } + +step "Checking prerequisites" +if ! command -v sbx >/dev/null; then + echo " sbx isn't installed. See https://docs.docker.com/ai/sandboxes/install/" >&2 + exit 1 +fi +sbx_version="$(sbx version 2>/dev/null | awk '{print $3}')" +sbx_version="${sbx_version#v}" +if [ "$(printf '%s\n' "$MIN_SBX_VERSION" "$sbx_version" | sort -V | head -n1)" != "$MIN_SBX_VERSION" ]; then + echo " sbx $sbx_version is too old: this sandbox needs $MIN_SBX_VERSION or newer." >&2 + echo " Upgrade it, for example with: brew upgrade --cask sbx" >&2 + exit 1 +fi +ok "sbx $sbx_version" +missing="" +for app in $("$SCRIPT_DIR/forms-dev/bin/forms-apps"); do + [ -d "$WS/$app" ] || missing="$missing $app" +done +if [ -n "$missing" ]; then + echo " Clone these next to forms-deploy in $WS first:$missing" >&2 + exit 1 +fi +ok "repos found in $WS" + +secrets="$(sbx secret ls 2>/dev/null || true)" + +step "GitHub token (optional)" +if echo "$secrets" | grep -Eq '^\(global\)[[:space:]]+service[[:space:]]+github[[:space:]]'; then + ok "already set" +else + if [ "$github" = ask ]; then + note "With a token, Claude in the sandbox can push and open PRs." + note "Without one, it commits locally and you push from your Mac." + read -r -p " Let the sandbox push and open PRs with a GitHub token? [y/N] " answer + case "$answer" in [yY]*) github=yes ;; *) github=no ;; esac + fi + if [ "$github" = yes ]; then + note "Paste a fine-grained PAT with Contents and Pull requests (read and" + note "write) on the govuk-forms repos." + sbx secret set github + ok "stored" + else + note "skipped. To add one later: sbx secret set github" + fi +fi + +step "Model gateway key" +if echo "$secrets" | grep -Eq '^\(global\)[[:space:]]+service[[:space:]]+ai-gateway[[:space:]]'; then + ok "already set" +else + read -r -s -p " Paste your LiteLLM key (sk-...): " key + echo + # Pasted keys often pick up whitespace or quotes, including curly ones, + # which the gateway rejects + key="$(printf '%s' "$key" | tr -d '[:space:]' | + sed -e "s/[\"']//g" -e 's/“//g' -e 's/”//g' -e "s/‘//g" -e "s/’//g")" + case "$key" in + sk-?*) ;; + *) echo " That doesn't look like a LiteLLM key: it should start with sk-" >&2; exit 1 ;; + esac + # The ai-gateway kit declares this service, and sbxenv.yaml approves it for + # the gateway host only + printf '%s' "$key" | sbx secret set ai-gateway >/dev/null + unset key + ok "stored as the ai-gateway secret" +fi +# Earlier versions of this script stored the key as a custom secret +if echo "$secrets" | grep -Eq "^\(global\)[[:space:]]+${GATEWAY_HOST}[[:space:]]"; then + note "The old custom secret for $GATEWAY_HOST is no longer used." + # Removing by placeholder takes every custom secret that shares it, so + # only suggest that when the old secret is the only one + if [ "$(echo "$secrets" | awk '$4 == "proxy-managed"' | wc -l)" -eq 1 ]; then + note "Remove it with: sbx secret rm --placeholder proxy-managed" + else + note "Remove it with sbx secret rm, choosing the one for $GATEWAY_HOST." + fi +fi + +step "Commit message skill" +if sbx skills ls 2>/dev/null | grep -Eq '^commit-messages[[:space:]]'; then + ok "already installed" +else + sbx skills add "$SKILLS_REPO" --skill commit-messages >/dev/null + ok "installed" +fi + +step "Git identity for commits in the sandbox" +name="$(git config --global user.name || true)" +email="$(git config --global user.email || true)" +defaults="$HOME/.sbxenv.yaml" +# Single-quoted YAML scalars only need ' doubled, so names containing quotes +# or backslashes stay intact +yaml_quote() { + local quote="'" + printf "'%s'" "${1//$quote/$quote$quote}" +} +# A defaults file with only some of the four variables leaves commits without +# a full identity, so it doesn't count as set up +has_identity() { + local var + for var in GIT_AUTHOR_NAME GIT_AUTHOR_EMAIL GIT_COMMITTER_NAME GIT_COMMITTER_EMAIL; do + grep -Eq "^[[:space:]]+$var:" "$defaults" || return 1 + done +} +yaml_name="$(yaml_quote "$name")" +yaml_email="$(yaml_quote "$email")" +if [ -z "$name" ] || [ -z "$email" ]; then + note "set git config --global user.name and user.email, then re-run this" +elif [ ! -f "$defaults" ]; then + cat > "$defaults" <)" +elif has_identity; then + ok "already in $defaults" +else + note "$defaults exists, so it wasn't changed. Add these under env: in it:" + for var in AUTHOR COMMITTER; do + echo " GIT_${var}_NAME: $yaml_name" + echo " GIT_${var}_EMAIL: $yaml_email" + done +fi + +step "Status line (optional)" +claude_dir="$WS/.claude" +local_settings="$claude_dir/settings.local.json" +# Claude starts in $WS on your machine and in the sandbox, which mounts it at +# the same path, so this one setting works in both +# shellcheck disable=SC2016 # Claude expands CLAUDE_PROJECT_DIR, not this script +statusline_command='bash "${CLAUDE_PROJECT_DIR:-.}/.claude/statusline.sh"' +current="$(jq -r '.statusLine.command // ""' "$local_settings" 2>/dev/null || true)" +if [ "$current" = "$statusline_command" ] && + cmp -s "$SCRIPT_DIR/statusline.sh" "$claude_dir/statusline.sh"; then + ok "already set up" +else + if [ "$statusline" = ask ]; then + note "Shows the model, folder, git branch and state, context use and cost" + note "under Claude's prompt, in the sandbox and when you run Claude in $WS." + read -r -p " Add the status line? [y/N] " answer + case "$answer" in [yY]*) statusline=yes ;; *) statusline=no ;; esac + fi + if [ "$statusline" = no ]; then + note "skipped. To add it later: ./setup.sh --statusline" + elif ! command -v jq >/dev/null; then + note "skipped: the status line needs jq. Install it, then re-run this." + else + mkdir -p "$claude_dir" + cp "$SCRIPT_DIR/statusline.sh" "$claude_dir/statusline.sh" + [ -s "$local_settings" ] || echo '{}' > "$local_settings" + new="$(jq --arg c "$statusline_command" \ + '.statusLine = {type: "command", command: $c}' "$local_settings")" + printf '%s\n' "$new" > "$local_settings" + ok "wrote $claude_dir/statusline.sh and set it in settings.local.json" + fi +fi + +step "Done. Start the sandbox with:" +echo " cd $SCRIPT_DIR && sbx env run" diff --git a/local/sbx/statusline.sh b/local/sbx/statusline.sh new file mode 100755 index 000000000..f3b245b4b --- /dev/null +++ b/local/sbx/statusline.sh @@ -0,0 +1,62 @@ +#!/bin/bash +# Claude Code status line: model | dir branch ✓, with used/window $cost anchored right +input=$(cat) +g() { printf '%s' "$input" | jq -r "$1" 2>/dev/null; } + +# Columns Claude Code keeps free at the right edge of the status line +right_margin=4 + +model=$(g '.model.display_name // ""') +cwd=$(g '.workspace.current_dir // .cwd // ""') +used=$(g '.context_window.total_input_tokens // 0') +size=$(g '.context_window.context_window_size // 0') +cost=$(g '.cost.total_cost_usd // 0') + +esc=$'\033' +reset="${esc}[0m" +dim="${esc}[2m" +red="${esc}[31m" +green="${esc}[32m" +yellow="${esc}[33m" +blue="${esc}[34m" +magenta="${esc}[35m" +cyan="${esc}[36m" +sep=" ${dim}|${reset} " + +case "$cwd" in + "$HOME"*) dir="~${cwd#"$HOME"}" ;; + *) dir="$cwd" ;; +esac + +out="${cyan}${model}${reset}${sep}${blue}${dir}${reset}" +# Visible width of out, without the colour codes +left_len=$((${#model} + 3 + ${#dir})) + +if [ -n "$cwd" ] && git -C "$cwd" --no-optional-locks rev-parse --is-inside-work-tree >/dev/null 2>&1; then + branch=$(git -C "$cwd" --no-optional-locks symbolic-ref --short -q HEAD 2>/dev/null || + git -C "$cwd" --no-optional-locks rev-parse --short HEAD 2>/dev/null) + if [ -n "$(git -C "$cwd" --no-optional-locks status --porcelain 2>/dev/null)" ]; then + mark="${red}✗${reset}" + else + mark="${green}✓${reset}" + fi + out="${out} ${magenta}${branch}${reset} ${mark}" + left_len=$((left_len + 1 + ${#branch} + 2)) +fi + +# Context colour follows how full the window is +right=$(awk -v u="${used:-0}" -v s="${size:-0}" -v c="${cost:-0}" \ + -v g="$green" -v y="$yellow" -v r="$red" -v reset="$reset" 'BEGIN { + p = (s > 0) ? u / s : 0 + col = (p >= 0.8) ? r : (p >= 0.5) ? y : g + printf "%s%6.1fk/%4dk%s %s%7s%s", col, u / 1000, s / 1000, reset, y, sprintf("$%.2f", c), reset + }') + +# Fill the gap so the right block ends at the window edge; fall back to a separator when it will not fit +right_len=21 +gap=$((${COLUMNS:-0} - right_margin - left_len - right_len)) +if [ "$gap" -ge 2 ]; then + printf '%s%*s%s' "$out" "$gap" '' "$right" +else + printf '%s%s%s' "$out" "$sep" "$right" +fi