Skip to content

migration.md

Latest commit

 

History

History
524 lines (384 loc) · 16.1 KB

migration.md

File metadata and controls

524 lines (384 loc) · 16.1 KB
Raw

Migration Guide: stackctl.sh to stackctl

DRAFT -- Pending CLI Stabilization

This document tracks the migration from stackctl.sh to the standalone stackctl binary. The CLI contracts, release workflow, and GitHub Actions integration are still evolving and have not yet reached a stable 1.0. Sections marked with ⚠️ may change before the first stable release.

This guide documents the migration from the repository-local ./stackctl.sh script to the standalone stackctl binary. It covers configuration migration, command mapping, behavior differences, and rollback instructions.

Overview

AniTrend/local-stack historically shipped a tools/stackctl.sh script plus Python-based generation and rendering tools (generate_stacks.py, render_compose.py). The stackctl binary replaces this entire toolchain with a single Deno-compiled binary, eliminating the Python and script dependencies.

Before After
./stackctl.sh up stackctl up
Python 3 + dependencies Single binary, no runtime
Per-repo local script System-wide install (Homebrew)
Shell-based config via env vars ~/.stackctl YAML config
Manual profile switching Built-in profile overlays

Prerequisites

  • Docker with Swarm mode enabled (same as before)
  • stackctl binary — installed via one of:
    • Homebrew: brew install AniTrend/tap/stackctl
    • GitHub Releases: download the appropriate tarball (stackctl-v<version>-<target-triple>.tar.gz) from the latest release. Supported triples: x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-apple-darwin, aarch64-apple-darwin.
    • Manual: deno install -n stackctl --allow-read --allow-write --allow-env --allow-run --allow-sys jsr:@anitrend/stackctl
  • SOPS + age (optional) — only needed for stackctl secrets commands

Quick Start

# Verify installation
stackctl --version

# Initialize config in your project
stackctl init

# Deploy all stacks
stackctl sync

# Check environment
stackctl doctor

Configuration Migration

Before: Environment Variables

The old stackctl.sh used shell environment variables and .env files:

export COMPOSE_DIR="./docker-compose"
export RENDER_DIR="./.rendered"
export STACKS_DIR="./stacks"
export STACK_PREFIX="mystack"
export STACKCTL_PROFILE="dev"

After: YAML Config File

Create a .stackctl file (generated via stackctl init):

project: myproject

stack:
  # Directory used by doctor and sync to locate generated stack files
  # (generate defaults to <repoRoot>/stacks instead)
  directory: ./stack
  # Stack names to manage (must contain at least one name)
  names:
    - app
  # Default Docker network
  network: myproject_default
  # Reserved for future use; currently does not affect generated output
  networkDriver: overlay

render:
  # Output directory for rendered YAML
  outputDirectory: ./.rendered

secrets:
  # One encrypted dotenv filename, or multiple filenames to discover
  encryptedFileName: [".env.enc", ".env.production.enc"]

Converting Environment Variables

Old Environment Variable New Config Field Example
COMPOSE_DIR No direct equivalent Repository-root scan
RENDER_DIR render.outputDirectory ./.rendered
STACKS_DIR No equivalent (generated to stacks/) N/A
STACK_PREFIX project mystack
STACKCTL_PROFILE --profile flag or STACKCTL_PROFILE env dev

stack.directory is not the source directory for generation. The generate command recursively discovers exact docker-compose.yml and docker-compose.yaml files from the repository root and writes generated files to <repoRoot>/stacks by default, unless --output-dir is provided. The configured stack.directory is used by doctor and sync when locating generated stack files.

The secrets.encryptedFileName field accepts either one encrypted dotenv filename or a list of filenames. When a list is configured, secrets commands discover all matching files recursively, subject to the usual skipped directories. When no .stackctl config file exists, encrypt, decrypt, clean, and check fall back to the default [".env.enc"] list. secrets deploy requires a valid .stackctl config. When secrets encrypt is run without explicit file arguments, it derives plaintext filenames by stripping .enc from each configured encrypted name, walks for those plaintext files, and encrypts only the ones without an existing encrypted counterpart.

Env status, env audit, and doctor encrypted-file checks currently support only .env.enc and do not discover multiple configured encrypted filenames. This is a known limitation planned for a future release.

Generation validates service network references against the final top-level declarations. A service referencing an undeclared network produces an error and the stack is not generated or deployed. A service declaring both network_mode and explicit networks produces an error because Docker Compose considers these mutually exclusive. Services using network_mode without explicit networks are allowed and skip network validation.

Command Parity

These commands have reached functional parity with the old stackctl.sh script:

Old (./stackctl.sh) New (stackctl) Notes
./stackctl.sh up stackctl up Replaces shell-based deploy
./stackctl.sh down stackctl down —
./stackctl.sh status stackctl status Now with --json output
./stackctl.sh logs stackctl logs Improved streaming
./stackctl.sh reload stackctl reload Full config-aware pipeline
./stackctl.sh doctor stackctl doctor More comprehensive checks

New Capabilities (No stackctl.sh Equivalent)

The standalone binary adds capabilities that were previously handled by separate Python scripts or not available at all:

Command Purpose
stackctl generate Explicit stack regeneration
stackctl render Explicit environment interpolation
stackctl secrets SOPS/age integration
stackctl env .env scaffolding
stackctl plan Inspect operations without executing
stackctl init Config file generation
stackctl sync Full pipeline (generate → render → deploy)

Step-by-Step Migration

Step 1: Export Current Configuration

Record your current stackctl.sh environment:

echo "COMPOSE_DIR=${COMPOSE_DIR:-./docker-compose}"
echo "RENDER_DIR=${RENDER_DIR:-./.rendered}"
echo "STACK_PREFIX=${STACK_PREFIX}"
echo "STACKCTL_PROFILE=${STACKCTL_PROFILE:-dev}"

Step 2: Run stackctl init

# Detect repository layout (scans for compose files)
stackctl init --detect

# Or with explicit values
stackctl init --preset standard

This creates .stackctl in your project root. Edit it to match your recorded configuration from Step 1.

Step 3: Verify Configuration

stackctl doctor

Fixes any issues reported:

  • Missing Docker or Swarm mode
  • Invalid or missing .stackctl config
  • Missing override files
  • Missing stack directories

The doctor --fix-volumes option is exposed but not yet implemented. It reports this status and does not create missing external volumes.

Step 4: Dry-Run a Deployment

# See what would happen without making changes
stackctl sync --dry-run
stackctl up --dry-run

Review the output carefully. The pipeline is:

Config → Discover → Generate → Override → Render → Deploy

Step 5: Deploy

# Deploy all stacks
stackctl sync

# Or deploy incrementally
stackctl up my-stack-name

Step 6: Verify

stackctl status
stackctl logs my-service

Profile Handling

Before

STACKCTL_PROFILE=prod ./stackctl.sh up

After

Profiles use separate config overlays:

# Using flag
stackctl up --profile prod

# Using environment variable
STACKCTL_PROFILE=prod stackctl up

Profile overlays are loaded in this order (later wins):

  1. Built-in defaults
  2. .stackctl (base)
  3. .stackctl.<profile> (e.g., .stackctl.prod)
  4. .stackctl.local (local overrides, gitignored)
  5. .stackctl.local.<profile> (local profile overrides)

Override File Support

stackctl supports explicit override files in addition to profile overlays. Override files use Docker Compose override semantics:

  • Scalars: replaced
  • Maps: deep-merged
  • Sequences: appended
stackctl up --override ./overrides/production.yml --override ./overrides/region-eu.yml

Override files are applied after profile merging but before render.

Compose Metadata (x-stack)

stackctl uses the x-stack compose metadata key to group docker-compose files into named stacks during discovery and generation. Every compose file must declare which stack it belongs to.

Discovery recursively scans the repository root only for files named exactly docker-compose.yml or docker-compose.yaml. Files named compose.yml, compose.yaml, or other variants such as docker-compose.dev.yml are not discovered by generation. Matching files must also declare x-stack metadata.

Supported Forms

Two forms are supported:

Scalar (legacy) -- a plain stack name:

services:
  api:
    image: myapp/api
x-stack: api

Object (v1) -- a map with a name field:

services:
  api:
    image: myapp/api
x-stack:
  name: api

Both forms are equivalent and normalize to the same stack name. Scalar and object forms can coexist within the same stack group; they are merged by normalized name.

Object Form Constraints

The object form currently accepts the name field only. Adding unknown fields will cause an error:

# Invalid -- "labels" is not a recognized field
x-stack:
  name: api
  labels: [production]

During discovery, files with invalid x-stack metadata are skipped for grouping and reported as errors. During explicit loading (e.g. stackctl generate), invalid metadata causes the command to fail with a descriptive message.

The x-stack key is source-only metadata. It is stripped from generated and rendered stack output and never appears in deployed compose files.

Rollback

Rollback a Deployment

# Remove a specific stack
stackctl down my-stack-name

# Re-deploy previous version
docker stack deploy --compose-file .rendered/my-stack-name.rendered.yml my-stack-name

Rollback stackctl Binary

# Homebrew
brew switch stackctl <previous-version>

# Manual
cp /usr/local/bin/stackctl /usr/local/bin/stackctl.new
# ... download previous version
mv stackctl.previous /usr/local/bin/stackctl

Revert to stackctl.sh

The old stackctl.sh remains in your repository and is unaffected by stackctl installation. To revert:

  1. Uninstall stackctl: brew uninstall stackctl
  2. Delete .stackctl config: rm .stackctl
  3. Continue using ./stackctl.sh as before

Generated files (stacks/*.yml, .rendered/*.yml) are compatible between both tools for the same configuration.

Troubleshooting

Docker Not Running

✗ Docker is not running or not accessible

Ensure Docker is running and your user has access:

docker info

Swarm Mode Not Active

✗ Docker Swarm mode is not active

Initialize Swarm mode:

docker swarm init

Stack Not Found

✗ Stack "myapp" not found in /path/to/project

Check that your compose files use the exact names docker-compose.yml or docker-compose.yaml, declare x-stack metadata, and are located under the repository root. stack.directory is used by doctor and sync for generated stack files, not as the generation source directory. See Compose Metadata (x-stack) for supported forms.

# docker-compose.yml -- either form works:
services:
  api:
    image: myapp/api
x-stack: myapp              # scalar form
# or:
# x-stack:
#   name: myapp             # object form

Config Validation Errors

stackctl validates configuration at startup. Run stackctl doctor for a complete diagnostic. Common issues:

  • Missing project: Set the project name in .stackctl
  • Missing stack.network: Set the Docker network name
  • Empty stack.names: Provide at least one stack name; an empty list is rejected
  • Invalid render.outputDirectory: Must be a valid path

Unresolved Environment Variables

In strict mode (stackctl render --strict), unresolved variables cause failure. Strictness is a CLI option, not a config field. Run without --strict for non-strict mode or provide the variables:

# Provide variable
export MY_VAR=value
stackctl up

Permission Issues

stackctl requires these permissions:

  • --allow-read — read compose files, config, env files
  • --allow-write — write generated/rendered stacks
  • --allow-env — read environment variables
  • --allow-run — execute Docker, sops, age
  • --allow-sys — system info for doctor

When installed via Homebrew, permissions are pre-configured.

Behavior Differences

Generated Stack Paths

  • Old: Relative paths in generated stacks reference the repo root
  • New: Paths are absolutized to the project root during rendering

This means .rendered/*.yml files are self-contained and can be used independently of the working directory.

⚠️ Generated files are not safe to deploy raw. Stack files in stacks/ (generated) and .rendered/ (rendered) contain ${VAR} placeholders that must be resolved through the render pipeline before deployment. Deploying a generated stack file directly without running stackctl render or stackctl sync will result in unresolved environment variables in your running services.

Deterministic Output

stackctl produces deterministic YAML output:

  • Keys are sorted alphabetically
  • Stack files are ordered by stack name
  • Runs produce identical output for identical input

This enables drift detection in CI.

Error Reporting

  • Old: First error stops the pipeline
  • New: All errors are collected and reported at once
  • Exit codes: 0=success, 1=validation/drift failure, 2=config error, 3=missing dependency, 4=unexpected error

Signal Handling

  • Old: Ctrl-C may leave processes running
  • New: Commands that stream child process output (e.g. up --logs, logs) forward SIGINT to the child. Most commands use non-streaming execution and do not forward signals. Secrets cleanup is not automatic after an interrupted deployment; run stackctl secrets clean when needed.

Using stackctl in GitHub Actions

⚠️ The GitHub Actions integration is under active development and its location may change before the first stable release.

Add the setup-stackctl composite action to your workflow to install the stackctl binary on any GitHub Actions runner (Linux x64/arm64, macOS x64/arm64):

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup stackctl
        uses: AniTrend/stackctl/.github/actions/setup-stackctl@v1
        with:
          version: latest # or a specific version like "0.1.0"

      - name: Verify installation
        run: stackctl --version

      - name: Run stackctl sync
        run: stackctl sync

The action selects the correct tarball (stackctl-v<version>-<target-triple>.tar.gz) for the runner's OS and architecture, verifies the SHA256 checksum, and adds the binary to PATH for all subsequent steps.