Skip to content

Latest commit

 

History

History
236 lines (183 loc) · 20.3 KB

reference.md

File metadata and controls

236 lines (183 loc) · 20.3 KB

Maestro reference

Details that the README leaves out to stay short.

Why Maestro instead of…

Tool What it is Why Maestro instead
Claude Code Dynamic Workflows The model generates the orchestration on the fly — different every run Maestro's flow is human-authored and identical every run: reviewable, diffable, ownable by a lead
Conductor / Temporal Server-side durable orchestration engines Maestro runs locally in your editor — no runtime, no infra, no ops
CrewAI / AutoGen / Swarm Agent frameworks that need their own model credentials Maestro uses your existing host + subscription; it holds no keys and calls no APIs itself

Maestro isn't trying to replace a durable production orchestrator or an autonomous agent. It owns the desk-side layer: the reviewed, repeatable flow a developer runs while building.

How the engine works

       you: /maestro or $maestro my-feature
                     │
       ┌─────────────▼──────────────┐    .maestro/engine/maestroctl.py (stdlib python3)
       │  LEAD AGENT (your session) │───► next → ONE action as JSON
       │  dispatches, never decides │◄─── complete / gate-record / input-record / fail
       └──┬─────────┬──────────┬────┘
          ▼         ▼          ▼
      subagents   scripts    you (gates + interview)
      (skill +    (validators,
       model per   stubs)
       step)
  • workflow.yaml — the graph: 6 node types (agent, interview, gate, script, parallel, subworkflow), per-node routes with tiny conditions, and back-edges for loops (an arrow to any earlier step; the engine cascade-resets downstream work and enforces a per-node visit cap so loops can't run away). Spec: docs/workflow-spec.md.
  • .maestro/runs/<slug>/state.yaml — the run ledger. Only the engine writes it. Resume, revise-cascades, gate history, parallel-join bookkeeping all live here.
  • The lead agent never interprets the graph. The deterministic resolver serves one fully-rendered action at a time; the LLM just dispatches it. That's what makes an LLM-driven orchestrator reliable — and it's all plain, tested Python (see CONTRIBUTING.md; no LLM in the loop).
  • Agent steps are instruction-first: write what the step should do; optionally pin a skill and model. Worker execution is the default; execution: lead keeps small context-continuous steps in the current session. Claude Code and Codex run independent worker steps with native subagents; Cursor uses the same engine inline and sequentially.

Install details

Installs: our skills/commands/agents into .claude/ or .cursor/, or repo-scoped Codex skills into .agents/skills/; the engine + workflows/ + ui/ + docs/ all live under a single .maestro/ parent. The engine is stdlib-only python3 — no execution CLI, config file, or additional package to install. When it finishes it prints host-specific next steps and points at ./maestro help. Upgrade by re-running the same command with --ref <tag|sha|branch> (without it, the version recorded in .maestro/install.json is reinstalled, so installs are reproducible); it overwrites .maestro/engine//.maestro/ui/ and never overwrites a workflow or skill you edited — see Upgrading in detail). (Private-repo fork? The piped curl can't authenticate — clone it and run ./install.sh from the checkout.)

Maestro init

The one-liner above installs the core pack. To finish setting the repo up, open your host and run /maestro-init (Claude Code/Cursor) or $maestro-init (Codex) once. It:

  1. detects your stack (detect-stack) and re-runs the installer with the matching --stack …, so only the core pack + the per-stack skills your code actually uses get installed;
  2. builds the living-docs knowledge base (build-knowledge) — docs/architecture.md plus per-domain functional (business rules) and technical (invariants, change scenarios) docs under docs/, with endpoints, events, jobs, tables and config generated from the code by living_docs.py, plus one codebase map per repo at docs/codebase-maps/<name>.md (the standing grounding each HLD reads before choosing an architecture), which the design and review steps read as context.

The codebase map is commit-tracked and incremental: the engine (codebase_scan.py) records, in each map's marker, the commit of the repo it reflects, so re-running (and the post-feature refresh in the archive phase) only re-examines the diff since that commit — not the whole repo. The maps are committed with the rest of docs/.

/maestro-init                 # detect stack → install matching packs → build docs
$maestro-init                 # the same operation in Codex

It's idempotent — safe to re-run after a stack change (it refreshes, never duplicates). Newly installed skills become active once your host reloads its config; then start a feature with /maestro <slug> or $maestro <slug>.

Added a repository later? Run /maestro-sync (or $maestro-sync) instead of /maestro-init. It is only for a newly added repository: it adds that repo's maestro.config.yaml entry, detects its stack, fills its build/test commands, installs matching stack skills, and writes its docs with build-knowledge — its docs/codebase-maps/<name>.md, new or extended domain pages, and its node, connections and domains in docs/architecture.md. Existing entries are never changed or removed, and other repos' docs are only touched where a shared page gains the new repo; a repo already out of date stays flagged for /build-knowledge. ./maestro sync adds just the config entry, without a model (--dry-run to preview). Details: project-config.md.

Prefer to pick stacks by hand? The pack ships per-stack reference skills + reviewers (Go, Java/Spring, Kotlin, Python/Django, React, Vue, Angular, Node, Rust, Flutter, Android, DB, Terraform, Kubernetes/Helm, Docker…); install only what your repo uses instead of running /maestro-init:

./maestro install codex --stack go,react,db   # core + those stacks only

Anything without a stack: tag is core and always installs; --stack all (or no flag) installs every stack.

Prerequisites in detail

  • Recommended: an umbrella workspace. For the best SDLC output, don't run Maestro inside one isolated service repo — stand up a parent/umbrella repo per project, clone the service repos you're changing (gitignored) underneath it, and install Maestro into that parent. The lead agent then sees the whole stack at once. Full setup: docs/umbrella-workspace.md. A single repo works too; the umbrella just gives better cross-repo results.
  • python3 ≥ 3.8 (stdlib only — nothing to pip install). Check: python3 --version.
  • git and, to install, bash + curl. Every repository selected for implementation must have at least one commit — each repo's implementation runs in its own isolated git worktree, which an uninitialized repo cannot provide (git init && git add -A && git commit once in each selected repo if needed).
  • An AI coding harness that supports skills — Claude Code (parallel subagents and Claude model aliases), Codex (native parallel subagents; invoke repo skills with $), or Cursor (inline sequential fallback).
  • Optional: node/npx, only for auto-installing the six external Superpowers helper skills. Without it the flow still runs (skills fall back to inline behavior).
  • OS: macOS and Linux are first-class. On Windows use WSL — the installer is bash and the engine's file lock uses fcntl (a msvcrt fallback exists, but WSL is the tested path).

The visual builder

./maestro ui serves the builder on a local port (default 8422, --port to change). Because it knows your repo, the Runs view and Open/Save work over HTTP in any browser — or open .maestro/ui/builder.html directly and it runs fully offline from file:// (the folder view needs Chrome/Edge).

  • Author visually — New blank canvas, Open any workflow in the repo, or Paste YAML. Drag nodes and arrows; an arrow pointing back is a loop (dashed, with its repeat-limit badge). Nodes are instruction-first (skill defaults to Auto), gates' options are their edges, and parallel branches drill in. Live validation, and Save (⌘S) exports YAML the engine accepts.
  • Runs (top-right) — a read-only live status board: pick a slug and the graph colours in as it progresses (green done, orange running, red failed, grey pending).

The UI server answers only requests addressed to 127.0.0.1 or localhost on its port, and saves only from the builder page it served (a per-session token), so another web page you have open cannot read your repository or write a workflow file through it.

./maestro ui, ./maestro install, ./maestro init and ./maestro sync are dev tooling only — the wrapper never drives a run or writes .maestro/**/state.yaml (that stays the Maestro skill + engine's job).

Engine commands

You almost never call the engine yourself. The lead agent issues its verbs — init, next, complete, gate-record, gate-input-record, fail, runs, … — for you as it drives the graph. Only two are worth running by hand, for inspection:

python3 .maestro/engine/maestroctl.py status --slug my-feature              # step table, gates, active steps
python3 .maestro/engine/maestroctl.py validate .maestro/workflows/my.yaml   # lint a workflow before running it
python3 .maestro/engine/maestroctl.py preflight                             # check maestro.config.yaml (docs/project-config.md)

Run modes. init --mode human|design|auto (the lead passes it when you ask for a mode) decides which gates the engine answers itself from each gate's auto: policy — see Run modes. status shows the mode and marks autopilot decisions.

Debug profile. init --debug (on a new or a resumed run) records every visit of every step and keeps .maestro/runs/<slug>/profile.md and profile.json current: per stage and step, the visits, work time (agents, scripts), human time (gates, interviews), queued time (ready but not yet picked up) and idle stretches when nothing ran and nobody was asked. profile --slug <slug> prints it. Off by default: a normal run keeps only each step's last visit.

If a run gets stuck, reset --slug <slug> --step <id> --cascade and rebase --slug <slug> are your recovery levers — see Working as a team.

Pausing & resuming a run

A run is durable and does not have to finish in one turn. All progress lives in the engine ledger (.maestro/runs/<slug>/state.yaml), so you can close the session anytime and pick up later — resume with /maestro <slug> or $maestro <slug> and it continues from the exact step, re-running nothing.

Sometimes the lead agent ends its turn mid-run — after a long step, or because the harness ended the turn. That's normal and loses nothing; just resume. Whether the loop auto-continues past such a pause depends on your harness:

  • Claude Code — auto-continue with an opt-in Stop hook. Maestro ships .maestro/engine/stop_hook.py: while a run still has autonomous work pending it keeps the loop going, and it steps aside (lets the turn end) at every human gate and when the run finishes — the human stays in charge at gates. It's read-only and off by default. Enable it by adding to .claude/settings.json:

    {
      "hooks": {
        "Stop": [
          { "hooks": [ { "type": "command", "command": "python3 .maestro/engine/stop_hook.py" } ] }
        ]
      }
    }
  • Cursor — resume manually. Cursor has no hook system, so the loop can't auto-continue; when it pauses, just re-run the /maestro <slug> command. Tip: run phase-by-phase (/maestro <slug> .maestro/workflows/design.yaml, then implementation) to keep each session short — a long single-session run is the most common cause of a mid-run stop.

  • Codex — resume from the durable ledger. Native subagents can run independent Maestro actions in parallel. If the lead turn pauses or you close the task, invoke $maestro <slug> again; the engine serves the exact pending action. If Codex subagents are disabled in your configuration, Maestro automatically uses the serial inline path.

Working as a team

Maestro is a shared, paved road, not a per-developer toy: keep the flows in a central, PR-reviewed repo owned by your leads/platform team (changes go through review, same as code), and governance falls out of the graph — route the "approve for release" gate to a lead while anyone runs the steps up to it, no extra RBAC layer needed.

The run ledger .maestro/runs/<slug>/state.yaml is written only by the engine and is git-tracked on purpose (so a run resumes on any machine). Team ownership follows ledger ownership:

  • One owner per slug at a time. One developer drives a feature's run; its ledger, documents and every repository's LLD live under .maestro/runs/<slug>/. A team that wants people working in parallel can copy the workflow and add parallel branches.
  • Resolving a state conflict: never hand-merge state.yaml. The engine refuses to run on a ledger with conflict markers and never overwrites it. Take one side, then run python3 .maestro/engine/maestroctl.py status --slug <slug> to see where it stands and continue, or reset --slug <slug> --step <id> --cascade to redo from a known-good step.
  • If you edit a workflow file mid-run, the engine halts on the next command with a hash mismatch — nobody silently diverges from the lead's flow. Accept the edit with maestroctl rebase --slug <slug> (it re-validates first) or start over with reset --slug <slug> --all.

Upgrading in detail

  • Upgrade: re-run the install one-liner (add --dry-run to preview). It overwrites .maestro/engine/, .maestro/ui/ and the installed skills; docs/provenance.yaml provenance and .maestro/runs/ are left alone. Workflows upgrade file by file, never over a local edit:

    Your .maestro/workflows/<name>.yaml Upgrade does
    missing adds it
    byte-identical to a stock copy (the last install or any past release) replaces it
    edited, upstream changed it keeps yours, writes upstream's as <name>.yaml.upstream
    edited, upstream unchanged keeps yours
    your own file name (e.g. my-sdlc.yaml) never touched

    Matching is strict: any change — even a comment, re-formatting or a re-save from the builder — counts as an edit, so only untouched stock copies are ever replaced (no backups needed). Reconcile each .upstream into your file — or mv it over yours to take upstream, after which the file auto-updates again — then delete it. Commit .maestro/workflow-manifest.json (the record of what the last upgrade offered). Every workflow is validated afterwards and failures are listed. Customising a copy under your own name avoids .upstream reconciliation entirely. A workflow that changes still halts an in-flight run until maestroctl rebase --slug <slug>. On the next /maestro resume, an old run is detected and upgraded once only after your confirmation.

  • Uninstall: delete the installed dirs (.claude/skills|commands|agents, .cursor/…, .agents/skills/, .maestro/engine/, .maestro/ui/). Keep .maestro/runs/<slug>/ — that's your work.

  • "workflow changed" halt: see Working as a team above (rebase or reset).

  • A step won't complete (exit 4): the engine refuses to advance without the declared artifact (non-empty) and output fields — re-run the step or maestroctl fail it.

  • status any time: python3 .maestro/engine/maestroctl.py status --slug <slug> prints the step table, visit counts, and gate history.

Customizing

The engine is generic; the SDLC pack is just one workflow. To make it yours:

  • Write your own workflow — a YAML file with nodes: (agent / interview / gate / script / parallel / subworkflow), routes, and back-edges for loops. The builder writes it for you. Spec: docs/workflow-spec.md. Run it with /maestro <slug> path/to/your.yaml or $maestro <slug> path/to/your.yaml.
  • Bring your own skills — an agent node names a skill; that can be one of ours, one you author (.claude/skills/<name>/SKILL.md or .agents/skills/<name>/SKILL.md), or a third-party pack (Obra, Superpowers, …). Because the workflow node supplies the instruction, inputs, artifact path and output fields at runtime, a skill only has to describe how to do its job — so swapping one for another is a one-line skill: change, or omit skill: entirely and let the harness auto-pick from installed skills by description.
  • Change a shipped step's behaviour — edit its installed skill; the flow is untouched.
  • Models — per node or per workflow (defaults.model); values are host-specific and passed only when the host accepts the exact identifier. haiku / sonnet / opus work in Claude Code; use valid Codex model IDs in a Codex-specific workflow rather than relying on guessed alias translation.
  • Loop bounds — per node max_visits (+ on_exhausted), backstopped by defaults.max_visits (default 10).
  • Delivery — after release approval, delivery.mode in maestro.config.yaml decides: branch (default) reports each verified branch and commit; pull-request pushes them and opens a pull request per repository with gh (a draft when autopilot approved the release). Maestro never merges. Implementation and QA do not fake-merge independent repos: the engine verifies and passes each repo's exact worktree, branch, and commit to QA.
  • Memory (improves over time). Run /build-knowledge or $build-knowledge once per workspace to seed .maestro/memory/knowledge/ from your codebase; the design and review steps read it, frozen at init. After a feature, the pre-merge archival phase (once you confirm the archive) distills lessons and consolidates them — a lesson becomes trusted (and injected into future runs) only once ≥3 runs corroborate it. See docs/memory.md.

File layout

After install, in your project repo:

.claude/  (or .cursor/ — set by the installer for those hosts)
  skills/     one SKILL.md per SDLC step + maestro (the lead agent) + bootstrap
              (detect-stack, maestro-init, maestro-sync, build-knowledge, retrospect)
              + per-stack reference packs tagged stack:<x>
  agents/     core roles (product, architect, implementer, reviewer, qa, analyst, general,
              discussant; planner = the architect's older name)
  commands/   /maestro, /maestro-init, /maestro-sync and /maestro-discuss slash-command shims (steps are invoked as skills)
.agents/skills/  Codex repo-scoped copy of the same flattened skill pack; invoke with `$`
.maestro/
  engine/     the deterministic engine (validate · init · next · complete · gate-record
              · interview-record · interview-record-batch · interview-brief · gate-input-record · fail · reset · rebase · status · todo · graph · note · runs)
              + ui_server.py + schemas
              + stop_hook.py (opt-in Claude Code auto-continue hook — see "Pausing & resuming")
              + codebase_scan.py (per-repo codebase-map commit tracking for incremental refresh)
  workflows/  the example pack: sdlc-main / workspace-sync / design / impl / qa,
              split into small stages: sdlc-main's delivery,
              design's design-intake / -prd / -hld / -contract / -lld,
              plus archive / build-knowledge / retrospect — customize or replace
  ui/         builder.html (single-file visual editor)
  docs/       workflow-spec.md (the workflow spec)
  memory/     knowledge/ (seeded by /build-knowledge)
  index/      sync plans / locks (living-doc provenance lives in docs/provenance.yaml)
  runs/<slug>/  everything for one feature: requirement/ + all artifacts + state.yaml
docs/
  provenance.yaml  shared living-doc provenance (commits, branches, docs hash)
maestro       repo-local dev wrapper: `maestro ui` (serve the builder) + `maestro install` + `maestro help`