Skip to content

plan engine: the start phase (epic #14081, Lot 1) - #14156

Draft
ndeloof wants to merge 3 commits into
docker:mainfrom
ndeloof:plan-learns-to-start
Draft

plan engine: the start phase (epic #14081, Lot 1)#14156
ndeloof wants to merge 3 commits into
docker:mainfrom
ndeloof:plan-learns-to-start

Conversation

@ndeloof

@ndeloof ndeloof commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Lot 1 of #14081, built as three commits — each one CI-validated before the next lands on the branch (the PR stays draft until all three are green):

  1. feat: reconciler plans the start phase (this push) — the plan vocabulary: OpWaitCondition (deduplicated per awaited service+condition, best-effort when every dependent is optional), OpRunPreStart (plan-time gating: only when no replica was running at observation), OpRunPostStart, PlanNode.Phase, replica chains preserving today's sequential start order, and ReconcileOptions.Scope (zero value = today's create-only plans, byte-identical — pinned by the existing 42 golden tests). Golden tests only, no consumer.
  2. feat: executor runs start-phase operations — to follow once 1 is green.
  3. refactor: split create() into preparePlan + execute — to follow once 2 is green.

🤖 Generated with Claude Code

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟡 NEEDS ATTENTION

Comment thread pkg/compose/reconcile.go Outdated
@codecov

codecov Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 76.44628% with 57 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pkg/compose/executor_ops.go 49.01% 21 Missing and 5 partials ⚠️
pkg/compose/reconcile.go 86.70% 12 Missing and 9 partials ⚠️
pkg/compose/executor_events.go 66.66% 6 Missing ⚠️
pkg/compose/executor.go 42.85% 4 Missing ⚠️

📢 Thoughts on this report? Let us know!

@ndeloof
ndeloof force-pushed the plan-learns-to-start branch from 4bffd21 to 9275a7e Compare September 1, 2026 08:50
@ndeloof
ndeloof requested a review from docker-agent September 1, 2026 08:50

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🔴 CRITICAL

One high-severity bug was found in the start-phase planning code introduced by this PR.

Comment thread pkg/compose/reconcile.go Outdated
@ndeloof
ndeloof force-pushed the plan-learns-to-start branch from 9275a7e to 4a19d74 Compare September 1, 2026 09:37
@ndeloof
ndeloof requested a review from docker-agent September 1, 2026 09:37

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟢 APPROVE

The plan engine start phase implementation looks solid. The DAG-based approach to OpWaitCondition, OpRunPreStart, OpRunPostStart, and the replica chain ordering is well-structured. The zero-value ReconcileOptions.Scope preserves byte-identical behavior for existing create-only plans (confirmed by the 42 golden tests). No high or medium confidence bugs found in the changed code.

Lower-confidence findings (not posted inline)

  • [medium] pkg/compose/reconcile.go:1119 — pre_start node placed in replica-1's start group causes Starting event to fire during pre_start hook, not during actual container start (confidence: weak 52/100)

    The preStart node is assigned the same group name as replica-1's container start node (fmt.Sprintf("start%s", strings.TrimPrefix(first.resID, "service"))). In buildGroupTracker, onNodeStart fires Starting when the first node in a group starts (gs.started == 1). With pre_start in the same group as start and post_start, the Starting progress event fires when the pre_start hook begins execution — potentially seconds before the container actually starts. The doneText = "Started" comment in the new code says this is intentional ("Started is emitted after the hooks ran — word-for-word the imperative sequence"), but the Starting signal fires earlier than users might expect. Worth a conscious decision on whether the group boundary should begin at the hook or at the container start.

@ndeloof
ndeloof force-pushed the plan-learns-to-start branch from 4a19d74 to ba3c166 Compare September 2, 2026 06:44
@ndeloof
ndeloof requested a review from docker-agent September 2, 2026 06:44

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟢 APPROVE

The plan engine changes (OpWaitCondition, OpRunPreStart, OpRunPostStart, PlanNode.Phase, ReconcileOptions.Scope) are well-structured. The drafter identified one potential slice alias hazard in planServiceStart around the pre_start dep construction, but verification confirmed it is self-contained and harmless: the original prev slice is overwritten with a fresh single-element slice on line 1121 immediately after, so any backing-array mutation from the conditional append is never observable by subsequent code. The replica loop correctly uses slices.Clone(prev) on the updated slice. No real bugs introduced by this PR.

@ndeloof
ndeloof requested a review from docker-agent September 2, 2026 06:55

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review covers only the commits pushed since ba3c16642ed0fd78dba6dee06ce687523c02c857.

Assessment: 🟢 APPROVE

The refactor cleanly splits create() into preparePlan() and a thin caller. All error paths in preparePlan return nil, nil, nil, err consistently, the relative ordering of every step is preserved, and emitRunningEvents is correctly moved to the caller in create() — same position in the call sequence as before. No bugs introduced by this incremental change.

@ndeloof
ndeloof force-pushed the plan-learns-to-start branch from f6f8ecd to 794aff8 Compare September 2, 2026 07:03
@ndeloof
ndeloof requested a review from docker-agent September 2, 2026 07:03

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟡 NEEDS ATTENTION

One medium-severity finding: a latent nil pointer dereference in newly introduced code. All other drafter hypotheses (slice aliasing in pre-start deps, missing OpRunPreStart/OpRunPostStart in event switch statements) were verified to be false positives — either the code flow is safe by construction or the cases are unreachable.

Lower-confidence findings (not posted inline)

(none — all below-threshold findings were dismissed)

Comment thread pkg/compose/reconcile.go
The plan learns the start vocabulary — inert until a caller opts in
(ReconcileOptions.Scope, zero value keeps today's create-only plans
byte-identical):

- OpWaitCondition, one node per (awaited service, condition),
  deduplicated across dependents like networkNodes deduplicates
  networks; required:false marks the shared node best-effort, one
  required dependent upgrades it. service_started needs no node — a
  plain DAG edge to the dependency's chain end expresses it. Health is
  deliberately re-observed at execution time: the plan encodes what to
  wait for, never a stale observation.
- OpRunPreStart, emitted at plan time only when no replica was running
  at observation — the imperative gating — targeting the
  lowest-numbered replica.
- OpRunPostStart per container, after its start.
- replica chains: inject+start+post_start of replica n+1 depends on
  the end of replica n's chain, today's sequential start order made
  visible in golden plans; startChainEnds points at the chain end so a
  service_started dependent waits for the whole service, matching
  InDependencyOrder semantics.
- scope Start plans starting observed exited/created containers
  without converging them (the future compose start); scope
  CreateStart appends the start phase to the create plan, start nodes
  resolving their target from the create node that materializes the
  replica (CreateNodeID, the mechanism OpRenameContainer already
  uses).

Golden tests only; no executor support yet and no caller passes the
scope. Epic docker#14081, Lot 1 (1/3).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
The executor learns the start vocabulary planned by the previous
commit — still without any caller passing the scope:

- execStartContainer, enriched for start-phase nodes: secret and
  config injection folds in (they always run as a pair right before
  the start — part of starting a container, not a separate decision),
  the target resolves from the observed summary or from the result of
  the create node that materialized the replica (CreateNodeID), and
  the ContainerStart goes through the call site holding startMx.
  Create-phase starts (exceptional states) keep the bare historical
  behavior.
- execWaitCondition re-observes the depends_on condition at execution
  time by delegating to the imperative waitDependency primitive — no
  polling rewrite, and the Waiting/Healthy/Exited/Skipped events are
  word-for-word the vocabulary users see today. A best-effort node
  (every dependent optional) absorbs a missing dependency as a Skipped
  event.
- execRunPreStart/execRunPostStart delegate to the runPreStart/runHook
  primitives, with listener plumbing for hook output.
- start:<service>:<n> event groups: Starting on the chain's first
  node, Started once the whole chain completed — after post_start,
  exactly the imperative sequence (groupTracker texts are now derived
  from the group family instead of hardcoding recreate's).

Epic docker#14081, Lot 1 (2/3).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Pure extraction: preparePlan holds everything that precedes execution —
model preparation (images, models, networks, volumes, use_api_socket),
state observation, and reconciliation — and returns the canonical
project, the observed snapshot, and the plan. create() becomes
preparePlan → emitRunningEvents → executePlan.

No behavior change; this hands a future caller (up) the three values it
needs to plan create+start as one DAG and execute it once.

Epic docker#14081, Lot 1 (3/3).

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
@ndeloof
ndeloof force-pushed the plan-learns-to-start branch from 794aff8 to 57a0683 Compare September 2, 2026 14:11
@ndeloof
ndeloof requested a review from docker-agent September 2, 2026 14:12

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟡 NEEDS ATTENTION

This PR introduces the start-phase plan vocabulary and executor operations (OpWaitCondition, OpRunPreStart, OpRunPostStart) along with the ScopeCreateStart/ScopeStart reconcile scopes. The implementation is well-structured, with comprehensive golden tests confirming byte-identical output for the existing create-only scope. One medium-confidence issue was found in the new wait-condition error path.

exec.compose.events.On(skippedEvent("Service "+op.Name, "no container to wait for"))
return nil
}
return fmt.Errorf("required dependency %q has no container to wait for", op.Name)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[medium] execWaitCondition error path emits internal resource ID instead of user-friendly service name

When a required OpWaitCondition node fails, the error propagates to groups.onNodeErroremitErrorEvent, which uses op.ResourceID ("wait:db:service_healthy") as the progress-UI event ID. The best-effort skip path two lines above correctly uses "Service " + op.Name:

// best-effort skip: user sees "Service db: Skipped: no container to wait for"
exec.compose.events.On(skippedEvent("Service "+op.Name, "no container to wait for"))
return nil

// required fail: error propagates; emitErrorEvent fires with ID="wait:db:service_healthy"
return fmt.Errorf("required dependency %q has no container to wait for", op.Name)

Because OpWaitCondition nodes are always ungrouped (empty Group), onNodeError delegates to emitErrorEvent. The default branch there (id = op.ResourceID) produces "wait:db:service_healthy: required dependency "db" has no container to wait for" in the UI — an internal key, not a display name.

Fix: emit a named error event before returning, mirroring the skip path:

exec.compose.events.On(api.Resource{ID: "Service " + op.Name, Status: api.Error, Text: "no container to wait for"})
return fmt.Errorf("required dependency %q has no container to wait for", op.Name)

Or extend emitErrorEvent's switch with a case for op.Name != "" (when Container is nil but Name is set).

Confidence Score
🟡 moderate 75/100

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants