Skip to content

fix(pod): refuse a runtime handler the venue cannot launch - #49

Merged
github-actions[bot] merged 1 commit into
mainfrom
fix/venue-runtime-handler
Nov 5, 2026
Merged

github-actions[bot] merged 1 commit into
mainfrom
fix/venue-runtime-handler

Conversation

@aitrawog

@aitrawog aitrawog commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Summary

charly config for a pod: deploy on engine: containerd emitted a systemd unit whose
ExecStart named a runtime handler chosen from the CAPABILITY ROW alone
(engineDefaultRuntime → RuntimeHandlers[0] → crun) and never asked the VENUE whether it
can launch it. On a venue whose containerd has no crun, the daemon rejects the unit, systemd
restart-loops, the failed attempt leaves the container CREATED with its name held, and every
retry dies name-store error: name ... already used (opencharly/plugin-containerd#16).

This change consults the venue:

  • venueRuntimeHandlers() (candy/plugin-deploy-pod/engine_ops.go) reports the subset of the
    engine's DECLARED handlers this venue can launch, asking the SAME resolution the launch
    performs: exec.LookPath over the runtime executable, mirroring nerdctl's
    generateRuntimeCOpts (a bare word is the OCI-runtime binary) and containerd's
    pkg/shim.BinaryName (a runtime type is containerd-shim-<second-last>-<last>).
  • ensureVenueRuntimeHandler() refuses the config/start step when the selected handler is not
    in that set, naming (a) the handler charly selected, (b) the handlers the venue reports, and
    (c) the remedy: install the runtime on the venue (layer-containerd installs crun and krun)
    or point the deploy at a venue that has it. It refuses instead of falling back to another
    handler — a silent switch to a different OCI runtime is a different substrate (R3).
  • It runs at runPodConfigSetup (before any file is written or image work runs), at
    runConfigDirect's launch, and at resolvePodStartDirect (start). An engine with an empty
    runtime_handlers list (podman, docker) takes exactly today's path: no probe, no
    --runtime, no new failure mode.
  • A failed start now cleans up: podContainerStart stops the unit (so Restart=always cannot
    re-create the container) and removes the partial container through the SAME teardown argv
    charly remove uses (engineRemoveArgv); runConfigDirect removes it on a failed launch
    too. resolvePodStartPlan now returns the selection it built the plan from, so the cleanup
    addresses the same address/namespace the launch used.

Measured, not assumed: on this host (rootless containerd 2.3.5, nerdctl 2.4.1) nerdctl info
carries no Runtimes map in either mode; the resolution nerdctl actually performs for a bare
--runtime word is exec.LookPath, which is what the probe mirrors.

The second copy of the default (plugin-containerd/candy/plugin-containerd/plugin.go,
defaultRuntimeHandler = "crun") is not on this code path and is not changed here; it is
tracked by the same issue #16, which now carries this PR's evidence.

Refs opencharly/plugin-containerd#16 — the issue lives in another repository and is NOT closed
by this PR; this PR fixes the pod substrate's consumer side of it.

How tested

Go gate and structural validate on the committed head (038d0323b553c3568a342966c162740ba2ae33c0,
tree 7079076c80ccb952f4833d43e8553c09d11d6a26):

# head 038d0323b553c3568a342966c162740ba2ae33c0
$ gofmt -l .
rc=0
$ go build ./...
rc=0
$ go vet ./...
rc=0
$ go test -race -count=1 ./...
ok  	github.com/opencharly/plugin-deploy-pod/candy/plugin-deploy-pod	3.276s
?   	github.com/opencharly/plugin-deploy-pod/candy/plugin-deploy-pod/cmd/serve	[no test files]
rc=0
# head 038d0323b553c3568a342966c162740ba2ae33c0
$ charly box validate
WARNING: charly is a dev/worktree build (not installed at /usr/bin or /usr/local/bin); NOT loading the installed package's plugins from /usr/lib/charly/plugins (they may be built against a different charly version). Set CHARLY_PLUGIN_DIR to load plugins from a custom location.
charly box validate: OK — checked 1 candy, 3 deploys, 7 distros, 7 builders; 0 warnings, 0 errors
rc=0

The venue fact the fix is built on, measured live:

$ command -v crun
rc=1
$ nerdctl --address /run/containerd/containerd.sock --namespace default run --rm --pull never --runtime crun --entrypoint /bin/true alpine:latest
time="2026-11-05T22:17:13+01:00" level=fatal msg="failed to start shim: failed to resolve runtime path: invalid runtime name crun, correct runtime name should be either format like `io.containerd.runc.v2` or a full path to the binary"
rc=1
$ nerdctl --address /run/containerd/containerd.sock --namespace default run --rm --pull never --runtime runc --entrypoint /bin/true alpine:latest
rc=0

The changed path executed LIVE on this venue — the config step refuses before ensure-image
is reached, naming the selected handler, the observed set and the remedy:

# head 038d0323b553c3568a342966c162740ba2ae33c0
$ CHARLY_RUN_ENGINE=containerd CHARLY_RUN_MODE=systemd-unit charly config pod-ensure-gate-present
WARNING: charly is a dev/worktree build (not installed at /usr/bin or /usr/local/bin); NOT loading the installed package's plugins from /usr/lib/charly/plugins (they may be built against a different charly version). Set CHARLY_PLUGIN_DIR to load plugins from a custom location.
plugin plugin-deploy-pod: served from /home/atrawog/.cache/charly/plugin-build/views/plugin-deploy-pod-ee67890b6502f95f (build sha256:9d35885b97ad824d72272ac235070147a9800a8e97e35340c4fa5850e4ef9f14); source /home/atrawog/Sync/Atrapub/mo/claude/opencharly/.worktrees/containerd-crabbox-program/rt16/plugin-deploy-pod/candy/plugin-deploy-pod
charly: error: command "config": deploy:pod config: InvokeProvider deploy:pod op=config-setup: rpc error: code = Unknown desc = engine "containerd": the selected runtime handler "crun" is not installed on this venue (the venue reports: runc); remedy: install a "crun" runtime handler on the venue (the layer-containerd candy installs crun and krun), or point the deploy at a venue that has it
rc=1

The unmodified base proceeds instead — it never consults the venue and reaches ensure-image,
where the bed's pinned image is denied by ghcr.io (see ## Filed issues):

# head 54c7c0c90aadffd64cc58a5b886bde31cd9f9e94 (origin/main, unmodified)
$ CHARLY_RUN_ENGINE=containerd CHARLY_RUN_MODE=systemd-unit charly config pod-ensure-gate-present
WARNING: charly is a dev/worktree build (not installed at /usr/bin or /usr/local/bin); NOT loading the installed package's plugins from /usr/lib/charly/plugins (they may be built against a different charly version). Set CHARLY_PLUGIN_DIR to load plugins from a custom location.
plugin plugin-deploy-pod: served from /home/atrawog/.cache/charly/plugin-build/views/plugin-deploy-pod-2956c3ebb9b8f839 (build sha256:8eeb729c22320d0a5bc73f082d58d664335ee77bbbdab9827ba985a4ae5ebacb); source /tmp/pdp-origin-main/candy/plugin-deploy-pod
Auto-detected devices: /dev/kvm, /dev/vhost-net, /dev/vhost-vsock, /dev/fuse, /dev/net/tun, /dev/hwrng
ensure-image: pulling ghcr.io/opencharly/check-firecracker-app:2026.287.1522
time="2026-11-05T22:16:39+01:00" level=fatal msg="failed to resolve image: failed to authorize: failed to fetch anonymous token: unexpected status from GET request to https://ghcr.io/token?scope=repository%3Aopencharly%2Fcheck-firecracker-app%3Apull&service=ghcr.io: 403 Forbidden"
ensure-image: pull ghcr.io/opencharly/check-firecracker-app:2026.287.1522 failed: exit status 1
charly: error: command "config": deploy:pod config: InvokeProvider deploy:pod op=config-setup: rpc error: code = Unknown desc = image not found in local storage: ghcr.io/opencharly/check-firecracker-app:2026.287.1522: ensure-image "ghcr.io/opencharly/check-firecracker-app:2026.287.1522": not present locally, pull failed, and no buildable short-name match in charly.yml — make the registry public, log in to the registry, or pre-build the image manually
rc=1

Negative control. TestRunPodConfigSetup_RefusesHandlerAbsentFromVenue fails on the
unmodified base and passes on this head. On the base I applied that single test plus its PATH
helper (the other new tests reference symbols that do not exist there) and ran it:

# head 54c7c0c90aadffd64cc58a5b886bde31cd9f9e94 (origin/main, unmodified) + the new test file applied
$ go test -run TestRunPodConfigSetup_RefusesHandlerAbsentFromVenue -v ./...
=== RUN   TestRunPodConfigSetup_RefusesHandlerAbsentFromVenue
warning: gpu probe detect-host-devices: no host reverse channel
    venue_gate_negative_control_test.go:65: charly config proceeded into the config walk on a venue without crun; it must refuse first (plugin-containerd#16)
--- FAIL: TestRunPodConfigSetup_RefusesHandlerAbsentFromVenue (0.00s)
FAIL
FAIL	github.com/opencharly/plugin-deploy-pod/candy/plugin-deploy-pod	0.012s
?   	github.com/opencharly/plugin-deploy-pod/candy/plugin-deploy-pod/cmd/serve	[no test files]
FAIL
rc=1

R10 bed — run, and it did NOT pass. charly check run check-pod-ensure-gate (the repo's own
disposable: true bed) ran on this head; its pe-config-path-completes step failed in
ensure-image because the bed's pinned image is anonymously denied by ghcr.io, and the SAME
failure reproduces on unmodified origin/main (evidence in ## Filed issues). The run's own
artifact says:

$ grep -E '^(bed|calver|status|ok|skipped_checks):' .check/check-pod-ensure-gate/2026.309.2114/summary.yml
bed: check-pod-ensure-gate
calver: 2026.309.2114
skipped_checks: 0
ok: false
status: FAIL
$ tail -n 3 .check/check-pod-ensure-gate/2026.309.2114/check-live.log
2 steps: 1 passed, 1 failed, 0 skipped
command "check": 1 check(s) failed
charly subcommand exited 2

No R10 pass is claimed: the bed failed in setup on an artifact this change does not touch, so
it judges nothing about this diff. The attribution tier is lowered accordingly.

Rulebook compliance

  • R0 — loaded before the first edit, by path: the umbrella AGENTS.md, this repo's
    AGENTS.md, /charly-internals:git-workflow, /charly-internals:strict-policy,
    /charly-internals:root-cause-analyzer, /charly-check:check.
  • R1 — the bed failure was RCA'd before any remediation: anonymous ghcr.io access to the
    bed's pinned image is denied (403 DENIED), reproduced on unmodified origin/main, filed
    as check-pod-ensure-gate cannot run: its pinned 'public' ghcr.io image is anonymously denied (403) #48. Go gate and charly box validate report 0 warnings.
  • R2 — one probe/decision pair (venueRuntimeHandlers / ensureVenueRuntimeHandler) used
    at every config/start site; the cleanup reuses engineRemoveArgv and podContainerStop, no
    second teardown.
  • R3 — no sleep, retry, or silent fallback: a handler mismatch REFUSES instead of selecting
    another OCI runtime; no ad-hoc engine command was added to the product.
  • R4 — the product was fixed; no prose routes around the defect and no documented command
    changed.
  • R5 — nothing was deleted or renamed, and no legacy identifier is introduced; the change
    adds no transitional path.
  • R6 — git status clean before and after; work in
    .worktrees/containerd-crabbox-program/rt16/plugin-deploy-pod; no destructive git, no
    force-push, PR-only.
  • R7 — coverage fails without the change (the pasted base FAIL); the changed path executed
    live on the venue (the pasted refusal).
  • R8 — no service is mocked: the venue probe and the nerdctl runtime measurements are
    real.
  • R9 — the worktree-local binary was rebuilt with ./charly/scripts/bootstrap-charly.sh;
    charly version prints 2026.309.1009, equal to git -C charly describe --tags --abbrev=0
    (v2026.309.1009); no new runtime OS dependency.
  • R10 — no pass claimed: the disposable bed ran on this head and failed in setup on the
    externally denied image (reproduced on origin/main, filed check-pod-ensure-gate cannot run: its pinned 'public' ghcr.io image is anonymously denied (403) #48); the tier is lowered rather
    than inflated.

Change classification

  • Kind: runtime behavior fix in the pod deploy substrate (config emission, direct
    launch, start cleanup). No schema, flag, wire, or pin change; charly box validate still
    checks the same candy and deploys.
  • What a consumer sees: a containerd deploy whose venue cannot launch the selected handler
    now fails charly config/charly start with the selected handler, the venue's handlers and
    the remedy, instead of writing a unit that can never start; a venue that has the handler
    produces the same argv as before.
  • Breaking change + rollback: no schema or emitted-artifact shape changes; the one
    deliberate behavior change is the loud refusal on a venue that cannot launch the handler
    (previously a unit that restart-looped and poisoned the container name). Rollback: revert
    this commit — no state, schema, volume, or generated artifact migrates, and any unit already
    written stays exactly as it is.

Filed issues

Agent: containerd-crabbox-program/rt16 · session 8b1c01cf-7f79-4a75-bff8-240415a06208
Assisted-by: Claude Code deepseek-v4.1-flash:cloud (analysed on a live system)

The containerd capability row declares runtime_handlers [crun, krun, runc,
nvidia] and engineDefaultRuntime names the FIRST one unconditionally, so
every containerd launch argv (and the generated systemd unit's ExecStart)
carried `--runtime crun` — even on a venue that has no crun. There the unit
restart-loops, the failed attempt leaves the container CREATED with its name
held, and every retry dies `name-store error: name ... already used`
(opencharly/plugin-containerd#16).

Nothing consulted the VENUE. This change does: venueRuntimeHandlers() asks
the same resolution the launch performs (exec.LookPath over the runtime
executable, mirroring nerdctl's generateRuntimeCOpts and containerd's
pkg/shim.BinaryName), and ensureVenueRuntimeHandler() refuses a config/start
step whose selected handler the venue cannot launch — naming the selected
handler, the handlers the venue reports, and the remedy (install the crun
runtime; layer-containerd installs crun and krun). It never falls back to a
different handler: a silent switch to another OCI runtime is a different
substrate (R3). An engine with an empty runtime_handlers list (podman,
docker) takes exactly the path it took before.

A failed start now also cleans up after itself: the unit is stopped (so
Restart=always cannot re-create the container) and the partial container is
removed through the SAME teardown argv `charly remove` uses
(engineRemoveArgv), so a retry starts from a free name. The direct-mode
config launch applies the same removal on failure.

MEASURED on this host's rootless containerd 2.3.5 (nerdctl 2.4.1):
/usr/bin/crun is absent; `nerdctl run --runtime crun` fails `failed to
resolve runtime path: invalid runtime name crun`; `--runtime runc` succeeds.

The repo's R10 bed could not complete on this host: its pinned image is
anonymously denied by ghcr.io (filed as plugin-deploy-pod#48), reproduced on
unmodified origin/main.

Refs opencharly/plugin-containerd#16

Assisted-by: Claude Code deepseek-v4.1-flash:cloud (analysed on a live system)
@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

Review — PASS

Head SHA: 038d0323b553c3568a342966c162740ba2ae33c0 (branch: fix/venue-runtime-handler)

Security screen: PASS — no scope mismatch, secrets, weakened guardrails, new deps, or embedded directives; the change is confined to the pod-deploy candy and matches its stated intent.

Checklist: All applicable rules PASS or NA. The change ships unit coverage that fails without it (TestRunPodConfigSetup_RefusesHandlerAbsentFromVenue negative control, pasted base FAIL) and demonstrates the changed config path executing live (the pasted venue refusal), with the tier honestly lowered. The check-pod-ensure-gate bed red is a genuinely external image-pull 403 (reproduced on unmodified origin/main) and is discharged by a real filed issue (#48) per R1/CONC — not a forbidden-framing dodge. Filed issues #48 and #16 are present with descriptions; placement is correct (capability code in the candy, no core/sdk coupling, no alias-file growth); no schema/wire/dependency changes. Comment thread is empty, so no comment intake applies.

Verdict: PASS


ollama-cloud/deepseek-v4.1-flash — action-review.

View action run

@github-actions
github-actions Bot enabled auto-merge (squash) November 5, 2026 21:22
@github-actions
github-actions Bot merged commit 6e9e883 into main Nov 5, 2026
3 checks passed
charly-auto-merge Bot added a commit that referenced this pull request Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants