Skip to content

Reaper discovery adopts another language binding's Ryuk, then leaks every container (findReaperContainers matches only org.testcontainers.ryuk) #1442

Description

@markanthonyortega

Expected Behaviour

Reaper discovery should only adopt a Ryuk container that belongs to testcontainers-node, or — if it adopts one belonging to another language binding — it should register its session with that reaper so the containers it creates are still reaped.

Actual Behaviour

findReaperContainers matches any running Ryuk on the host, regardless of which language binding started it. On a CI host shared between a testcontainers-node project and a testcontainers-python project, the Node workers adopt the Python binding's Ryuk and then leak every container they create.

packages/testcontainers/src/reaper/reaper.ts (verified in the published build/reaper/reaper.js of both 11.14.0 and 12.1.0 — the code is identical in the two):

async function findReaperContainers(client) {
    const containers = await client.container.list();
    return containers
        .filter((container) => container.State === "running" &&
        container.Labels[LABEL_TESTCONTAINERS_RYUK] === "true" &&
        container.Labels["TESTCONTAINERS_RYUK_TEST_LABEL"] !== "true")
        .sort((a, b) => b.Created - a.Created);
}

The only positive predicate is the image label org.testcontainers.ryuk=true. That label is on the testcontainers/ryuk image itself, so every binding's reaper carries it — confirmed on two versions in use side by side:

$ docker image inspect testcontainers/ryuk:0.14.0 --format '{{json .Config.Labels}}'
{"org.testcontainers.ryuk":"true"}
$ docker image inspect testcontainers/ryuk:0.8.1  --format '{{json .Config.Labels}}'
{"org.testcontainers.ryuk":"true"}

There is no check on org.testcontainers.lang, on org.testcontainers.version, or on the image tag — even though this library writes org.testcontainers.lang: "node" on everything it creates (utils/labels.ts, createLabels()).

Why adoption then loses the session. A reaper started by another binding carries no org.testcontainers.session-id label, so this line mints a fresh id per worker:

const existingSessionId = reaperContainer.Labels[LABEL_TESTCONTAINERS_SESSION_ID] ?? new RandomUuid().nextUuid();

The adopted reaper is then asked to watch a session it was never told about in a form it durably owns, and the containers created under that id are never reaped.

Testcontainer Logs

From the foreign (Python-started) reaper's own log while Node workers were running against it. Its own session id is a full UUID; the 12-hex ids are the ones this library minted per worker:

Adding {"label":{"org.testcontainers.session-id=cf5e8a51-7464-4627-9ede-6c092b479713":true}}
Adding {"label":{"org.testcontainers.session-id=3297709e6c31":true}}
Adding {"label":{"org.testcontainers.session-id=57a593d04394":true}}

The observable signature is one leaked container per distinct session id, with no reaper alive that ever owned them. We saw six containers survive with six distinct session ids and no test runner process alive, then reproduced the id-per-worker pattern at 13/13 and 23/23.

Ruled out by measurement, so these are not the cause: TESTCONTAINERS_RYUK_DISABLED is unset everywhere (the reaper is running and doing its job for live sessions), and there was no OOM kill.

Steps to Reproduce

  1. On one Docker host, start a testcontainers-python suite so that its testcontainers/ryuk container is running.
  2. While it is up, run any testcontainers-node suite against the same host.
  3. Observe the Node run log Reusing existing Reaper for session "<id>" and that the adopted reaper is the Python one.
  4. Let the Node suite end. Its containers are not reaped.

Environment

  • testcontainers 11.14.0; the same code is present in 12.1.0 (latest at time of writing), so this is not fixed by upgrading.
  • Node 22.18.0, Linux, Docker with the containerd image store.
  • testcontainers/ryuk 0.14.0 (this library) and 0.8.1 (the other binding) both present.

Suggested fix

Narrow the discovery predicate so a reaper is only adopted when this binding can actually own the session. Any of:

  • require org.testcontainers.lang === "node" on the reaper container, matching what createLabels() already writes; or
  • require the reaper container to carry an org.testcontainers.session-id label at all, rather than falling back to a freshly minted id — adopting a reaper whose session cannot be identified is the step that silently breaks reaping; or
  • match on the expected REAPER_IMAGE tag.

Happy to open a PR if you would like one, and to test a candidate against the mixed-binding host where this reproduces.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions