Skip to content

test(testkit): use one pullable test image with the tools ported tests need #5352

Open

Description

@matthewgrossman

User Story

I'm moving e2e/rust tests onto testkit and running conformance against gateways other than the ones our lanes set up. I need one test image that any gateway can pull and that has the tools the ported tests use. Then no lane has to build or load a test image, and a gateway outside this repo can run conformance as is.

Description

Today sandbox tests use two different images:

  • openshell/e2e-python:dev, built from e2e/python/Dockerfile.workload. It has system Python, a uv Python and virtualenv, curl, git, openssl, requests, google-auth, cloudpickle, and a non-root sandbox user. It isn't published, so it exists only where a lane builds it (the Docker and Podman e2e/rust scripts, mise run e2e:workload:build) or loads it (the tmachine installers).
  • ghcr.io/astral-sh/uv:0.12.17-python3.12-trixie-slim@sha256:…, a public image pinned by digest. It has the Python standard library only: no curl, git or openssl, and no non-root user.

As of #5318, testkit::python_image() defaults to the public uv image, so conformance works against any gateway. OPENSHELL_TEST_PYTHON_IMAGE overrides it. That covers the current tests, which use only the standard library. Many e2e/rust tests still to be ported (#5281, #5297) use curl, git, openssl or requests, and would fail on the uv image.

Pick one image to replace both:

  1. A trusted public image with more tools. It should be maintained by a known publisher, pinned by digest, and include Python plus curl, git and openssl. That means nothing for us to publish or maintain.
  2. Our own test image, published. Publish e2e/python/Dockerfile.workload (or a slimmer version) to ghcr from CI, pinned by digest in testkit. We control its contents, at the cost of a publish step and keeping its dependencies up to date.

Context

  • tests/testkit/testkit/src/image.rs: python_image(), PYTHON_BINARIES.
  • The legacy harness has the same split: E2E_WORKLOAD_IMAGE in e2e/rust/src/harness/sandbox.rs.
  • tmachine builds and loads e2e-python:dev (tests/tmachine/artifacts.nix, config.nix, and the openshell_gateway role and openshell-k3s.yaml playbook). Those Ansible comments say the testkit uses it, which is no longer true. The Podman driver e2e playbook (drivers/podman/e2e.yaml) still uses it directly.
  • openshell/e2e-python:dev has no registry in its name, so a pull of it resolves to Docker Hub. Any default or override must name a registry and a digest.
  • The image must work on every driver without a named workload user in the test policy. Podman rejects a policy user the image doesn't have.

Definition of Done

  • One image is chosen, with the reasoning recorded here.
  • testkit::python_image() defaults to it, pinned by registry and digest. PYTHON_BINARIES covers its Python paths.
  • Conformance passes on every conformance row and host lane with no OPENSHELL_TEST_PYTHON_IMAGE set.
  • tmachine and the host lanes no longer build or load an image for testkit suites, or a remaining use is documented.
  • If we publish our own image, CI publishes it and something keeps its base and packages updated.
  • tests/testkit/README.md and TESTING.md describe the image and the override.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions