Skip to content

test: replace MinIO with RustFS as the S3-compatible object store - #1109

Merged
mnencia merged 11 commits into
mainfrom
dev/624
Sep 24, 2026
Merged

mnencia merged 11 commits into
mainfrom
dev/624

Conversation

@armru

@armru armru commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

The MinIO images are no longer maintained. Replace them with RustFS in the e2e harness, in hack/ and in the docs, as cloudnative-pg did in cloudnative-pg/cloudnative-pg#10865, and name the resources in provider-neutral terms (object-store, s3).

The e2e Deployment mirrors core: an init container makes the data and log subdirectories writable for the non-root RustFS user, no fsGroup, glibc image, probes on /health, containers run with dropped capabilities and non-root where possible. RustFS requires at least 3 characters for the access key and 8 for the secret; the e2e credentials meet that. hack/minio becomes hack/object-store, keeping the cert-manager TLS setup, and the mc pod becomes an AWS CLI Deployment. The docs replace the MinIO operator section with a RustFS one; versioned docs are untouched.

Verified with task e2e-ephemeral and task e2e-external-kind (all S3 specs green, the skipped ones are the pre-existing GCS specs), and by applying hack/object-store with cert-manager on kind: a plugin backup completed over TLS and the cleanup script emptied the bucket.

Closes #624

@armru
armru marked this pull request as ready for review September 21, 2026 12:13
@armru
armru requested a review from a team as a code owner September 21, 2026 12:13
The MinIO images are no longer maintained. Replace them with RustFS in the e2e harness, in hack/ and in the docs, as cloudnative-pg did in #10865, and name the resources in provider-neutral terms (object-store, s3).

The e2e Deployment mirrors core: an init container makes the data and log subdirectories writable for the non-root RustFS user, no fsGroup, glibc image, probes on /health. Credentials are longer because RustFS requires at least 3 characters for the access key and 8 for the secret. hack/minio becomes hack/object-store, keeping the cert-manager TLS setup, and the mc pod becomes an AWS CLI pod. The docs replace the MinIO operator section with a RustFS one; versioned docs are untouched.

Verified with task e2e-ephemeral and task e2e-external-kind (all S3 specs green, the skipped ones are the pre-existing GCS specs), and by applying hack/object-store with cert-manager on kind: a plugin backup completed over TLS and the cleanup script emptied the bucket.

Closes #624

Signed-off-by: Armando Ruocco <armando.ruocco@enterprisedb.com>
Signed-off-by: Niccolò Fei <niccolo.fei@enterprisedb.com>
Comment thread hack/object-store/object-store-deployment.yaml Outdated
Comment thread hack/object-store/object-store-deployment.yaml Outdated
Comment thread hack/object-store/s3-client.yaml Outdated
Signed-off-by: Niccolò Fei <niccolo.fei@enterprisedb.com>
Signed-off-by: Niccolò Fei <niccolo.fei@enterprisedb.com>
Signed-off-by: Niccolò Fei <niccolo.fei@enterprisedb.com>
Mimic the definition of object-store-deployment.yaml.
The PVC is ReadWriteOnce: a rolling update would try to schedule
the new pod while the old one still holds it.

Signed-off-by: Niccolò Fei <niccolo.fei@enterprisedb.com>
…yment

Signed-off-by: Niccolò Fei <niccolo.fei@enterprisedb.com>
Signed-off-by: Niccolò Fei <niccolo.fei@enterprisedb.com>
Comment thread renovate.json5 Outdated
Comment thread test/e2e/internal/tests/walrestore/fixtures.go Outdated
Comment thread web/docs/migration.md Outdated
Comment thread web/docs/migration.md Outdated
Comment thread hack/object-store/object-store-deployment.yaml Outdated
mnencia and others added 3 commits September 24, 2026 09:51
Apply Tao's review suggestions.

Co-authored-by: Tao Li <tao.li@enterprisedb.com>
Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
Drop all Linux capabilities on the init, object-store and s3-client
containers, none of them need any. Add runAsNonRoot to object-store
and s3-client; the init container still needs root to chmod the
PVC-owned data and log directories.

This is a dev/testing deployment, but anyone copying it as a
starting point should get a secure default, not a baseline one.

Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
A subPath mount doesn't pick up an updated Secret when cert-manager
renews the certificate, so the cached CA would go stale until the
pod restarts. The secret's items already remap tls.crt to ca.crt, so
mounting the whole volume at /certs keeps AWS_CA_BUNDLE pointing at
the same file while staying live-updated.

Signed-off-by: Marco Nenciarini <marco.nenciarini@enterprisedb.com>
@mnencia
mnencia merged commit ab99236 into main Sep 24, 2026
9 checks passed
@mnencia
mnencia deleted the dev/624 branch September 24, 2026 08:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Chore]: Migrate from deprecated MinIO Docker images for S3 archive testing

5 participants