test(e2e): migrate the blockdevice suite to the custom e2e-br image#2633
Draft
Isteb4k wants to merge 21 commits into
Draft
test(e2e): migrate the blockdevice suite to the custom e2e-br image#2633Isteb4k wants to merge 21 commits into
Isteb4k wants to merge 21 commits into
Conversation
Point every VirtualDiskCreation data source (HTTP, Upload, registry, VirtualImage, ClusterVirtualImage) at the minimal custom e2e-br image: a Selectel qcow2 for HTTP/Upload and a ttl.sh container-disk for the registry source. Drop the cloud-init provisioning for the test's VM — it only needs the VM as a disk consumer with a live guest agent and never logs in, so no user has to be created. New object symbols (ImageURLCustomBIOS, ImageURLCustomContainer, PrecreatedCVICustomBIOS) are additive; existing constants are left untouched, so other e2e tests keep their behavior. Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
The custom e2e-br image is ~35 MiB and grows its root filesystem to the disk on first boot, so the 400Mi disks are oversized. Use a 50Mi vdCreationImageSize for the image-backed VirtualDisks instead. Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
Switch every Alpine-based bootable-qcow2 source in the blockdevice suite to the custom e2e-br image: HTTP (ImageURLAlpineBIOS -> ImageURLCustomBIOS), registry (ImageURLContainerImage -> ImageURLCustomContainer) and the CVI ObjectRef (PrecreatedCVIAlpineBIOS -> PrecreatedCVICustomBIOS). Ubuntu-based tests (data_exports, virtual_disk_resizing, vd_snapshots Ubuntu cases — they need systemd/SSH/cloud-init), the ISO/CD-ROM format tests (ImageURLUbuntuISO / PrecreatedCVIUbuntuISO) and the non-bootable testdata images are left untouched: the custom image is a bootable qcow2, not a drop-in for those. Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
The quota-exhausted test only needs a valid CVI source to trigger the QuotaExceeded path on PVC creation; use the custom e2e-br image instead of the non-bootable testdata qcow2. Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
data_exports does not SSH — it waits for the guest agent and exports/restores data over the API. Use the custom e2e-br image (agent auto-starts) and drop the Ubuntu cloud-init provisioning. Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
- add observer/vmbda package with BeAttached()/BeFailed() predicates; - add vd observer BeResizing() predicate; - source virtual_disk_resizing from the custom e2e-br image, log in as root (baked key) instead of the cloud user, and drop sudo (custom has neither cloud user nor sudo) via blockdevice-local SSH/lsblk helpers so the shared util helpers stay untouched; - replace resource waits (UntilObjectPhase/UntilSSHReady/UntilDisksAreAttached) with observer + WaitFor + predicate; the only remaining Eventually is the guest-side lsblk-grows check, deliberately kept and commented as the sole sanctioned exception (no Kubernetes resource to observe). Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
Add additive NewHTTPVDCustomBIOS/NewGeneratedHTTPVICustomBIOS constructors (the AlpineBIOS variants stay for other suites), switch importer_network_policy to them, and replace VI/VD phase waits with observer+WaitFor. The Project 'Deployed' wait is kept via UntilObjectState (Project has no typed client/Observer) with an explicit comment. Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
Replace UntilObjectPhase/UntilVMAgentReady resource waits with observer+WaitFor (vi/vd/vm), source the HTTP case from the custom image (NewHTTPVDCustomBIOS), and disable cloud-init provisioning on the consumer VMs (custom image, agent-only). Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
Source all disks from the custom e2e-br image (drop Ubuntu), disable cloud-init provisioning, and replace resource waits with observers: agent/disk/attachment via WaitFor, snapshot readiness via a vdsnapshot observer per snapshot, VM deletion via observer.WaitForDeleted, disks-attached via a VM WaitFor predicate. The transient filesystem-freeze check now uses a new vm.BeFilesystemFrozen() predicate observed from before the snapshot is created (replacing the Eventually poll). Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
…vers Add blockdevice-local root/no-sudo guest filesystem helpers (guest_fs.go) — the util.* originals stay cloud+sudo for other suites (vmop/restore). Convert data_exports resource waits to observers (agent/stopped/vmop-completed/disk-phase/snapshot-ready/delete), add vm.BeStopped(); the upload-endpoint retry stays as a commented Eventually exception (external HTTP, not a resource). Document the Project 'Deployed' wait exception in setupProject (Project has no typed Observer). Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
Add ImageURLCustomISO/PrecreatedCVICustomISO (HTTP CVI for the custom EFI ISO). Point the ISO/CD-ROM boot and format-expectation tests at the custom ISO instead of the Ubuntu live-server ISO, and drop the VNC subiquity-installer screen assertion (Ubuntu-specific) — the CD-ROM boot test now only asserts the VM reaches Running with a bootable device (Never NoBootableDevice), which is what booting a custom ISO as CD-ROM verifies. Removes the RFB/VNC framebuffer machinery that check required. Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
…tection
ProtectionService.RemoveProtection built its merge patch from a possibly stale
(informer cache) copy and resent the whole finalizers list. On a terminating
PersistentVolumeClaim this could re-add kubernetes.io/pvc-protection that the
built-in pvc-protection controller had already dropped, which the API server
rejects with 422 ("no new finalizers can be added if the object is being
deleted"). The VM deletion handler surfaced this as "failed to release PVC
protection" and never removed the VM cleanup finalizer, so VM deletion hung.
Re-read a fresh copy and patch it under an optimistic lock, retrying on
conflict, so only our finalizer is removed and no other finalizer is
resurrected.
Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
…rt skip Blockdevice suite hardening on top of the custom e2e-br image: - Introduce label.SIGDescribe and SIG labels (mirroring Kubernetes' [sig-*] ownership labels) and group the blockdevice specs under sig-storage, so the group can be run with `--label-filter='sig-storage'`. - Resolve guest block devices by their SCSI VPD serial read straight from sysfs (/sys/block/*/device/vpd_pg80). The minimal e2e-br image runs no udev, so lsblk's SERIAL column and /dev/disk/by-id are empty; this also drops the awk one-liner whose single quotes clashed with d8 ssh -c '...'. - Add vd observer predicate BeResized and wait on it after a resize instead of BeReady, which errors on the transient Resizing phase. - Skip DataExports when the suite runs off-cluster: publish-mode export is currently broken by a bug in the storage-volume-data-manager module (origin Ingress looked up in d8-user-authn instead of kube-system). TODO to drop the skip once fixed. - Filter the benign pvc-protection cleanup error and raise the VM-deletion wait to LongTimeout. Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
…tion-custom-image
Isteb4k
changed the title
- drop the redundant copyloopvar copy in the resize goroutine loop (unparam/copyloopvar) - drop the always-constant bdKind param from getBlockDeviceLsblkSizeAsRoot - download DataExports artifacts into a Ginkgo per-spec temp dir instead of the working directory, so a run leaves nothing untracked in the repo Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
The disk is deleted right after being exported; its exporter pod keeps the PVC mounted until the export tears down, so deletion can exceed the 1m MiddleTimeout (it finishes in ~15s once released). Use LongTimeout. Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
In-cluster measurement shows vd-data deletion after export is fast (~0-16s on healthy storage). The earlier LongTimeout bump was unjustified: the >1m hang was caused by transient LINSTOR degradation, and a larger timeout would have masked that flaky storage rather than surfacing it. Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
…sk in DataExports The test deleted vd-data and blocked on observer.WaitForDeleted, which timed out flakily. The wait is unnecessary: the restored disks use different names and the VM's block devices are reassigned away from vd-data, so nothing depends on the original being gone. vd-data is removed by the deferred cleanup at teardown. Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
… in the no-consumer snapshot test The no-consumer case deletes the throwaway VM to leave the disk without a consumer, then snapshots it. It waited on observer.WaitForDeleted(VM), which gated on VM-object teardown and timed out flakily. Wait instead for the actual precondition — vdobs.BeDetached() (the disk has no attached VM) — which is what the snapshot needs and does not depend on VM teardown latency. Signed-off-by: Dmitry Rakitin <dmitry.rakitin@flant.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Move the whole
blockdevicee2e suite from the ~350 MiB Alpine image to theminimal from-scratch
e2e-brimage (~35 MiB), and fix what that surfaced:Selectel, registry →
ttl.shcontainer-disk, VI/CVI → precreatedv12n-e2e-custom-bios); ISO tests use a custom EFIcustom.iso.no udev, so
lsblk SERIAL/by-idare empty; SSH logs in asroot, nosudo.Eventually(newsig-storageSIGlabel via
SIGDescribe,vdBeResized,vmbdaobserver).ProtectionService.RemoveProtectionno longer resurrects afinalizer on a terminating PVC, which previously errored and hung VM deletion.
DataExportsis skipped off-cluster (publish-mode export bug in thestorage-volume-data-managermodule).New
internal/objectsymbols are additive; other suites keep the Alpine image.Why do we need it, and what problem does it solve?
e2e-bris ~10× smaller than the Alpine image the suite pulls today. Thismigrates the blockdevice group to it and fixes the guest/observer/controller
issues that only appear on a truly minimal (no-udev, no-cloud-init) image.
What is the expected result?
The
sig-storage(blockdevice) group passes on the custom image: disksprovision from every source, consumer VMs reach
Running+AgentReady, resizeand snapshots work.
DataExportsruns on a cluster node / in CI. Other suitesare unaffected.
Checklist
Changelog entries