[cloudstack-csi-driver] PVC resize stuck for project-owned volumes: ExpandVolume omits projectid (Count:0)

[tromob]

problem

This is a bug in the CloudStack CSI driver (repo cloudstack/cloudstack-csi-driver, image ghcr.io/cloudstack/cloudstack-csi-driver). Filing it here because Issues are disabled on that repository. Related open PR: cloudstack/cloudstack-csi-driver#7.

Symptom: Expanding a PVC (StorageClass with allowVolumeExpansion: true) never completes when the backing CloudStack volume is owned by a project. The external-resizer retries indefinitely and the PVC stays in Resizing forever; status.capacity never updates.

On every attempt the cloudstack-csi-controller logs:

ControllerExpandVolume ... volume_id="<volume-uuid>"
rpc error: code = Internal desc = Could not resize volume "<volume-uuid>" to size N:
failed to retrieve volume '<volume-uuid>': No match found for <volume-uuid>: &{Count:0 Volumes:[]}

Root cause: In pkg/cloud/volumes.go, ExpandVolume() looks up the volume with the raw cloudstack-go SDK call c.Volume.GetVolumeByID(volumeID), which does not include projectid. For a project-owned volume, a project-scoped API key then gets Count:0 ("No match found") — even though project-id is correctly set in the cloud-config. The driver's own project-aware wrapper c.GetVolumeByID(ctx, volumeID) (which calls SetProjectid) is not used in this path.

Secondary issue: ExpandVolume calls ResizeVolume(newSize) unconditionally, without checking currentSize >= newSize, so it also cannot converge if the CloudStack volume is already at the requested size.

versions

• Apache CloudStack: 4.22.0.0
• cloudstack-csi-driver: v3.0.0 (same code in release cloudstack-csi-3.0.1 and current main); image ghcr.io/cloudstack/cloudstack-csi-driver
• Kubernetes: v1.33; sidecar external-resizer v1.11.1
• Hypervisor: KVM
• Storage backend: Linstor/DRBD (note: the bug is at the CloudStack API layer and is storage-agnostic)

The steps to reproduce the bug

1. A Kubernetes cluster whose CSI volumes live in a CloudStack project, with a project-scoped API key in cloudstack-secret (cloud-config has project-id set).
2. Provision a PVC via the cloudstack-csi StorageClass (works fine).
3. Increase the PVC's spec.resources.requests.storage (e.g. 20Gi -> 40Gi).
4. external-resizer loops with ... No match found for <uuid>: &{Count:0 Volumes:[]}; the PVC never leaves Resizing.

What to do about it?

Make ExpandVolume use a project-aware lookup (the existing c.GetVolumeByID(ctx, ...) wrapper), and/or set the project id globally on the cloudstack-go client so every SDK call includes it.

Open PR cloudstack/cloudstack-csi-driver#7 already adds the global approach via csClient.DefaultOptions(cloudstack.WithProject(config.ProjectID)), which would also cover ExpandVolume — but it is still open, and its description only mentions the getVMByName / node-metadata symptom, not volume resize. Merging that (and noting it also fixes resize) would resolve this. Additionally, guard ExpandVolume to skip the resize when the volume is already at the requested size (idempotency).

Workaround for others hitting this: grow the CloudStack volume, then online-grow the filesystem on the node (resize2fs on the mounted ext4 volume — non-disruptive), and reconcile the Kubernetes objects (PV .spec.capacity and PVC .status.capacity to the new size, and clear the Resizing condition) so the resizer sees requests == capacity and stops.

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!