Skip to content

docs(operations): backup-classes claims the lookup-gated strategy skip self-heals on the HelmRelease interval, and suggests a reconcile that does not re-render #639

Description

@mattia-eleuteri

The backup-classes page tells operators that a missed install-time render self-heals on the HelmRelease interval, and offers a recovery command that does not work. Both are wrong, and the second one actively wastes an operator's time during an incident.

content/en/docs/next/operations/services/backup-classes.md, "Cozy-default Bucket bootstrap":

The HelmRelease re-reconciles on its interval (5 minutes by default — set by the cozystack operator's helmrelease-interval flag, not a Flux default), at which point the populated BucketClaim status causes the missing strategy templates to materialise.

If you need the BackupClass functional immediately (e.g. an e2e), trigger a Flux reconcile (flux reconcile helmrelease backupstrategy-controller -n cozy-backup-controller) once you see [...] non-empty.

helm-controller re-renders a release only when its chart or values change. The interval reconcile is a no-op for a healthy release, and drift detection is off on operator-generated HelmReleases, so the skip is permanent, not a bootstrap window — and flux reconcile helmrelease does not re-render either, so the suggested escape hatch has no effect. Recovery needs a real Helm upgrade: reconcile.fluxcd.io/forceAt and requestedAt.

This is not theoretical. It was reported from a cluster that lost the race by 4 seconds at install and still had BackupClass cozy-default with no Strategy CRs and no Velero BSL months later — see cozystack/cozystack#3518. It also fail-closes the pre-adoption snapshot in the v1.6.0 etcd migration, so an affected cluster cannot be upgraded at all.

Upstream fix: cozystack/cozystack#3524. It corrects the same paragraph in this repo's docs/operations/backup-classes.md, adds the two-release manual recovery procedure, and makes backupstrategy-controller force the upgrade itself so the condition converges.

Two changes wanted here, once the upstream PR settles (the exact wording depends on the shape it lands in, which is why this is an issue and not a PR):

  1. operations/services/backup-classes.md — replace the "re-reconciles on its interval" claim and the flux reconcile suggestion. The upstream page now documents the forced-upgrade recovery, including the easy-to-miss detail that the <bucket>-<user>-credentials Secret is rendered by the bucket-<name>-system release, not by bucket-<name>, so an affected cluster needs two releases forced, in order.
  2. operations/configuration/platform-package.md — the backupStorage row enumerates the forwarded keys (provisionBucket, bucketName, endpoint, region, forcePathStyle, systemSecretName, systemNamespaces). fix(backupstrategy-controller): repair lookup-gated backup objects cozystack#3524 adds reconcileDefaultObjects to that block, and since the whole block is deep-merged into the component it is admin-settable through spec.components.platform.values.backupStorage, so the list becomes incomplete.

v1.5 and v1.6 carry the same stale paragraph. Whether to backport is a maintainer call: the claim is wrong on those versions too, and an operator on v1.5 or v1.6 is exactly who hits this, but released version directories are normally left frozen.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions