Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
365dd6d
perf(store): answer the node-scoped question with a node-scoped read
kvaps Sep 2, 2026
2c31bb1
perf(cli): read the volume sizes for `resource list` in one request
kvaps Sep 2, 2026
0dd8b7d
perf(cli): pick the volume-size read to match how narrow the listing is
kvaps Sep 7, 2026
6de2cb9
fix(store): key ListAll so a caller with the replica's spelling finds it
kvaps Sep 7, 2026
d46e1fd
fix(store): register the field indexes the scoped reads need
kvaps Sep 7, 2026
26b8584
fix(store): find the storage pools on a node that were applied by hand
kvaps Sep 7, 2026
d8d5415
perf(cli): answer the node commands with the node-scoped read
kvaps Sep 7, 2026
5f5528a
style(store): satisfy the linters on the scoped-read work
kvaps Sep 7, 2026
f3c387c
fix(store): tie the field indexes to the manager, and the fallback to…
kvaps Sep 8, 2026
89e9a8a
fix(store): find the snapshots of a definition that were adopted with…
kvaps Sep 8, 2026
463def9
fix(cli): degrade per definition, and count what node delete reads
kvaps Sep 8, 2026
e13e0c3
fix(store): decide a node's fate on a read the cache cannot be behind on
kvaps Sep 8, 2026
55d4025
fix(store): surface a failed parent read instead of listing without it
kvaps Sep 8, 2026
3e9be40
test(integration): read what `node lost` answered instead of assuming…
kvaps Sep 9, 2026
3b4db2e
fix(store): read the status a node's fate turns on from the API server
kvaps Sep 9, 2026
c9ecaec
fix(cli): stop answering a refused bulk read with N refused reads
kvaps Sep 9, 2026
4f6fa1a
docs(cli): name the input the bulk-read cutoff does not have
kvaps Sep 9, 2026
278b9cd
docs(store): say where FoldName's equality stops
kvaps Sep 9, 2026
b5611e0
test(harness): make a simulated-offline node offline in the way the c…
kvaps Sep 9, 2026
cc4c86a
fix(store): build every manager-backed store the same way
kvaps Sep 10, 2026
d50ad9e
fix(rest): say when the orphan-snapshot sweep did not run
kvaps Sep 10, 2026
6172eb2
fix(cli): treat a throttled server as the budget case it is
kvaps Sep 10, 2026
68cf497
chore(store): keep the conformance suite under the complexity budget
kvaps Sep 10, 2026
9dd8240
fix(store): make the manager, its indexes and its store one call
kvaps Sep 14, 2026
f17b917
perf(cli): answer a narrowed resource list with the scoped read
kvaps Sep 14, 2026
8afecc2
fix(store): hold the remaining node and snapshot reads to the same rule
kvaps Sep 14, 2026
055ad10
style(apiserver): gofmt after dropping the separate store construction
kvaps Sep 14, 2026
8fc9fa2
fix(cli): list every replica a narrowed filter matches
kvaps Sep 14, 2026
266216d
fix(store): keep the uncached snapshot read uncached when it falls back
kvaps Sep 14, 2026
27475be
fix(store): ask a node's objects under the spelling it is registered as
kvaps Sep 14, 2026
0996147
test(store): close the wiring guard's skip list and helper escape
kvaps Sep 14, 2026
40fc53a
fix(store): build the manager without asking the API server anything
kvaps Sep 14, 2026
b9234dd
style(store): satisfy the linters on the round's store changes
kvaps Sep 14, 2026
47bcfff
Revert "fix(store): build the manager without asking the API server a…
kvaps Sep 14, 2026
ff18809
fix(store): wait for the API server while registering the indexes
kvaps Sep 14, 2026
29daa1c
test(store,cli): isolate the three terms no fixture discriminated
kvaps Sep 15, 2026
eabf5d8
fix(store): end the index registration wait before liveness kills
kvaps Sep 15, 2026
945af86
refactor(store): index only the fields the cache is asked for
kvaps Sep 15, 2026
3aca5d4
fix(rest): re-walk a node delete under the spellings seen before it
kvaps Sep 15, 2026
1efb936
style(rest): satisfy the linters on the node delete spelling change
kvaps Sep 15, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions api/v1alpha1/resource_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -523,6 +523,12 @@ type ResourceVolumeStatus struct {
// +kubebuilder:subresource:status
// +kubebuilder:resource:scope=Cluster
// +kubebuilder:validation:XValidation:rule="oldSelf.hasValue() || self.metadata.name.lowerAscii() == (self.spec.resourceDefinitionName + '.' + self.spec.nodeName).lowerAscii()",message="metadata.name must equal <spec.resourceDefinitionName>.<spec.nodeName> (case-insensitive)",optionalOldSelf=true
// Server-side field selectors. Without these the API server refuses a
// selector on these paths outright, which is the failure mode we want: a
// selector that silently returned a subset is the Bug 038 shape, where a
// partial-but-correct answer hid the replicas an operator applied by hand.
// +kubebuilder:selectablefield:JSONPath=`.spec.nodeName`
// +kubebuilder:selectablefield:JSONPath=`.spec.resourceDefinitionName`

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MINOR] spec.resourceDefinitionName is declared selectable and nothing selects on it

The only MatchingFields in the tree is spec.nodeName. ListByDefinition deliberately scans (pkg/store/k8s/resources.go:105-127) because the label selector was Bug 038, and a field selector would not have that problem, so the declaration reads like an intended follow-up that did not land. A selectable field is close to permanent once shipped, since a client may start relying on it. Either wire ListByDefinition to it in this PR or drop the marker until something uses it.

// +kubebuilder:printcolumn:name="Definition",type=string,JSONPath=`.spec.resourceDefinitionName`
// +kubebuilder:printcolumn:name="Node",type=string,JSONPath=`.spec.nodeName`
// +kubebuilder:printcolumn:name="Pool",type=string,JSONPath=`.spec.storagePool`
Expand Down
6 changes: 6 additions & 0 deletions api/v1alpha1/snapshot_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -274,6 +274,12 @@ type SnapshotPerNodeStatus struct {
// +kubebuilder:subresource:status
// +kubebuilder:resource:scope=Cluster
// +kubebuilder:validation:XValidation:rule="oldSelf.hasValue() || self.metadata.name.lowerAscii() == (self.spec.resourceDefinitionName + '.' + self.spec.snapshotName).lowerAscii()",message="metadata.name must equal <spec.resourceDefinitionName>.<spec.snapshotName> (case-insensitive)",optionalOldSelf=true
// spec.resourceDefinitionName is selectable so a definition-scoped snapshot
// read is a definition-scoped query. A field, not the label the objects
// usually carry: a Snapshot adopted from a LINSTOR dump has no labels, and a
// label selector would answer partial-but-correct on the read that refuses
// `rd d` and sweeps the leftovers after it.
// +kubebuilder:selectablefield:JSONPath=`.spec.resourceDefinitionName`
// +kubebuilder:printcolumn:name="Definition",type=string,JSONPath=`.spec.resourceDefinitionName`
// +kubebuilder:printcolumn:name="Snapshot",type=string,JSONPath=`.spec.snapshotName`
// +kubebuilder:printcolumn:name="Nodes",type=string,JSONPath=`.spec.nodes`
Expand Down
8 changes: 8 additions & 0 deletions api/v1alpha1/storagepool_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,14 @@ type StoragePoolStatus struct {
// +kubebuilder:subresource:status
// +kubebuilder:resource:scope=Cluster
// +kubebuilder:validation:XValidation:rule="oldSelf.hasValue() || self.metadata.name.lowerAscii() == (self.spec.poolName + '.' + self.spec.nodeName).lowerAscii()",message="metadata.name must equal <spec.poolName>.<spec.nodeName> (case-insensitive)",optionalOldSelf=true
// spec.nodeName is selectable so a node-scoped read is a node-scoped query.
// The pools on a node are what a `node delete` is refused on and what the
// cascade removes, and answering that by listing every pool in the cluster is
// the read this replaces. A field, not the label the objects usually carry: a
// pool created by an operator (piraeus writes them with `kubectl apply`) has
// no label, and a label selector would answer partial-but-correct — which on
// the refusal path means deleting a node the cluster still has pools on.
// +kubebuilder:selectablefield:JSONPath=`.spec.nodeName`
// +kubebuilder:printcolumn:name="Node",type=string,JSONPath=`.spec.nodeName`
// +kubebuilder:printcolumn:name="Pool",type=string,JSONPath=`.spec.poolName`
// +kubebuilder:printcolumn:name="Provider",type=string,JSONPath=`.spec.providerKind`
Expand Down
22 changes: 5 additions & 17 deletions cmd/apiserver/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -138,8 +138,8 @@ func newScheme() *runtime.Scheme {
// election off — every apiserver replica serves reads
// independently. Caches still warm up so the REST server's
// cached-client reads are cheap.
func buildManager(flags *apiserverFlags) (manager.Manager, error) {
mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
func buildManager(flags *apiserverFlags) (manager.Manager, *storek8s.Store, error) {
mgr, st, err := storek8s.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
Scheme: newScheme(),
Metrics: metricsserver.Options{
BindAddress: flags.metricsAddr,
Expand All @@ -150,10 +150,10 @@ func buildManager(flags *apiserverFlags) (manager.Manager, error) {
LeaderElection: false,
})
if err != nil {
return nil, errors.Wrap(err, "new manager")
return nil, nil, errors.Wrap(err, "new manager")
}

return mgr, nil
return mgr, st, nil
}

// resolveNamespace mirrors the controller's namespace-resolution
Expand Down Expand Up @@ -248,24 +248,12 @@ func main() {
flags := parseFlags()
namespace := resolveNamespace(flags.controllerNamespace)

mgr, err := buildManager(flags)
mgr, st, err := buildManager(flags)
if err != nil {
setupLog.Error(err, "Failed to start manager")
os.Exit(1)
}

// CRD-backed store is the only supported persistence layer
// post-Phase-11 — the apiserver/controller split made
// in-process state pointless across replicas.
//
// BUG-048: pass the manager's direct (uncached) API reader so the
// atomic VolumeNumber allocation re-reads live RD state on each
// conflict-retry. With only the informer-cached client, two
// concurrent `vd c` against one RD both retry against a stale cache,
// re-derive the same number, exhaust the retry budget, and silently
// drop the second volume.
st := storek8s.NewWithAPIReader(mgr.GetClient(), mgr.GetAPIReader())

ready := newReadyState()

// Bug 219: `ctrl.SetupSignalHandler` is one-shot — a second call
Expand Down
21 changes: 21 additions & 0 deletions cmd/blockstor/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,10 @@ import (
"os"
"strings"

ctrl "sigs.k8s.io/controller-runtime"
ctrlclient "sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/client/config"
"sigs.k8s.io/controller-runtime/pkg/log/zap"

corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime"
Expand All @@ -55,6 +57,25 @@ import (
)

func main() {
// The store logs when a scoped read falls back to reading everything. Only
// a cluster whose CRD predates the selectable fields takes that branch,
// and both uncached readers reach it there: this client, and the servers'
// node-scoped reads, which go to the manager's API reader and are refused
// on the wire by the same API server. What the servers do not reach is the
// cache half of it, since they register the indexes.
//
// Without a root logger controller-runtime buffers the line, then after
// thirty seconds promotes to a null sink and prints its own "SetLogger
// was never called" stack trace into operator-facing stderr instead. So
// set one: quiet at the default level, and BLOCKSTOR_DEBUG turns the
// V(1) lines on for the operator who is asking why a large cluster
// crawls.
logOpts := zap.Options{
Development: os.Getenv("BLOCKSTOR_DEBUG") != "",
DestWriter: os.Stderr,
}
ctrl.SetLogger(zap.New(zap.UseFlagOptions(&logOpts)))

app := &cli.App{
Out: os.Stdout,
Err: os.Stderr,
Expand Down
13 changes: 6 additions & 7 deletions cmd/controller/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ func main() {
metricsServerOptions.KeyName = metricsCertKey
}

mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
mgr, st, err := storek8s.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
Scheme: scheme,
Metrics: metricsServerOptions,
WebhookServer: webhookServer,
Expand All @@ -190,12 +190,11 @@ func main() {
os.Exit(1)
}

// Construct the store before reconciler wiring so the
// NodeReconciler can drive eviction-triggered migration via the
// shared placer. CRD-backed is the only supported persistence
// layer since Phase 11.x — the apiserver/controller split makes
// in-process state pointless across replicas.
st := storek8s.New(mgr.GetClient())
// The store came back with the manager, before reconciler wiring, so the
// NodeReconciler can drive eviction-triggered migration via the shared
// placer. CRD-backed is the only supported persistence layer since Phase
// 11.x — the apiserver/controller split makes in-process state pointless
// across replicas.

if err := (&controller.NodeReconciler{
Client: mgr.GetClient(),
Expand Down
3 changes: 3 additions & 0 deletions config/crd/bases/blockstor.cozystack.io_resources.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -781,6 +781,9 @@ spec:
optionalOldSelf: true
rule: oldSelf.hasValue() || self.metadata.name.lowerAscii() == (self.spec.resourceDefinitionName
+ '.' + self.spec.nodeName).lowerAscii()
selectableFields:
- jsonPath: .spec.nodeName
- jsonPath: .spec.resourceDefinitionName
served: true
storage: true
subresources:
Expand Down
2 changes: 2 additions & 0 deletions config/crd/bases/blockstor.cozystack.io_snapshots.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,8 @@ spec:
optionalOldSelf: true
rule: oldSelf.hasValue() || self.metadata.name.lowerAscii() == (self.spec.resourceDefinitionName
+ '.' + self.spec.snapshotName).lowerAscii()
selectableFields:
- jsonPath: .spec.resourceDefinitionName
served: true
storage: true
subresources:
Expand Down
2 changes: 2 additions & 0 deletions config/crd/bases/blockstor.cozystack.io_storagepools.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,8 @@ spec:
optionalOldSelf: true
rule: oldSelf.hasValue() || self.metadata.name.lowerAscii() == (self.spec.poolName
+ '.' + self.spec.nodeName).lowerAscii()
selectableFields:
- jsonPath: .spec.nodeName
served: true
storage: true
subresources:
Expand Down
Loading