Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 43 additions & 21 deletions test/e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ non-technical stakeholders.

## Project Structure

Component step packages are introduced incrementally. The layout below shows
their intended ownership after all component extractions land.

```
test/
├── e2e/
Expand All @@ -33,8 +36,12 @@ test/
└── internal/
├── e2e/
│ ├── runner.go # Godog options, scenario selection, summary wiring
│ ├── steps/ # All existing step definitions and hooks
│ ├── steps/ # Shared world, hooks, catalog, resource, RBAC, and condition helpers
│ │ └── testdata/ # RBAC and resource templates
│ ├── operator-controller/steps/ # ClusterExtension lifecycle and integration assertions
│ ├── catalogd/steps/ # HA/proxy/TLS steps and restoration hooks
│ ├── object-controller/steps/ # ClusterObjectSet/Deployment assertions
│ ├── platform/steps/ # Demo steps and recording
│ └── support-bundle.yaml # Shared diagnostics
└── summary/ # Summary generation and embedded templates
```
Expand All @@ -43,7 +50,7 @@ test/

### 1. Test Runner (`operator-controller/suite_test.go`)

The component entry point registers the full shared step set and hooks, then calls
The component entry point registers shared steps, hooks, and its component steps, then calls
`e2e.Run` from `test/internal/e2e` with its feature directory. The shared runner configures Godog options,
scenario selection, and summary generation.

Expand All @@ -53,10 +60,22 @@ controller unit tests live under `internal/testutil`, which is accessible throug
the repository.

Features follow the ownership in the [per-component suite design](../../docs/designs/e2e-per-component.md).
Each component has a thin entry point that still registers the full shared step set;
step ownership is separated in the next stage. The newer `cos_*`, `cod_*`, and
`group` features belong to object-controller. Feature contents and step behavior
are unchanged, including ClusterExtension integration assertions in object features.
The newer `cos_*`, `cod_*`, and `group` features belong to object-controller.
Suites register additional component steps when existing features contain integration
assertions: operator-controller uses object assertions, object-controller retains
ClusterExtension integration coverage, and catalogd proxy tests use ClusterExtension
steps. Platform registers ClusterExtension steps for demos. Feature contents,
scenario names, tags, and assertions are unchanged.

`test/internal/e2e/steps` owns `World` and the `Scenario(ctx)` accessor, generic resource and
condition helpers, catalog/registry prerequisites, RBAC templates, and cleanup.
`ListClusterObjectSetsInGroup` and `ResolveObjectRef` remain shared because
both ClusterExtension and ClusterObjectSet assertions consume them.
Catalogd owns deployment restoration and proxy state. Platform owns recording;
shared helpers observe commands through an optional `CommandRecorder` interface.
Pass extra lifecycle registration to `sharedsteps.RegisterHooks` to keep suite hooks
between shared scenario setup and cleanup. The upgrade helpers remain shared because
catalogd HA scenarios also consume component readiness and catalog reconciliation steps.

### 2. Feature Files (`operator-controller/features/*.feature`)

Expand Down Expand Up @@ -112,7 +131,7 @@ Feature: Install ClusterExtension
And ClusterExtension is available
```

### 3. Step Definitions (`test/internal/e2e/steps/steps.go`)
### 3. Step Definitions (`test/internal/e2e/<component>/steps/steps.go`)

Go functions that implement the steps defined in feature files. Each step is registered with a regex pattern that
matches the Gherkin text.
Expand All @@ -121,9 +140,8 @@ matches the Gherkin text.

```go
func RegisterSteps(sc *godog.ScenarioContext) {
sc.Step(`^OLM is available$`, OLMisAvailable)
sc.Step(`^bundle "([^"]+)" is installed in version "([^"]+)"$`, BundleInstalled)
sc.Step(`^ClusterExtension is applied$`, ResourceIsApplied)
sc.Step(`^ClusterExtension is applied$`, shared.ResourceIsApplied)
// ... more steps
}
```
Expand All @@ -132,7 +150,7 @@ sc.Step(`^ClusterExtension is applied$`, ResourceIsApplied)

Status assertions must read metadata and status from the same API response and
require the asserted condition's `observedGeneration` to equal `metadata.generation`.
Use `waitForCondition` for condition checks. Bundle metadata uses the `Installed`
Use `shared.WaitForCondition` for condition checks. Bundle metadata uses the `Installed`
condition, and COS phase digests and object counts use the `Ready` condition to
verify that the status reflects the current spec. Deployments instead expose
`status.observedGeneration` at the resource level.
Expand All @@ -146,18 +164,18 @@ That step accepts any current-generation `Progressing` status or reason, whereas

```go
func BundleInstalled(ctx context.Context, name, version string) error {
sc := scenarioCtx(ctx)
waitFor(ctx, func() bool {
v, err := k8sClient(ctx, "get", "clusterextension", sc.clusterExtensionName, "-o", "json")
sc := shared.Scenario(ctx)
shared.WaitFor(ctx, func() bool {
v, err := shared.K8sClient(ctx, "get", "clusterextension", sc.ClusterExtensionName, "-o", "json")
if err != nil {
return false
}
var ext ocv1.ClusterExtension
if err := json.Unmarshal([]byte(v), &ext); err != nil {
return false
}
installed := currentCondition(ext.Generation, ext.Status.Conditions, ocv1.TypeInstalled)
return conditionMatches(installed, metav1.ConditionTrue, nil, nil) &&
installed := shared.CurrentCondition(ext.Generation, ext.Status.Conditions, ocv1.TypeInstalled)
return shared.ConditionMatches(installed, metav1.ConditionTrue, nil, nil) &&
ext.Status.Install.Bundle.Name == name && ext.Status.Install.Bundle.Version == version
})
return nil
Expand Down Expand Up @@ -199,7 +217,7 @@ Feature: Your Feature Name

### 2. Implement Step Definitions

Add step implementations in `test/internal/e2e/steps/steps.go`:
Add component assertions in `test/internal/e2e/<component>/steps/`; add reusable prerequisites and helpers in `test/internal/e2e/steps/`:

```go
func RegisterSteps(sc *godog.ScenarioContext) {
Expand All @@ -208,7 +226,7 @@ func RegisterSteps(sc *godog.ScenarioContext) {
}

func YourStepFunction(ctx context.Context, param string) error {
sc := scenarioCtx(ctx)
sc := shared.Scenario(ctx)
// Implementation
return nil
}
Expand All @@ -230,7 +248,7 @@ Use these variables in YAML templates:
- `${NAME}`: Scenario-specific ClusterExtension name (e.g., `ce-123`)
- `${COS_NAME}`: Scenario-specific ClusterObjectSet name (e.g., `cos-123`; for applying ClusterObjectSets directly)
- `${TEST_NAMESPACE}`: Scenario-specific namespace (e.g., `ns-123`)
- `${SCENARIO_ID}`: Unique scenario identifier used for resource name isolation
- `${SCENARIO_ID}`: Feature-scoped scenario identifier used for resource isolation across concurrent Go suites
- `${PACKAGE:<name>}`: Parameterized package name (e.g., `${PACKAGE:test}` expands to `test-<scenario-id>`)
- `${CATALOG:<name>}`: Catalog resource name (e.g., `${CATALOG:test}` expands to `test-catalog-<scenario-id>`)

Expand Down Expand Up @@ -427,8 +445,12 @@ go test -count=1 ./test/e2e/operator-controller/... -args --log.debug --k8s.cli=

Each scenario runs in its own namespace with unique resource names, ensuring complete isolation:

- Namespace: `ns-{scenario-id}`
- ClusterExtension: `ce-{scenario-id}`
- Namespace: `ns-{feature-hash}-{godog-id}`
- ClusterExtension: `ce-{feature-hash}-{godog-id}`

Godog restarts numeric IDs in each process. The feature hash prevents collisions
between concurrent component suites and stays stable when a feature moves. Feature
and scenario annotations retain their existing values.

### 2. Test-Identifying Annotations

Expand Down Expand Up @@ -476,7 +498,7 @@ When ClusterExtension is applied
All asynchronous operations use `waitFor` with consistent timeout (300s) and tick (1s):

```go
waitFor(ctx, func() bool {
shared.WaitFor(ctx, func() bool {
// Check condition
return conditionMet
})
Expand Down
3 changes: 3 additions & 0 deletions test/e2e/catalogd/suite_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"github.com/cucumber/godog"

"github.com/operator-framework/operator-controller/test/internal/e2e"
catalogsteps "github.com/operator-framework/operator-controller/test/internal/e2e/catalogd/steps"
"github.com/operator-framework/operator-controller/test/internal/e2e/steps"
)

Expand All @@ -15,7 +16,9 @@ func TestMain(m *testing.M) {
tc.BeforeSuite(steps.BeforeSuite)
}, func(sc *godog.ScenarioContext) {
steps.RegisterSteps(sc)
catalogsteps.RegisterSteps(sc)
steps.RegisterDemoSteps(sc)
catalogsteps.RegisterHooks(sc)
steps.RegisterHooks(sc)
}))
}
5 changes: 4 additions & 1 deletion test/e2e/features_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import (
"github.com/cucumber/godog"

"github.com/operator-framework/operator-controller/test/internal/e2e"
catalogsteps "github.com/operator-framework/operator-controller/test/internal/e2e/catalogd/steps"
"github.com/operator-framework/operator-controller/test/internal/e2e/steps"
)

Expand Down Expand Up @@ -41,7 +42,9 @@ func TestMain(m *testing.M) {
tc.BeforeSuite(steps.BeforeSuite)
}, func(sc *godog.ScenarioContext) {
steps.RegisterSteps(sc)
catalogsteps.RegisterSteps(sc)
steps.RegisterDemoSteps(sc)
steps.RegisterHooks(sc)
catalogsteps.RegisterHooks(sc)
steps.RegisterHooks(sc, steps.RegisterAsciiCastHooks)
}))
}
2 changes: 1 addition & 1 deletion test/e2e/platform/suite_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@ func TestMain(m *testing.M) {
}, func(sc *godog.ScenarioContext) {
steps.RegisterSteps(sc)
steps.RegisterDemoSteps(sc)
steps.RegisterHooks(sc)
steps.RegisterHooks(sc, steps.RegisterAsciiCastHooks)
}))
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,33 +5,27 @@ import (
"fmt"
"strings"

"k8s.io/component-base/featuregate"
shared "github.com/operator-framework/operator-controller/test/internal/e2e/steps"
)

// catalogdHAFeature gates scenarios that require a multi-node cluster.
// It is set to true in BeforeSuite when the cluster has at least 2 nodes,
// which is the case for the experimental e2e suite (kind-config-2node.yaml)
// but not the standard suite.
const catalogdHAFeature featuregate.Feature = "CatalogdHA"

// CatalogdLeaderPodIsForceDeleted force-deletes the catalogd leader pod to simulate leader loss.
// The pod is identified from sc.leaderPods["catalogd"] (populated by a prior
// The pod is identified from sc.LeaderPods["catalogd"] (populated by a prior
// "catalogd is ready to reconcile resources" step). Force-deletion is equivalent to
// an abrupt process crash: the lease is no longer renewed and the surviving pod
// acquires leadership after the lease expires.
//
// Note: stopping the kind node container is not used here because both nodes in the
// Note: stopping the Kind node container is not used here because both nodes in the
// experimental 2-node cluster are control-plane nodes that run etcd — stopping either
// would break etcd quorum and make the API server unreachable for the rest of the test.
func CatalogdLeaderPodIsForceDeleted(ctx context.Context) error {
sc := scenarioCtx(ctx)
leaderPod := sc.leaderPods["catalogd"]
sc := shared.Scenario(ctx)
leaderPod := sc.LeaderPods["catalogd"]
if leaderPod == "" {
return fmt.Errorf("catalogd leader pod not found in scenario context; run 'catalogd is ready to reconcile resources' first")
}

logger.Info("Force-deleting catalogd leader pod", "pod", leaderPod)
if _, err := k8sClient(ctx, "delete", "pod", leaderPod, "-n", namespaceForComponent("catalogd"),
shared.Log().Info("Force-deleting catalogd leader pod", "pod", leaderPod)
if _, err := shared.K8sClient(ctx, "delete", "pod", leaderPod, "-n", shared.NamespaceForComponent("catalogd"),
"--force", "--grace-period=0"); err != nil {
return fmt.Errorf("failed to force-delete catalogd leader pod %q: %w", leaderPod, err)
}
Expand All @@ -40,13 +34,13 @@ func CatalogdLeaderPodIsForceDeleted(ctx context.Context) error {

// NewCatalogdLeaderIsElected polls the catalogd leader election lease until the holder
// identity changes to a pod other than the deleted leader. It updates
// sc.leaderPods["catalogd"] with the new leader pod name.
// sc.LeaderPods["catalogd"] with the new leader pod Name.
func NewCatalogdLeaderIsElected(ctx context.Context) error {
sc := scenarioCtx(ctx)
oldLeader := sc.leaderPods["catalogd"]
sc := shared.Scenario(ctx)
oldLeader := sc.LeaderPods["catalogd"]

waitFor(ctx, func() bool {
holder, err := k8sClient(ctx, "get", "lease", leaseNames["catalogd"], "-n", namespaceForComponent("catalogd"),
shared.WaitFor(ctx, func() bool {
holder, err := shared.K8sClient(ctx, "get", "lease", "catalogd-operator-lock", "-n", shared.NamespaceForComponent("catalogd"),
"-o", "jsonpath={.spec.holderIdentity}")
if err != nil || holder == "" {
return false
Expand All @@ -55,8 +49,8 @@ func NewCatalogdLeaderIsElected(ctx context.Context) error {
if newPod == oldLeader {
return false
}
sc.leaderPods["catalogd"] = newPod
logger.Info("New catalogd leader elected", "pod", newPod)
sc.LeaderPods["catalogd"] = newPod
shared.Log().Info("New catalogd leader elected", "pod", newPod)
return true
})
return nil
Expand Down
85 changes: 85 additions & 0 deletions test/internal/e2e/catalogd/steps/hooks.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
package steps

import (
"context"
"fmt"

"github.com/cucumber/godog"

shared "github.com/operator-framework/operator-controller/test/internal/e2e/steps"
)

// deploymentRestore records the original state of a deployment so it can be
// rolled back after a test that modifies deployment configuration.
type deploymentRestore struct {
Name string // deployment name
Namespace string
containerName string // container to patch (for env var restores)
patchedArgs bool // true when container args were modified (for TLS profile patches)
originalArgs []string // original container args; may be nil if args were unset
originalEnv []string // original env vars as "NAME=VALUE" (for proxy patches)
}

type catalogContextKey struct{}
type catalogContext struct {
proxy *recordingProxy
deploymentRestores []deploymentRestore
}

func catalogScenario(ctx context.Context) *catalogContext {
return ctx.Value(catalogContextKey{}).(*catalogContext)
}

// RegisterHooks installs catalog-specific state and unconditional deployment restoration.
// Register it before shared hooks so modified deployments are restored before resource cleanup.
func RegisterHooks(sc *godog.ScenarioContext) {
sc.Before(func(ctx context.Context, _ *godog.Scenario) (context.Context, error) {
return context.WithValue(ctx, catalogContextKey{}, &catalogContext{}), nil
})
sc.After(func(ctx context.Context, _ *godog.Scenario, err error) (context.Context, error) {
sc := catalogScenario(ctx)
// Stop the in-process recording proxy if one was started.
if sc.proxy != nil {
sc.proxy.stop()
}
// Restore any deployments that were modified during the scenario. Runs
// unconditionally (even on failure) to prevent a misconfigured deployment
// from bleeding into subsequent scenarios. Restored in LIFO order so that
// multiple patches to the same deployment unwind to the true original.
for i := len(sc.deploymentRestores) - 1; i >= 0; i-- {
dr := sc.deploymentRestores[i]
if dr.patchedArgs {
if err2 := patchDeploymentArgs(dr.Namespace, dr.Name, dr.originalArgs); err2 != nil {
shared.Log().Info("Error restoring deployment args", "name", dr.Name, "error", err2)
} else if _, err2 := shared.K8sClient(ctx, "rollout", "status", "-n", dr.Namespace,
fmt.Sprintf("deployment/%s", dr.Name), "--timeout=2m"); err2 != nil {
shared.Log().Info("Timeout waiting for deployment rollout after restore", "name", dr.Name)
}
}
if err2 := restoreDeployment(dr); err2 != nil {
shared.Log().Info("Error restoring deployment env", "deployment", dr.Name, "namespace", dr.Namespace, "error", err2)
}
}

return ctx, err
})
}

// RegisterSteps registers catalogd HA, proxy, and TLS assertions.
func RegisterSteps(sc *godog.ScenarioContext) {
sc.Step(`^(?i)the "([^"]+)" deployment is configured with custom TLS minimum version "([^"]+)"$`, ConfigureDeploymentWithCustomTLSVersion)
sc.Step(`^(?i)the "([^"]+)" deployment is configured with custom TLS version "([^"]+)", ciphers "([^"]+)", and curves "([^"]+)"$`, ConfigureDeploymentWithCustomTLSFull)
sc.Step(`^(?i)the "([^"]+)" metrics endpoint accepts a TLS 1\.3 connection$`, MetricsEndpointAcceptsTLS13)
sc.Step(`^(?i)the "([^"]+)" metrics endpoint rejects a TLS 1\.2 connection$`, MetricsEndpointRejectsTLS12)
sc.Step(`^(?i)the "([^"]+)" metrics endpoint negotiates cipher "([^"]+)" over TLS 1\.2$`, MetricsEndpointNegotiatesTLS12Cipher)
sc.Step(`^(?i)the "([^"]+)" metrics endpoint rejects a TLS 1\.2 connection offering only cipher "([^"]+)"$`, MetricsEndpointRejectsTLS12ConnectionWithCipher)
sc.Step(`^(?i)the "([^"]+)" metrics endpoint accepts a TLS 1\.2 connection with cipher "([^"]+)" and curve "([^"]+)"$`, MetricsEndpointAcceptsTLS12ConnectionWithCurve)
sc.Step(`^(?i)the "([^"]+)" metrics endpoint rejects a TLS 1\.2 connection with cipher "([^"]+)" and only curve "([^"]+)"$`, MetricsEndpointRejectsTLS12ConnectionWithCurve)
sc.Step(`^(?i)the "([^"]+)" metrics endpoint accepts a connection using only curve "([^"]+)"$`, MetricsEndpointAcceptsConnectionUsingCurve)
sc.Step(`^(?i)the "([^"]+)" metrics endpoint rejects a connection using only curve "([^"]+)"$`, MetricsEndpointRejectsConnectionUsingOnlyCurve)
sc.Step(`^(?i)the "([^"]+)" component is configured with HTTPS_PROXY "([^"]+)"$`, ConfigureDeploymentWithHTTPSProxy)
sc.Step(`^(?i)the "([^"]+)" component is configured with HTTPS_PROXY pointing to a recording proxy$`, StartRecordingProxyAndConfigureDeployment)
sc.Step(`^(?i)the recording proxy received a CONNECT request for the catalogd service$`, RecordingProxyReceivedCONNECTForCatalogd)
sc.Step(`^(?i)the catalogd leader pod is force-deleted$`, CatalogdLeaderPodIsForceDeleted)
sc.Step(`^(?i)a new catalogd leader is elected$`, NewCatalogdLeaderIsElected)
}
Loading
Loading