Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 43 additions & 21 deletions test/e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ non-technical stakeholders.

## Project Structure

Component step packages are introduced incrementally. The layout below shows
their intended ownership after all component extractions land.

```
test/
├── e2e/
Expand All @@ -33,8 +36,12 @@ test/
└── internal/
├── e2e/
│ ├── runner.go # Godog options, scenario selection, summary wiring
│ ├── steps/ # All existing step definitions and hooks
│ ├── steps/ # Shared world, hooks, catalog, resource, RBAC, and condition helpers
│ │ └── testdata/ # RBAC and resource templates
│ ├── operator-controller/steps/ # ClusterExtension lifecycle and integration assertions
│ ├── catalogd/steps/ # HA/proxy/TLS steps and restoration hooks
│ ├── object-controller/steps/ # ClusterObjectSet/Deployment assertions
│ ├── platform/steps/ # Demo steps and recording
│ └── support-bundle.yaml # Shared diagnostics
└── summary/ # Summary generation and embedded templates
```
Expand All @@ -43,7 +50,7 @@ test/

### 1. Test Runner (`operator-controller/suite_test.go`)

The component entry point registers the full shared step set and hooks, then calls
The component entry point registers shared steps, hooks, and its component steps, then calls
`e2e.Run` from `test/internal/e2e` with its feature directory. The shared runner configures Godog options,
scenario selection, and summary generation.

Expand All @@ -53,10 +60,22 @@ controller unit tests live under `internal/testutil`, which is accessible throug
the repository.

Features follow the ownership in the [per-component suite design](../../docs/designs/e2e-per-component.md).
Each component has a thin entry point that still registers the full shared step set;
step ownership is separated in the next stage. The newer `cos_*`, `cod_*`, and
`group` features belong to object-controller. Feature contents and step behavior
are unchanged, including ClusterExtension integration assertions in object features.
The newer `cos_*`, `cod_*`, and `group` features belong to object-controller.
Suites register additional component steps when existing features contain integration
assertions: operator-controller uses object assertions, object-controller retains
ClusterExtension integration coverage, and catalogd proxy tests use ClusterExtension
steps. Platform registers ClusterExtension steps for demos. Feature contents,
scenario names, tags, and assertions are unchanged.

`test/internal/e2e/steps` owns `World` and the `Scenario(ctx)` accessor, generic resource and
condition helpers, catalog/registry prerequisites, RBAC templates, and cleanup.
`ListClusterObjectSetsInGroup` and `ResolveObjectRef` remain shared because
both ClusterExtension and ClusterObjectSet assertions consume them.
Catalogd owns deployment restoration and proxy state. Platform owns recording;
shared helpers observe commands through an optional `CommandRecorder` interface.
Pass extra lifecycle registration to `sharedsteps.RegisterHooks` to keep suite hooks
between shared scenario setup and cleanup. The upgrade helpers remain shared because
catalogd HA scenarios also consume component readiness and catalog reconciliation steps.

### 2. Feature Files (`operator-controller/features/*.feature`)

Expand Down Expand Up @@ -112,7 +131,7 @@ Feature: Install ClusterExtension
And ClusterExtension is available
```

### 3. Step Definitions (`test/internal/e2e/steps/steps.go`)
### 3. Step Definitions (`test/internal/e2e/<component>/steps/steps.go`)

Go functions that implement the steps defined in feature files. Each step is registered with a regex pattern that
matches the Gherkin text.
Expand All @@ -121,9 +140,8 @@ matches the Gherkin text.

```go
func RegisterSteps(sc *godog.ScenarioContext) {
sc.Step(`^OLM is available$`, OLMisAvailable)
sc.Step(`^bundle "([^"]+)" is installed in version "([^"]+)"$`, BundleInstalled)
sc.Step(`^ClusterExtension is applied$`, ResourceIsApplied)
sc.Step(`^ClusterExtension is applied$`, shared.ResourceIsApplied)
// ... more steps
}
```
Expand All @@ -132,7 +150,7 @@ sc.Step(`^ClusterExtension is applied$`, ResourceIsApplied)

Status assertions must read metadata and status from the same API response and
require the asserted condition's `observedGeneration` to equal `metadata.generation`.
Use `waitForCondition` for condition checks. Bundle metadata uses the `Installed`
Use `shared.WaitForCondition` for condition checks. Bundle metadata uses the `Installed`
condition, and COS phase digests and object counts use the `Ready` condition to
verify that the status reflects the current spec. Deployments instead expose
`status.observedGeneration` at the resource level.
Expand All @@ -146,18 +164,18 @@ That step accepts any current-generation `Progressing` status or reason, whereas

```go
func BundleInstalled(ctx context.Context, name, version string) error {
sc := scenarioCtx(ctx)
waitFor(ctx, func() bool {
v, err := k8sClient(ctx, "get", "clusterextension", sc.clusterExtensionName, "-o", "json")
sc := shared.Scenario(ctx)
shared.WaitFor(ctx, func() bool {
v, err := shared.K8sClient(ctx, "get", "clusterextension", sc.ClusterExtensionName, "-o", "json")
if err != nil {
return false
}
var ext ocv1.ClusterExtension
if err := json.Unmarshal([]byte(v), &ext); err != nil {
return false
}
installed := currentCondition(ext.Generation, ext.Status.Conditions, ocv1.TypeInstalled)
return conditionMatches(installed, metav1.ConditionTrue, nil, nil) &&
installed := shared.CurrentCondition(ext.Generation, ext.Status.Conditions, ocv1.TypeInstalled)
return shared.ConditionMatches(installed, metav1.ConditionTrue, nil, nil) &&
ext.Status.Install.Bundle.Name == name && ext.Status.Install.Bundle.Version == version
})
return nil
Expand Down Expand Up @@ -199,7 +217,7 @@ Feature: Your Feature Name

### 2. Implement Step Definitions

Add step implementations in `test/internal/e2e/steps/steps.go`:
Add component assertions in `test/internal/e2e/<component>/steps/`; add reusable prerequisites and helpers in `test/internal/e2e/steps/`:

```go
func RegisterSteps(sc *godog.ScenarioContext) {
Expand All @@ -208,7 +226,7 @@ func RegisterSteps(sc *godog.ScenarioContext) {
}

func YourStepFunction(ctx context.Context, param string) error {
sc := scenarioCtx(ctx)
sc := shared.Scenario(ctx)
// Implementation
return nil
}
Expand All @@ -230,7 +248,7 @@ Use these variables in YAML templates:
- `${NAME}`: Scenario-specific ClusterExtension name (e.g., `ce-123`)
- `${COS_NAME}`: Scenario-specific ClusterObjectSet name (e.g., `cos-123`; for applying ClusterObjectSets directly)
- `${TEST_NAMESPACE}`: Scenario-specific namespace (e.g., `ns-123`)
- `${SCENARIO_ID}`: Unique scenario identifier used for resource name isolation
- `${SCENARIO_ID}`: Feature-scoped scenario identifier used for resource isolation across concurrent Go suites
- `${PACKAGE:<name>}`: Parameterized package name (e.g., `${PACKAGE:test}` expands to `test-<scenario-id>`)
- `${CATALOG:<name>}`: Catalog resource name (e.g., `${CATALOG:test}` expands to `test-catalog-<scenario-id>`)

Expand Down Expand Up @@ -427,8 +445,12 @@ go test -count=1 ./test/e2e/operator-controller/... -args --log.debug --k8s.cli=

Each scenario runs in its own namespace with unique resource names, ensuring complete isolation:

- Namespace: `ns-{scenario-id}`
- ClusterExtension: `ce-{scenario-id}`
- Namespace: `ns-{feature-hash}-{godog-id}`
- ClusterExtension: `ce-{feature-hash}-{godog-id}`

Godog restarts numeric IDs in each process. The feature hash prevents collisions
between concurrent component suites and stays stable when a feature moves. Feature
and scenario annotations retain their existing values.

### 2. Test-Identifying Annotations

Expand Down Expand Up @@ -476,7 +498,7 @@ When ClusterExtension is applied
All asynchronous operations use `waitFor` with consistent timeout (300s) and tick (1s):

```go
waitFor(ctx, func() bool {
shared.WaitFor(ctx, func() bool {
// Check condition
return conditionMet
})
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/features_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,6 @@ func TestMain(m *testing.M) {
}, func(sc *godog.ScenarioContext) {
steps.RegisterSteps(sc)
steps.RegisterDemoSteps(sc)
steps.RegisterHooks(sc)
steps.RegisterHooks(sc, steps.RegisterAsciiCastHooks)
}))
}
2 changes: 1 addition & 1 deletion test/e2e/platform/suite_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@ func TestMain(m *testing.M) {
}, func(sc *godog.ScenarioContext) {
steps.RegisterSteps(sc)
steps.RegisterDemoSteps(sc)
steps.RegisterHooks(sc)
steps.RegisterHooks(sc, steps.RegisterAsciiCastHooks)
}))
}
2 changes: 1 addition & 1 deletion test/internal/e2e/steps/asciicast_recorder.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import (
type recorderContextKey struct{}

func WithRecorder(ctx context.Context, rec *AsciiCastRecorder) context.Context {
return context.WithValue(ctx, recorderContextKey{}, rec)
return context.WithValue(WithCommandRecorder(ctx, rec), recorderContextKey{}, rec)
}

func RecorderFromContext(ctx context.Context) *AsciiCastRecorder {
Expand Down
14 changes: 7 additions & 7 deletions test/internal/e2e/steps/conditions.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ type resourceConditions struct {
} `json:"status"`
}

func readResourceConditions(ctx context.Context, resourceType, resourceName string) (*resourceConditions, error) {
out, err := k8sClient(ctx, "get", resourceType, resourceName, "-o", "json")
func ReadResourceConditions(ctx context.Context, resourceType, resourceName string) (*resourceConditions, error) {
out, err := K8sClient(ctx, "get", resourceType, resourceName, "-o", "json")
if err != nil {
return nil, fmt.Errorf("reading %s %q: %w", resourceType, resourceName, err)
}
Expand All @@ -30,21 +30,21 @@ func readResourceConditions(ctx context.Context, resourceType, resourceName stri
return &resource, nil
}

func generationObserved(generation, observedGeneration int64) bool {
func GenerationObserved(generation, observedGeneration int64) bool {
return generation > 0 && observedGeneration == generation
}

// currentCondition returns only the requested condition, and only if it reflects
// CurrentCondition returns only the requested condition, and only if it reflects
// the current generation. An unrelated up-to-date condition is not sufficient.
func currentCondition(generation int64, conditions []metav1.Condition, conditionType string) *metav1.Condition {
func CurrentCondition(generation int64, conditions []metav1.Condition, conditionType string) *metav1.Condition {
condition := apimeta.FindStatusCondition(conditions, conditionType)
if condition == nil || !generationObserved(generation, condition.ObservedGeneration) {
if condition == nil || !GenerationObserved(generation, condition.ObservedGeneration) {
return nil
}
return condition
}

func conditionMatches(condition *metav1.Condition, conditionStatus metav1.ConditionStatus, reason *string, msgCmp msgMatchFn) bool {
func ConditionMatches(condition *metav1.Condition, conditionStatus metav1.ConditionStatus, reason *string, msgCmp MsgMatchFn) bool {
return condition != nil && condition.Status == conditionStatus &&
(reason == nil || condition.Reason == *reason) &&
(msgCmp == nil || msgCmp(condition.Message))
Expand Down
Loading
Loading