Skip to content

OCPBUGS-105193: extract HCCO webhook validation into a dedicated controller - #9239

Merged
openshift-merge-bot[bot] merged 2 commits into
openshift:mainfrom
bryan-cox:OCPBUGS-105193
Aug 7, 2026
Merged

OCPBUGS-105193: extract HCCO webhook validation into a dedicated controller#9239
openshift-merge-bot[bot] merged 2 commits into
openshift:mainfrom
bryan-cox:OCPBUGS-105193

Conversation

@bryan-cox

@bryan-cox bryan-cox commented Aug 6, 2026

Copy link
Copy Markdown
Member

What this PR does / why we need it:

Extracts ensureGuestAdmissionWebhooksAreValid() from the monolithic HCCO resources controller into a dedicated webhook-validation controller. The [Feature:WebhookValidation] e2e test has a 47% pass rate on e2e-v2-azure-self-managed because webhook validation runs at the tail of a 15+ sub-reconciler chain, causing 60+ second delays between webhook creation and deletion.

The new controller watches ValidatingWebhookConfiguration and MutatingWebhookConfiguration directly, so reconciliation triggers immediately on webhook events. It encodes the webhook type in the request namespace field so Reconcile() targets only the type that fired, avoiding a redundant Get for the other kind.

Which issue(s) this PR fixes:

Fixes https://issues.redhat.com/browse/OCPBUGS-105193

Special notes for your reviewer:

  • The HCCO manager cache already includes VWC and MWC with labels.Everything() (no label filter) — see operator/config.go:129-130 — so the resources controller retains its watches for ensureResourceCreationIsBlocked
  • Two commits: (1) the extraction refactor, (2) e2e timeout bump (separate rationale)
  • 13 unit tests: 7 table-driven reconcile cases + 2 error-path tests + 4 URL validation tests

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Always review AI generated responses prior to use.
Generated with Claude Code via openshift-developer plugin

Summary by CodeRabbit

  • Bug Fixes

    • Automatically removes admission webhooks that target disallowed hosted cluster control-plane services.
    • Preserves allowed external targets and direct service references.
    • Improves handling of missing resources and transient errors.
    • Revalidates webhook configurations when related services or configurations change.
  • Tests

    • Added coverage for webhook filtering, cleanup, preservation, and error scenarios.
    • Increased the end-to-end cleanup wait time to improve reliability.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 6, 2026
@openshift-ci

openshift-ci Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Aug 6, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@bryan-cox: This pull request references Jira Issue OCPBUGS-105193, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

What this PR does / why we need it:

Extracts ensureGuestAdmissionWebhooksAreValid() from the monolithic HCCO resources controller into a dedicated webhook-validation controller. The [Feature:WebhookValidation] e2e test has a 47% pass rate on e2e-v2-azure-self-managed because webhook validation runs at the tail of a 15+ sub-reconciler chain, causing 60+ second delays between webhook creation and deletion.

The new controller watches ValidatingWebhookConfiguration and MutatingWebhookConfiguration directly, so reconciliation triggers immediately on webhook events. It encodes the webhook type in the request namespace field so Reconcile() targets only the type that fired, avoiding a redundant Get for the other kind.

Which issue(s) this PR fixes:

Fixes https://issues.redhat.com/browse/OCPBUGS-105193

Special notes for your reviewer:

  • The HCCO manager cache already includes VWC and MWC with labels.Everything() (no label filter) — see operator/config.go:129-130 — so the resources controller retains its watches for ensureResourceCreationIsBlocked
  • Two commits: (1) the extraction refactor, (2) e2e timeout bump (separate rationale)
  • 13 unit tests: 7 table-driven reconcile cases + 2 error-path tests + 4 URL validation tests

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Always review AI generated responses prior to use.
Generated with Claude Code via openshift-developer plugin

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label Aug 6, 2026
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The operator now registers a dedicated webhook validation controller. The controller watches validating and mutating webhook configurations and control-plane Services. It builds disallowed service URLs and deletes matching guest webhook configurations. Storage reconciliation no longer performs this validation. Tests cover filtering, deletion, preservation, not-found handling, and error propagation. The end-to-end test waits up to three minutes for deletion.

Sequence Diagram(s)

sequenceDiagram
  participant WebhookConfiguration
  participant WebhookValidationController
  participant ControlPlaneServices
  participant GuestWebhookConfiguration
  WebhookConfiguration->>WebhookValidationController: trigger webhook event
  WebhookValidationController->>ControlPlaneServices: list services and build disallowed URLs
  WebhookValidationController->>GuestWebhookConfiguration: get configuration
  WebhookValidationController->>GuestWebhookConfiguration: delete configuration when a URL matches
Loading

Possibly related PRs


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error)

Check name Status Explanation Resolution
No-Sensitive-Data-In-Logs ❌ Error The new controller logs the full webhook URL in disallowed_url; this can expose internal service hostnames and other URL-embedded sensitive data. Do not log the full URL. Log only the webhook type and name, or redact and strictly sanitize the URL before logging.
✅ Passed checks (10 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: moving HCCO webhook validation into a dedicated controller.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The changed Ginkgo test keeps the static title "should be automatically deleted"; the PR only changes its timeout. New unit tests use static t.Run names, not Ginkgo titles.
Test Structure And Quality ✅ Passed The PR adds no Ginkgo It blocks; its only Ginkgo change sets a 3-minute Eventually timeout, and the created webhook already uses DeferCleanup.
Topology-Aware Scheduling Compatibility ✅ Passed The PR adds a controller that watches Services and webhook configurations, but adds no pod scheduling fields, replica logic, PDBs, affinity, topology spread, selectors, or tolerations.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The PR adds no new Ginkgo e2e test; its only e2e change increases an existing timeout. New tests are unit tests using fake clients and contain no external connectivity or IPv4-only assumptions.
No-Weak-Crypto ✅ Passed The complete PR diff adds no MD5, SHA1, DES, RC4, Blowfish, ECB, crypto APIs, custom crypto, or secret/token comparisons; new code only filters webhook URLs.
Container-Privileges ✅ Passed The PR adds no container or Kubernetes manifest privilege settings. No added privileged, hostPID, hostNetwork, hostIPC, SYS_ADMIN, or allowPrivilegeEscalation fields were found.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@openshift-ci

openshift-ci Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bryan-cox

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added approved Indicates a PR has been approved by an approver from all required OWNERS files. area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release area/testing Indicates the PR includes changes for e2e testing and removed do-not-merge/needs-area labels Aug 6, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@bryan-cox: This pull request references Jira Issue OCPBUGS-105193, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
Details

In response to this:

What this PR does / why we need it:

Extracts ensureGuestAdmissionWebhooksAreValid() from the monolithic HCCO resources controller into a dedicated webhook-validation controller. The [Feature:WebhookValidation] e2e test has a 47% pass rate on e2e-v2-azure-self-managed because webhook validation runs at the tail of a 15+ sub-reconciler chain, causing 60+ second delays between webhook creation and deletion.

The new controller watches ValidatingWebhookConfiguration and MutatingWebhookConfiguration directly, so reconciliation triggers immediately on webhook events. It encodes the webhook type in the request namespace field so Reconcile() targets only the type that fired, avoiding a redundant Get for the other kind.

Which issue(s) this PR fixes:

Fixes https://issues.redhat.com/browse/OCPBUGS-105193

Special notes for your reviewer:

  • The HCCO manager cache already includes VWC and MWC with labels.Everything() (no label filter) — see operator/config.go:129-130 — so the resources controller retains its watches for ensureResourceCreationIsBlocked
  • Two commits: (1) the extraction refactor, (2) e2e timeout bump (separate rationale)
  • 13 unit tests: 7 table-driven reconcile cases + 2 error-path tests + 4 URL validation tests

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Always review AI generated responses prior to use.
Generated with Claude Code via openshift-developer plugin

Summary by CodeRabbit

  • Bug Fixes

  • Added validation to remove disallowed admission webhooks targeting hosted cluster control-plane services.

  • Preserved allowed external targets and direct service references.

  • Improved handling of missing resources and transient errors.

  • Tests

  • Added comprehensive coverage for webhook filtering, cleanup, and error scenarios.

  • Increased the end-to-end cleanup wait time to improve reliability.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (3)
control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation.go (2)

67-71: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider an explicit type discriminator instead of req.Namespace.

Reconcile reads the webhook kind from req.Namespace. Both watched resources are cluster-scoped, so the field is unused and the encoding works. However, the contract is implicit and lives in setup.go. A reader of Reconcile alone cannot tell that Namespace is a kind selector.

An alternative is one controller instance per webhook kind, each with its own reconciler holding a webhookType field. That removes the map lookup and the silent no-op path for unknown values.

This is optional. The current code is documented by the comment on Lines 21-23.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation.go`
around lines 67 - 71, Optionally refactor Reconcile to use an explicit
webhookType field on each reconciler instance, configured by setup.go with one
controller per webhook kind. Replace the webhookTypesByName lookup and
unknown-namespace no-op path with direct use of the reconciler’s webhookType,
while preserving existing reconciliation behavior.

122-129: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

Substring matching on the bare service name can over-match.

buildDisallowedURLs adds https://<serviceName> with no delimiter. isAllowedWebhookURL then uses strings.Contains. A hosted-cluster webhook that points at an unrelated external host whose name begins with a control-plane service name is deleted. Example: a control-plane Service named etcd-client makes https://etcd-client.corp.example.com disallowed.

This logic appears to be carried over from the previous implementation, so it is likely pre-existing and out of scope for this PR. Confirm that the intent is a prefix-with-delimiter match rather than a plain substring match.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation.go`
around lines 122 - 129, Update isAllowedWebhookURL to avoid plain substring
matching against bare service names; match only URLs where the service-name
prefix is followed by an appropriate delimiter or otherwise represents the
intended host. Preserve allowing unrelated external hosts such as
etcd-client.corp.example.com, and keep buildDisallowedURLs’ existing inputs
unchanged.
control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation_test.go (1)

255-268: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add table cases for the remaining URL forms and the unknown webhook type.

The current cases exercise only the bare https://<serviceName> form. buildDisallowedURLs also produces https://<name>.<ns>.svc and https://<name>.<ns>.svc.cluster.local. Those two forms are only covered indirectly through TestIsAllowedWebhookURL, which bypasses buildDisallowedURLs.

Two gaps remain:

  1. No case uses a webhook URL in the .svc or .svc.cluster.local form.
  2. No case supplies an unrecognized value in req.Namespace, so the early return at webhookvalidation.go Lines 68-71 is untested.

A third gap is the Delete error path in reconcileWebhook. An interceptor.Funcs{Delete: ...} test would mirror the existing Get and List error tests.

🧪 Proposed additional table cases
 		{
 			name:        "When webhook config does not exist, it should return without error",
 			webhookType: validatingType.name,
 			cpServices: []corev1.Service{
 				{
 					ObjectMeta: metav1.ObjectMeta{
 						Name:      "etcd-client",
 						Namespace: hcpNamespace,
 					},
 				},
 			},
 			guestObjects:  []client.Object{},
 			reconcileName: "nonexistent-webhook",
 		},
+		{
+			name:        "When validating webhook targets a CP service by cluster-local FQDN, it should delete the webhook",
+			webhookType: validatingType.name,
+			cpServices: []corev1.Service{
+				{
+					ObjectMeta: metav1.ObjectMeta{
+						Name:      "etcd-client",
+						Namespace: hcpNamespace,
+					},
+				},
+			},
+			guestObjects: []client.Object{
+				&admissionregistrationv1.ValidatingWebhookConfiguration{
+					ObjectMeta: metav1.ObjectMeta{Name: "fqdn-validating-webhook"},
+					Webhooks: []admissionregistrationv1.ValidatingWebhook{
+						{
+							Name: "fqdn.webhook.io",
+							ClientConfig: admissionregistrationv1.WebhookClientConfig{
+								URL: ptr.To("https://etcd-client." + hcpNamespace + ".svc.cluster.local:2379"),
+							},
+						},
+					},
+				},
+			},
+			reconcileName:     "fqdn-validating-webhook",
+			expectWebhookGone: true,
+		},
+		{
+			name:        "When the request carries an unknown webhook type, it should preserve the webhook",
+			webhookType: "unknown",
+			cpServices: []corev1.Service{
+				{
+					ObjectMeta: metav1.ObjectMeta{
+						Name:      "etcd-client",
+						Namespace: hcpNamespace,
+					},
+				},
+			},
+			guestObjects: []client.Object{
+				&admissionregistrationv1.ValidatingWebhookConfiguration{
+					ObjectMeta: metav1.ObjectMeta{Name: "untouched-webhook"},
+					Webhooks: []admissionregistrationv1.ValidatingWebhook{
+						{
+							Name:         "untouched.webhook.io",
+							ClientConfig: admissionregistrationv1.WebhookClientConfig{URL: ptr.To("https://etcd-client:2379")},
+						},
+					},
+				},
+			},
+			reconcileName: "untouched-webhook",
+		},

The unknown-type case needs the assertion helpers to resolve a concrete type. Either assert inline for that case or default wtName to validatingType.name in assertWebhookExists.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation_test.go`
around lines 255 - 268, Extend the webhook reconciliation table tests to cover
both https://<name>.<ns>.svc and https://<name>.<ns>.svc.cluster.local URLs,
plus an unrecognized req.Namespace value that exercises the early return. Add a
reconcileWebhook case using interceptor.Funcs.Delete to verify Delete errors.
Update assertWebhookExists or the unknown-type assertion so it resolves a
concrete webhook type, defaulting wtName to validatingType.name where
appropriate.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@control-plane-operator/hostedclusterconfigoperator/cmd.go`:
- Line 78: The controller list used for IBMCloud must include
webhook-validation. Update the explicit controller list in the hosted control
plane deployment configuration to include the webhookvalidation controller, and
regenerate or update the corresponding IBMCloud fixture while preserving the
existing controller entries.

In
`@control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/setup.go`:
- Around line 33-39: Extend the controller setup around the existing
ValidatingWebhookConfiguration and MutatingWebhookConfiguration watches to also
watch control-plane Service events through opts.CPCluster.GetCache(). Enqueue
affected webhook configurations when matching Services change, ensuring
buildDisallowedURLs is reevaluated without waiting for the long cache resync;
alternatively, return a bounded RequeueAfter.

---

Nitpick comments:
In
`@control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation_test.go`:
- Around line 255-268: Extend the webhook reconciliation table tests to cover
both https://<name>.<ns>.svc and https://<name>.<ns>.svc.cluster.local URLs,
plus an unrecognized req.Namespace value that exercises the early return. Add a
reconcileWebhook case using interceptor.Funcs.Delete to verify Delete errors.
Update assertWebhookExists or the unknown-type assertion so it resolves a
concrete webhook type, defaulting wtName to validatingType.name where
appropriate.

In
`@control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation.go`:
- Around line 67-71: Optionally refactor Reconcile to use an explicit
webhookType field on each reconciler instance, configured by setup.go with one
controller per webhook kind. Replace the webhookTypesByName lookup and
unknown-namespace no-op path with direct use of the reconciler’s webhookType,
while preserving existing reconciliation behavior.
- Around line 122-129: Update isAllowedWebhookURL to avoid plain substring
matching against bare service names; match only URLs where the service-name
prefix is followed by an appropriate delimiter or otherwise represents the
intended host. Preserve allowing unrelated external hosts such as
etcd-client.corp.example.com, and keep buildDisallowedURLs’ existing inputs
unchanged.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: df0f3c87-a67f-41bf-a459-cc133329b083

📥 Commits

Reviewing files that changed from the base of the PR and between 1d48188 and 81a7168.

📒 Files selected for processing (7)
  • control-plane-operator/hostedclusterconfigoperator/cmd.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/resources/resources.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/resources/resources_test.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/setup.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation_test.go
  • test/e2e/v2/tests/hosted_cluster_security_test.go
💤 Files with no reviewable changes (2)
  • control-plane-operator/hostedclusterconfigoperator/controllers/resources/resources.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/resources/resources_test.go

Comment thread control-plane-operator/hostedclusterconfigoperator/cmd.go
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/setup.go`:
- Around line 67-95: Update serviceToWebhookHandler to enqueue a fixed
reconciliation request for Service events instead of listing webhook
configurations in the map function. Move both ValidatingWebhookConfigurationList
and MutatingWebhookConfigurationList operations into Reconcile, return either
guestClient.List error, and enqueue each discovered webhook configuration there
so transient failures trigger reconciliation retries.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 6c6c2c14-42fd-4b62-ac96-79317ed691d3

📥 Commits

Reviewing files that changed from the base of the PR and between 1d48188 and b898189.

📒 Files selected for processing (7)
  • control-plane-operator/hostedclusterconfigoperator/cmd.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/resources/resources.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/resources/resources_test.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/setup.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation_test.go
  • test/e2e/v2/tests/hosted_cluster_security_test.go
💤 Files with no reviewable changes (2)
  • control-plane-operator/hostedclusterconfigoperator/controllers/resources/resources.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/resources/resources_test.go
🚧 Files skipped from review as they are similar to previous changes (4)
  • test/e2e/v2/tests/hosted_cluster_security_test.go
  • control-plane-operator/hostedclusterconfigoperator/cmd.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation_test.go

@codecov

codecov Bot commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 66.40625% with 43 lines in your changes missing coverage. Please review.
✅ Project coverage is 44.98%. Comparing base (1d48188) to head (c9cb6d2).
⚠️ Report is 33 commits behind head on main.

Files with missing lines Patch % Lines
...figoperator/controllers/webhookvalidation/setup.go 0.00% 37 Missing ⚠️
...controllers/webhookvalidation/webhookvalidation.go 93.40% 4 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9239      +/-   ##
==========================================
+ Coverage   44.96%   44.98%   +0.01%     
==========================================
  Files         778      780       +2     
  Lines       97444    97528      +84     
==========================================
+ Hits        43819    43870      +51     
- Misses      50602    50636      +34     
+ Partials     3023     3022       -1     
Files with missing lines Coverage Δ
...-plane-operator/hostedclusterconfigoperator/cmd.go 0.00% <ø> (ø)
...rconfigoperator/controllers/resources/resources.go 57.54% <ø> (-0.30%) ⬇️
...controllers/webhookvalidation/webhookvalidation.go 93.40% <93.40%> (ø)
...figoperator/controllers/webhookvalidation/setup.go 0.00% <0.00%> (ø)
Flag Coverage Δ
cmd-support 38.64% <ø> (ø)
cpo-hostedcontrolplane 47.24% <ø> (ø)
cpo-other 45.76% <66.40%> (+0.09%) ⬆️
hypershift-operator 55.00% <ø> (ø)
other 34.32% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation_test.go (2)

39-80: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Add a host-boundary case.

The table covers exact and FQDN matches but not a host that only begins with a Service name. Add a case for https://etcd-client-external.example.com against https://etcd-client. With the current strings.Contains implementation that case returns false, which documents the over-broad matching flagged in webhookvalidation.go.

💚 Proposed test case
 		{
 			name:           "When disallowed list is empty, it should return true",
 			disallowedURLs: []string{},
 			url:            "https://anything",
 			expected:       true,
 		},
+		{
+			name:           "When URL host only shares a prefix with a disallowed service name, it should return true",
+			disallowedURLs: []string{"https://etcd-client"},
+			url:            "https://etcd-client-external.example.com",
+			expected:       true,
+		},
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation_test.go`
around lines 39 - 80, Add a table-driven case to TestIsAllowedWebhookURL where
disallowedURLs contains “https://etcd-client” and the URL is
“https://etcd-client-external.example.com”, expecting true. Keep the case
focused on verifying that a hostname merely beginning with the disallowed
service name is allowed.

357-366: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add coverage for aggregated per-webhook errors.

The sentinel table covers a failing List but not a failing Delete during bulk reconciliation. reconcileAllWebhooks collects per-webhook errors with errors.Join and continues the loop. Add a case with two disallowed configurations and a Delete interceptor that fails for one of them. Assert that the returned error mentions the failing configuration and that the other configuration is deleted.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation_test.go`
around lines 357 - 366, The webhook validation tests need coverage for
aggregated Delete errors during bulk reconciliation. Add a case with two
disallowed configurations, make the Delete interceptor fail for exactly one
configuration, and verify reconcileAllWebhooks returns an error mentioning that
configuration while confirming the other configuration is deleted.

Source: Coding guidelines

control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/setup.go (1)

47-52: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value

Consider a predicate on the control-plane Service watch.

This watch has no predicate. Every Service event in the HCP namespace, including status-only updates and periodic cache resyncs, enqueues two sentinel requests. Each sentinel lists all webhook configurations of one type. Only Service creation, deletion, name changes, and changes to the AllowGuestWebhooksServiceLabel label affect the disallowed set.

Add a predicate that admits create and delete events and admits update events only when the label set changes.

♻️ Proposed predicate
+	servicePredicate := predicate.Funcs{
+		UpdateFunc: func(e event.UpdateEvent) bool {
+			_, oldAllowed := e.ObjectOld.GetLabels()[hyperv1.AllowGuestWebhooksServiceLabel]
+			_, newAllowed := e.ObjectNew.GetLabels()[hyperv1.AllowGuestWebhooksServiceLabel]
+			return oldAllowed != newAllowed
+		},
+	}
-	if err := c.Watch(source.Kind[client.Object](opts.CPCluster.GetCache(), &corev1.Service{}, serviceEventHandler())); err != nil {
+	if err := c.Watch(source.Kind[client.Object](opts.CPCluster.GetCache(), &corev1.Service{}, serviceEventHandler(), servicePredicate)); err != nil {
 		return fmt.Errorf("failed to watch control plane Services: %w", err)
 	}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/setup.go`
around lines 47 - 52, Update the Service watch in the setup flow around
serviceEventHandler to apply a predicate that accepts all create and delete
events, while accepting update events only when the
AllowGuestWebhooksServiceLabel value changes; reject status-only updates and
resync events.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation.go`:
- Around line 142-168: The URL matching in isAllowedWebhookURL is over-broad:
update buildDisallowedURLs and isAllowedWebhookURL in
control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation.go
(lines 142-168) to return bare hostnames, parse each webhook URL, and compare
url.Hostname() for exact equality instead of using strings.Contains. In
control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation_test.go
(lines 39-80), add coverage confirming https://etcd-client-external.example.com
remains allowed when etcd-client is disallowed, and revise the existing
substring case for hostname equality semantics.

---

Nitpick comments:
In
`@control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/setup.go`:
- Around line 47-52: Update the Service watch in the setup flow around
serviceEventHandler to apply a predicate that accepts all create and delete
events, while accepting update events only when the
AllowGuestWebhooksServiceLabel value changes; reject status-only updates and
resync events.

In
`@control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation_test.go`:
- Around line 39-80: Add a table-driven case to TestIsAllowedWebhookURL where
disallowedURLs contains “https://etcd-client” and the URL is
“https://etcd-client-external.example.com”, expecting true. Keep the case
focused on verifying that a hostname merely beginning with the disallowed
service name is allowed.
- Around line 357-366: The webhook validation tests need coverage for aggregated
Delete errors during bulk reconciliation. Add a case with two disallowed
configurations, make the Delete interceptor fail for exactly one configuration,
and verify reconcileAllWebhooks returns an error mentioning that configuration
while confirming the other configuration is deleted.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: ca483eb0-318f-43f7-b03c-22ddcf10627b

📥 Commits

Reviewing files that changed from the base of the PR and between c183e79 and 56b9d36.

📒 Files selected for processing (7)
  • control-plane-operator/hostedclusterconfigoperator/cmd.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/resources/resources.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/resources/resources_test.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/setup.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/webhookvalidation/webhookvalidation_test.go
  • test/e2e/v2/tests/hosted_cluster_security_test.go
💤 Files with no reviewable changes (2)
  • control-plane-operator/hostedclusterconfigoperator/controllers/resources/resources.go
  • control-plane-operator/hostedclusterconfigoperator/controllers/resources/resources_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • test/e2e/v2/tests/hosted_cluster_security_test.go
  • control-plane-operator/hostedclusterconfigoperator/cmd.go

bryan-cox and others added 2 commits August 6, 2026 07:40
Move webhook validation logic from the monolithic HCCO resources
controller into its own webhookvalidation package with dedicated
watches, reconciler, and tests.

Service events now enqueue sentinel requests so that webhook config
listing happens inside Reconcile where errors are retried by the
work queue, rather than silently discarding list failures in the
event handler.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@bryan-cox

Copy link
Copy Markdown
Member Author

Local Testing

Tested locally against an AWS dev cluster with a hosted cluster (brcox-sm-dev-hc).

Setup

  1. Build a custom CPO image with the changes:

    quay.io/rh_ee_brcox/hypershift:OCPBUGS-105193-2026-08-06-1
    
  2. Override the CPO image on the hosted cluster:

    KUBECONFIG=<mgmt-kubeconfig> kubectl -n clusters annotate hc <hc-name> \
      hypershift.openshift.io/control-plane-operator-image=quay.io/rh_ee_brcox/hypershift:OCPBUGS-105193-2026-08-06-1 \
      --overwrite
  3. Wait for KAS and HCCO rollouts to complete.

E2E Test

Ran the [Feature:WebhookValidation] e2e test 10 times — all passed:

KUBECONFIG=<mgmt-kubeconfig> \
  E2E_HOSTED_CLUSTER_NAME=<hc-name> \
  E2E_HOSTED_CLUSTER_NAMESPACE=clusters \
  bin/test-e2e-v2 --ginkgo.focus='\[Feature:WebhookValidation\]' -test.v

Results: 10/10 passes, each completing in 0.40–0.48 seconds. Previously this test had a 47% pass rate with 60+ second delays waiting for the monolithic resources controller to cycle.

HCCO Logs

Confirmed the new controller registers at startup and all three watches are active:

setting up controller  controller="webhook-validation"
Starting EventSource   controller="webhook-validation"  source="kind source: *v1.Service"
Starting EventSource   controller="webhook-validation"  source="kind source: *v1.ValidatingWebhookConfiguration"
Starting EventSource   controller="webhook-validation"  source="kind source: *v1.MutatingWebhookConfiguration"

The resources controller continues reconciling normally alongside the new controller — no regressions observed.

@bryan-cox
bryan-cox marked this pull request as ready for review August 6, 2026 13:04
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 6, 2026
@openshift-ci
openshift-ci Bot requested review from devguyio and ironcladlou August 6, 2026 13:04
}

// Webhook configs are cluster-scoped so Namespace is normally empty; we repurpose it
// to carry the webhook kind ("validating"/"mutating") so Reconcile targets only the type that fired.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's clever but it redefines the semantics of the core controller constructs in service of an optimization and makes it harder to understand internally... not necessarily opposed but was there some empirical analysis that led to the decision to make the design so abstract given there are only 2 types of webhooks and I'm curious how many resources there are in a typical cluster

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you like me to drop the webhookType abstraction and just have two controllers (or one reconciler with two explicit code paths), each watching its own type directly. That eliminates both the namespace hack and the function-pointer indirection.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, I don't think the above is blocking feedback, if what's here is well tested and working I say go with it

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two controllers seems like it would be pretty straightforward FWIW

@ironcladlou

Copy link
Copy Markdown
Contributor

This makes sense to me and the test plan looks good, will give it a tag but also a hold in case someone else wants an opportunity to look before merge

/lgtm
/hold

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 6, 2026
@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Aug 6, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-azure-self-managed
/test e2e-v2-gke

@bryan-cox

Copy link
Copy Markdown
Member Author

/retest

@cwbotbot

cwbotbot commented Aug 6, 2026

Copy link
Copy Markdown

Test Results

e2e-aws

e2e-aks

@bryan-cox

Copy link
Copy Markdown
Member Author

/test e2e-v2-aws

@bryan-cox

Copy link
Copy Markdown
Member Author

/hold cancel

No further PR comments to address

@openshift-ci openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 6, 2026
@bryan-cox

Copy link
Copy Markdown
Member Author

/test e2e-v2-gke

1 similar comment
@bryan-cox

Copy link
Copy Markdown
Member Author

/test e2e-v2-gke

@bryan-cox

Copy link
Copy Markdown
Member Author

/verified by e2e passing

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Aug 7, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@bryan-cox: This PR has been marked as verified by e2e passing.

Details

In response to this:

/verified by e2e passing

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

@bryan-cox: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot
openshift-merge-bot Bot merged commit 9bf009a into openshift:main Aug 7, 2026
45 checks passed
@openshift-ci-robot

Copy link
Copy Markdown

@bryan-cox: Jira Issue OCPBUGS-105193: Some pull requests linked via external trackers have merged:

The following pull request, linked via external tracker, has not merged:

All associated pull requests must be merged or unlinked from the Jira bug in order for it to move to the next state. Once unlinked, request a bug refresh with /jira refresh.

Jira Issue OCPBUGS-105193 has not been moved to the MODIFIED state.

This PR is marked as verified. If the remaining PRs listed above are marked as verified before merging, the issue will automatically be moved to VERIFIED after all of the changes from the PRs are available in an accepted nightly payload.

Details

In response to this:

What this PR does / why we need it:

Extracts ensureGuestAdmissionWebhooksAreValid() from the monolithic HCCO resources controller into a dedicated webhook-validation controller. The [Feature:WebhookValidation] e2e test has a 47% pass rate on e2e-v2-azure-self-managed because webhook validation runs at the tail of a 15+ sub-reconciler chain, causing 60+ second delays between webhook creation and deletion.

The new controller watches ValidatingWebhookConfiguration and MutatingWebhookConfiguration directly, so reconciliation triggers immediately on webhook events. It encodes the webhook type in the request namespace field so Reconcile() targets only the type that fired, avoiding a redundant Get for the other kind.

Which issue(s) this PR fixes:

Fixes https://issues.redhat.com/browse/OCPBUGS-105193

Special notes for your reviewer:

  • The HCCO manager cache already includes VWC and MWC with labels.Everything() (no label filter) — see operator/config.go:129-130 — so the resources controller retains its watches for ensureResourceCreationIsBlocked
  • Two commits: (1) the extraction refactor, (2) e2e timeout bump (separate rationale)
  • 13 unit tests: 7 table-driven reconcile cases + 2 error-path tests + 4 URL validation tests

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Always review AI generated responses prior to use.
Generated with Claude Code via openshift-developer plugin

Summary by CodeRabbit

  • Bug Fixes

  • Automatically removes admission webhooks that target disallowed hosted cluster control-plane services.

  • Preserves allowed external targets and direct service references.

  • Improves handling of missing resources and transient errors.

  • Revalidates webhook configurations when related services or configurations change.

  • Tests

  • Added coverage for webhook filtering, cleanup, preservation, and error scenarios.

  • Increased the end-to-end cleanup wait time to improve reliability.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release area/testing Indicates the PR includes changes for e2e testing jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants