feat: add MergeBackends option for backend cluster deduplication

[muwaqar-cflt]

What this PR does / why we need it:

Adds an opt-in EnvoyProxy.spec.mergeBackends option: routes referencing the identical backend can share a single Envoy cluster instead of Envoy Gateway generating one cluster per route rule. This reduces xDS size, active health-check traffic, and stats cardinality on gateways with many routes fanning out to a small set of shared backends. Disabled by default and marked experimental.

Cluster sharing is skipped wherever it would be unsafe:

• multi-backendRef rules using session persistence, ConsistentHash load balancing, or a fallback backend (all need the full backend pool kept together in one cluster, not split per identity)
• dynamic-resolver/custom (extension-provided) backends
• a backendRef carrying any per-backendRef filter (e.g. CredentialInjection, header modification) or a UseClientProtocol override, both of which are baked into the shared cluster rather than the per-route destination
• a route-level BackendTrafficPolicy cluster setting, or a RoutingType diverging from the gateway's baseline
• TCP/UDP/TLS rules with more than one backendRef, since those route types have no way to represent a rule split across more than one cluster

Approach: enabling this required restructuring how a route's backend data is stored. Previously, every RouteDestination embedded its own copy of the resolved DestinationSettings, so two routes sharing a backend could never share anything at the IR level - deduplication had nowhere to attach. This PR introduces a per-gateway backend cluster registry (ir.Xds.Backends, ir.BackendCluster): each distinct backend is resolved once, and every RouteDestination holds only a BackendClusterRef (a name) into that registry. RouteDestination.Settings is removed once BackendClusterRefs is the sole source.

Beyond enabling MergeBackends, this registry is what makes a future backend-level BackendTrafficPolicy straightforward: since a backend's settings now live in exactly one place instead of being copied into every route that references it, a policy targeting the backend directly can be applied once at the registry entry, rather than needing to be threaded through and reconciled across every route-level copy.

Reviewer notes: this is a large diff; start here:

• api/v1alpha1/envoyproxy_types.go - the new user-facing MergeBackends/MergeBackendsConfig field
• internal/ir/xds.go - the BackendCluster/BackendClusterRef types and the Xds.Backends registry
• internal/gatewayapi/route.go - processBackendRef is the single entry point every route type (and the RequestMirror filter) uses to resolve a backendRef's destination and cluster assignment together. It wraps resolveBackendCluster (decides the cache key/name/merge outcome), shouldMergeBackend (owns the full eligibility decision), mergeIncompatibleForWeightedRule/mergeIncompatibleForSingleClusterRule (the per-rule-level exclusions, split by whether the route type supports weighted-cluster route actions), and getOrCreateBackendCluster (the find-or-create registry lookup)
• internal/xds/translator/backend_registry.go - registry resolution (getBackendClusters, singleResolvedClusterName) consumed by the HTTP/GRPC route-action and TCP/UDP listener builders, and cluster.go's cluster-per-setting/weight logic

A good portion of the remaining diff is golden testdata: internal/gatewayapi/testdata/*.out.yaml fixtures re-derived to reflect destinations serializing as backendClusterRefs + a top-level backends: registry instead of inline settings:, plus new fixtures (mergebackends-* in internal/gatewayapi/testdata, merge-backends-* in internal/xds/translator/testdata) covering the merge-eligibility rules above at both the IR-building layer and the xDS translation layer - several of the latter are derived directly from the former's real output, tracing a single Gateway API resource all the way through to the generated Envoy config.

Which issue(s) this PR fixes:
Fixes #5307

[netlify]

✅ Deploy Preview for cerulean-figolla-1f9435 ready!

Name	Link
🔨 Latest commit	fff4d8f
🔍 Latest deploy log	https://app.netlify.com/projects/cerulean-figolla-1f9435/deploys/6a5cc75e4c92f80008412f44
😎 Deploy Preview	https://deploy-preview-9477--cerulean-figolla-1f9435.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[muwaqar-cflt]

/retest

[codecov]

Codecov Report

❌ Patch coverage is 97.03390% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.60%. Comparing base (2df4a96) to head (31fbfa0).

Files with missing lines	Patch %	Lines
internal/gatewayapi/backendtrafficpolicy.go	95.30%	4 Missing and 3 partials ⚠️
internal/gatewayapi/ext_service.go	86.66%	2 Missing ⚠️
internal/gatewayapi/route.go	99.12%	2 Missing ⚠️
internal/gatewayapi/securitypolicy.go	88.23%	0 Missing and 2 partials ⚠️
internal/gatewayapi/runner/runner.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9477      +/-   ##
==========================================
+ Coverage   75.41%   75.60%   +0.18%     
==========================================
  Files         252      253       +1     
  Lines       41660    42116     +456     
==========================================
+ Hits        31417    31840     +423     
- Misses       8113     8139      +26     
- Partials     2130     2137       +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6011972ddb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[zhaohuabing]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 31fbfa0fa5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[zhaohuabing]

We may want to default this to true eventually. The fallback approach won't break current user-facing routing behaviors, and it helps with the Envoy memory issue.

It may be too risky to turn this on by default in 1.9, as it does break EPPs and extension servers, but we could keep it opt-in for the first release and consider flipping the default in 1.10 after we have more coverage and user feedback.

[muwaqar-cflt]

resolving because not seeing an action item for now.

[zhaohuabing]

Let's keep it unresolved for now, so other reviewer can see and discuss on this. cc @arkodg @guydc

[arkodg]

is this flag needed ? mergeBackends: {} should imply we are turning on the feature

[zhaohuabing]

yeah, not necessarily. A bool flag is more explicit than mergeBackends: {}, I'm fine with both.

[cnvergence]

lint fails :(

[zhaohuabing]

This PR looks good. Great work!

I left a few minor comments.

@muwaqar-cflt Could you please fix the link and check whether all e2e tests pass? I’d like to get this merged before 1.9.0 if possible. Thanks!

[arkodg]

can we avoid testdata changes to the default case

[muwaqar-cflt]

@arkodg Splitting the Codex findings into individual threads for tracking, so each can be resolved independently as it's fixed.

Multi-rule TLSRoute can reference a nonexistent cluster

Merge safety is decided independently per rule here, but a TLSRoute can have up to 16 rules — two rules that each have a single backend can each look mergeable on their own. That produces one TCP proxy route referencing multiple backend-cluster names, and singleResolvedClusterName() falls back to a route-scoped name that was never actually created as a Cluster — so the listener ends up referencing a missing cluster.

Merge compatibility needs to be computed across the whole TLSRoute, not independently per rule.

[muwaqar-cflt]

@arkodg

Listener-specific ClientTrafficPolicy settings leak or disappear

shouldMergeBackend doesn't account for listener-specific HTTP settings (e.g. preserveHeaderCase). If two listeners with different listener-level HTTP settings both route to the same backend, they end up sharing one merged cluster; whichever listener's route gets processed first wins (addXdsCluster's existing-cluster-name skip), silently dropping or leaking the other listener's settings depending on processing order.

Effective listener HTTP settings need to either be folded into the merge identity, or make the backend ineligible for merging.

[muwaqar-cflt]

@arkodg

Merging can break active HTTP health checks

For every merged cluster, this clears the route hostname unconditionally (routeHostname = ""), even when only one route currently references the cluster. An HTTP health check without an explicit Host previously used that route hostname; once merged, no Host is emitted, so Envoy falls back to the cluster name instead — a virtual-host-sensitive health endpoint can return 404 and mark the backend unhealthy.

Should only merge when the health-check Host is explicit/invariant across the merged routes, or keep separate clusters when route hostname affects health checking.

[muwaqar-cflt]

@arkodg

clusterStatName is silently ignored for merged clusters

Same block also clears statName unconditionally for merged clusters, so a configured EnvoyProxy.spec.telemetry.metrics.clusterStatName stops producing alt_stat_name once merging is enabled.

Should either prevent merging when a stat name is configured, use a documented backend-based stat name, or explicitly reject/document this as an incompatible combination rather than silently discarding it.

[arkodg]

The current shape makes the backend registry part of every translation, even when mergeBackends is unset. In a three-sample local run of the large translator benchmark with the option unset, this branch was about +4.1% ns/op, +1.9% B/op, and +1.4% allocs/op versus the base branch.

I think we can keep the universal IR registry while making merge-specific work pay for itself:

1. Keep the default path local. Build the nonmerged BackendCluster in the route-rule builder, append its destination settings there, and add it to the registry once. Use BackendClusterMap only for shared clusters and initialize it lazily. This should also let us remove ResolvedBackendCluster, routeScopedAdded, and the route-specific fields from the shared key.
2. Exit earlier when merging is disabled. Check the class/Gateway setting at the start of shouldMergeBackend, before custom-backend lookup, filter inspection, and policy compatibility checks. Only build the merge-safety BackendTrafficPolicy indexes if an accepted Gateway can enable merging, and resolve each policy's targets once rather than once per index.
3. Look up shared backends before fully resolving them. processBackendRef currently runs processDestination before consulting the registry, so endpoint, TLS, and destination work is repeated and then discarded on a shared-cache hit. Compute the stable identity first and cache the invariant resolution.
4. Deduplicate earlier in xDS translation. Check an O(1) emitted-cluster set before calling asClusterArgs. The current path builds the arguments, applies traffic policy, and then performs a linear scan to discover that the cluster already exists.
5. Resolve route backend refs once. Pass one resolved {refs, clusters, weights, singleName} result through route and cluster generation instead of repeatedly looking up and allocating the same slices.
6. Make missing registry references explicit errors. The synthesized empty backend and rd.Name fallback make invariant violations harder to detect. Once all fixtures use the registry, failing at the translator boundary would be simpler and safer.

The first two changes are the ones I would prioritize in this PR because they directly address the measured cost when the feature is not enabled. The remaining items could be follow-ups if keeping this change smaller is preferable.

[zhaohuabing]

Consider using BackendCluster only for the merged case, and keeping inline Settings on RouteDestination for non-merged.

Right now BackendCluster serves double duty — merged (shared, in Xds.Backends) and non-merged (route-scoped) — carried by a Merged flag. That makes the xDS translation layer harder to follow and it churns every non-merged golden fixture.

I think we can confine the new machinery to the merged path:

type RouteDestination struct {
      Name     string  `json:"name" yaml:"name"`
      StatName *string `json:"statName,omitempty" yaml:"statName,omitempty"`

      // A destination is either route-local (Settings) or merged (BackendClusterRefs) — never
      // both. Both empty means it has no resolvable backend. 
      //
      // Settings holds the endpoints of this route's OWN, non-merged cluster (the default case).
      // A rule's backendRefs collapse into this one route-scoped cluster, so it can hold multiple
      // settings. It is never shared with another route.
      Settings []*DestinationSetting `json:"settings,omitempty" yaml:"settings,omitempty"`

      // BackendClusterRefs references shared, identity-deduplicated clusters in Xds.Backends (the
      // merged case) — one ref per backend. Each referenced cluster is a single backend identity
      // and may be shared across routes; its data lives in the registry.
      BackendClusterRefs []*BackendClusterRef `json:"backendClusterRefs,omitempty" yaml:"backendClusterRefs,omitempty"`

      IsDynamicResolver bool              `json:"isDynamicResolver,omitempty" yaml:"isDynamicResolver,omitempty"`
      Metadata          *ResourceMetadata `json:"metadata,omitempty" yaml:"metadata,omitempty"`
}

// A shared, identity-deduplicated backend cluster: one backend identity, one setting.
type BackendCluster struct {
      Name     string
      Setting  *DestinationSetting   // singular — holding the endpoints for a shared backend
      Metadata *ResourceMetadata
}

Benefits:

• Non-merged IR and xDS output revert to their previous shape → non-merged fixtures stay untouched, so the diff shows only the merged feature.
• Merged becomes structural ("in the registry ⇔ merged") rather than a flag, so the type stops doing double duty.
• len(rd.BackendClusterRefs) > 1 becomes unambiguous — non-merged has zero refs, so > 1 cleanly means "merged multi-backend."
• Easier to review since the number of changed files would be much less.

WDYT? @muwaqar-cflt @arkodg ?