Antalya 26.6: Allow empty object storage cluster - #2221
Open
ianton-ru wants to merge 21 commits into
Open
Conversation
Share cluster resolution via resolveClusterRead so getQueryProcessingStage matches read when object_storage_cluster_fallback_if_empty is enabled, and skip local object_storage_cluster lookup when object_storage_remote_initiator and object_storage_remote_initiator_cluster are both set. Co-authored-by: Cursor <cursoragent@cursor.com>
Cover pure fallback on unknown cluster, aggregate planning, remote-initiator interaction, and integration scenarios with locally unknown object_storage_cluster. Co-authored-by: Cursor <cursoragent@cursor.com>
Cover stateless and integration cases where object_storage_cluster is missing locally and remote initiator falls back to non-cluster execution. Co-authored-by: Cursor <cursoragent@cursor.com>
…uster functions. Distinguish s3() with object_storage_cluster setting from s3Cluster() argument so fallback works for the former but explicit cluster names still fail when unknown. Co-authored-by: Cursor <cursoragent@cursor.com>
Distinguish alternative-syntax table functions from table reads so remote initiator keeps s3() for fallback and uses cluster path for Iceberg tables. Co-authored-by: Cursor <cursoragent@cursor.com>
… cluster setting. Track explicit *Cluster function arguments separately from table engine and query object_storage_cluster settings so fallback works for persistent tables. Co-authored-by: Cursor <cursoragent@cursor.com>
Split storage policy from the setting check and route all local-fallback decisions through one helper so the resolve/read path is easier to follow. Co-authored-by: Cursor <cursoragent@cursor.com>
… syntax. Pure-send to a remote initiator requires object_storage_remote_initiator_cluster; otherwise keep the clustered path that defaults the initiator cluster to object_storage_cluster. Co-authored-by: Cursor <cursoragent@cursor.com>
…ster. Under remote-initiator deferral, an empty local cluster name must fall back to pure send so ENGINE=S3/Iceberg without object_storage_cluster does not hit LOGICAL_ERROR. Co-authored-by: Cursor <cursoragent@cursor.com>
Drop the redundant pure-send gate, flatten fallback_to_pure, and reuse the already resolved remote-initiator cluster on the clustered path. Co-authored-by: Cursor <cursoragent@cursor.com>
Local fallback applies only to object_storage_cluster; a bad remote-initiator cluster must still report CLUSTER_DOESNT_EXIST. Co-authored-by: Cursor <cursoragent@cursor.com>
Empty OSC with remote_initiator and no remote_initiator_cluster must keep BAD_ARGUMENTS even when fallback is enabled; extend tests for cases 1-3 and this regression. Co-authored-by: Cursor <cursoragent@cursor.com>
Pure-send ENGINE tables under remote-initiator deferral (preserving OSC in SETTINGS/context) so fallback matches s3() alternative syntax; extend TF and ENGINE tests for cases 1-3. Co-authored-by: Cursor <cursoragent@cursor.com>
Merge the two policy hooks into usesObjectStorageClusterSettingSyntax, share local-vs-remote fallback decisions between read and getQueryProcessingStage, and dedupe remote-initiator send. Co-authored-by: Cursor <cursoragent@cursor.com>
…TTINGS. object_storage_cluster may be unknown locally and defined on the remote (or the reverse); *Cluster would bake the name into the function argument and skip remote fallback. Co-authored-by: Cursor <cursoragent@cursor.com>
…cluster. Align flag semantics with s3()/iceberg() alternative syntax when the cluster name comes from SETTINGS rather than a *Cluster argument. Co-authored-by: Cursor <cursoragent@cursor.com>
…ter. AST rewrite and setting-syntax checks only need cluster_name_from_function_argument. Co-authored-by: Cursor <cursoragent@cursor.com>
ianton-ru
marked this pull request as ready for review
August 17, 2026 16:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebase of #2028
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Allow empty object storage cluster
Documentation entry for user-facing changes
With 'object_storage_cluster' setting query to s3,iceberg and some other sources are executed as cluster request.
But with swarm cluster, when initiator is not a cluster member, may be situation when no one swarm node is alive at the moment. In this case query is failed with
CLUSTER_DOESNT_EXISTerror.New setting
object_storage_cluster_fallback_if_emptyallow to execute read query on local node in this case.Write query is not executed on cluster right now, so attempt to write is still failed in this case to avoid situation when query is success when swarm is empty and failed when has some nodes alive.
PR is a little bit complex because:
s3(...)- can fall back ifobject_storage_clusteris empty (cluster does not have active nodes, not 'empty setting value')s3(...) SETTINGS object_storage_remote_initiator=1- failed on local node ifobject_storage_clusteris emptys3(...) SETTINGS object_storage_remote_initiator=1, object_storage_remote_initiator_cluster='...'- decision about falling back must be made on remote initiator, on local nodeobject_storage_clustercan be unknown.But behavior is not changed for
Clusterfunctions:s3Cluster(...)- can't fall backs3Cluster(...) SETTINGS object_storage_remote_initiator=1- must failed on remote initiator ifobject_storage_clusteris empty.CI/CD Options
Exclude tests:
Regression jobs to run: