Add an opt-in query mod that scopes Expr paren suppression - #4
Open
IanHuangRubrik wants to merge 1 commit into
Open
Add an opt-in query mod that scopes Expr paren suppression#4IanHuangRubrik wants to merge 1 commit into
IanHuangRubrik wants to merge 1 commit into
Conversation
whereClause decides once per query whether to bracket each WHERE condition: a single paren marker anywhere turns automatic bracketing off for the entire query. Because AND binds tighter than OR, a condition holding a bare OR then becomes a top-level branch and neutralises every other filter in the statement. Add ScopedExprParens, a query mod that confines the suppression to the conditions inside the Expr that requested it, deciding per condition from the nesting depth instead. The flag defaults to off, so a query that does not apply the mod renders byte-for-byte as before. Covered by queries/scoped_expr_parens_test.go, which pins both halves: the unopted rendering is unchanged, and the opted-in rendering brackets only at depth zero, including the WhereIn and empty-IN paths.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the builder behaviour behind SPARK-954255, tracked by SPARK-959739.
Problem
whereClausedecides once per query whether to bracket eachWHEREcondition: a single paren marker anywhere (i.e. anyqm.Expr) turns automatic bracketing off for the entire query. SinceANDbinds tighter thanOR, a condition holding a bareORthen becomes a top-level branch and neutralises every other condition in the statement.In RSC this shipped a P1: the Objects page's "Failed" recovery filter returned workloads that had succeeded, because
RECOVERY_STATUSusesqm.Exprand a sibling filter contained a bareor. 3 correct rows became 13. The two conditions were written by different teams, years apart, and neither is wrong on its own.The library documents the behaviour without justifying it: "When Expr is used, the entire query will stop doing automatic parentheses." That is coherent for a query written by one author. It fails when a query is assembled from ~190 independently-written filter functions.
Change
Adds a
scopedExprParensflag onQuery, aSetScopedExprParenssetter, and aqm.ScopedExprParens()mod. When set,whereClauseskips the whole-query pre-scan and instead decides per condition from nesting depth:The six
!manualParensreads becomeautoWrap.depthis incremented after theLeftParenwrite and decremented after theRightParenwrite. The zero-argINpath that writes(1=0)and returns early is deliberately untouched, so it does not gain a second pair of brackets.Safety
The flag defaults to
falseand nothing in the fork sets it, so this is inert for every existing caller. Verified rather than assumed: the fork'squeriessuite was run with the change stashed and unstashed, and 79 test results are byte-for-byte identical.A condition at depth 0 under the flag renders exactly as every condition already renders in any query containing no
qm.Expr, where the suppression is already off. The change introduces no new rendering path; it applies the existing, overwhelmingly common one to more conditions.Tests
New
queries/scoped_expr_parens_test.go(externalpackage queries_test, sincequeries/qmimportsqueries) pins both halves: that an unopted query renders unchanged, and that an opted-in query brackets only at depth 0, covering nestedExpr,WhereIn, and the empty-INpath.Note for reviewers running the suite:
TestTypedArgBigQueryandTestResolveTypedArgsSlowPathalready panic atv3.7.1-scaledata.4on a missing BigQueryArgConverter, unrelated to this change. Because a panic kills the test binary, it also truncates every result after it, so exclude those two with-runor a before/after comparison is meaningless.Merge request
This is a single commit, so a squash gains nothing. Please "Create a merge commit" rather than squash, so
ef0f389is preserved and thev3.7.1-scaledata.5tag can point at it. All five existing tags are lightweight and are ancestors ofv3.7.1-scaledata; keeping that property is the reason for the ask.