Skip to content

Add an opt-in query mod that scopes Expr paren suppression - #4

Open
IanHuangRubrik wants to merge 1 commit into
v3.7.1-scaledatafrom
scoped-expr-parens-IanHuangRubrik
Open

Add an opt-in query mod that scopes Expr paren suppression#4
IanHuangRubrik wants to merge 1 commit into
v3.7.1-scaledatafrom
scoped-expr-parens-IanHuangRubrik

Conversation

@IanHuangRubrik

Copy link
Copy Markdown
Collaborator

Fixes the builder behaviour behind SPARK-954255, tracked by SPARK-959739.

Problem

whereClause decides once per query whether to bracket each WHERE condition: a single paren marker anywhere (i.e. any qm.Expr) turns automatic bracketing off for the entire query. Since AND binds tighter than OR, a condition holding a bare OR then becomes a top-level branch and neutralises every other condition in the statement.

In RSC this shipped a P1: the Objects page's "Failed" recovery filter returned workloads that had succeeded, because RECOVERY_STATUS uses qm.Expr and a sibling filter contained a bare or. 3 correct rows became 13. The two conditions were written by different teams, years apart, and neither is wrong on its own.

The library documents the behaviour without justifying it: "When Expr is used, the entire query will stop doing automatic parentheses." That is coherent for a query written by one author. It fails when a query is assembled from ~190 independently-written filter functions.

Change

Adds a scopedExprParens flag on Query, a SetScopedExprParens setter, and a qm.ScopedExprParens() mod. When set, whereClause skips the whole-query pre-scan and instead decides per condition from nesting depth:

autoWrap := !manualParens
if q.scopedExprParens {
    autoWrap = depth == 0   // inside a group, the group owns the brackets
}

The six !manualParens reads become autoWrap. depth is incremented after the LeftParen write and decremented after the RightParen write. The zero-arg IN path that writes (1=0) and returns early is deliberately untouched, so it does not gain a second pair of brackets.

Safety

The flag defaults to false and nothing in the fork sets it, so this is inert for every existing caller. Verified rather than assumed: the fork's queries suite was run with the change stashed and unstashed, and 79 test results are byte-for-byte identical.

A condition at depth 0 under the flag renders exactly as every condition already renders in any query containing no qm.Expr, where the suppression is already off. The change introduces no new rendering path; it applies the existing, overwhelmingly common one to more conditions.

Tests

New queries/scoped_expr_parens_test.go (external package queries_test, since queries/qm imports queries) pins both halves: that an unopted query renders unchanged, and that an opted-in query brackets only at depth 0, covering nested Expr, WhereIn, and the empty-IN path.

Note for reviewers running the suite: TestTypedArgBigQuery and TestResolveTypedArgsSlowPath already panic at v3.7.1-scaledata.4 on a missing BigQuery ArgConverter, unrelated to this change. Because a panic kills the test binary, it also truncates every result after it, so exclude those two with -run or a before/after comparison is meaningless.

Merge request

This is a single commit, so a squash gains nothing. Please "Create a merge commit" rather than squash, so ef0f389 is preserved and the v3.7.1-scaledata.5 tag can point at it. All five existing tags are lightweight and are ancestors of v3.7.1-scaledata; keeping that property is the reason for the ask.

whereClause decides once per query whether to bracket each WHERE
condition: a single paren marker anywhere turns automatic bracketing
off for the entire query. Because AND binds tighter than OR, a
condition holding a bare OR then becomes a top-level branch and
neutralises every other filter in the statement.

Add ScopedExprParens, a query mod that confines the suppression to the
conditions inside the Expr that requested it, deciding per condition
from the nesting depth instead. The flag defaults to off, so a query
that does not apply the mod renders byte-for-byte as before.

Covered by queries/scoped_expr_parens_test.go, which pins both halves:
the unopted rendering is unchanged, and the opted-in rendering brackets
only at depth zero, including the WhereIn and empty-IN paths.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant