Repository navigation
Conversation
…ors and E'' strings - predicate: `= ANY(...)` / `<> ALL(...)` / `LIKE ANY(...)` used to need two operators (comparisonOperator + subqueryOperator); use subqueryOperator only - primaryExpression: allow `expr COLLATE name` - mathOp: add `<<` and `>>` (lexed as LESS_LESS / GREATER_GREATER, so the generic Operator token never matched them) - lexer: pop back to the default mode after an E'...' string; the old AfterEscapeStringConstantMode only matched whitespace, so every token after the string was an error Closes DTStack#504
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #504
Root cause
Four separate bugs in the PostgreSQL grammar:
= ANY(...)/<> ALL(...)/LIKE ANY(...)– thequantifiedComparisonalternative ofpredicatewascomparisonOperator subqueryOperator subType (...).subqueryOperatoralready contains every comparison operator (throughqualAllOp→allOp→mathOp), so the rule needed two operators in a row (id = = ANY(...)was accepted,id = ANY(...)was not).expr COLLATE "C"– the expression rules had noCOLLATEalternative (the olda_expr KW_COLLATE anyNamewas lost in the rewrite).<</>>– these are lexed asLESS_LESS/GREATER_GREATER(for PL/pgSQL<<label>>), so they never become anOperatortoken, andmathOpdidn't list them.E'...'– after the closing quote the lexer switched toAfterEscapeStringConstantMode, which only has a whitespace rule and never leaves. Every token after an escape string (;,FROM,||, ...) became an error.Changes
predicate:quantifiedComparisonis nowsubqueryOperator subType (selectWithParens | OPEN_PAREN expression CLOSE_PAREN).primaryExpression: newprimaryExpression collateClausealternative, next toTYPECAST(same precedence level as in the PostgreSQL docs).mathOp: addedLESS_LESSandGREATER_GREATER. This also fixesCREATE OPERATOR << (...)andOPERATOR(pg_catalog.<<).EscapeStringConstant/InvalidEscapeStringConstantnowpopModeinstead of switching toAfterEscapeStringConstantMode. (That mode andAfterEscapeStringConstantWithNewlineModeare now unused; I left them in to keep the diff small. Happy to remove them if you prefer.)src/lib/postgresqlwithpnpm antlr4 --lang postgresql(Compile postgresql succeeded!), ranantlr-formaton the grammar.Tests
test/parser/postgresql/syntax/fixtures/select.sql: 17 new statements (ANY/SOME/ALL with arrays, params and subqueries,LIKE ANY,@> ANY,OPERATOR(...) ANY,COLLATEin ORDER BY / WHERE / after a cast,<</>>,E''strings followed by more tokens).test/parser/postgresql/lexer.test.ts: tokens afterE'a\nb'are lexed normally.mainand pass with this change.SELECT 1 = = ANY(ARRAY[1]),WHERE id = ANY;,SELECT a COLLATE FROM t. PL/pgSQL<<label>>blocks andCREATE TABLE ... COLLATE "C" DEFAULT ...still pass.Commands run:
pnpm test– 235 suites, 6898 tests passedpnpm check-types– OKnpx prettier --check src/grammar/postgresql test/parser/postgresql– OKpnpm benchmark --lang postgresql(cold), twice each onmainand on this branch – same numbers within noise (e.g.validateon Query Collection: 1043/1054 ms on main vs 1106/1056 ms here; Collect Entities 591/586 vs 610/588 ms).