Skip to content

fix(postgresql): accept quantified comparisons, COLLATE, shift operators and E'' strings - #506

Open
bardrop wants to merge 1 commit into
DTStack:mainfrom
bardrop-oss:fix/postgresql-any-collate-shift-escape
Open

bardrop wants to merge 1 commit into
DTStack:mainfrom
bardrop-oss:fix/postgresql-any-collate-shift-escape

Conversation

@bardrop

@bardrop bardrop commented Nov 13, 2026

Copy link
Copy Markdown

Closes #504

Root cause

Four separate bugs in the PostgreSQL grammar:

  1. = ANY(...) / <> ALL(...) / LIKE ANY(...) – the quantifiedComparison alternative of predicate was comparisonOperator subqueryOperator subType (...). subqueryOperator already contains every comparison operator (through qualAllOp → allOp → mathOp), so the rule needed two operators in a row (id = = ANY(...) was accepted, id = ANY(...) was not).
  2. expr COLLATE "C" – the expression rules had no COLLATE alternative (the old a_expr KW_COLLATE anyName was lost in the rewrite).
  3. << / >> – these are lexed as LESS_LESS / GREATER_GREATER (for PL/pgSQL <<label>>), so they never become an Operator token, and mathOp didn't list them.
  4. E'...' – after the closing quote the lexer switched to AfterEscapeStringConstantMode, which only has a whitespace rule and never leaves. Every token after an escape string (;, FROM, ||, ...) became an error.

Changes

  • predicate: quantifiedComparison is now subqueryOperator subType (selectWithParens | OPEN_PAREN expression CLOSE_PAREN).
  • primaryExpression: new primaryExpression collateClause alternative, next to TYPECAST (same precedence level as in the PostgreSQL docs).
  • mathOp: added LESS_LESS and GREATER_GREATER. This also fixes CREATE OPERATOR << (...) and OPERATOR(pg_catalog.<<).
  • Lexer: EscapeStringConstant / InvalidEscapeStringConstant now popMode instead of switching to AfterEscapeStringConstantMode. (That mode and AfterEscapeStringConstantWithNewlineMode are now unused; I left them in to keep the diff small. Happy to remove them if you prefer.)
  • Regenerated src/lib/postgresql with pnpm antlr4 --lang postgresql (Compile postgresql succeeded!), ran antlr-format on the grammar.

Tests

  • test/parser/postgresql/syntax/fixtures/select.sql: 17 new statements (ANY/SOME/ALL with arrays, params and subqueries, LIKE ANY, @> ANY, OPERATOR(...) ANY, COLLATE in ORDER BY / WHERE / after a cast, << / >>, E'' strings followed by more tokens).
  • test/parser/postgresql/lexer.test.ts: tokens after E'a\nb' are lexed normally.
  • The 18 new tests fail on main and pass with this change.
  • Checked that invalid SQL is still rejected: SELECT 1 = = ANY(ARRAY[1]), WHERE id = ANY;, SELECT a COLLATE FROM t. PL/pgSQL <<label>> blocks and CREATE TABLE ... COLLATE "C" DEFAULT ... still pass.

Commands run:

  • pnpm test – 235 suites, 6898 tests passed
  • pnpm check-types – OK
  • npx prettier --check src/grammar/postgresql test/parser/postgresql – OK
  • pnpm benchmark --lang postgresql (cold), twice each on main and on this branch – same numbers within noise (e.g. validate on Query Collection: 1043/1054 ms on main vs 1106/1056 ms here; Collect Entities 591/586 vs 610/588 ms).

…ors and E'' strings

- predicate: `= ANY(...)` / `<> ALL(...)` / `LIKE ANY(...)` used to need
  two operators (comparisonOperator + subqueryOperator); use subqueryOperator only
- primaryExpression: allow `expr COLLATE name`
- mathOp: add `<<` and `>>` (lexed as LESS_LESS / GREATER_GREATER, so the
  generic Operator token never matched them)
- lexer: pop back to the default mode after an E'...' string; the old
  AfterEscapeStringConstantMode only matched whitespace, so every token after
  the string was an error

Closes DTStack#504
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: PostgreSQL validate() rejects = ANY(...), COLLATE, <<, and E'...' escape strings

1 participant