Skip to content

SQL: shapes DataFusion supports that the lowering rejects #658

Open

Description

@zzylol

Found by the DataFusion differential check of #539. Each of these is rejected with an explicit error, so nothing is computed wrongly, but DataFusion supports them.

Rejected shapes

  • ClickHouse scalar builtins such as startsWith, match, positionCaseInsensitive, splitByChar fail with "unregistered scalar function": the IR function registry (ir/scalar.rs) does not know them, although the frontend registers their DataFusion stubs and a code comment says registering the stub is the entire fix. toDate / toStartOf* likely have the same gap (not run: no timestamp data in the check).
  • INTERSECT / EXCEPT [ALL] and LEFT SEMI / LEFT ANTI JOIN are rejected (frontend-sql/src/sql/mod.rs, set-op and join arms), although the IR has these set-op kinds and the Semi / Anti join kinds.
  • Two window functions sharing one window spec are rejected; the error message suggests this is deliberate, but the shape is common.
  • Others: PARTITION BY <expression>; IS [NOT] DISTINCT FROM (also in JOIN ON); IS TRUE / IS NOT TRUE / IS UNKNOWN; ||; SIMILAR TO; ~; substr; ntile; percent_rank; bool_and; array_agg; aggregate first_value; SUM(DISTINCT); GROUPING(); two-argument uniqExact; ClickHouse quantile(q)(x).

Rejections the #539 description already calls deliberate are not listed (DISTINCT ON, GROUPS frames, non-literal LIMIT, correlated subqueries other than a filter-conjunct EXISTS, count(nullable) under ROLLUP).

Follow-up

Support the shapes the workloads need, starting with the ClickHouse builtins, which the BGP corpora use.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions