Skip to content

fix(snowflake-driver): map TIMESTAMP_TZ/TIMESTAMP_LTZ to generic timestamp so they can be pre-aggregated - #11404

Open
npwalker wants to merge 3 commits into
cube-js:masterfrom
npwalker:fix/snowflake-timestamp-tz-generic-type
Open

fix(snowflake-driver): map TIMESTAMP_TZ/TIMESTAMP_LTZ to generic timestamp so they can be pre-aggregated#11404
npwalker wants to merge 3 commits into
cube-js:masterfrom
npwalker:fix/snowflake-timestamp-tz-generic-type

Conversation

@npwalker

Copy link
Copy Markdown

Check List

  • Tests has been run in packages where changes made if available
  • Linter has been run for changed code
  • Tests for the changes have been added if not covered yet
  • Docs have been added / updated if required

Issue Reference this PR resolves

Fixes #11403

Description of Changes Made

TIMESTAMP_TZ and TIMESTAMP_LTZ columns can't be used as pre-aggregation time dimensions on Snowflake — the rollup build fails in Cube Store with Custom type 'timestamp_tz' is not supported.

SnowflakeToGenericType maps only timestamp_ntz, and BaseDriver.toGenericType() falls through as DbTypeToGenericType[columnType.toLowerCase()] || columnType — i.e. unknown types are returned verbatim. DbTypeToGenericType has no timestamp_tz/timestamp_ltz entry either, so the raw Snowflake type name is handed to Cube Store as though it were a generic type. Both paths that type a pre-aggregation column are affected: queryColumnTypes() (INFORMATION_SCHEMA.COLUMNS.DATA_TYPETIMESTAMP_TZ) and the snowflake-sdk path (column.getType()timestamp_tz). Both normalise through toLowerCase(), so one map entry covers each.

This adds the two missing entries:

   fixed: 'decimal',
-  timestamp_ntz: 'timestamp'
+  timestamp_ntz: 'timestamp',
+  timestamp_ltz: 'timestamp',
+  timestamp_tz: 'timestamp'
 };

Why mapping all three to the same generic type is correct

This is a type-map omission, not a change in value semantics:

  • src/type-parsers.ts already lists timestamp_tz and timestamp_ltz in its hydrator and normalises both via formatUtcTimestamp.
  • test/SnowflakeDriver.test.ts already asserts that a TIMESTAMP_TZ column hydrates to the same string as the equivalent TIMESTAMP_NTZ column.
  • Since perf(snowflake-driver): Combine session init into a single ALTER SESSION #11010, initConnection() pins TIMEZONE = 'UTC' on every connection, so all three variants render deterministically as UTC.

The driver already treats TIMESTAMP_TZ as a UTC timestamp everywhere except the type map.

I also confirmed Cube Store handles the offset the unload emits (TIMESTAMP_FORMAT = 'YYYY-MM-DD"T"HH24:MI:SS.FF3TZH:TZM'). Against cubestored v1.7.8, via both INSERT and the CSV-location ingest path used by the export bucket:

Ingested Stored
2025-08-02T17:00:00.000 (no offset) 2025-08-02T17:00:00.000
2025-08-02T17:00:00.000+00:00 2025-08-02T17:00:00.000
2025-08-02T17:00:00.000-07:00 2025-08-03T00:00:00.000
2025-08-02T17:00:00.000+05:30 2025-08-02T11:30:00.000

date_trunc('day', ts) grouped correctly throughout — the parser converts offsets to UTC rather than truncating wall-clock time, so a +00:00 value is byte-equivalent to the TIMESTAMP_NTZ case that already works.

Precedent

timestamp_ntz: 'timestamp' was itself added to this exact map to fix the same class of breakage (#2592 / #2601); timestamp_tz and timestamp_ltz were never added alongside it. Same remedy in #1796 (int8: 'bigint'), #6458 (Firebolt timestamptz) and #10175. The class is still open elsewhere: #7017 and #5277 (identical Cube Store error, Postgres), #7127 (DuckDB hugeInt), #10363 (BigQuery precision/scale), #10289 (ClickHouse).

Tests

New test/type-mapping.test.ts — 19 assertions, no Snowflake connection required, so they run without credentials:

  • each of TIMESTAMP_NTZ / TIMESTAMP_LTZ / TIMESTAMP_TZ maps to timestamp;
  • the same in lower case, covering both the INFORMATION_SCHEMA (upper) and snowflake-sdk (lower) spellings;
  • a regression guard that no TIMESTAMP variant is returned as its raw Snowflake name — the specific fall-through this PR fixes;
  • the pre-existing mappings (NUMBER, fixed, object, DATE, TEXT, VARCHAR, BOOLEAN) are unchanged, so the additions can't shadow anything.

I verified all 19 pass with the patch and that the 4 new timestamp assertions fail without it.

test/SnowflakeDriver.test.ts gains TIMESTAMP_TZ and TIMESTAMP_LTZ columns carrying a non-UTC offset (-0700), asserting they hydrate to the UTC instant (13:07:42.123 -07:0020:07:42.123) rather than the wall clock. That pins down the property that makes the shared mapping correct.

One caveat, in the interest of transparency: I have no Snowflake account to run the integration suite against, so test/SnowflakeDriver.test.ts has not been executed locally — the expected values there are derived from formatUtcTimestamp formatting the UTC components of the SDK's Date. The new unit tests were run and pass. Please flag if CI disagrees and I'll correct them.

Note

real has the same gap — present in the type-parsers.ts hydrator next to fixed, absent from both type maps, so toGenericType('real') returns 'real'. Left out to keep this focused; happy to fold it in.

npwalker added 3 commits July 28, 2026 16:49
Leaving them unmapped made BaseDriver.toGenericType() fall through to its
`|| columnType` default, handing Cube Store the literal Snowflake type name.
Cube Store rejects that when creating the pre-aggregation table:
"Custom type 'timestamp_tz' is not supported".
…variants

Credential-free unit tests, so they run without a Snowflake connection.
Adds non-UTC-offset columns, documenting that all TIMESTAMP variants carry the
same UTC instant - which is what makes the shared generic mapping correct.
@npwalker
npwalker requested a review from a team as a code owner July 28, 2026 21:50
@github-actions github-actions Bot added driver:snowflake Issues relating to the Snowflake driver javascript Pull requests that update Javascript code data source driver pr:community Contribution from Cube.js community members. labels Jul 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data source driver driver:snowflake Issues relating to the Snowflake driver javascript Pull requests that update Javascript code pr:community Contribution from Cube.js community members.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Snowflake TIMESTAMP_TZ/TIMESTAMP_LTZ cannot be pre-aggregated: type map falls through, Cube Store rejects "Custom type 'timestamp_tz' is not supported"

1 participant