Repository navigation
feat: report hasResultSet in result metadata - #584
vuanhphung wants to merge 4 commits into
Conversation
DDL such as CREATE TABLE and an empty SELECT looked identical: zero rows and, on Thrift, the same schema, because the server reports a placeholder `Result STRING` column for statements without a result set and sets hasResultSet on every handle. The Arrow schema the server sends is truthful (no fields for DDL), and the kernel returns a zero-column schema. Add an optional `hasResultSet` to `getResultMetadata()` derived from those schemas. Catalog operations report true; Thrift without Arrow results reports undefined. No extra requests; the fetch path is unchanged. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
There was a problem hiding this comment.
Verdict: 1 Nit
Looks good, no blocking issues. The Thrift backend works out hasResultSet from the Arrow schema (decoded once and cached against the stable metadata buffer), always reports true for catalog operations, and leaves it undefined when there's no Arrow schema. The kernel backend works it out from the field count of a schema it already decodes. Both have unit tests built from captured schema bytes and an e2e spec, and the optional field doesn't change any existing behavior.
Other findings
- ⚪ Nit — In
tests/e2e/result_set_presence.test.ts, the Arrow-disabled test callsopenSession(false, { arrowEnabled: false })before itstry. If connecting fails, thegetConfigstub is never restored bysinon.restore()(the stub is on a throwaway client, so the impact is negligible). Moving the call inside thetry, or adding anafterEach(() => sinon.restore()), would make cleanup unconditional.
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
|
Addressed the nit in f171c1c: the e2e suite now restores sinon stubs in |
There was a problem hiding this comment.
Verdict: 1 Low
Looks good, with one low concern: on Thrift, the new hasResultSet ignores an explicit false that the server already reported on the handle. The Arrow-schema derivation, the kernel field-count check and the catalog/EmptyOperationBackend true paths are consistent with the documented contract. Unit and e2e tests cover these paths.
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
There was a problem hiding this comment.
Verdict: 1 Nit
Looks good: no correctness concerns found. DBSQLOperation.getResultMetadata() waits for the operation to be ready before reading the handle's hasResultSet flag, so the flag is current. Catalog and primary-key fallback operations return true early, and the memoized Arrow decode is keyed on the buffer reference. Unit tests (with captured schema bytes) and e2e tests cover the DDL, empty-SELECT, catalog, Arrow-disabled and decode-failure cases.
Other findings
- ⚪ Nit —
resultSetPresencedrops Arrow schema decode failures without logging anything (catch { value = undefined; }). Logging the failure atLogLevel.debugwould make an unexpectedundefinedeasier to diagnose in the field. The contract still holds without it.
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
|
Addressed the round-3 nit in 8381b44: a failed Arrow schema decode is now logged at debug level (operation id + error message), and the unit test asserts it. |
DDL such as
CREATE TABLEand an empty query such asSELECT 'sample' AS Result WHERE 1 = 0look the same: zero rows and, on Thrift, the same schema. Live probes show why. The server setshasResultSeton every Thrift handle and reports a placeholderResult STRINGcolumn inTTableSchemafor statements without columns (DDL,USE,CREATE VIEW). The Arrow schema it sends alongside is accurate: it has no fields for DDL, while the empty SELECT keeps itsResultfield. The kernel (SEA) returns a zero-column schema for DDL.This adds an optional
hasResultSettogetResultMetadata(). It is derived from the Arrow schema on Thrift and from the decoded schema on the kernel. An explicithasResultSet: falsefrom the Thrift server is honored. Catalog operations reporttrue. On Thrift witharrowEnabled: falsethere is no Arrow schema, so it staysundefinedrather than guessing. Existing behavior (getSchema(), the Thrift-shapedgetMetadata(), fetch) is unchanged. DML reportstrue, since the server returnsnum_affected_rows.Verification:
tests/e2e/result_set_presence.test.ts, passes live on Thrift and kernel. The related arrow, cloudfetch, primary-keys, staging and kernel smoke specs also pass (24 tests).npm test(1534 passing), lint, prettier, type-check and build are clean.Performance: request counts are identical to
main, 1 RPC per operation for every workload on Thrift. The fetch path doesn't touch the new code. On Thrift,getResultMetadata()decodes the Arrow schema once per operation (memoized): about 5 µs for a narrow schema (local micro-benchmark, medians of 3 alternating runs, main 0.2–0.4 µs vs branch 4.6–5.5 µs). The kernel readsfields.lengthon a schema it already decodes. Live medians in ms (main vs branch, 3 fresh processes each, alternating, 10 iterations):Fixes #309
This pull request and its description were written by Isaac.