[gurungabit]

Fixes #31.

Root cause

When a parameter targets a CLOB/BLOB/DBCLOB declared larger than 32,767 bytes, DESCRIBE INPUT reports its declared length (clamped to u16). build_sqldta then described the parameter as VARCHAR/VARBINARY/VARGRAPHIC with that length, for example 33 80 00 for CLOB(32768) or 29 FF FF for BLOB(1M). FD:OCA varying-length descriptors only have 15 bits of length. A set high bit declares a LOB placeholder instead, so Db2 LUW 11.5.9 and 12.1 rejected the SQLDTA with DSCINVRM (0x220A, invalid description).

The driver did not classify 0x220A, so process_execute_reply skipped it. The chained commit then succeeded and the statement reported rowCount=0 with no error. CAST(? AS VARCHAR(100)) worked because its descriptor stayed below 0x8000. Statements with more parameters likely raised an error on 1.0.24 because SQLDARD parsing rejected large LOB declarations there. 1.0.25 accepts them (404|408|412|988 up to 0x7FFF_FFFF), which exposed the encoding gap.

Wire capture on 12.1 before the fix:

SQLDTA GDA: 76 D0 33 80 00          (nullable VARCHAR, length 0x8000)
reply cp=0x220A ... ERROR MSG = ... (DSCINVRM, SVRCOD 8)  -> rowCount 0

Fix

• LUW LOB input descriptors beyond 0x7FFF are now described with DRDA LOB types: CF LOBCMIXED, C9 BLOB, CD DBCLOB (nullable variants). Each uses the same 9-byte reference LUW returns for LOB result columns (0x8009): status 0x02 plus an eight-byte big-endian length. DBCLOB lengths count UTF-16 code units, matching LUW VARGRAPHIC input.
• Values are sent after SQLDTA as same-correlation EXTDTA objects, in parameter order. Each starts with the nullable indicator, as LUW EXTDTA results do. Large objects use the existing DDM extended-length and DSS continuation framing.
  • NULL sends only the FDODTA null indicator.
  • An empty value is complete in its zero-length placeholder and sends no EXTDTA. LUW pairs EXTDTA objects only with nonempty LOBs: an indicator-only EXTDTA for an empty value shifted every later LOB value and was rejected ("lob lengths don't match").
• XML inputs declare no length, so values beyond 32,767 bytes could not fit the VARCHAR descriptor and failed client-side ("string parameter too large for SQLDTA"). On LUW they are now always sent as LOBCMIXED with EXTDTA, which Db2 parses into the XML target like the VARCHAR form. Malformed or empty XML still returns Db2's SQLSTATE 2200M.
• All SQLDTA writers (direct DML/query, prepared execute/query, batch, CALL) go through a new SqlDta::write.
• LOB targets ≤ 32,767 bytes keep the existing VARCHAR/VARBINARY/VARGRAPHIC encoding. z/OS descriptors are left unchanged because no live z/OS server is available.
• DSCINVRM is now a classified protocol reply error. It is also listed as a terminal reply and in diagnostics names, so a rejected descriptor can no longer report success.

How the wire format was found: a 4- or 8-byte placeholder produced DTAMCHRM ... Parser: Data mismatch with SUBCODE2=<our width>, SUBCODE3=9. Status bytes 0x00/0x01 were rejected. Omitting the EXTDTA indicator or using a DBCLOB byte length produced SYNTAXRM. UTF-16LE DBCLOB data stored the wrong characters.

Binary parameter performance

The fix makes multi-megabyte BLOB parameters usable, which exposed a pre-existing cost. normalizeParam in index.cjs turned every Buffer/typed array/ArrayBuffer into a JS array via Array.from. The addon then read it one element at a time into one JSON value per byte, mostly on the event loop.

• Byte views now reach the addon as Uint8Array. Other typed arrays and DataView are re-viewed without copying. The addon copies the bytes once into a Vec<u8>.
• Parameters move into Db2Value instead of being cloned.
• A 50 MB BLOB UPDATE drops from ~3.0 s to ~0.5 s, on par with CLOB. A batch of 1,000 × 256 KB BLOBs goes from 20.4 s to under 3 s.

Tests

• Rust unit tests (connection.rs):
  • Exact GDA triplets, FDODTA placeholders and EXTDTA payloads for 32767/32768/1M CLOB, BLOB and DBCLOB (with a surrogate pair), plus NULL, an empty BLOB placed before nonempty LOBs, and a following INTEGER.
  • XML descriptors (sqltype 989, length 0) are externalized for small, large and NULL values.
  • z/OS descriptors are not externalized.
  • Chain flags and correlation for 100 KB/40 KB EXTDTA, round-tripped through DssReader.
  • DSCINVRM fails process_execute_reply.
• Live Rust (lob_regressions_test.rs):
  • A direct update at exactly 32,768 bytes plus a 70 KB BLOB and a DBCLOB with surrogates. One byte over the limit gives classified SQLSTATE 22001.
  • Prepared updates with empty and NULL values.
  • Empty LOBs before a nonempty DBCLOB. This fails without the empty-value fix.
• Live Node (lob-regressions.test.ts):
  • The reporter's reproducer, plus DBCLOB and both CAST forms.
  • Boundary and 600 KB values with several LOB parameters before an INTEGER.
  • The 22001 overflow.
  • Prepared execute, prepared executeBatch with NULL/empty, and the OPNQRY path with a LOB parameter.
  • Every empty/NULL/small/large ordering of four externalized columns (256 rows) through executeBatch, alternating direct/prepared updates, and a CALL with LOB IN parameters.
  • XML: 120 KB values through direct insert/update, executeBatch with small/NULL/large, XMLEXISTS(... CAST(? AS XML)), and a malformed-document 2200M.
  • Binary inputs: Buffer slices with offsets, Uint8Array views, Int16Array, DataView, ArrayBuffer, plain byte arrays, an 8 MB Buffer, and invalid byte-array rejection.

Validation

All run against local Docker Db2 LUW 12.1.0.0 and 11.5.9.0:

• cargo fmt --check, cargo clippy --workspace --all-targets -D warnings, and cargo test --workspace --lib + -p db2-proto all pass. The db2-napi JS tests pass 17/17.
• 12.1 and 11.5: cargo test --workspace --test '*' passes.
• Node npm test: 12.1 has 129 passed, 0 failed, 7 skipped (credential/TLS fixtures not supplied). 11.5 has 14 failures, all SQL0204 for the pre-existing missing EMPLOYEES/TEST_NULLS/TEST_STRINGS/ACCOUNTS fixtures.
• Ad-hoc probes on both servers: an 8 MB CLOB(2G) and 5 MB BLOB(2G) update, NOT NULL LOB columns, VALUES CAST(? AS CLOB/BLOB), INSERT … SELECT, and MERGE with a CLOB parameter.
• Performance (examples/performance suites on 12.1, branch vs. main): CRUD, batch, pipeline, blob, edge-case and pool numbers match within run-to-run noise, with no regressions. LOB parameter throughput at 50 MB is about 95 MB/s for CLOB and BLOB writes and about 100 MB/s for reads.

Limits

• No live z/OS server was available, so z/OS LOB and XML input encoding is unchanged. A z/OS DSCINVRM for the same shape would now surface as an error instead of silent success.
• LOB parameters are materialized in memory; values are not streamed.
• Locator inputs are unchanged.

🤖 Generated with Claude Code