Skip to content

perf: complete LUW replies at the DRDA chain end instead of timed drains - #34

Merged
gurungabit merged 1 commit into
mainfrom
claude/deterministic-reply-completion
Oct 5, 2026
Merged

gurungabit merged 1 commit into
mainfrom
claude/deterministic-reply-completion

Conversation

@gurungabit

Copy link
Copy Markdown
Owner

Problem

Nearly all of the driver's request latency came from waiting, not from Db2:

  • After every prepare, execute and describe reply, read_*_reply_frames waited frameDrainTimeout (default 25 ms) for follow-up frames before returning.
  • Before the first fetch of any result with LOB columns, query_frame_drain_timeout waited 250 ms, the z/OS LOB drain applied regardless of server.

Neither wait ever received anything on LUW, so every operation paid for them:

Operation (Db2 12.1, localhost) Before
VALUES 1 30 ms
SELECT with a parameter 61 ms
SELECT of a CLOB column 285 ms
Autocommit UPDATE with parameters 87 ms

How replies actually end

DRDA marks every reply DSS except the last one in a chain with the chain bit (0x40). A logging TCP proxy between the driver and Db2 12.1 confirmed this for every flow: connect, prepare, describe input, OPNQRY/CNTQRY with and without LOBs, EXTDTA, execute plus commit, batches, errors, CALL, transactions and TLS.

C> RQS C-S corr=37 EXCSQLSTT   (batch of 3)
...
S< RPY C-S corr=37 RDBUPDRM
S< OBJ C-- corr=37 SQLCARD
S< OBJ C-- corr=38 SQLCARD
S< OBJ --- corr=39 SQLCARD     <- chain ends; nothing follows until the next request

Running the full Rust and Node suites through the proxy (358 connections) found no case where the driver pipelines independent request chains, and no reply that arrived without an outstanding request. Once the unchained DSS arrives, the reply is complete.

Change

  • Reply reading: off z/OS, read_reply_frames reads through the DSS that clears the chain bit. A reply split across TCP reads, such as a large QRYDTA or EXTDTA chain or a commit flushed separately, is no longer cut off by a timer.
  • Drains: the prepare, execute and describe-input drains, and the initial OPNQRY drain (the 250 ms LOB wait), return immediately once the chain is complete. The LUW LOB fetch path already relied on the chain bit; this applies the same rule everywhere.
  • DssReader: a frame merged from same-correlation segments now reports the last segment's chain bit, so a merged frame can't hide the end of a chain.
  • z/OS keeps its existing timed drains, because no z/OS server was available to verify against. frameDrainTimeout is now documented as z/OS-only. Before the server is identified, behavior is unchanged.

Results

examples/performance suites plus a LOB-parameter benchmark on Db2 12.1 (Docker, Apple M4 Pro), merged main vs. this branch, single runs:

Benchmark main this PR
1,000 × VALUES 1 30.3 s 0.51 s
1,000 queries returning 0 rows 30.2 s 0.47 s
500 prepare → execute → close 29.8 s 0.36 s
500 single-row prepared UPDATEs (tx) 14.9 s 0.14 s
100 sequential 500-row queries 9.7 s 0.42 s
Paginate 50,000 rows (page = 1,000) 5.5 s 0.56 s
200 pool acquire → query → release 6.1 s 0.11 s
50 connect/query/disconnect 3.5 s 0.22 s
Batch INSERT 10,000 rows 480 ms 166 ms
SELECT * 10,000 rows 140 ms 52 ms
1 KB CLOB/BLOB write / read 92 / 287 ms 1.0 / 0.9 ms
1 MB CLOB write / read 100 / 300 ms 9.9 / 5.7 ms
50 MB BLOB write / read 496 / 483 ms 393 / 200 ms

The Node test suite drops from 46.7 s to 4.0 s, and the Rust integration suite from several minutes to 7 s. The only benchmark that doesn't move is "10,000 prepared executions", which is an unindexed COUNT(*) over 50,000 rows and is bound by Db2.

Tests

  • Mock-server regression (lob_wire_regressions.rs): an LUW server whose OPNQRY reply chain spans two DSSes and two writes 50 ms apart, then an execute plus commit chain, with frame_drain_timeout = 5s. Both operations must finish in under 1 s, which they cannot if any drain wait remains. Fails on main.
  • DssReader unit test: a merged same-correlation continuation reports the last segment's cleared chain bit. Fails without the change.
  • Live, Db2 12.1:
    • cargo test --workspace --test '*' passes.
    • Node npm test has 137 passed, 0 failed, 7 skipped, with the TLS suite (8/8) on port 50001.
  • Live, Db2 11.5:
    • cargo test --workspace --test '*' passes.
    • Node has the same 14 pre-existing SQL0204 failures as main (missing seed fixtures).
  • Proxy audit: both suites run through the DSS-logging proxy with no pipelined chains or unsolicited replies.
  • Mid-chain stall: a 1,000-row prepared batch blocked mid-chain on a row lock held for 500 ms by another connection returns the correct rowCount, and later queries on the connection stay in sync.
  • cargo fmt --check, cargo clippy --workspace --all-targets -D warnings, workspace unit tests and the db2-napi JS tests all pass.

Notes

  • If a server sent a chained DSS and then stalled, the read now waits for the rest of the chain, bounded by queryTimeout (or the existing 30 s fetch deadline), just like a server that never replies. Before, the driver gave up after 25 ms and returned a partial reply, leaving the rest to be misread by the next request.
  • Found while profiling, not changed here: a prepared executeBatch under autocommit sends no RDBCMM, so its rows stay uncommitted until the connection's next commit.

🤖 Generated with Claude Code

After every reply the driver waited frame_drain_timeout (25 ms) for
follow-up frames, and 250 ms before fetching any result with LOB
columns, even though nothing more ever arrives. DRDA clears the chain
bit on the last DSS of a reply chain, so off z/OS read_reply_frames now
reads through that DSS and the speculative drains return immediately.

A DSS capture of the full Rust and Node suites (358 connections) showed
every LUW reply chain ends this way, with no pipelined request chains
and no unsolicited replies. Same-correlation segments merged by
DssReader now carry the last segment's chain bit, so a merged frame
cannot hide the end of a chain.

On Db2 12.1, VALUES 1 drops from ~30 ms to ~0.5 ms and a LOB SELECT from
~285 ms to ~1 ms. z/OS keeps its timed drains, and frame_drain_timeout
is documented as z/OS-only.
@gurungabit
gurungabit merged commit 7233a9e into main Oct 5, 2026
3 checks passed
@gurungabit
gurungabit deleted the claude/deterministic-reply-completion branch October 5, 2026 03:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant