fix(storage): rebind postgres tx prepares and guard stale retries

[ddbaron]

Problem

On the PostgreSQL backend the stale cleanup path does nothing. Both reapers return n=0 with:

prepare stale execution update: ERROR: syntax error at or near "," (SQLSTATE 42601)

sqlTx embeds *sql.Tx and overrides Exec, Query, and QueryRow so ? placeholders are rebound to $n, but it never overrode PrepareContext or Prepare. A prepared statement created inside a transaction therefore went to the driver with ? placeholders intact and failed to prepare on PostgreSQL, which is why MarkStaleExecutions and MarkStaleWorkflowExecutions never reaped anything in cloud mode.

RetryStaleWorkflowExecutions has a related gap. It selects on the workflow clock alone and runs before the reaper, so once the path works it would start re-dispatching workflows whose paired execution is still active.

Change

• Adds sqlTx.PrepareContext and the deprecated sqlTx.Prepare, both rebinding the query before delegating to *sql.Tx, matching what sqlDatabase already did.
• Makes RetryStaleWorkflowExecutions consult the paired executions activity clock, the same way MarkStaleWorkflowExecutions does.
• Repeats the status, retry-count, workflow-clock, and paired-execution-clock predicates in the conditional retry update, so activity arriving between selection and update leaves the workflow untouched. RowsAffected > 0 gating and retry-count semantics are unchanged.

Verification

Focused validation passes:

cd control-plane
go test ./internal/storage ./internal/handlers -count=1
go vet ./...
gofmt -d internal/storage/sql_helpers.go internal/storage/execution_records.go internal/storage/coverage_misc_helpers_test.go internal/storage/retry_stale_test.go
git diff --check

The full control-plane suite was also run. Storage and handlers pass; unrelated pre-existing failures remain in internal/core/services and internal/packages (environment and path expectations). No unrelated package was changed.

Live PostgreSQL

Run against a throwaway PostgreSQL 16 container, loopback only on a dynamically allocated port, per-test throwaway database, container and volume removed afterwards.

The storage suite reads POSTGRES_TEST_URL. With it set, go test ./internal/storage/ -run TestPostgres -v -count=1 gives 18 passed, 0 failed, 1 skipped (the skip is the pre-existing unconditional TestPostgresStorage_DatabaseCreation). The whole storage package also passes with a live database configured.

Evidence that this pins the fix rather than merely passing next to it:

• Pre-fix reproduction: sending the reaper's ? placeholders straight to PostgreSQL fails with SQLSTATE 42601, the error reported above.
• Post-fix, the text handed to the driver is rebound, with $1..$8 for the workflow and retry UPDATEs and $1..$6 for the execution sync UPDATE.
• End to end on the live database: a workflow stale on both clocks is reaped (status=timeout), a workflow whose paired execution has fresh activity survives (status=running), the retry guard re-dispatches a genuinely stale workflow (status=pending retry_count=1) and leaves a freshly heartbeating one alone, and the terminal-paired-execution branch still reaps.
• Removing the two sqlTx.Prepare* overrides on this branch makes the three new live tests fail with the original error.

Known harness gaps left untouched: coverage_postgres_test.go sits behind the integration build tag with a hardcoded DSN, and locks_test.go's setupPostgresTestStorage returns nil unconditionally, so both skip regardless of POSTGRES_TEST_URL. go test -race for this package also aborts in the boltdb/bolt v1.3.1 dependency during SQLite initialization, independent of this change.

One deployment note: the workflow-side reaper only started consulting the execution clock in #1046, so this fix changes behaviour only on a control plane that includes that change.

[ddbaron]

@santoshkumarradha This closes the gap that came up when #1046 merged. In that thread you noted that on the PostgreSQL backend neither stale reaper runs at all, because sqlTx has no PrepareContext override, so the ? placeholders in these prepared UPDATEs were never rebound to $n, and you asked whoever picked it up to look at RetryStaleWorkflowExecutions in the same pass.

What I understood from that:

1. Give sqlTx the same prepare-time rebinding that sqlDatabase already has, so prepared statements created inside a transaction work on PostgreSQL.
2. Check the retry path, since it still selected on the workflow clock alone and runs before the reaper, so switching the path on would start re-dispatching live runs wherever retries are enabled.

What I did:

• Added sqlTx.PrepareContext and the deprecated sqlTx.Prepare, both rebinding before delegating to *sql.Tx.
• Made RetryStaleWorkflowExecutions consult the paired execution activity clock the same way the workflow reaper does, and repeat the stale predicates in its conditional update so activity arriving between selection and update is respected. RowsAffected gating and retry-count semantics are unchanged.
• Verified it against a live PostgreSQL 16 rather than leaving the connection-backed tests skipped. I reproduced the original SQLSTATE 42601 failure first, then drove both reapers and the retry path end to end and confirmed that a row stale on both clocks is reaped while one with a fresh paired execution survives. Removing the two Prepare* overrides makes the new live tests fail with the original error. Counts and the captured statements are in the description.

Two pre-existing gaps I deliberately left alone: a Postgres test file sits behind the integration build tag with a hardcoded DSN, and locks_test.go's setupPostgresTestStorage returns nil unconditionally, so those tests skip however POSTGRES_TEST_URL is set.

If you would rather this were done differently, tell me and I will change it.

[github-actions]

📊 Coverage gate

Thresholds from .coverage-gate.toml: per-surface ≥ 84%, aggregate ≥ 85%, max per-surface regression ≤ 1.0 pp, max aggregate regression ≤ 0.50 pp.

Surface	Current	Baseline	Δ
control-plane	87.80%	87.40%	↑ +0.40 pp	🟡
sdk-go	93.10%	92.00%	↑ +1.10 pp	🟢
sdk-python	94.72%	93.73%	↑ +0.99 pp	🟢
sdk-typescript	91.72%	90.42%	↑ +1.30 pp	🟢
web-ui	84.76%	84.79%	↓ -0.03 pp	🟡
aggregate	85.88%	85.75%	↑ +0.13 pp	🟡

✅ Gate passed

No surface regressed past the allowed threshold and the aggregate stayed above the floor.

[github-actions]

📐 Patch coverage gate

Threshold: 80% on lines this PR touches vs origin/main (from .coverage-gate.toml:thresholds.min_patch).

Surface	Touched lines	Patch coverage	Status
control-plane	44	100.00%	✅
sdk-go	0	—	➖ no changes
sdk-python	0	—	➖ no changes
sdk-typescript	0	—	➖ no changes
web-ui	0	—	➖ no changes

✅ Patch gate passed

Every surface whose lines were touched by this PR has patch coverage at or above the threshold.