[famarting]

Description

This follows @JoshVanL's review suggestion on #1312: keep one gRPC channel for the worker's lifetime instead of replacing it every time the work-item stream ends.

Why the worker replaced its channel. Recreating the channel on every reconnect came from microsoft/durabletask-python#50 (2025), which was vendored here in #963. Neither PR says why. Upstream has since moved toward this design: microsoft/durabletask-python#135 keeps the channel, re-opens only the stream, and recreates the channel only after 5 transport failures in a row. A grpc.Channel reconnects by itself, re-resolving DNS when its connection drops, so only GetWorkItems has to be re-sent.

What changes

• start() opens one channel and stop() closes it. On reconnect the run loop only sends Hello and a new GetWorkItems on that channel.
• Removed: the retire/publish handshake, the per-channel call counts, the deferred-close threads, _RESULT_RECONNECT_WAIT and the stream-reconnect wait on result sends.
• Result sends and the history fetch now pass wait_for_ready=True with their existing 30s deadline. A result that is ready during a reconnect waits for the channel, instead of the worker tracking connections.
• The connect Hello also uses wait_for_ready=True. Without it, a Hello sent while gRPC is between reconnect attempts fails at once. The run loop's retry delay then adds to gRPC's own backoff, and in testing the worker took 8.7s to reconnect after the sidecar came back, against 0.7s on fix(workflow): keep the workflow worker working across sidecar reconnects #1312.
• gRPC's reconnect backoff is capped at 15s (grpc.max_reconnect_backoff_ms), matching the run loop's retry cap. The default is 120s, which could leave a worker idle for 2 minutes after a long outage. A caller's channel_options still override it.

What stays

• A small guard (_sidecar_stub with one in-flight counter) so stop() never closes the channel while a call is starting on it. With interceptors, grpcio crashes with SIGSEGV on a call started on a closed channel (see fix(workflow): keep the workflow worker working across sidecar reconnects #1312), and one channel does not remove that risk at shutdown.
• stop() keeps fix(workflow): keep the workflow worker working across sidecar reconnects #1312's order: wait up to 5s for calls in flight, close the channel, then join the run loop. The channel has to close first because closing it is the only way to end the work-item stream. Through an interceptor, GetWorkItems returns a plain generator with no cancel().

Found along the way, not fixed here: for the same reason, _make_stream_teardown (used when a history fetch fails) does nothing when the channel has interceptors, which is always the case under WorkflowRuntime. Its cancel() raises AttributeError, which is logged at debug level. This predates both PRs, and I can open an issue for it.

Checked against a real gRPC server (in-process, with the SDK's metadata interceptor): start the worker, kill the sidecar while an activity runs, let the activity finish while the sidecar is down, then restart the sidecar on the same port after 6s.

	#1312	this PR
Channels created	one per connect attempt	1
Result delivered after the restart	yes, 0.7s after it	yes, 2.7s after it (gRPC's backoff)
Stream re-opened	0.7s after the restart	2.7s after the restart
stop()	0.0s	0.0s

On #1312 the 0.7s depends on when the run loop's retry happens to land. On both branches the worst case is about 15s.

Issue reference

No issue filed. This is a follow-up to review feedback on #1312.

Checklist

• Code compiles correctly
• Created/updated tests
• Extended the documentation (dapr/ext/workflow/AGENTS.md)

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 96.47059% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.62%. Comparing base (ba2b297) to head (4bcb9e7).

Files with missing lines	Patch %	Lines
dapr/ext/workflow/_durabletask/worker.py	96.47%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1317      +/-   ##
==========================================
+ Coverage   87.35%   87.62%   +0.26%     
==========================================
  Files         163      163              
  Lines       13069    13046      -23     
==========================================
+ Hits        11416    11431      +15     
+ Misses       1653     1615      -38     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[CasperGN]

Thanks @famarting for this contribution and for following up on the #1312 review so quickly! 🙌 Keeping one channel for the worker's lifetime makes the code much simpler. The PR description explains clearly why the old design existed, and checking against a real gRPC server with the interceptor in place was a nice touch.

I left one comment inline about how long stop() can take while the sidecar is down, please take a look. 👀

A few more notes:

• CI is red on this head: build (windows-latest, 3.13), validate (3.14), validate-complete and both codecov checks fail. Could you check whether any of them come from this change?

_{Drafted with AI assistance and reviewed by me.}