Repository navigation
feat(workflow): add a per-client default deadline to DaprWorkflowClient - #1343
Conversation
DaprWorkflowClient (sync and async) takes a new api_timeout_seconds argument: a default deadline in seconds for its unary calls to the sidecar. It overrides DAPR_API_TIMEOUT_SECONDS for that client only, so an app can bound its workflow calls without also bounding every other Dapr call in the process, such as a long Conversation API call. DaprClientTimeoutInterceptor and DaprClientTimeoutInterceptorAsync take an optional timeout_seconds. Without it they behave exactly as before. Calls that set their own timeout keep it, and WaitForInstanceStart / WaitForInstanceCompletion stay unbounded unless the caller passes one. Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1343 +/- ##
==========================================
+ Coverage 87.62% 87.83% +0.20%
==========================================
Files 163 163
Lines 13046 13065 +19
==========================================
+ Hits 11432 11475 +43
+ Misses 1614 1590 -24 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
CasperGN
left a comment
There was a problem hiding this comment.
Thanks @famarting for your interest in the project and for this contribution! 🙌 A per-client deadline that leaves DAPR_API_TIMEOUT_SECONDS and the workflow waits alone is a clean way to fix the hung schedule_new_workflow / get_workflow_state case. I really like the silent TCP server tests, which exercise the real deadline end to end on both the sync and async clients. 🧪 I left one small nit inline about non-finite values.
A few more notes:
- nit:
SilentTcpServeris the same intests/ext/workflow/test_workflow_client.pyandtests/ext/workflow/test_workflow_client_aio.py. It could live in one shared test helper, but I'm fine with it as is.
Drafted with AI assistance and reviewed by me.
NaN slipped past the `<= 0` check because every comparison with it is False, and infinity passed too; either became the gRPC deadline. Both are now rejected with a ValueError, matching the documented contract. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com>
Description
DaprWorkflowClient(sync and async) gets a new optional constructor argument,api_timeout_seconds: a default deadline, in seconds, for the client's unary calls to the sidecar (schedule_new_workflow,get_workflow_state,raise_workflow_event,terminate_workflow,purge_workflow, and so on).Why. Today the only bound on these calls is
DAPR_API_TIMEOUT_SECONDS, which applies to every Dapr unary call in the process. An app that sets it to bound its workflow calls also bounds everything else, such as a Conversation API call to an LLM that can legitimately run for minutes. So in practice apps leave it unset, and the workflow client calls have no deadline at all. We hit this for real: after a network change, aschedule_new_workflow/get_workflow_statecall hung for hours (gRPC keepalive was on, yet the call never returned), and the app's loop that made the call stalled with it.What the argument does.
api_timeout_seconds=None(the default) keeps today's behavior exactly: useDAPR_API_TIMEOUT_SECONDSif it is set, otherwise no deadline.DAPR_API_TIMEOUT_SECONDSfor this client only. Other clients and the rest of the process are not affected.wait_for_workflow_startandwait_for_workflow_completionnever get this deadline. They stay unbounded unless the caller passestimeout_in_seconds, the same as they do withDAPR_API_TIMEOUT_SECONDStoday (_UNBOUNDED_WAIT_METHODS).ValueError. Read as a gRPC deadline, 0 would make every call fail at once, which is never what the caller means.How.
DaprClientTimeoutInterceptorandDaprClientTimeoutInterceptorAsynctake an optionaltimeout_seconds. When it is given, the interceptor uses it in place ofDAPR_API_TIMEOUT_SECONDS. Called with no argument, as every other caller in the SDK does, they behave as before and still read the setting on every call. The workflow clients passapi_timeout_secondsto the interceptor they already install. No stubs are patched.Why this name. The constructor arguments that set a per-client default follow the setting they override:
DaprClient(http_timeout_seconds=...)overridesDAPR_HTTP_TIMEOUT_SECONDS. This argument overridesDAPR_API_TIMEOUT_SECONDS, so it isapi_timeout_seconds. I did not reusetimeout_in_seconds: onwait_for_workflow_start/wait_for_workflow_completionit is a per-call bound on a wait, and those waits are exactly the calls this default does not touch. Giving both the same name would suggest the constructor value applies to the waits too. I'm happy to rename it if maintainers prefer something else.Tests.
None, and invalid values are rejected.api_timeout_seconds=0.5andDAPR_API_TIMEOUT_SECONDSunset, the call fails withDEADLINE_EXCEEDEDin under 5 seconds. For comparison, without the argument the same sync call against that server only failed after about 20 seconds, withUNAVAILABLEfrom gRPC's connect timeout. When the connection is already up and the server stops answering, which is the case we hit, nothing ends the call at all.Docs. Constructor docstrings for both clients and the
DaprWorkflowClientsection ofdapr/ext/workflow/AGENTS.md.Note: the async
DaprWorkflowClienthas no publicclose(), so its silent-server test closes the underlying channel through the private attribute. I left adding aclose()for a separate change.Issue reference
No upstream issue. I found no open PR or issue for this. #1246 is related, but it is about retrying waits when the server cancels them.
Checklist
Checks run locally:
uv run ruff checkanduv run ruff format --check: cleanuv run mypy: no issues in 228 source filesuv run pytest -m "not e2e" ./tests --ignore=tests/integration --ignore=tests/examples: 2121 passeduv run python -m unittest discover ./tests/ext/workflow: 247 tests OKuv run pytest -m "not e2e" ./tests/ext/workflow/durabletask/: 333 passed🤖 Generated with Claude Code