Skip to content

feat(workflow): add a per-client default deadline to DaprWorkflowClient - #1343

Merged
CasperGN merged 2 commits into
dapr:mainfrom
famarting:feat/workflow-client-api-timeout
Oct 23, 2026
Merged

CasperGN merged 2 commits into
dapr:mainfrom
famarting:feat/workflow-client-api-timeout

Conversation

@famarting

Copy link
Copy Markdown
Contributor

Description

DaprWorkflowClient (sync and async) gets a new optional constructor argument, api_timeout_seconds: a default deadline, in seconds, for the client's unary calls to the sidecar (schedule_new_workflow, get_workflow_state, raise_workflow_event, terminate_workflow, purge_workflow, and so on).

Why. Today the only bound on these calls is DAPR_API_TIMEOUT_SECONDS, which applies to every Dapr unary call in the process. An app that sets it to bound its workflow calls also bounds everything else, such as a Conversation API call to an LLM that can legitimately run for minutes. So in practice apps leave it unset, and the workflow client calls have no deadline at all. We hit this for real: after a network change, a schedule_new_workflow / get_workflow_state call hung for hours (gRPC keepalive was on, yet the call never returned), and the app's loop that made the call stalled with it.

What the argument does.

  • api_timeout_seconds=None (the default) keeps today's behavior exactly: use DAPR_API_TIMEOUT_SECONDS if it is set, otherwise no deadline.
  • A number overrides DAPR_API_TIMEOUT_SECONDS for this client only. Other clients and the rest of the process are not affected.
  • A call that already sets its own timeout keeps it.
  • wait_for_workflow_start and wait_for_workflow_completion never get this deadline. They stay unbounded unless the caller passes timeout_in_seconds, the same as they do with DAPR_API_TIMEOUT_SECONDS today (_UNBOUNDED_WAIT_METHODS).
  • A value of 0 or less raises ValueError. Read as a gRPC deadline, 0 would make every call fail at once, which is never what the caller means.

How. DaprClientTimeoutInterceptor and DaprClientTimeoutInterceptorAsync take an optional timeout_seconds. When it is given, the interceptor uses it in place of DAPR_API_TIMEOUT_SECONDS. Called with no argument, as every other caller in the SDK does, they behave as before and still read the setting on every call. The workflow clients pass api_timeout_seconds to the interceptor they already install. No stubs are patched.

Why this name. The constructor arguments that set a per-client default follow the setting they override: DaprClient(http_timeout_seconds=...) overrides DAPR_HTTP_TIMEOUT_SECONDS. This argument overrides DAPR_API_TIMEOUT_SECONDS, so it is api_timeout_seconds. I did not reuse timeout_in_seconds: on wait_for_workflow_start / wait_for_workflow_completion it is a per-call bound on a wait, and those waits are exactly the calls this default does not touch. Giving both the same name would suggest the constructor value applies to the waits too. I'm happy to rename it if maintainers prefer something else.

Tests.

  • Interceptor tests (sync and async): an explicit timeout applies when the setting is unset, it overrides the setting, a per-call timeout is kept, workflow waits are skipped, and 0 or a negative value is rejected. One more sync test shows that the no-argument interceptor still reads the setting on every call.
  • Workflow client tests (sync and async): the argument reaches the interceptor, it defaults to None, and invalid values are rejected.
  • One test per client starts a TCP server that accepts connections and never answers. With api_timeout_seconds=0.5 and DAPR_API_TIMEOUT_SECONDS unset, the call fails with DEADLINE_EXCEEDED in under 5 seconds. For comparison, without the argument the same sync call against that server only failed after about 20 seconds, with UNAVAILABLE from gRPC's connect timeout. When the connection is already up and the server stops answering, which is the case we hit, nothing ends the call at all.

Docs. Constructor docstrings for both clients and the DaprWorkflowClient section of dapr/ext/workflow/AGENTS.md.

Note: the async DaprWorkflowClient has no public close(), so its silent-server test closes the underlying channel through the private attribute. I left adding a close() for a separate change.

Issue reference

No upstream issue. I found no open PR or issue for this. #1246 is related, but it is about retrying waits when the server cancels them.

Checklist

  • Code compiles correctly
  • Created/updated tests
  • Extended the documentation

Checks run locally:

  • uv run ruff check and uv run ruff format --check: clean
  • uv run mypy: no issues in 228 source files
  • uv run pytest -m "not e2e" ./tests --ignore=tests/integration --ignore=tests/examples: 2121 passed
  • uv run python -m unittest discover ./tests/ext/workflow: 247 tests OK
  • uv run pytest -m "not e2e" ./tests/ext/workflow/durabletask/: 333 passed

🤖 Generated with Claude Code

DaprWorkflowClient (sync and async) takes a new api_timeout_seconds
argument: a default deadline in seconds for its unary calls to the
sidecar. It overrides DAPR_API_TIMEOUT_SECONDS for that client only, so
an app can bound its workflow calls without also bounding every other
Dapr call in the process, such as a long Conversation API call.

DaprClientTimeoutInterceptor and DaprClientTimeoutInterceptorAsync take
an optional timeout_seconds. Without it they behave exactly as before.
Calls that set their own timeout keep it, and WaitForInstanceStart /
WaitForInstanceCompletion stay unbounded unless the caller passes one.

Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.83%. Comparing base (5fd4cfe) to head (47a8ef0).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1343      +/-   ##
==========================================
+ Coverage   87.62%   87.83%   +0.20%     
==========================================
  Files         163      163              
  Lines       13046    13065      +19     
==========================================
+ Hits        11432    11475      +43     
+ Misses       1614     1590      -24     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@famarting
famarting marked this pull request as ready for review October 23, 2026 15:18
@famarting
famarting requested review from a team as code owners October 23, 2026 15:18
@CasperGN CasperGN self-assigned this Oct 23, 2026
CasperGN
CasperGN previously approved these changes Oct 23, 2026

@CasperGN CasperGN left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @famarting for your interest in the project and for this contribution! 🙌 A per-client deadline that leaves DAPR_API_TIMEOUT_SECONDS and the workflow waits alone is a clean way to fix the hung schedule_new_workflow / get_workflow_state case. I really like the silent TCP server tests, which exercise the real deadline end to end on both the sync and async clients. 🧪 I left one small nit inline about non-finite values.

A few more notes:

  • nit: SilentTcpServer is the same in tests/ext/workflow/test_workflow_client.py and tests/ext/workflow/test_workflow_client_aio.py. It could live in one shared test helper, but I'm fine with it as is.

Drafted with AI assistance and reviewed by me.

Comment thread dapr/clients/grpc/interceptors.py Outdated
@CasperGN
CasperGN added this pull request to the merge queue Oct 23, 2026
NaN slipped past the `<= 0` check because every comparison with it is
False, and infinity passed too; either became the gRPC deadline. Both are
now rejected with a ValueError, matching the documented contract.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Fabian Martinez <46371672+famarting@users.noreply.github.com>
Merged via the queue into dapr:main with commit 461d576 Oct 23, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants