Skip to content

Add API proto support for NexusHandler callbacks - #863

Merged
chrsmith merged 10 commits into
mainfrom
feature/worker-callbacks
Sep 8, 2026
Merged

Add API proto support for NexusHandler callbacks#863
chrsmith merged 10 commits into
mainfrom
feature/worker-callbacks

Conversation

@chrsmith

@chrsmith chrsmith commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

What changed?

Adds protobufs to support a new variant of commonpb.Callback called NexusHandler.

This also includes additional types to support the new type of callbacks being exposed from the server:

  • A new variant commonpb.Link_Callback, to distinguish links to a particular callback on a resource.
  • temporal.api.notificationservice.v1.OnComplete{ Request, Response } protos, which are the envelope types that will be used sent to Nexus handlers expecting to receive a NexusHandler callback.
  • New fields on StartNexusOperationExecutionRequest to allow attaching callbacks to SANO executions.
  • A new field on DescribeNexusOperationExecutionResponse (and temporal.api.nexusoperation.v1) to retrieve the state of the callbacks attached to a SANO.

Why?

The end goal is to provide a streamlined way for adding completion handlers to operations. It's something that in specific scenarios would be more costly to do in other ways.

Breaking changes

None. These fields are purely additive.

Server PR

The server-side support for these new protos are in a set of stacked PRs still being reviewed/developed.

Starting with temporalio/temporal#11380 and ending with temporalio/temporal#11589.

@chrsmith
chrsmith requested review from a team, VegetarianOrc and bergundy August 29, 2026 19:49
@chrsmith
chrsmith force-pushed the feature/worker-callbacks branch from ff34ca1 to 2dc2ef6 Compare September 1, 2026 21:07
@chrsmith chrsmith changed the title Add support for NexusHandler callbacks Add API proto support for NexusHandler callbacks Sep 3, 2026
@chrsmith chrsmith mentioned this pull request Sep 4, 2026
Merged
5 tasks
chrsmith and others added 10 commits September 8, 2026 08:33
@chrsmith
API surface area of SANO worker callbacks
ae2f7d3
@chrsmith
Remove 'Outcome' types, moving fields to parent protos (#849)
4861804 
**What changed?**
**Why?**

There are two changes in this PR, **both breaking changes**. (But since
the feature is under development, this won't impact users in any way.)

(1) Remove the `callbackpb.CallbackOutcome` proto, and instead move the
`success`/`failure` fields directly into `callbackpb.CallbackInfo`. Do
the same for `OnCompleteRequest`, removing the
`OnCompleteRequest.Outcome` message.

It's extremely unlikely we would ever add a new variant for the
callback's result, and having the extra layer of indirection made the
SDK and server-side code cumbersome.

(2) Have `OnCompleteRequest.result` variant `success` be a
`commonpb.Payload` instead of a `commonpb.Payloads`.

The Temporal API makes it look as if a Workflow can return multiple
values. (i.e. a `commonpb.Payloads`.) But in reality, the SDK only ever
deals with a single value (`commonpb.Payload`).

And the extra layer of indirection via `commonpb.Payloads` also made the
SDK and server-side code more cumbersome.

**Breaking changes**

Yes. But this is still a WIP feature branch. The only person being
broken is, alas, @chrsmith .
@chrsmith
Add links to StartNexusOperationExecutionRequest
e84644a
@chrsmith
Wire request_id through CallbackInfo for linking (#851)
37d4497 
⚠️ This is to be merged into the `feature/worker-callbacks` branch, and
not `main`. Only after the feature is complete will that branch be
rebased and merged into `main`.

---

This PR makes three changes, all so that resources spanwed from the
invocation of a worker callback can be linked correctly.

(1) Remove the `Link_NexusOperationCallback` variant with a more general
`Link_Callback` proto

Previously we were scoping the feature to only be applicable for SANO
callbacks. But if we are going to support worker callbacks for any async
operation, having a general link type (that uses the existing [Execution
proto](https://github.com/temporalio/api/blob/0066de621239ca9ddc6c976e091e27a6bc474752/temporal/api/common/v1/message.proto#L73-L77))
will avoid needing to create additional link variants in the future.

(2) Add a `callbackpb.CallbackInfo::request_id` field

This type is used in the `Describe-` operations for standalone
Activities and standalone Nexus operations. Without it, there would be
no way to determine _which_ completion callback is being referred to.
(Instead, we couldn't be any more accurate than to have the link point
to "one of these N" callbacks.)

(3) Add `workflowpb.CallbackInfo::{request_id, result}`

The `workflowpb` namespace forked rather than embedded the
`callbackpb.CallbackInfo` message. The changes here add the missing
fields, so that `DescribeWorkflowExecution` can disambiguate callbacks
as well. (In addition to carrying the result of those callbacks.)

**Why?**

With these changes, the server will be able to properly cross-link
resources spawned from completion callbacks.

On the Caller-side, any resources spawned from the completion callbacks
would be available on the `commonpb.Callback::links` field. (*)

```graphql
query GetSpawnedResourceLinks(workflowID: string {
  DescribeWorkflowExecution(workflowID) {
    completion_callbacks {
      callback {
        links
      }
    }
  }
}
```

> (*) Only the resources _initially_ created from the worker callback
invocation will be present. e.g. the Workflow that backs an asynchronous
Nexus handler. It would not contain links for any subsequent resources
created.

On the Handler-side, a single `Link_Callback` would be supplied to the
Nexus handler receiving the worker callback. (This would be in the form
of a `nexuspb.Link`.)

**Breaking changes**

Yes, this PR contains breaking proto changes. However, in the context of
a PR into a long-lived feature branch for an unshipped feature this is
safe. (The protos haven't ever been persisted by a production service.)

**Server PR**

It isn't out yet, but will be stacked on top of this:
temporalio/temporal#11589
@chrsmith
Add Link_Callback.component_id for Workflow Update references
2893d87
@chrsmith
Rename Worker to NexusHandler
76eece2
@chrsmith
Update the dynamic config fields controlling NexusHandler callbacks
8724481
@chrsmith
Rebase onto 'main' and rebuild
260ec2d
@chrsmith
Minor last minute adjustments
005b252
@chrsmith
Rerun 'make'
c24351d
@chrsmith
chrsmith force-pushed the feature/worker-callbacks branch from 2dc2ef6 to c24351d Compare September 8, 2026 15:47
@chrsmith

chrsmith commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

After a conversation with @bergundy, made a few last minute tweaks:

On commonpb.Link variant Callback:

  • Replaced component_id with a repeated string component_path. This is a little less janky and more future proof. (Currently it only really matters for callbacks attached to workflow updates.) We had discussed introducing a proper ComponentType enum, but since that would only have one value for the time being, just having a repeated string component_path seemed preferable.

On callbackpb.CallbackInfo:

  • Removed the oneof result field. (Carrying an emptypb for success, or a failurepb.Failure for failure.) For now, the terminal failure of a callback can be obtained from the last_attempt_failure field. So duplicating that for some future world where we do add support for timeouts or cancellation of callbacks doesn't make sense.

On workflowpb.CallbackInfo:

  • Added a string request_id field. This is how the receiver of callbacks can disambiguate server-initiated retries from different callbacks attached to the same workflow.

@chrsmith

chrsmith commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Merging to unblock SDK development. The proto changes are purely additive, so it won't cause any problems if picked up into the server repo as-is. But I'm trying to get all the server bits squared away and merged in the next day or two.

@chrsmith
chrsmith merged commit 46ee8b8 into main Sep 8, 2026
4 checks passed
@chrsmith
chrsmith deleted the feature/worker-callbacks branch September 8, 2026 20:35
This was referenced Sep 9, 2026
Merged
Merged
chrsmith added a commit to temporalio/temporal that referenced this pull request Sep 14, 2026
@chrsmith
Merge support for NexusHandler-variant callbacks (#12023)
95dd734 
## What changed?

This PR contains the series of PRs for adding `NexusHandler`-variant
completion callbacks to Workflows, Workflow Updates, standalone
Activities, as well as standalone Nexus operations. (Which previously
didn't support completion callbacks at all.)

> This will introduce a breaking change in the `saas-temporal` repo,
since this makes a breaking change to the `common/callbacks.Validator`
interface. temporalio/saas-temporal#8783 is
ready for when this gets merged to get that addressed.

## Why?

The earlier PRs were merged into the `feature/worker-callbacks` branch
so that the server-side changes could stabilize and be merged shortly
after the API changes. (Which landed in
temporalio/api#863.)

## How did you test it?

- [x] built
- [x] run locally and tested manually
- [x] covered by existing tests
- [x] added new unit test(s)
- [x] added new functional test(s)

## Potential risks

Only that this new feature might be _too_ amazing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bergundy bergundy bergundy approved these changes
@VegetarianOrc VegetarianOrc VegetarianOrc approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@chrsmith @bergundy @VegetarianOrc