feat(idl): task aliases — mutable named pointers to immutable versions - #8013
Draft
dansola wants to merge 1 commit into
Draft
feat(idl): task aliases — mutable named pointers to immutable versions#8013dansola wants to merge 1 commit into
dansola wants to merge 1 commit into
Conversation
Signed-off-by: Daniel Sola <daniel.sola@union.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are the changes needed?
There is no first-class way to say "production runs this version" and change it later. An external caller — a Lambda, an event bus, a scheduler outside Flyte — must name something stable, and an operator must be able to move what that name resolves to.
Flyte 1 had this as launch plan activation: registering a new version was inert, and activation was the deliberate act that changed what production ran. Flyte 2 has no equivalent. The only way to build it today is to exploit task-version mutability — deploy the same code twice, once under an immutable tag and once under a label like
prod, and have the caller pinversion="prod". That is an overwrite rather than a pointer: nothing linksprodto the version it stands for, there is no history of who moved it, rolling back means redeploying an old commit under the label, and a run's storedversion: "prod"no longer identifies the code that ran.The same shape exists across the ecosystem — Lambda versions and aliases, image digests and tags, git commits and branches. In every case the mutable name is a separate object from the immutable artifact.
Raised by a team running event-driven pipelines where an AWS Lambda fires a pipeline by name and operators need to control which code version it reaches.
What changes were proposed in this pull request?
Two proto files; the rest of the diff is generated (10 files).
TaskIdentifiergains an alias selector (flyteidl2/task/task_definition.proto)with a message-level CEL rule requiring exactly one of the two. Putting the selector on the identifier means every RPC that already takes one resolves an alias for free —
GetTaskDetails,CreateRun, and later triggers.Two deliberate choices worth reviewing:
oneof. Folding the existingversioninto one is wire-compatible but changes the generated Go and Python accessors, breaking every caller. A second field plus a CEL constraint gets the same guarantee without the churn.versionloses its per-fieldmin_len. An alias-selected identifier leavesversionunset, and proto3 renders that as"", whichmin_len = 1rejects. The message-level rule now carries that requirement instead, so the constraint is preserved rather than dropped.Five RPCs on
TaskService(flyteidl2/task/task_service.proto):SetTaskAlias,GetTaskAlias,ListTaskAliases,DeleteTaskAlias,GetTaskAliasHistory, plusTaskAliasName,TaskAlias, andTaskAliasRevision.SetTaskAliasResponsecarriesprevious_versionso a client can renderprod: v1.4.0 -> v1.7.0without a second read. Aliases are scoped per(org, project, domain, task_name), soprodin staging andprodin production are distinct pointers.Server-side implementation is in the Union cloud repo; this PR is the contract only.
How was this patch tested?
Generated with the repo's own CI image (
ghcr.io/flyteorg/flyte/ci:latest) viamake buf-goandmake buf-python, socheck-generateshould be satisfied.buf lintandbuf format -wclean.The contract was exercised end to end against a live backend built from these protos (a k3d devbox running the cloud implementation): a Go integration test drives all five RPCs plus alias-selected
GetTaskDetails, and a Python script drives the same flow through real deploys and real task runs. Both pass. Details and output are in the companion cloud PR's test plan.Labels
Please add one or more of the following labels to categorize your PR:
This is important to improve the readability of release notes.
Setup process
Screenshots
Check all the applicable boxes
Related PRs
flyteorg/flyte-sdk#1565
https://github.com/unionai/cloud/pull/18303
Stack
If you do use
git townto manage PR Stacks, the stack relevant to this PRwill show below. Otherwise, you can ignore this section.
Docs link