fix(executorch): register device copy kernels in the packaged runner - #4581
Merged
lanluo-nvidia merged 2 commits intoAug 26, 2026
Merged
Conversation
An exported program that is fully delegated to TensorRT still runs two
operators outside the delegate: the host to device copy of its inputs and
the device to host copy of its outputs. The device placement pass inserts
them at export time.
The Bazel build of the example ExecuTorch runner links the core runtime
only, and the core runtime carries no kernels, so loading any exported
program fails:
kernel 'et_copy::_h2d_copy.out' not found
load_method('forward') failed with error 0x14
Compile the two kernel implementations from the pinned ExecuTorch tree and
register them from a small registration-only library that executables
depend on directly. Only these two are registered, not the whole portable
kernel set, since a fully delegated program never calls the rest.
Registration is kept out of the backend library because the kernel registry
aborts when the same kernel is registered twice, and the backend can be
linked into both a shared library and an executable. It is also kept out of
the CMake source list, since that build already links ExecuTorch's own
kernel library.
Tested by compiling both translation units against the pinned ExecuTorch
headers and checking that the registration object's undefined symbols match
the kernel object's defined symbols, and that the registered names are
exactly et_copy::_h2d_copy.out and et_copy::_d2h_copy.out. The end to end
run is covered by the existing CI job that runs the example runner.
shoumikhin
force-pushed
the
fix/packaged-runner-device-copy-kernels
branch
from
August 26, 2026 01:38
596dd81 to
058c53d
Compare
The repository lint job runs `black --check .` across the whole tree, so any file that does not match the formatter fails CI for every open pull request, not only the one that touched it. `tests/py/dynamo/conversion/test_cumsum_aten.py` is currently not black-conformant on main, which turns the Python Linting check red here. Reformat that one file with black. This is a formatting-only change: two statements that fit on a single line are un-wrapped. No test logic changes. Verified by running `black --check .` on the full tree: all files pass.
7 tasks
lanluo-nvidia
pushed a commit
that referenced
this pull request
Aug 26, 2026
…4581) Co-authored-by: shoumikhin <shoumikhin@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is broken
The
executorch-runtime-buildjobs are red onmain, 10 of them. The example runner builds fine and then fails at run time:Why
An exported program that is fully delegated to TensorRT still runs two operators outside the delegate: the host to device copy of its inputs and the device to host copy of its outputs. The device placement pass inserts them at export time.
The Bazel build of the example runner links the core ExecuTorch runtime only, and the core runtime carries no kernels, so those two operators have nothing to dispatch to and loading any exported program fails. The CMake build does not hit this because it links ExecuTorch's own kernel library.
Fix
Compile the two kernel implementations from the pinned ExecuTorch tree and register them from a small registration-only library that executables depend on directly.
Two deliberate choices:
Verification
No GPU was available for an end to end run, so this was checked at the symbol level: both translation units compile against the pinned ExecuTorch headers, the registration object's undefined symbols match the kernel object's defined symbols, and the registered names are exactly
et_copy::_h2d_copy.outandet_copy::_d2h_copy.out.The end to end path is covered by the CI job that runs the example runner, which is the job that is currently failing.