Skip to content

fix(dynamo): keep the complex decomposition inside the TRT op set - #4580

Open
shoumikhin wants to merge 2 commits into
pytorch:mainfrom
shoumikhin:fix/complex-subgraph-lowering
Open

fix(dynamo): keep the complex decomposition inside the TRT op set#4580
shoumikhin wants to merge 2 commits into
pytorch:mainfrom
shoumikhin:fix/complex-subgraph-lowering

Conversation

@shoumikhin

Copy link
Copy Markdown
Contributor

What is broken

tests/py/dynamo/lowering/test_aten_lowering_passes.py::TestComplexSubgraph::test_complex_subgraph is red on main. The test compiles a rotary attention module and expects one TensorRT engine; it gets three, with PyTorch blocks in between.

Why

The complex lowering pass hands the interior expansion to PyTorch's decompose_complex_in_graph, which re-traces the module with make_fx. The module that comes back is therefore built from whatever the complex tensor subclass dispatched to, not from the ops that survived the earlier stages of the same pass list. Three kinds of leftovers reach the partitioner:

  • aten.stack and aten.alias, which the decomposition table run at export time expands and drops, are back;
  • constants that constant folding had already reduced to one frozen tensor are live chains of ops again, and the converters for those ops want a TensorRT tensor rather than a weight;
  • aten.view_as_real and aten.view_as_complex are left at the seams.

The last one only shows up when the complex region is interior, meaning the graph takes real tensors and returns real tensors, which is what a rotary embedding does. The boundary normalizer only knew about aten.complex, aten.real and aten.imag, which is what a graph with complex inputs or outputs meets at its edges.

None of these have a TensorRT converter, so the partitioner cuts the graph around them.

Fix

Three parts, all in the lowering pass:

  1. pass the compilation settings' decomposition table to the retrace, so the retraced graph lands back in the same op set the rest of the pass list works in;
  2. re-run the constant fold exclusion marking and the constant fold after the retrace, so the constants that were frozen before are frozen again;
  3. teach the boundary normalizer the two view_as_real seams: view_as_real(view_as_complex(x)) folds to x, and view_as_real(complex(re, im)) folds to the interleaved real layout the rest of the flow expects.

Tests

The failing test passes again: the rotary attention module builds a single engine and matches eager output.

Full runs: tests/py/dynamo/lowering/ 326 passed, and the complex, rope and graph break suites 189 passed.

Three new unit tests cover the added folds and the post-retrace constant folding. All three fail without this change and pass with it.

The complex lowering pass hands the interior expansion to PyTorch's
decompose_complex_in_graph, which re-traces the module with make_fx.
The module that comes back is therefore built from whatever the complex
tensor subclass dispatched to, not from the ops that survived the
earlier stages of the same pass list, and three kinds of leftovers reach
the partitioner:

* aten.stack and aten.alias, which the decomposition table run at export
  time expands and drops, are back;
* constants that constant folding had already reduced to one frozen
  tensor are live chains of ops again, and the converters for those ops
  want a TRT tensor rather than a weight;
* aten.view_as_real and aten.view_as_complex are left at the seams.

The last one only shows up when the complex region is interior, that is
when the graph takes real tensors and returns real tensors, as a rotary
embedding does. The boundary normalizer only knew about aten.complex,
aten.real and aten.imag, which is what a graph with complex inputs or
outputs meets upstream at.

None of these have a TensorRT converter, so the partitioner cut the
graph around them: a rotary attention module that used to build one
engine came back as three, with PyTorch blocks in between.

Pass the decomposition table to the retrace, fold constants again after
it, and teach the normalizer the two view_as_real seams. The rotary
attention module builds a single engine again and matches eager output.
@meta-cla meta-cla Bot added the cla signed label Aug 26, 2026
@github-actions github-actions Bot added component: tests Issues re: Tests component: lowering Issues re: The lowering / preprocessing passes component: core Issues re: The core compiler component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Aug 26, 2026
@github-actions
github-actions Bot requested a review from zewenli98 August 26, 2026 01:24
@lanluo-nvidia lanluo-nvidia added this to the v2.14.0 milestone Aug 26, 2026
The repository lint job runs `black --check .` across the whole tree, so any
file that does not match the formatter fails CI for every open pull request,
not only the one that touched it. `tests/py/dynamo/conversion/test_cumsum_aten.py`
is currently not black-conformant on main, which turns the Python Linting check
red here.

Reformat that one file with black. This is a formatting-only change: two
statements that fit on a single line are un-wrapped. No test logic changes.

Verified by running `black --check .` on the full tree: all files pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci: full cla signed component: api [Python] Issues re: Python API component: core Issues re: The core compiler component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: lowering Issues re: The lowering / preprocessing passes component: tests Issues re: Tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants