[CuTeDSL] Add llc{...} compile-option token and NvvmOptions to pass LLVM codegen flags to the NVVM backend

[zkyue]

TL;DR

There is currently no supported way to hand an LLVM codegen (llc) flag to the DSL's JIT pipeline. This PR adds one, strictly opt-in:

CUTE_DSL_COMPILER_OPT="llc{aggressive-machine-cse=1}"

compiled = cute.compile(fn, *args, options="llc{aggressive-machine-cse=1}")
# or programmatically:
compiled = cute.compile[NvvmOptions("-Xllc -aggressive-machine-cse=1")](fn, *args)

Each brace item is forwarded to LLVM codegen as an -Xllc -<flag> pair through the nvvm-options option of the cute-to-nvvm pipeline. Nothing is forwarded unless the user asks for it: with no llc{...} token the new option contributes no text to the pipeline (see Limitations for the one-space registry side effect every added option has).

Motivation

On a large production sparse-attention backward kernel on B200 (sm_100a, CUDA 13), a single llc flag, -aggressive-machine-cse=1, gives a measurable, numerics-preserving win:

• Static SASS: −3.9% instructions on the stock kernel (5984 → 5752) and −6.4/−6.6% on a tuned variant (5984/5976 → 5592), dominated by IMAD (−45/−84) and MOV (−49/−52) dedup — the classic redundant-address-math CSE signature.
• Runtime (nsys pure-kernel time, i.e. kernel duration only — no wrapper/launch/sync; n=150 per arm from 3 interleaved rounds): −1.2 to −2.1% median, −1.4 to −1.7% mean, −2.0 to −2.2% min across all four pairings of {stock, tuned} × {cutlass-dsl 4.6.1, 4.7.0}. Honest headline: ~1.5–2% pure-kernel on B200 at the production shape.
• Numerics: the deterministic output is bitwise-identical with the flag on vs. off in all four pairings; the fp32-atomic outputs stay inside their intrinsic run-to-run envelope. The flag only affects instruction selection/scheduling of integer/address math.

That flag is emphatically not a universal win, which is exactly why this has to be an opt-in knob rather than a pipeline default. A follow-up sweep of the same flag over five other production CuTe DSL kernels (B200, one toolchain pin, nsys pure-GPU medians, n=100/arm, outputs bitwise or within the fp32-atomic run-to-run envelope in all five) found effects spanning −15.2% to +0.33%, with sign flips:

kernel	Δ time	notes
sparse-attention indexer forward, fp8	−15.2%	static spill ops halved (21→12 LDL / 21→12 STL); this kernel sits in a bistable codegen regime where an unrelated schedule perturbation costs +15%
top-k selection	−1.5%	−88 static instructions
mean-pool scoring	−0.8%	−208 static instructions; already compiled at --opt-level 2 and the flag still helps on top
sparse-attention indexer forward, bf16	+0.25%	at the measurement floor
sparse-attention indexer backward	+0.33%	small but reproducible regression; picks up an 8-byte stack frame and one LDL/STL pair

So the flag is safe with respect to numerics but must be chosen per kernel with a perf gate behind it — a scoped, per-compile passthrough (what this PR adds), never a default change.

Today the only way to do this is to monkeypatch CompileOptions.to_str and splice nvvm-options='-Xllc ...' into its return value — fragile, version-dependent, and unsupported. The pipeline plumbing for it already exists (the cute-to-nvvm pass accepts nvvm-options, verified on the 4.6.1 and 4.7.0 wheels); what is missing is purely the user-facing vocabulary on the Python side.

What this adds

1. NvvmOptions — a registered StringCompileOption with _option_name = "nvvm-options", the exact analog of the existing PtxasOptions (ptx-options). It serializes as nvvm-options='<value>' into the cute-to-nvvm{...} brace built by _get_pipeline and, because get_module_hash hashes CompileOptions.to_str(), is automatically part of the compile-cache key. Exported next to PtxasOptions (cute.NvvmOptions).
2. llc{<flag>[,<flag>...]} — a compact token in CompileOptions._apply_opt_string, following the existing brace-token pattern (debug{...}, warnings{...}, remarks{...}). Each item is validated for shape ([A-Za-z0-9][\w.-]*(?:=[\w.-]+)?, spelled without the leading -) and appended to NvvmOptions as an -Xllc -<flag> pair. Multiple items, and repeated llc{...} tokens, accumulate. The token is accepted everywhere compact tokens already are: CUTE_DSL_COMPILER_OPT, cute.compile(..., options=...) (pure-compact and mixed compact+legacy paths), and _apply_opt_string.

Small enabling details:

• NvvmOptions sets _suppress_when_absent, so an absent --nvvm-options argparse flag on the mixed legacy path does not clobber a value set by the llc{...} token (this is exactly what _suppress_when_absent exists for).
• --nvvm-options joins --ptxas-options in the hyphen-value argparse workaround in _parse_compile_options_from_str.
• Appending is idempotent: a flag already present is not added again. CUTE_DSL_COMPILER_OPT is re-applied to the (reused) CompileOptions on every compile, so a non-idempotent append would grow the value — and the compile-cache key — on each compile.
• Precedence, documented in the docstring, the .rst, and a test: compact tokens are applied before the legacy string API, so an explicit --nvvm-options <value> replaces flags contributed by llc{...} in the same options string, regardless of order.

Docs: media/docs/pythonDSL/cute_dsl_general/dsl_jit_compilation_options.rst gains an nvvm-options table row, both string-API and Python-type examples next to the ptxas-options ones, and a short section on forwarding llc flags (including the env-var form).

Injection safety

A user-supplied string ends up inside a quoted slot of the pass-pipeline spec, so both entry points validate:

• The llc{...} token accepts only [A-Za-z0-9][\w.-]*(?:=[\w.-]+)? per comma-separated item — no quotes, spaces, braces, or backslashes can get through, and the flag is composed as -Xllc -<flag> by the parser itself.
• NvvmOptions (constructor and .value setter) rejects any value whose quote-stripped core contains ', ", {, }, \, or control whitespace — i.e. anything that could terminate the nvvm-options='...' slot or the pipeline nesting. Surrounding quotes are tolerated because serialize() strips one layer before re-quoting (that is how the legacy string path delivers values).

Error behavior (documented, matches what libNVVM actually does)

• Malformed token syntax (llc, llc=1, llc{}, llc{-foo}, llc{a b}, unclosed brace, embedded quote) fails at parse time with a ValueError carrying a targeted message.
• Well-formed flags that libNVVM rejects surface as a compile-time diagnostic (CompilerDiagnosticError, "NVVM backend compilation failed") — verified with -time-passes.
• Flags/values libNVVM does not recognize are silently ignored by the backend (verified empirically). The docstrings say so: shape validation happens in Python, semantic validation is the backend's.

Tests

test/python/CuTeDSL/test_compile_options_llc.py (unittest, follows sibling files):

• Parse: default contributes no option text; single/multiple/valueless flags; accumulation across tokens; idempotence under repeated application; documented --nvvm-options precedence; pure-compact and mixed compact+legacy string API; coexistence with warnings{...}.
• Rejection: 9 malformed-input cases raise ValueError with the expected message.
• Programmatic: NvvmOptions direct use, export identity, quote/brace rejection on constructor and assignment, surrounding-quote tolerance.
• Compilation (end-to-end on GPU): options="llc{aggressive-machine-cse=1}" compiles; cute.compile[NvvmOptions(...)] compiles; a backend-rejected flag surfaces as CompilerDiagnosticError mentioning NVVM (skipped, not failed, if a future libNVVM accepts that flag).

Verified on B200 (sm_100a): 29/29 pass with the patch (backend-touching tests run in
disposable subprocesses because accepted -Xllc flags are process-global; the new
process-global-state test skips itself if its witness flag is invisible on a future
libNVVM); on stock, llc{...} is rejected with option 'llc' does not take {...} sub-options and NvvmOptions does not exist. The env-var route was additionally verified end-to-end in a fresh process, including that three successive compiles produce a stable option string. The three pre-existing test files in test/python/CuTeDSL/ behave identically with and without the patch.

Limitations

• Process-global flag state (measured; documented and pinned by a test).
  -Xllc flags are parsed into LLVM's process-global option registry (cl::opt) by
  the in-process backend, and the registry is only re-parsed by compiles that
  themselves pass an nvvm-options token. A compile using llc{...} therefore
  leaves the flag in effect for every subsequent cute.compile in the same
  process that does not pass its own nvvm-options — the flag does not expire with
  the compile that set it. Verified on B200: a kernel compiled with no options after
  a flagged compile of a different kernel is byte-identical (PTX, cubin and SASS)
  to an explicitly flagged build of itself, and differs from a never-flagged process
  (witness flag -nvptx-sched4reg, which is PTX-visible even on trivial kernels).
  Two follow-on hazards are documented with it: (1) compile caches key on the
  requested option string (get_module_hash), not on inherited registry state, so
  an unflagged cached JIT compilation in a flagged process can store flag-affected
  code under the clean key — including in the persistent file cache, from which it
  can be served to later processes (explicit cute.compile always recompiles and is
  unaffected); (2) a paired reset compile that passes the flag's default value
  explicitly is only best-effort — it needs a value-expressible default, must
  actually reach the backend (cache-served compiles do not re-parse), and cannot undo
  CUTE_DSL_COMPILER_OPT, which is re-applied every compile; full isolation is a
  fresh process. This is a limitation of the current in-process backend and cannot be
  scoped from Python; automatic reset semantics are unimplementable in general (the
  DSL cannot know an arbitrary flag's default, and libNVVM exposes no registry-reset
  entry point). The .rst gained a "Process-global flag state" section;
  TestLlcFlagStateIsProcessGlobal pins the behavior (one fresh interpreter per
  compile ordering, victim always the second compile so warm-state matches) and the
  pre-existing end-to-end compile tests now also run in disposable subprocesses so
  accepted flags cannot poison later compiles in a test worker.

• llc flags are inherently tied to the LLVM version inside libNVVM; this is a power-user escape hatch (same contract as ptxas-options / nvcc -Xptxas), not a stable API. Opt-in only: with no llc{...} token, NvvmOptions serializes to nothing.

• Registering a new option adds one separator space to the default serialized option string (to_str appends a space per option, including empty ones), so default compile-cache keys change once, as they do whenever an option is added — the DSL's own default string went from 88 to 106 bytes between 4.6.1 and 4.7.0. The extra whitespace is inert in the pass-pipeline string.

• The backend silently ignores unrecognized flags (libNVVM behavior, not controllable from Python), so a typo in a flag name does not error; flags the backend rejects do error.

• No SASS-diff regression test: on toy kernels -aggressive-machine-cse=1 is a no-op (small kernels get re-CSEd downstream regardless), so an instruction-count assertion would be fragile. The compile-success and backend-rejection tests pin the passthrough contract instead.

[zkyue]

While integrating this into our production stack we found a behavior of the underlying
mechanism that deserves prominent disclosure, so I have amended the PR rather than
leave it implicit.

-Xllc flags forwarded through nvvm-options are parsed into LLVM's
process-global cl::opt registry, and the registry is only re-parsed by compiles
that themselves pass an nvvm-options token. So a single llc{...} compile leaves the
flag in effect for every subsequent cute.compile in the same process that doesn't
pass its own nvvm-options — it does not expire with the compile that set it. We hit
this for real: a kernel compiled after our flagged kernel, with no options of its
own, came out byte-identical to an explicitly flagged build (PTX, cubin and SASS;
reproducible with -nvptx-sched4reg, which is visible even on trivial kernels).

Two follow-on consequences worth knowing:

• Compile caches key on the requested option string, not on inherited registry state,
  so an unflagged cached JIT compilation performed while a flag is active can persist
  flag-affected code under the clean cache key — including in the file cache, i.e.
  beyond the process. (Explicit cute.compile always recompiles and is unaffected.)
• A paired "reset" compile passing the flag's default value works, but only
  best-effort: it needs a value-expressible default, must actually reach the backend,
  and cannot undo CUTE_DSL_COMPILER_OPT (re-applied every compile). Full isolation is
  a fresh process.

I considered making the DSL auto-reset after each flagged compile and rejected it as
unsound: the DSL cannot know an arbitrary flag's default value to reset to, and libNVVM
exposes no registry-reset entry point. What the amendment does instead:

• documents the semantics, both hazards, and the best-effort reset recipe in the
  .rst ("Process-global flag state") and, in short form, in the NvvmOptions
  docstring;
• adds TestLlcFlagStateIsProcessGlobal, which pins the behavior — one fresh
  interpreter per compile ordering, the victim kernel always the process's second
  compile so backend warm-state is identical across arms, PTX byte-compare against
  clean and explicitly-flagged builds. It self-skips if the witness flag is invisible
  on a future libNVVM, and fails with a targeted message if the backend ever gains
  per-compile scoping;
• hardens the existing end-to-end tests to run in disposable subprocesses too, so
  their accepted flags cannot poison later compiles in a shared test worker (the child
  env drops CUTE_DSL_COMPILER_OPT; PYTHONHASHSEED pinned).

No functional code change; the feature stays opt-in and its serialization with the
option unset remains byte-identical to stock. The file is now 29 tests, all passing on
B200 against the 4.7.0 wheel with the branch overlaid. If you would rather pair this
with conservative cache behavior (e.g. tainting/disabling the file cache once an
nvvm-options compile has run in the process), I am happy to implement that in a
follow-up — it is a functional change, so I kept it out of this amendment.

(edited: removed an internal build-log section that was accidentally included below the technical content — no changes above this line)