feat(cutile): register six backends and harden CE/SwiGLU contracts

[arde171]

Summary

• Register opt-in/last-fallback cuTile adapters for CrossEntropy, GeGLU, SwiGLU, KLDiv, RoPE, and FusedAddRMSNorm. Retain existing Triton/CuTeDSL/Ascend registrations and preference ranks.
• Fix native edge cases exposed through the adapters: remask CE vocabulary padding after softcapping; normalize strided SwiGLU inputs/gradients; materialize zero-stride RoPE inputs/reduction gradients before in-place rotation; preserve mixed query/key output dtypes.
• Bind all six operators' native launches to the input tensor's CUDA device and its current stream. Guard fused-add RMSNorm autotuning as well as its final launch. This follows the device-safe launch pattern already used by other cuTile dispatcher implementations.
• Isolate the SwiGLU global-pin test from higher-priority environment overrides while preserving its exact equality assertion and restoring prior global state.
• Retain the CE validation-statistics batching optimization. The specialized Hopper-only cuTile FusedLinearCE adapter is not included.

This extends:
#1420

Testing Done

Exact revision 1abc899e4d38bf769d9e3be88c788761cecadb61: 840 passed, 0 failed, 20 skipped on an exclusively leased NVIDIA B300, compute capability 10.3; PyTorch 2.13.0+cu130, Triton 3.7.1, cuda-tile 1.5.0.

LIGER_KERNEL_IMPL=cutile PYTHONPATH=src python -m pytest \
  test/cutile/test_device.py test/cutile/test_backend.py \
  test/cutile/test_cross_entropy.py test/cutile/test_rope.py \
  test/cutile/test_swiglu.py test/transformers/test_cross_entropy.py \
  test/ops/test_fused_add_rms_norm.py test/ops/test_geglu.py \
  test/ops/test_kl_div.py test/ops/test_rope.py test/ops/test_swiglu.py \
  -q -p no:randomly

The 20 skips are explicitly gated non-current-device cases requiring two CUDA GPUs. Physical multi-GPU execution remains unqualified. The 38 passing single-B300 device/stream witnesses observe guard entry, input-device stream selection, and real native kernel execution on a nondefault stream; outputs and gradients are compared with PyTorch references. They are not presented as physical cross-device tests.

Regression evidence:

• All 38 type-safe launch-guard witnesses fail on the preceding published source 3759be61bdff9a14c7adb1a8c27d06484a7f952a with missing-input-device-guard assertions, and pass after the fix.
• The SwiGLU pin-test failure is reproduced on that preceding source when LIGER_KERNEL_IMPL=cutile is set. Dispatch correctly gives environment selection precedence over the programmatic pin; the corrected pin-specific test clears those overrides locally without changing dispatcher behavior or loosening equality.
• The prior 48 RoPE regression/control cases remain covered. Earlier reproduction against 01392c2e821f80e756f8a539bf54f9b2e05b37b9 yielded 24 actual value/dtype failures and 24 controls.
• CE coverage includes singleton/padded vocabularies, ignore indices, weighted all-ignore behavior, invalid targets, strided arguments, smoothing, auxiliary outputs, inference/training, and non-unit gradients. Transformer witnesses assert native cuTile calls.

Changed-file Ruff lint and formatting pass. The compiler's ignored worker-warp-hint warnings remain visible.

The historical native CE operator-latency comparison is retained here:
#1456 (comment)

That measurement predates the new device-guard host code. Performance has not been requalified for this latest revision; no model end-to-end or general-DSL speedup is claimed.

• Full make test (targeted suites above were run)
• Full make checkstyle (changed-file Ruff checks were run)
• make test-convergence

No Hopper, AMD/Intel, NPU, multi-GPU, or full-model convergence qualification is claimed. Maintainer approval and repository CI gates remain separate requirements.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[arde171]

B300 validation (NVIDIA B300 SXM6, sm_103) — full test run + kernel benchmarks

Fix in this revision

The cuTile fused_linear_cross_entropy adapter was registered with the generic Blackwell CUTILE_CAPABILITY (min_cc=(10,0)), but the underlying OSS kernel (ops/cutile/ops/fused_linear_cross_entropy.py) is a Hopper/SM90 kernel — _validate_inputs requires compute capability exactly (9,0) and raises otherwise. Restricted the adapter capability to Hopper (min_cc=max_cc=(9,0)) so the dispatcher never selects it on Blackwell (parity tests cleanly skip it there; it remains available on H100/H200).

Full test suite (B300)

pytest test/cutile test/cutedsl test/backends test/ops → 1737 passed, 350 skipped, 0 failed. ruff clean.

Kernel performance — triton vs cuTile vs cuTeDSL (fwd+bwd, bf16, median ms)

Shapes: T=8192, H=4096 (cross_entropy/kl_div/jsd T=4096, V=128256/32000).

op	triton	cuTile	cuTeDSL	fastest
rms_norm	0.298	0.483	0.227	cuTeDSL
layer_norm	0.315	0.628	0.261	cuTeDSL
softmax	0.190	0.220	0.178	cuTeDSL
swiglu	0.255	0.227	0.236	cuTile
geglu	0.252	0.244	0.249	cuTile
fused_add_rms_norm	0.414	0.372	0.364	cuTeDSL
rope	0.580	1.357	0.595	triton
cross_entropy	0.987	1.170	1.038	triton
kl_div	0.508	0.486	0.369	cuTeDSL
jsd	0.947	1.534	0.586	cuTeDSL

Notes: cuTeDSL is fastest on 6/10 (norms/softmax/kl_div/jsd, up to ~28–38% vs triton); cuTile wins the GLU activations; triton stays ahead on rope & cross_entropy. ⚠️ cuTile ran without its num_worker_warps tuning (pod tileiras 13.2, needs 13.3 — a UserWarning is emitted), so cuTile numbers are conservative here.

[arde171]

Review response (GPT-6 Astra audit) + B300 re-validation

Addressed the two Hopper-CI-blocking issues this PR owned:

• Dropped the SM90-only cuTile fused_linear_cross_entropy adapter. Its kernel requires Hopper cc==(9,0) and is BF16/contiguous/mean-or-sum-only, which does not fit the generic cross-backend FLCE test matrix (FP16/FP32/bias) — it would fail those parametrizations on Hopper. Deferred to a dedicated follow-up that integrates its narrow contract (BF16-only tolerances + supported/expected-error test split).
• Guarded the new dispatcher execution smoke test to Blackwell (test/cutile/test_backend.py::test_cutile_dispatch_transformers_execute now skipif cc < (10,0)), so it cleanly skips on H100 (where cuTile CE/RoPE are min_cc=(10,0) and not registered) instead of raising ImplNotAvailableError.

Re-validated on B300 (sm_103): test/cutile + test/ops + test/backends -> 472 passed / 0 failed, ruff clean.

Pre-existing OSS kernel edge cases flagged by the audit (in ops/cutile/ops/*, not the adapters this PR adds; current parity suite is green): CE softcap on a partial final tile (padded -inf -> finite -softcap after tanh) and SwiGLU .view(-1,N) before .contiguous() on non-contiguous inputs. These predate this PR (the kernels already live on main); tracking them for a separate kernel-hardening PR.

[arde171]

Kernel benchmarks — NVIDIA B300 SXM6 (sm_103), bf16, median of 30 (warmup 10)

forward (ms — lower is better)

op	shape	triton	cuTile	cuTeDSL
rms_norm	(2048, 4096)	0.062	0.046	0.029
rms_norm	(8192, 4096)	0.056	0.170	0.044
rms_norm	(16384, 8192)	0.131	0.564	0.104
layer_norm	(2048, 4096)	0.035	0.049	0.030
layer_norm	(8192, 4096)	0.055	0.183	0.052
layer_norm	(16384, 8192)	0.166	0.584	0.131
softmax	(2048, 4096)	0.026	0.035	0.023
softmax	(8192, 4096)	0.042	0.072	0.036
softmax	(16384, 8192)	0.110	0.305	0.095
swiglu	(2048, 4096)	0.034	0.025	0.023
swiglu	(8192, 4096)	0.056	0.050	0.045
swiglu	(16384, 8192)	0.141	0.152	0.135
geglu	(2048, 4096)	0.032	0.028	0.024
geglu	(8192, 4096)	0.062	0.057	0.045
geglu	(16384, 8192)	0.186	0.189	0.135
fused_add_rms_norm	(2048, 4096)	0.041	0.035	0.037
fused_add_rms_norm	(8192, 4096)	0.072	0.067	0.069
fused_add_rms_norm	(16384, 8192)	0.188	0.185	0.184
cross_entropy	(2048, 32000)	0.179	0.197	0.192
cross_entropy	(4096, 128256)	0.576	0.760	0.613
cross_entropy	(8192, 32000)	0.338	0.389	0.312
kl_div	(2048, 32000)	0.111	0.134	0.086
kl_div	(4096, 128256)	0.576	0.711	0.584
kl_div	(8192, 32000)	0.312	0.403	0.250
jsd	(2048, 32000)	0.416	0.730	0.223
jsd	(4096, 128256)	2.957	5.335	1.475
jsd	(8192, 32000)	1.508	2.630	0.780
rope	(1, 32, 2048, 128)	0.059	0.028	0.065
rope	(1, 32, 8192, 128)	0.217	0.063	0.221
rope	(1, 32, 16384, 128)	0.408	0.109	0.413

forward+backward (ms — lower is better)

op	shape	triton	cuTile	cuTeDSL
rms_norm	(2048, 4096)	0.241	0.183	0.188
rms_norm	(8192, 4096)	0.297	0.487	0.224
rms_norm	(16384, 8192)	0.780	1.920	0.644
layer_norm	(2048, 4096)	0.236	0.225	0.227
layer_norm	(8192, 4096)	0.306	0.623	0.242
layer_norm	(16384, 8192)	0.777	2.938	0.725
softmax	(2048, 4096)	0.158	0.156	0.166
softmax	(8192, 4096)	0.201	0.219	0.175
softmax	(16384, 8192)	0.573	0.867	0.551
swiglu	(2048, 4096)	0.197	0.149	0.164
swiglu	(8192, 4096)	0.256	0.227	0.235
swiglu	(16384, 8192)	0.757	0.753	0.788
geglu	(2048, 4096)	0.185	0.154	0.135
geglu	(8192, 4096)	0.253	0.243	0.236
geglu	(16384, 8192)	0.810	0.812	0.788
fused_add_rms_norm	(2048, 4096)	0.282	0.244	0.233
fused_add_rms_norm	(8192, 4096)	0.412	0.367	0.358
fused_add_rms_norm	(16384, 8192)	1.125	1.066	1.092
cross_entropy	(2048, 32000)	0.293	0.306	0.327
cross_entropy	(4096, 128256)	0.987	1.167	1.032
cross_entropy	(8192, 32000)	0.575	0.626	0.561
kl_div	(2048, 32000)	0.310	0.295	0.221
kl_div	(4096, 128256)	1.714	1.552	1.721
kl_div	(8192, 32000)	0.908	0.859	0.663
jsd	(2048, 32000)	0.523	0.837	0.331
jsd	(4096, 128256)	3.472	5.860	1.980
jsd	(8192, 32000)	1.783	2.929	1.059
rope	(1, 32, 2048, 128)	0.240	0.689	0.248
rope	(1, 32, 8192, 128)	0.577	1.409	0.590
rope	(1, 32, 16384, 128)	1.082	2.654	1.089

Notes: cuTeDSL leads most norms/softmax/jsd and is strong on large kl_div; cuTile wins RoPE forward ~2-4x (but its RoPE backward is costly, so triton/cuTeDSL lead RoPE fwd+bwd); triton stays best for cross_entropy. cuTile ran without num_worker_warps tuning (pod tileiras 13.2 < required 13.3). cuTeDSL auto-falls back to triton above its fwd vocab limit (32768) for kl_div/CE-family at V=128256.

[arde171]

Measurement qualification: the previously posted timing tables are exploratory, not a promotion-quality comparison. Earlier GPU tests and timing jobs were launched concurrently; the harness also did not consistently reset input/gradient state or reject internal fallback. Please do not use those tables to claim a native DSL speedup or choose new defaults. A controlled rerun is in progress with exclusive GPU execution, pinned source/toolchain, PyTorch-reference forward/backward checks, strict fallback attribution, restored input/gradient buffers and five randomized-order repetitions. Unsupported regions will be labeled explicitly. The cuTile CE softcap-padding and non-contiguous SwiGLU findings remain open until their regression fixes are GPU-validated.

[arde171]

Confirmed B300 native cuTile CE improvement

Baseline: ea08307; revision: 01392c2. Only host-side validation-statistics handling changes; CUDA kernel math, bounds checks, weighted/ignored-target semantics and backend preference ranks are retained.

Hardware: NVIDIA B300 (sm_103), PyTorch 2.13.0+cu130, cuda-tile 1.5.0. These are eager full-operator measurements, not model end-to-end speedups.

Rows x vocabulary	dtype	Forward ms: before -> after	Backward ms: before -> after	Full ms: before -> after	Full latency reduction
32 x 32000	float32	0.125152 -> 0.096176	0.092896 -> 0.093984	0.217936 -> 0.192192	11.81%
32 x 32000	bfloat16	0.119008 -> 0.091088	0.092304 -> 0.093872	0.215776 -> 0.193536	10.31%
1024 x 128256	float32	0.401344 -> 0.370976	0.383456 -> 0.384656	0.767136 -> 0.738576	3.72%
1024 x 128256	bfloat16	0.294352 -> 0.264320	0.337696 -> 0.338464	0.613264 -> 0.584704	4.66%
1024 x 131073	float32	0.473488 -> 0.442912	0.391200 -> 0.391360	0.844288 -> 0.815872	3.37%
1024 x 131073	bfloat16	0.406240 -> 0.374256	0.344576 -> 0.344512	0.729936 -> 0.700448	4.04%

Method and gates

• One uncontended GPU lease; no tests/profilers overlapped timing.
• Five repetitions with randomized baseline/candidate order; 30 measured CUDA-event iterations after 10 warmups for each cell/repetition.
• Identical seeded inputs and fixed non-unit scalar upstream gradient (1.5). Input and gradient restoration and compilation are outside timing; backward uses a fresh forward graph.
• Forward includes CE gradient preparation; full means forward plus backward. Peak PyTorch allocated memory for full operation is unchanged in every measured case.
• All confirmatory phases meet CV <=5% and no >5% latency/memory regression. Full-call CV is <=1.4%. Target small-batch improvement exceeds5%.
• The initial sweep was preserved; its small-FP32 baseline CV was5.064%, so one confirmatory run with a different source ordering was used rather than promoting that noisy row. The table above is the confirmation, not the best of repeated attempts.
• Independent review found no static blocker. Native CE validation on this revision: 229 passed,0 failed, including bounds errors, positive/negative ignore indices, all-ignore, weighted normalization and softcap regressions.

The measured saving comes from combining the count/min/max validation metadata into one device-to-host transfer. No validation was removed. Older broad DSL timing tables remain exploratory; this qualified comparison is specifically native cuTile CE before/after.