[Triton/Gluon] Support caller-defined padding cache slot in fused MLA writer - #5010
[Triton/Gluon] Support caller-defined padding cache slot in fused MLA writer#5010tanth47 wants to merge 1 commit into
Conversation
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
PR title tags: |
|
@k50112113 Could you take a look and approve the fork workflows? This adds an opt-in padding-slot guard to the fused MLA writer while preserving the existing default. It also updates the gfx1250 Gluon mirror; please add |
Summary
Add an optional
pad_slot_idtofused_qk_rope_cat_and_cache_mlaso callers can reserve a non-negative cache slot for padding without changing the default AITER behavior.The argument defaults to
-1, so slot 0 remains writable for existing callers. When a caller opts in, both the decode/mixed and extra-prefill KV-write branches skip the selected slot. The behavior is implemented consistently in the Triton and gfx1250 Gluon kernels.Motivation
Some runtimes reserve a physical KV-cache slot while using padding rows in
slot_mapping. The generic SGLang MLA writers already protect this invariant in sgl-project/sglang#36003, but the fused AITER writer bypasses those writers and can still overwrite the reserved slot.This change keeps AITER generic: no slot is reserved by default, and callers explicitly select one when needed. Downstream context is tracked in sgl-project/sglang#36207.
Tests
Added focused coverage that verifies:
pad_slot_id=0preserves slot 0 in the decode/mixed branch;pad_slot_id=0preserves slot 0 in the extra-prefill branch;The new value is a compile-time kernel argument, so the default predicate can be constant-folded to the existing behavior.
Local checks: Python compilation, Black, Ruff, and
git diff --check. GPU execution is left to AITER CI.