Skip to content

[Triton/Gluon] Support caller-defined padding cache slot in fused MLA writer - #5010

Open
tanth47 wants to merge 1 commit into
ROCm:mainfrom
tanth47:fix/fused-mla-reserved-cache-slot
Open

[Triton/Gluon] Support caller-defined padding cache slot in fused MLA writer#5010
tanth47 wants to merge 1 commit into
ROCm:mainfrom
tanth47:fix/fused-mla-reserved-cache-slot

Conversation

@tanth47

@tanth47 tanth47 commented Aug 26, 2026

Copy link
Copy Markdown

Summary

Add an optional pad_slot_id to fused_qk_rope_cat_and_cache_mla so callers can reserve a non-negative cache slot for padding without changing the default AITER behavior.

The argument defaults to -1, so slot 0 remains writable for existing callers. When a caller opts in, both the decode/mixed and extra-prefill KV-write branches skip the selected slot. The behavior is implemented consistently in the Triton and gfx1250 Gluon kernels.

Motivation

Some runtimes reserve a physical KV-cache slot while using padding rows in slot_mapping. The generic SGLang MLA writers already protect this invariant in sgl-project/sglang#36003, but the fused AITER writer bypasses those writers and can still overwrite the reserved slot.

This change keeps AITER generic: no slot is reserved by default, and callers explicitly select one when needed. Downstream context is tracked in sgl-project/sglang#36207.

Tests

Added focused coverage that verifies:

  • default behavior still writes slot 0;
  • pad_slot_id=0 preserves slot 0 in the decode/mixed branch;
  • pad_slot_id=0 preserves slot 0 in the extra-prefill branch;
  • ordinary positive slots are still written.

The new value is a compile-time kernel argument, so the default predicate can be constant-folded to the existing behavior.

Local checks: Python compilation, Black, Ruff, and git diff --check. GPU execution is left to AITER CI.

@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:gfx1250-ffm-triton Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5010 --add-label <label>

PR title tags:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf] and op tags like [MLA] are left untouched. Add the no-auto-title label to opt this PR out of title tagging.

@tanth47

tanth47 commented Aug 26, 2026

Copy link
Copy Markdown
Author

@k50112113 Could you take a look and approve the fork workflows? This adds an opt-in padding-slot guard to the fused MLA writer while preserving the existing default. It also updates the gfx1250 Gluon mirror; please add ci:gfx1250-ffm-triton if that suite is appropriate (contributors cannot apply the label).

@zufayu
zufayu requested review from a team and Dewei-Wang-sh August 27, 2026 01:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant