[ROCm] Add Qwen3 Next FP8 QKV preparation - #5002
Draft
nholmber wants to merge 5 commits into
Draft
Conversation
Assisted-by: OpenAI Codex Signed-off-by: Nico Holmberg <nico.holmberg@amd.com>
Assisted-by: OpenAI Codex Signed-off-by: Nico Holmberg <nico.holmberg@amd.com>
Assisted-by: OpenAI Codex Signed-off-by: Nico Holmberg <nico.holmberg@amd.com>
Assisted-by: OpenAI Codex Signed-off-by: Nico Holmberg <nico.holmberg@amd.com>
Contributor
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
PR title tags: |
Assisted-by: OpenAI Codex Signed-off-by: Nico Holmberg <nico.holmberg@amd.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Add the Qwen3 Next FP8 QKV preparation kernel and allow varlen FMHA callers to
forward precomputed FP8 descales.
The kernel combines Q/K RMSNorm, RoPE, output-gate extraction, and FP8
quantization while preserving BF16 model-visible outputs. It supports the
vLLM prequantized-QKV attention integration without requiring attention to
requantize Q/K/V.
Changes
tests.
Dependency
The end-to-end model integration is in the matching vLLM draft:
Duplicate-work check
No issue number was provided. Searches of open ROCm/AITER PRs for
Qwen3 Next FP8 QKVandFP8 descales varlen FMHAfound no matching PR.Validation
Post-rebase checks on current
origin/main:Previously completed MI355 GPU validation for the unchanged feature code:
End-to-end validation with the paired vLLM branch:
AI assistance disclosure
This change was developed with OpenAI Codex assistance. The human submitter
must review every changed line and confirm they understand and can defend the
implementation before marking the PR ready.