[TRTLLM-15177][chore] Kimi K3: inline MLA module into modeling_kimi_linear - #18903
[TRTLLM-15177][chore] Kimi K3: inline MLA module into modeling_kimi_linear#18903brnguyen2 wants to merge 1 commit into
Conversation
…inear Move the K3-specific KimiK3MLAAttention (and its rope/dtype helpers) from the modules/kimi_k3_mla package directly into modeling_kimi_linear.py, matching the per-model modeling_xxx.py convention (e.g. DeepSeek-V3), and delete the now-empty package. The moved code is unchanged apart from following the file's Linear-as-TrtllmLinear alias. kimi_kda stays a standalone module (general enough to warrant it). Deferred cleanup from PR NVIDIA#17269, tracked in TRTLLM-15177. Supersedes PR NVIDIA#18159, which inlined the pre-refactor version of the module and no longer rebases cleanly after the shared-MLA output-gate-hook refactor (NVIDIA#18375) and attention-domain consolidation (NVIDIA#17968). Signed-off-by: Brian Nguyen <brnguyen@nvidia.com>
|
/bot run |
|
PR_Github #72195 [ run ] triggered by Bot. Commit: |
|
PR_Github #72195 [ run ] completed with state
|
|
Closing this out. This is a pure code-move cleanup (inlining The module is still a single-consumer helper and the per-model inline convention (cf. DeepSeek-V3) still applies, so the cleanup remains valid and can be reopened/reworked later if we want to prioritize it. Not worth carrying as a perpetually-conflicting draft in the meantime. |
Description
Deferred cleanup carried over from PR #17269 (Kimi K3 / KimiLinear model support), tracked in TRTLLM-15177.
Inline the
kimi_k3_mlamodule intomodeling_kimi_linear.py, matching the per-modelmodeling_xxx.pyconvention (e.g. DeepSeek-V3).KimiK3MLAAttentionand its rope/dtype helpers move directly into the model file and the standalonemodules/kimi_k3_mlapackage is deleted. The moved code is unchanged apart from following the file's existingLinear as TrtllmLinearalias.kimi_kdastays a standalone module (general enough to warrant it).No functional change: the moved code is identical; only its location changes.
Supersedes #18159
This replaces #18159, which is being closed rather than rebased. #18159 inlined the pre-refactor version of the module and no longer rebases cleanly onto
main: after it was opened, upstream substantially rewrote the very module it deletes — the shared-MLA output-gate-hook refactor (#18375, which removed the K3-local backend-selection machinery and moved the gate to a base-class hook) and the attention-domain consolidation (#17968, which relocated theattention.*import paths). Carrying the old inlined copy forward would silently revert that work, so the delta is different enough to warrant a fresh review.This PR is scoped to the inline only. The other two items from #18159 — removing the
communication_methodparameter chain and theKimiLinearConfigtuple default — are dropped from this PR: the comm-method chain is now wired throughConfigurableMoE.determine_communication_methodonmainand deserves its own evaluation. Those can follow as separate one-concern PRs.Test Coverage
No new code paths are introduced — this is a pure move. Existing Kimi K3 parity suites under
tests/unittest/_torch/modeling/constructKimiK3MLAAttentionvia the model and exercise the moved code unchanged.PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (no new code paths here; pure move).
If PR introduces API changes, an appropriate PR label is added (no API change; internal
_torchonly).Any new dependencies have been scanned (none added).
CODEOWNERS updated if ownership changes (no ownership change).
Documentation updated as needed.
Update tava architecture diagram if there is a significant design change (none).
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.