Skip to content

[TRTLLM-15177][chore] Kimi K3: inline MLA module into modeling_kimi_linear - #18903

Closed
brnguyen2 wants to merge 1 commit into
NVIDIA:mainfrom
brnguyen2:k3/15177-inline-mla
Closed

[TRTLLM-15177][chore] Kimi K3: inline MLA module into modeling_kimi_linear#18903
brnguyen2 wants to merge 1 commit into
NVIDIA:mainfrom
brnguyen2:k3/15177-inline-mla

Conversation

@brnguyen2

Copy link
Copy Markdown
Collaborator

Description

Deferred cleanup carried over from PR #17269 (Kimi K3 / KimiLinear model support), tracked in TRTLLM-15177.

Inline the kimi_k3_mla module into modeling_kimi_linear.py, matching the per-model modeling_xxx.py convention (e.g. DeepSeek-V3). KimiK3MLAAttention and its rope/dtype helpers move directly into the model file and the standalone modules/kimi_k3_mla package is deleted. The moved code is unchanged apart from following the file's existing Linear as TrtllmLinear alias. kimi_kda stays a standalone module (general enough to warrant it).

No functional change: the moved code is identical; only its location changes.

Supersedes #18159

This replaces #18159, which is being closed rather than rebased. #18159 inlined the pre-refactor version of the module and no longer rebases cleanly onto main: after it was opened, upstream substantially rewrote the very module it deletes — the shared-MLA output-gate-hook refactor (#18375, which removed the K3-local backend-selection machinery and moved the gate to a base-class hook) and the attention-domain consolidation (#17968, which relocated the attention.* import paths). Carrying the old inlined copy forward would silently revert that work, so the delta is different enough to warrant a fresh review.

This PR is scoped to the inline only. The other two items from #18159 — removing the communication_method parameter chain and the KimiLinearConfig tuple default — are dropped from this PR: the comm-method chain is now wired through ConfigurableMoE.determine_communication_method on main and deserves its own evaluation. Those can follow as separate one-concern PRs.

Test Coverage

No new code paths are introduced — this is a pure move. Existing Kimi K3 parity suites under tests/unittest/_torch/modeling/ construct KimiK3MLAAttention via the model and exercise the moved code unchanged.

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (no new code paths here; pure move).

  • If PR introduces API changes, an appropriate PR label is added (no API change; internal _torch only).

  • Any new dependencies have been scanned (none added).

  • CODEOWNERS updated if ownership changes (no ownership change).

  • Documentation updated as needed.

  • Update tava architecture diagram if there is a significant design change (none).

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

…inear

Move the K3-specific KimiK3MLAAttention (and its rope/dtype helpers) from
the modules/kimi_k3_mla package directly into modeling_kimi_linear.py,
matching the per-model modeling_xxx.py convention (e.g. DeepSeek-V3), and
delete the now-empty package. The moved code is unchanged apart from
following the file's Linear-as-TrtllmLinear alias. kimi_kda stays a
standalone module (general enough to warrant it).

Deferred cleanup from PR NVIDIA#17269, tracked in TRTLLM-15177. Supersedes
PR NVIDIA#18159, which inlined the pre-refactor version of the module and no
longer rebases cleanly after the shared-MLA output-gate-hook refactor
(NVIDIA#18375) and attention-domain consolidation (NVIDIA#17968).

Signed-off-by: Brian Nguyen <brnguyen@nvidia.com>
@brnguyen2

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #72195 [ run ] triggered by Bot. Commit: 04086b8 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #72195 [ run ] completed with state SUCCESS. Commit: 04086b8
/LLM/main/L0_MergeRequest_PR pipeline #59237 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@brnguyen2

Copy link
Copy Markdown
Collaborator Author

Closing this out. This is a pure code-move cleanup (inlining KimiK3MLAAttention into modeling_kimi_linear.py) with no functional change, and the kimi_k3_mla module it targets is under active development — each change there (e.g. #18728) turns this into a modify/delete conflict that needs a fresh semantic re-fold, which already forced one respin (this superseded #18159 for the same reason).

The module is still a single-consumer helper and the per-model inline convention (cf. DeepSeek-V3) still applies, so the cleanup remains valid and can be reopened/reworked later if we want to prioritize it. Not worth carrying as a perpetually-conflicting draft in the meantime.

@brnguyen2 brnguyen2 closed this Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants