Skip to content

Fix DeepSeek4 shape mismatch in mhc.py and moe.py - #4783

Closed
olufiyin19 wants to merge 1 commit into
fix-validation-pipeline-deepseek4-284b-dag_verify_checkpoint_shape-manual__dbb8774c-checkpoint_shape_validationfrom
fix-validation-pipeline-deepseek4-284b-dag_verify_checkpoint_shape-manual__2026-08-07T20-00-31-322598-00-00-checkpoint_
Closed

Fix DeepSeek4 shape mismatch in mhc.py and moe.py#4783
olufiyin19 wants to merge 1 commit into
fix-validation-pipeline-deepseek4-284b-dag_verify_checkpoint_shape-manual__dbb8774c-checkpoint_shape_validationfrom
fix-validation-pipeline-deepseek4-284b-dag_verify_checkpoint_shape-manual__2026-08-07T20-00-31-322598-00-00-checkpoint_

Conversation

@olufiyin19

Copy link
Copy Markdown
Collaborator

Fix DeepSeek4 shape mismatch in mhc.py and moe.py

@google-cla

google-cla Bot commented Aug 7, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors DeepSeek4HyperHead in mhc.py to inherit from nnx.Module with custom parameters, updates the vocab size lookup for DeepSeek V4 Hash Routing in moe.py, and integrates deepstack_process in nnx_decoders.py while cleaning up duplicate code. Feedback highlights an unreachable duplicate return statement in DeepSeek4HyperHead.call and a duplicated comment line in moe.py.

Comment thread src/maxtext/layers/mhc.py
Comment on lines +359 to +360
return x
return jnp.sum(x, axis=2)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

There is an unreachable duplicate return statement at the end of the __call__ method. The second return statement should be removed.

Suggested change
return x
return jnp.sum(x, axis=2)
return x

Comment thread src/maxtext/layers/moe.py
Comment on lines +423 to 424
# DeepSeek V4 Hash Routing
# DeepSeek V4 Hash Routing

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The comment # DeepSeek V4 Hash Routing is duplicated. Please remove the duplicate line.

Suggested change
# DeepSeek V4 Hash Routing
# DeepSeek V4 Hash Routing
# DeepSeek V4 Hash Routing

@olufiyin19 olufiyin19 closed this Aug 8, 2026
@olufiyin19
olufiyin19 deleted the fix-validation-pipeline-deepseek4-284b-dag_verify_checkpoint_shape-manual__2026-08-07T20-00-31-322598-00-00-checkpoint_ branch August 8, 2026 00:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant