Fix DeepSeek4 shape mismatch in mhc.py and moe.py - #4783
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
There was a problem hiding this comment.
Code Review
This pull request refactors DeepSeek4HyperHead in mhc.py to inherit from nnx.Module with custom parameters, updates the vocab size lookup for DeepSeek V4 Hash Routing in moe.py, and integrates deepstack_process in nnx_decoders.py while cleaning up duplicate code. Feedback highlights an unreachable duplicate return statement in DeepSeek4HyperHead.call and a duplicated comment line in moe.py.
| return x | ||
| return jnp.sum(x, axis=2) |
| # DeepSeek V4 Hash Routing | ||
| # DeepSeek V4 Hash Routing |
Fix DeepSeek4 shape mismatch in mhc.py and moe.py