Skip to content

fix(llama4): honour skip_logits in lce_forward - #1445

Open
yupengtang wants to merge 2 commits into
linkedin:mainfrom
yupengtang:fix-llama4-skip-logits
Open

yupengtang wants to merge 2 commits into
linkedin:mainfrom
yupengtang:fix-llama4-skip-logits

Conversation

@yupengtang

@yupengtang yupengtang commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Summary

llama4.lce_forward is the only one of the 42 forwards under transformers/model/ that has no skip_logits parameter. It still decides on its own:

if self.training and (labels is not None or shift_labels is not None):

#704 added skip_logits precisely so a caller could override that, and swept every model that existed on 2025-05-23. llama4 arrived a month later in #740 and never picked it up; everything added since has it.

Because the signature does not name it, a skip_logits argument lands in **kwargs and is silently ignored, so skip_logits=False during training still takes the fused path and returns logits=None. That is the one thing the flag exists to prevent, and the convergence harness relies on it, setting eval_batch["skip_logits"] = False before reading eval_output.logits.

The change mirrors mllama, which has the identical LigerForCausalLMLoss shape. skip_logits=None evaluates to the previous condition, so nothing moves for callers that do not pass it.

Testing Done

New test/transformers/test_llama4.py. Both cases stay on the non-fused branch, so they run without a GPU:

  • skip_logits=False in training mode returns logits and a loss. On main this fails: the flag is ignored, the fused path runs, and on a CPU box Triton raises RuntimeError: 0 active drivers. On a GPU box it fails on logits is None instead.

  • skip_logits=True with no labels raises ValueError, as it does on every other model. On main this is DID NOT RAISE.

  • pytest test/transformers/test_monkey_patch.py test/transformers/test_llama4.py: 65 passed / 1 skipped (63 + the 2 new; the skip is muse_glimmer missing from the installed transformers).

  • ruff check . and ruff format --check . with the v0.14.11 pinned in .pre-commit-config.yaml: clean.

Hardware Type: CPU. Ran on transformers 5.9.0 / torch 2.11.0.

yupengtang and others added 2 commits September 7, 2026 08:22
linkedin#704 added `skip_logits` so callers can pick between the fused loss and
materialised logits, and swept every model that existed at the time. llama4
landed a month later in linkedin#740 and never picked it up - it is now the only one
of the 42 forwards under transformers/model/ without the parameter, and it
still hardcodes `self.training and (labels is not None or shift_labels is not
None)`.

Because the signature does not name it, `skip_logits` falls into `**kwargs`
and is ignored, so `skip_logits=False` during training takes the fused path
anyway and returns `logits=None`.

Mirrors mllama, which has the same LigerForCausalLMLoss shape. The default
(`skip_logits=None`) evaluates to the previous condition, so nothing changes
for callers that do not pass it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants