Skip to content

[shardformer]: fix qwen2/qwen3 causal mask with pipeline + sequence parallelism and no flash attention - #6451

Open
LiRunGuo wants to merge 2 commits into
hpcaitech:mainfrom
LiRunGuo:fix/qwen-pp-sp-attn-mask
Open

[shardformer]: fix qwen2/qwen3 causal mask with pipeline + sequence parallelism and no flash attention#6451
LiRunGuo wants to merge 2 commits into
hpcaitech:mainfrom
LiRunGuo:fix/qwen-pp-sp-attn-mask

[pre-commit.ci] auto fixes from pre-commit.com hooks

1593654
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs