[void-mckenzie]

Serves llama.cpp gpt-oss GGUFs (gpt-oss-20b, gpt-oss-120b) on the family's own model classes. Part of #671 (Model Support: gpt-oss). Rebased onto current main (e328cf6, clean).

What it does

• Config (gpt_oss/gguf.py): parse_gguf_config from the GGUF KVs - yarn rope (factor 32, original 4096; the betas default to 32/1, which llama.cpp hardcodes for gpt-oss - its loader never reads the keys - when a file omits them; the unsloth UD and F16 releases omit them), attention.sliding_window, and the alternating sliding_attention/full_attention layout from layer 0 (every release's layer_types). attention.sliding_window.pattern is read the way llama.cpp's load_swa_pattern does: the per-layer array verbatim, else the scalar's il % p < p - 1, default 2 - no release writes the key, and a wrong-length array refuses. The swiglu's alpha 1.702 / limit 7 are family constants the GGUF does not store, as in llama.cpp. Attention sinks, router bias and q/k/v/o biases load with the dense state dict; the packed embedding takes the GGUF quant (model.py now passes quant_config through, as the qwen3 model does). A GGUF missing any bias tensor llama.cpp requires (openai-moe.cpp creates attn_output.bias, ffn_gate_inp.bias, the three ffn_*_exps.bias and a qkv bias as required tensors) refuses at config parse naming the file and tensors, instead of serving silently bias-free.
• Activation. GGUF banks store gate|up split, so the experts build as swigluoai (the uninterleaved spelling of gpt_oss_swiglu, same math; the ggml swiglu_oai epilogue and the cpu executor already know it) with interleaved=False.
• Per-expert biases - the first GGUF family whose experts carry them (ffn_{gate,up,down}_exps.bias). They ride the expert banks as two roles (gate_up_bias [E, 2I], down_bias [E, H] bf16), so each bias follows its expert through the slot cache on decode (where routing ids become slot ids) and the expert-ordered prefill/resident views. The ggml kernel adds gate/up before the gated activation and down before the routing weight, as llama.cpp's build_moe_ffn does; the NVFP4 weight_scale_2 multiply comes before the bias, as there too (no public gpt-oss file reaches that combination). bank_schema widens for the two roles on the gguf formats. A file carrying the biases on only some layers refuses at bank load. cpu/hybrid decode refuses bias files (no CPU executor format), auto resolves to offload.
• Float experts (F16/BF16/F32 files): banks load and run bf16 (the fused MoE's dtype; on main only BF16 float files ever served - the others failed the dtype check against bf16 activations), both fused impls take the optional biases, and cpu_spec sends every float type to the bf16 executor format.

Verification (RTX 5090 32 GB, driver 580.95.05, Ryzen 9 7950X3D, 187 GiB RAM)

• gguf_check dry load: unsloth UD-Q4_K_XL, ggml-org MXFP4, unsloth F16 - 3/3 pass. tok_compare vs openai/gpt-oss-20b tokenizer.json: 0 differing texts (edges, full vocab, 20k combos) on the UD and MXFP4 files.
• Served all three end to end over /v1/chat/completions: correct math (17*23+45/9 = 396 with steps) and SQL on each; harmony reasoning parses into reasoning_content; 12 concurrent requests all complete; a repeated prompt hits the radix through the sliding/full mix (0 -> 268 cached; the SWA pool tracks both groups). UD-Q4_K_XL: prefill ~19.2k tok/s (11.8k prompt), decode 145 tok/s, experts offloaded (9.5 GiB host banks).
• llama.cpp parity (llama-server b11850/9ae73e68f, CUDA, same UD-Q4_K_XL file): prompt tokens identical on all 12 probes (87/87, 94/94, ...); discrete-answer probes verbatim identical 6/6 (cube root of 343, leap year, Fe, 1989, 15% of 240, Mercury); the 6 long-form probes give the same answers and facts, differing in phrasing (two quantized-MoE kernel stacks diverge in low-order bits, and greedy decoding amplifies that into wording).
• Bias numerics: both fused impls vs a plain-torch reference with the swigluoai formula from kernel/triton/activation.py - 0.008 max rel err with the biases, 0.26 without (pinned by a test now).
• Missing-bias refusal, verified by re-running the failure scenario: the real UD-Q4_K_XL's tensor index with the bias tensors stripped refuses at config parse with the file and tensor names (previously this served silently bias-free).
• Sweep (fetch_headers --search gpt-oss, 1,093 stubs / 128 repos, kernels probed on the 5090): 969 gpt-oss trunks, 948 pass. Failures: 20 base/abliterated files with no chat template, 1 broken upload (no token_embd). Nothing crashes and nothing mis-loads silently.
• Tests: tests/moe/test_gpt_oss.py, tests/moe/test_fused_moe.py, tests/models/test_models_registry.py, tests/models/test_models_loader.py - 60 passed, including a new per-expert-bias reference check over both fused impls (test_fused_moe.py extends its existing reference helper with the swigluoai formula and bias params); tests/moe/test_offload.py (the rebased main's seams) passes too. No per-family GGUF test file, per the feat(gguf): serve GLM-5.3-Flash GGUF checkpoints on glm5-next #721 review direction.

Not covered

• SoAIHQ's gpt-oss-*-VISION* repos: mainline gpt-oss text plus an internvl projector mmproj (clip.projector_type=internvl, 436 v.* tensors) - a community graft no FreeToken tower implements; the text files of those repos serve.
• cpu/hybrid/--moe-cpu-layers for gpt-oss files: the per-expert biases have no CPU executor format, refused at kernel selection.

[jason-fxz]

Thanks for adding gpt-oss GGUF. Four things to fix before merge:

1. YaRN truncate (models/gpt_oss/gguf.py): the HF gpt-oss config sets "truncate": false, and transformers and FreeToken's HF gpt-oss path both honor it. The GGUF rope_scaling leaves it out, so GGUF files get the truncated correction range (8..18 instead of 8.09..17.40). Frequency dims 9-17 then differ from the HF path, and the two rope tables are already up to 0.23 apart at position 1024. llama.cpp always truncates, so a parity check against it does not catch this. Please add "truncate": False as a family constant next to alpha / limit.

2. bf16 cast of float experts (models/gguf/experts.py, _bank_spec): the cast also hits mixed float/quant layers in every family, which main keeps in their own dtype since #702. HauhauCS Qwen3.6-35B Q8_K_P (13 layers of F16 gate_up + Q8_0 down) serves on main with --moe-strategy hybrid. With this PR it fails at startup with ValueError: MoE layer 20 gate_up bank (256, 1024, 2048) torch.bfloat16 is not F16 rows of 2048 values, because cpu_spec still sends those layers to the gguf CPU format as F16. On GPU their F16 GEMM also drops to bf16. Please cast only when both gate_up and down are float (the layer runs the bf16 fused MoE) and keep mixed layers in their own dtype. If _expert_rows stays a view and banks[...].copy_(r) does the conversion, --load-format dummy also stops reading the expert data (three F16 layers now peak at +7.9 GiB RSS, vs +3.9 GiB on main).

3. NVFP4 down scale vs. down bias (layers/quantization/moe/gguf.py, non-MMVQ down branch): above mmvq_max_tokens, weight_scale_2 is folded into topk_weights and the bias is then added to the unscaled y, so the bias is scaled too: sum w*s*(y + b) instead of sum w*(s*y + b). No public gpt-oss file has NVFP4 experts yet, but with synthetic NVFP4 experts through GgmlMoEKernel.apply, prefill (T=256) is 94% off the reference while decode (T=4) is within 0.4%. With a bias, scale y in place instead; prefill then lands within 0.4% too:

   if down_scales is not None and down_bias is not None:
       # the bias adds after weight_scale_2, so the scale cannot fold into the routing weights
       y.mul_(down_scales[ids, 2].unsqueeze(-1))
   elif down_scales is not None:
       topk_weights = topk_weights * down_scales[ids, 2]

4. w2_bias with apply_router_weight_on_input (moe/fused.py, both impls): the guard checks only w1_bias, but w2_bias is always multiplied by the routing weight. In that mode both impls are 66% off your test reference, which adds it unweighted. Please make the assert cover both:

   # both bias adds assume the router weight scales the expert output
   assert (w1_bias is None and w2_bias is None) or not apply_router_weight_on_input, "per-expert biases need the router weight on the output"

Smaller cleanups:

• OffloadMoeCache.set_bank_sources widens the gguf schema at runtime when bias roles show up. The loader already knows the file is biased when it names the format (experts.py, ExpertBanks("gguf_scaled" if scaled else "gguf", ...)), so registering the biased layouts in _BANK_SCHEMAS keeps it the single place a format's bank layout is declared.
• _LAYER_LEAVES: mlp.gate_proj / up_proj / down_proj and mlp.gate match no gpt-oss module (no dense MLP; the router is mlp.router).
• The pattern is None branch repeats the scalar branch with p = 2; md.get(key, 2) covers it.

Not for this PR: fusing the expert biases into the kernels (the [T, k, 2I] gathers on prefill, the MMVQ glu path on decode) and bias support in the CPU executor. We will follow up on those separately.

[void-mckenzie]

All four and the cleanups are in 15b41ed, rebased onto main ee43b2a (the #823 conflict keeps both the inactive clamp and the bias kwargs).

1. truncate: _ROPE_TRUNCATE = False in the GGUF rope_scaling; the parsed dict matches HF field for field on the real UD file.
2. bf16 cast: per layer now, bf16 only when gate_up and down are both float; mixed layers keep the file's dtype. _expert_rows is a view again, so dummy loads skip the expert data. I don't have the HauhauCS file locally so no direct rerun of it; _bank_spec unit-checked for the three cases and the F16 stubs load in the sweep.
3. nvfp4 down scale: your snippet verbatim. Prefill vs decode on the same synthetic banks: 0.79% max rel diff.
4. w2_bias assert: your assert in both impls; fires for w2_bias + router-on-input, no-bias mode unchanged.

Cleanups: biased formats registered in _BANK_SCHEMAS (runtime widening gone), dead _LAYER_LEAVES dropped, pattern read is md.get(key, 2).

Tests: 113 passed (the PR's four files + test_offload.py). Sweep unchanged: 948/969, same failure classes.

[jason-fxz]

LGTM.