feat(gguf): serve Muse-Glimmer GGUF checkpoints

[jason-fxz]

Summary

Serves llama.cpp muse-glimmer GGUF files on the existing muse_glimmer model classes. Text only.

Changes

• models/muse_glimmer/gguf.py (new):
  • Builds the config from the GGUF metadata. llama.cpp hard-codes NoPE on the full-attention layers and a 1e-8 post-norm eps, so the loader sets the same.
  • The sliding-window pattern is one bool per layer or llama.cpp's period (every n-th layer is full attention). The sliding layers take rope.freq_base_swa. The final logit softcap defaults to llama.cpp's 30.
  • Subtracts the 1 the converter adds to the four norms of each block. The final norm is stored as is.
  • Undoes the converter's q/k row permute (llama.cpp runs interleaved rope), as for Llama.
  • The converter writes qk_scale_factor into constant q norms and sets the k norms to ones. The loader reads the factor back from the q norm and refuses a file whose q/k norms are not those constants.
  • Reads the single attn_qkv matrix of files converted with --fuse-qkv. attn_gate joins q, k and v in qkvg_proj.
  • Refuses prism files, per-tensor scales and metadata-only GGUFs (no q norm to read the factor from).
• models/muse_glimmer/model.py: embed_tokens takes the quant config, so a quantized token_embd is served packed.
• gguf/config.py, register.py, kernel/aot_models.py: register the muse-glimmer arch, share the packed-module map with the HF spec, and claim the arch in the AOT model table.

Tested models

RTX PRO 6000 Blackwell (sm_120), ft serve --model-path <file>. AIME25 problems 1–8 × 4 samples (temperature 1.0, top_k 64, top_p 0.95, from generation_config). llama.cpp f2cc728 ran with the same sampling and min_p 0. Also 48 staggered requests at concurrency 16.

Model	Quant	bs=1 tok/s	AIME FreeToken / llama.cpp f2cc728
ggml-org/Muse-Glimmer-30B-GGUF	Q4_K_M	69	19/32 / 16/32
ggml-org/Muse-Glimmer-30B-GGUF	BF16	26	–
unsloth/Muse-Glimmer-30B-GGUF	UD-Q4_K_XL	80	17/32 / –
bartowski/Muse-Glimmer-30B-GGUF	IQ2_XXS, mixed q/k/v/gate types	117	–

Staggered requests: no corrupted output on any file.

Results

• Weights: the BF16 GGUF loads bit-exact against meta-models/Muse-Glimmer-30B at layers 0, 3 and 51 (sliding and full) plus the embedding, final norm and lm_head, and the factor read from the q norms equals the HF qk_scale_factor. Dry load against the meta-built model matches every param of all four files and of a --fuse-qkv file (419, or 497 for bartowski's IQ2_XXS).
• Tokenizer: the GGUF tokenizer equals HF tokenizer.json on 11 strings with and without special tokens, and the stop ids equal the HF generation_config (200001, 200008).
• vs llama.cpp f2cc728: same file, greedy, 6 prompts × 128 tokens. Identical outputs: Q4_K_M 1/6, BF16 3/6. Every divergence is a near-tie in llama.cpp's own probabilities: both engines' picks are in its top 3, at most 0.07 apart (e.g. 0.212 vs 0.209).
• Coverage: header check (dry load, kernel coverage, tokenizer, chat template) of 1191 muse-glimmer files from 133 HF repos (draft models not counted): 1111 pass. The rest: 21 IQ1_M, 20 derivatives with a pruned vocabulary, 19 in ggml types mainline llama.cpp lacks (ROCm forks), 10 GGUF NVFP4 (9 with per-tensor scales, refused), 6 without a chat template, 4 TQ1_0/TQ2_0.
• --fuse-qkv: the HF checkpoint converted to Q8_0 with and without --fuse-qkv loads to byte-identical state dicts (419 tensors, 52 attn_qkv).
• Unit tests: tests/models and tests/tokenizer pass, except 5 test_quant_config cases that also fail on main.

Not covered

• Image input (the GGUF mmproj).
• IQ1_M weights (no MMQ kernel), TQ1_0/TQ2_0, and GGUF NVFP4 files with per-tensor scales.
• TP > 1.