ModelPatchLoader: migrate legacy quant metadata before quant detection - #16221
Open
berryber09 wants to merge 1 commit into
Open
ModelPatchLoader: migrate legacy quant metadata before quant detection#16221berryber09 wants to merge 1 commit into
berryber09 wants to merge 1 commit into
Conversation
load_model_patch() calls comfy.utils.detect_layer_quantization(sd, "") directly on the
raw loaded state dict, which only recognizes the native per-tensor `.comfy_quant`
marker. A legacy/reference-format quantized checkpoint (metadata carrying
`_quantization_metadata`, per-tensor `weight_codebook`/`weight_s_channel`/
`weight_s_rel` siblings instead of the native marker) is therefore never detected as
quantized, so the loader builds a plain full-precision model and load_state_dict()
fails with "Unexpected key(s)" for every quant-suffix tensor plus K-dimension shape
mismatches (the packed W4A8 weight tensor is legitimately half the K-dim of the
unpacked model).
comfy/sd.py's own UNET/checkpoint loading paths already call
comfy.utils.convert_old_quants(sd, model_prefix, metadata) right after load, at five
separate call sites, specifically to promote this legacy format into native
`.comfy_quant` markers before any detection runs. nodes_model_patch.py's
ModelPatchLoader is the one loader in the codebase that loads a state dict via
load_torch_file() and skips this step, so any legacy-format quantized model patch
(e.g. a W4A8 MiniMax H3 Fun ControlNet Union checkpoint) is unloadable through it,
regardless of which of the loader's format branches it would otherwise match.
Reproduced against a real asym_w4a8_int8/ConvRot-256 quantized MiniMax H3 Fun
ControlNet Union checkpoint (in_dim halved from 5376 to 2688 by 4-bit packing, plus
the codebook/scale sibling tensors): load_model_patch() raises before construction.
Inserting `sd, metadata = comfy.utils.convert_old_quants(sd, model_prefix="",
metadata=metadata)` right after the load_torch_file() call — matching the placement
already used throughout comfy/sd.py — was replayed by hand against the same
checkpoint: detect_layer_quantization() now returns {"mixed_ops": True}, and the full
MiniMaxH3FunControl construction + load_state_dict() succeeds in under half a second
with all control_blocks present.
The fix is unconditional and format-agnostic (convert_old_quants no-ops when there is
nothing to migrate), so it does not change behavior for any currently-working
ModelPatchLoader input — only unblocks the previously-unloadable legacy-quant case.
berryber09
requested review from
Kosinkadink,
alexisrolland,
comfyanonymous,
guill,
kijai and
rattus128
as code owners
September 10, 2026 03:29
Contributor
|
🎉 Thank you for your contribution, we really appreciate it! 🎉 Like many open source projects, we require contributors to sign our Contributor License Agreement (CLA). A CLA makes the ownership of contributions explicit, so contributors and the project share a clear understanding of how the code can be used. By signing, you:
CLAs are standard practice across major open source projects including those under the Apache Software Foundation and the Linux Foundation. Ours is based on the Apache Software Foundation's CLA. Most importantly, it would enable us to relicense the project under a more permissive license in the future, giving the project and its community greater flexibility. ✍ To sign, please post a new comment on this PR with exactly the following text: ✍ I have read and agree to the Contributor License Agreement You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review. 📜 Recent review details⏰ Context from checks skipped due to timeout. (1)
|
| Check name | Status | Explanation | Resolution |
|---|---|---|---|
| Docstring Coverage | Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. | Write docstrings for the functions missing them to satisfy the coverage threshold. |
✅ Passed checks (4 passed)
| Check name | Status | Explanation |
|---|---|---|
| Title check | ✅ Passed | The title clearly and concisely describes the main change: migrating legacy quantization metadata before quantization detection in ModelPatchLoader. |
| Description check | ✅ Passed | The description is directly related to the changeset. It explains the legacy quantization detection failure, the applied fix, and the expected behavior. |
| Linked Issues check | ✅ Passed | Check skipped because no linked issues were found for this pull request. |
| Out of Scope Changes check | ✅ Passed | Check skipped because no linked issues were found for this pull request. |
- Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
- Create stacked PR
- Commit on current branch
Comment @coderabbitai help to get the list of available commands.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What breaks: `ModelPatchLoader.load_model_patch()` calls `comfy.utils.detect_layer_quantization(sd, "")` directly on the raw `load_torch_file()` output, which only recognizes the native per-tensor `.comfy_quant` marker — it never calls `comfy.utils.convert_old_quants(sd, model_prefix, metadata)` first, unlike every UNET/checkpoint loading path in `comfy/sd.py` (5 call sites). So a legacy/reference-format quantized model patch (metadata carrying `_quantization_metadata` + per-tensor `weight_codebook`/`weight_s_channel`/`weight_s_rel` siblings instead of the native marker) is never detected as quantized: the loader builds a plain full-precision model and `load_state_dict()` fails with "Unexpected key(s)" for every quant-suffix tensor plus K-dimension shape mismatches (the packed W4A8 weight tensor is legitimately half the K-dim of the unpacked model).
Replay proof / before-after: reproduced against a real `asym_w4a8_int8`/ConvRot-256 quantized MiniMax H3 Fun ControlNet Union checkpoint. Before the fix: `detect_layer_quantization` returns `None`, `MiniMaxH3FunControl` is built at full precision, and `load_state_dict()` raises immediately. After inserting `sd, metadata = comfy.utils.convert_old_quants(sd, model_prefix="", metadata=metadata)` right after the `load_torch_file()` call (same placement `comfy/sd.py` already uses everywhere): `detect_layer_quantization` returns `{"mixed_ops": True}`, and the full `MiniMaxH3FunControl` construction + `load_state_dict()` succeeds in under half a second with all control_blocks present. The fix is unconditional and format-agnostic (`convert_old_quants` no-ops when there's nothing to migrate), so it changes nothing for currently-working `ModelPatchLoader` inputs — it only unblocks the previously-unloadable legacy-quant case.