Skip to content

[Feature] Add Qwen3.8-27B local-Hessian PTQ recipes - #2363

Draft
realAsma wants to merge 2 commits into
asma/layerwise-lm-headfrom
asma/qwen38-day0-local-hessian-recipes
Draft

[Feature] Add Qwen3.8-27B local-Hessian PTQ recipes#2363
realAsma wants to merge 2 commits into
asma/layerwise-lm-headfrom
asma/qwen38-day0-local-hessian-recipes

Conversation

@realAsma

@realAsma realAsma commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Type of change: new feature

Adds a reusable NVFP4 W4A4 local-Hessian PTQ recipe and a checkpoint-specific recipe for Qwen/Qwen3.8-27B.

The checkpoint recipe reproduces the validated 5.5-bit NVFP4-max AutoQuantize assignment: NVFP4 W4A4 for MLP projections and lm_head, FP8 W8A8 for self-attention and large linear-attention projections, and no quantization for the vision tower, MTP layers, KV cache, or linear_attn.in_proj_a/b.

This PR is stacked on #2339.

Usage

cd examples/hf_ptq
python hf_ptq.py \
  --pyt_ckpt_path Qwen/Qwen3.8-27B \
  --recipe models/Qwen/Qwen3.8-27B/ptq/autoquant_nvfp4_max_5p5bits_local_hessian \
  --export_path <quantized_ckpt_path>

Local-Hessian calibration currently requires a single-rank run.

Testing

  • pre-commit run --files <five changed paths>
  • pytest tests/unit/recipe/test_loader.py tests/unit/recipe/test_recipe_docs.py -q -x — 321 passed
  • Loaded and resolved both added recipes.
  • Compared the Qwen recipe against the final checkpoint hf_quant_config.json; all 401 quantized modules matched the expected FP8/NVFP4 formats.

Before your PR is "Ready for review"

  • Is this change backward compatible?: ✅
  • If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: ✅
  • Did you write any new necessary tests?: N/A — existing recipe discovery and documentation tests cover both additions.
  • Did you update Changelog?: ✅
  • Did you get Claude approval on this PR?: N/A

@copy-pr-bot

copy-pr-bot Bot commented Sep 9, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

@realAsma
realAsma force-pushed the asma/layerwise-lm-head branch from 5ff8457 to a776897 Compare September 10, 2026 18:42
Signed-off-by: realAsma <akuriparambi@nvidia.com>
@realAsma
realAsma force-pushed the asma/qwen38-day0-local-hessian-recipes branch from ac4c17c to b78e589 Compare September 10, 2026 21:15
Signed-off-by: realAsma <akuriparambi@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant