Qualcomm: 8-bit per-channel weight scales are floored at the 16-bit eps, and the HTP miscomputes near-zero channels

[msluszniak]

get_ptq_per_channel_quant_config floors every per-channel weight scale at DEFAULT_EPS_16BIT (1.5e-9), 8-bit weights included. A conv output channel that BatchNorm has all but zeroed gets that scale, ~1e6x below its neighbours, and the HTP computes the channel as garbage. Host fake-quant is correct.

16a8w on SM8850 (Galaxy S26 Ultra, QAIRT 2.47), 200 VOC2012 val images, mIoU:

	default eps	eps=DEFAULT_EPS_8BIT	XNNPACK int8
DeepLabV3-ResNet101	0.10%	78.90%	78.82%
FCN-ResNet101	28.69%	74.93%	74.99%

Repro, one conv. Writes both .pte files, the input and the fp32 reference to the directory given as the first argument:

import sys
import torch
from executorch.backends.qualcomm.export_utils import make_quantizer
from executorch.backends.qualcomm.quantizer.qconfig import DEFAULT_EPS_8BIT
from executorch.backends.qualcomm.quantizer.quantizer import QuantDtype
from executorch.backends.qualcomm.serialization.qc_schema import QcomChipset
from executorch.backends.qualcomm.utils.utils import (
    generate_htp_compiler_spec, generate_qnn_executorch_compiler_spec,
    get_qnn_context_binary_alignment, to_edge_transform_and_lower_to_qnn)
from executorch.exir.capture._config import ExecutorchBackendConfig
from executorch.exir.passes.memory_planning_pass import MemoryPlanningPass
from torchao.quantization.pt2e.quantize_pt2e import convert_pt2e, prepare_pt2e

torch.manual_seed(0)
conv = torch.nn.Conv2d(16, 16, 3, padding=1)
with torch.no_grad():
    conv.weight[0] *= 1e-8  # what BatchNorm with gamma ~ 0 leaves after folding
    conv.bias[0] = 0.0
model = torch.nn.Sequential(conv).eval()
calib = [torch.randn(1, 16, 32, 32) for _ in range(8)]
x = torch.randn(1, 16, 32, 32)
out = sys.argv[1]
x.numpy().tofile(f"{out}/input_0.raw")
with torch.no_grad():
    model(x).numpy().tofile(f"{out}/ref.raw")
for tag, eps in [("default", None), ("eps8bit", DEFAULT_EPS_8BIT)]:
    p = prepare_pt2e(torch.export.export(model, (x,)).module(),
                     make_quantizer(quant_dtype=QuantDtype.use_16a8w, soc_model="SM8850", eps=eps))
    for c in calib:
        p(c)
    q = convert_pt2e(p)
    spec = generate_qnn_executorch_compiler_spec(soc_model=QcomChipset.SM8850,
        backend_options=generate_htp_compiler_spec(use_fp16=False))
    prog = to_edge_transform_and_lower_to_qnn(q, (x,), spec).to_executorch(ExecutorchBackendConfig(
        memory_planning_pass=MemoryPlanningPass(alloc_graph_input=True, alloc_graph_output=True),
        segment_alignment=get_qnn_context_binary_alignment()))
    open(f"{out}/conv_dead_ch_{tag}.pte", "wb").write(prog.buffer)

Run each .pte with qnn_executor_runner on input_0.raw and read channel 0 of the output. It should be 0 (fp32 max 2.2e-8). With the default eps it comes out at 2.25; with eps=DEFAULT_EPS_8BIT it is 0.0. The other 15 channels match fp32 equally well in both runs (relative error 0.0069), and host fake-quant gives 0.0 in both cases.

cc @cccclai @winskuo-quic @shewu-quic @haowhsu-quic @DannyYuyang-quic @cbilgin @abhinaykukkadapu @psiddh

[msluszniak]

Second effect of the same floor: if the near-dead channel has a nonzero bias, its int32 bias scale (s_in * s_w) is too small to hold it and the bias saturates to ~0, on host fake-quant too. LR-ASPP MobileNetV3 16a8w loses 3.6 mIoU on SM8850 from 3 such channels in each of two depthwise convs (biases up to 0.93). Flooring at DEFAULT_EPS_8BIT is not enough for that; #23161 also gives floored channels the median live scale.