Summary
A W4A16 build succeeds while silently dequantizing FP4 weights to FP32. Nothing in the build output indicates the requested precision was not realized, so a user can benchmark an "FP4 engine" that is executing FP32.
Affects: TensorRT 10.16, still observed on 11.2.1.
Detail
Feeding a genuine FLOAT4E2M1 weight-QDQ graph to TensorRT:
- W4A4 fails loudly ("type not supported yet") — this is fine and is the correct behaviour.
- W4A16 succeeds and silently realizes FP32.
We only detected this because our device payload records precision_realized independently of the requested precision. Two independent observations on the same engine: the manifest reports realized precisions FP32, INT8 for a graph requested as fp4, and the same engine subsequently fails to complete a trtexec --loadEngine timing run.
The ask
Not an FP4 Conv kernel. The request is that the builder warn or refuse when an explicitly quantized precision is discarded, exactly as it already refuses W4A4. A silent downcast to FP32 is a correctness trap: it converts a precision request into a performance measurement of something else entirely.
Summary
A W4A16 build succeeds while silently dequantizing FP4 weights to FP32. Nothing in the build output indicates the requested precision was not realized, so a user can benchmark an "FP4 engine" that is executing FP32.
Affects: TensorRT 10.16, still observed on 11.2.1.
Detail
Feeding a genuine
FLOAT4E2M1weight-QDQ graph to TensorRT:We only detected this because our device payload records
precision_realizedindependently of the requested precision. Two independent observations on the same engine: the manifest reports realized precisionsFP32, INT8for a graph requested asfp4, and the same engine subsequently fails to complete atrtexec --loadEnginetiming run.The ask
Not an FP4 Conv kernel. The request is that the builder warn or refuse when an explicitly quantized precision is discarded, exactly as it already refuses W4A4. A silent downcast to FP32 is a correctness trap: it converts a precision request into a performance measurement of something else entirely.