Fix/converter dtype promotion - #4520
Conversation
Three related fixes for use_explicit_typing correctness: 1. scatter: replace `np.ones` with `np.full` to avoid float64 scalar promotion, and cast src_tensor dtype when it mismatches input (e.g. after argmax/topk emit INT32 instead of INT64). 2. promote_trt_tensors_to_same_dtype: preserve bool dtype when both operands are bool, rather than erroneously promoting to int32. 3. convert_binary_elementwise: use torch.result_type for scalar-tensor dtype promotion instead of unconditionally casting the scalar to the tensor's dtype (e.g. int64_tensor * float_scalar now correctly yields float32, not int64).
|
Hi @dongwoonhyun! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
|
scatter.value still crashes on a float scalar. The fix replaces np.ones(...) with np.full(index_shape_list, src), but np.full still infers dtype purely from the Python scalar's type True to bool, 1 to int64, 2.5 tof loat64 identical to the problem being fixed. Please add a float-scalar test case (e.g. scatter.value(..., 2.5)) that actually exercises this path. |
|
@dongwoonhyun could you please take a look and resolve the comments |
Description
This PR fixes three dtype-related bugs in converters.
scatter: fix scalar float64 promotion and src dtype mismatch
The
scatterconverter has 2 bugs that arise under use_explicit_typing:np.ones, which defaults todtype=np.float64, raising ValueError: TensorRT does not support float64 (double) precision. Fix: replacenp.oneswithnp.fullto get the right dtype.argmax,topk) break the dtype contract and emit INT32 ITensors; downstream consumers likescatter.srcthen crash because they are promised INT64 inputs but receive INT32. Fix: add a conditional cast in case of dtype mismatch.preserve bool dtype in promote_trt_tensors_to_same_dtype
promote_trt_tensors_to_same_dtypeerroneously promotes operations between (bool, bool) dtypes to int32 because it unconditionally promotes non-floating-point types to int32. This leads to problems when using use_explicit_typing. Fix: explicitly handle the (bool, bool) case to output bool, matching PyTorch's behavior.scalar coercion: preserve float scalar dtype in integer-typed elementwise ops
When
convert_binary_elementwiseis called with one Python scalar input and one tensor input, the scalar's dtype is always cast to the tensor's dtype. As a result, int64_tensor * float32_scalar -> int64, which conflicts with PyTorch's type promotion rules, which would be float32. This poses a problem under use_explicit_typing. Fix: Rather than unconditionally casting the scalar to the tensor's dtype, cast it to the dtype that PyTorch's scalar-tensor promotion rules dictate, viatorch.result_type.Type of change
Checklist: