Decode dictionary-encoded arrays before bucket transform - #3635
Conversation
febcf28 to
e46ded2
Compare
| assert output_arr == VoidTransform().pyarrow_transform(IntegerType())(input_arr) | ||
|
|
||
|
|
||
| def test_bucket_pyarrow_transform_dictionary_encoded() -> None: |
There was a problem hiding this comment.
These new tests fail if we revert changes in transforms.py:
def _transform(array: "ArrayLike") -> "ArrayLike":
def _cast_if_needed(arr: "ArrayLike") -> "ArrayLike":
if expected_type is not None:
return arr.cast(expected_type)
else:
return arr
if isinstance(array, pa.Array):
> return _cast_if_needed(transform_func(array, *args))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E ValueError: FeatureUnsupported => Unsupported data type for bucket transform: Dictionary(Int32, Utf8)
pyiceberg/transforms.py:149: ValueError
e46ded2 to
5434368
Compare
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
|
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. |
1 participant
Rationale for this change
Fix "Unsupported data type for bucket transform" error.
Are these changes tested?
Yes
Are there any user-facing changes?
Yes