Skip to content

Grouping operations on large datasets can overflow i32 offsets #23694

Description

@maxburke

Describe the bug

The ByteGroupValueBuilder can raise an offset overflow error if the columns (string or binary) have a large number of values.

Also see: apache/datafusion-comet#4718

To Reproduce

apache/datafusion-comet#4718 has a reproduction

Expected behavior

Grouping operations ideally should not overflow and return an error, even on large input data

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Fields

    No fields configured for Bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions