Skip to content

fix(arrow): support zero-length fixed-size lists - #1308

Open
jakthom wants to merge 1 commit into
apache:mainfrom
jakthom:fix/zero-size-fixed-size-list
Open

fix(arrow): support zero-length fixed-size lists#1308
jakthom wants to merge 1 commit into
apache:mainfrom
jakthom:fix/zero-size-fixed-size-list

Conversation

@jakthom

@jakthom jakthom commented Sep 9, 2026

Copy link
Copy Markdown

Rationale for this change

DataFusion can return FixedSizeList(0, Null) for an empty list. Arrow Go currently rejects the C data schema +w:0, and its fixed-size list constructors also panic for size zero. This prevents importing results such as:

select arrow_cast(a, 'FixedSizeList(0, Null)')
from values ([]), (NULL) t(a);

A zero-size list has no child values, but its parent array still has rows and a validity bitmap that distinguishes empty lists from null lists. This fixes the two failing cases in DataFusion's arrow_typeof.slt.

What changes are included in this PR?

  • Accept zero in the three fixed-size list type constructors and in C data schema imports.
  • Allow zero-size fixed-size list arrays to validate without dividing by zero. Negative sizes, invalid child schemas, and overflowing C schema sizes remain rejected.
  • Add constructor, array, C data, and IPC regression tests covering empty and null rows, bulk appends, nonzero slice offsets, and checked-allocator cleanup with null, integer, and nested-list element types.

Are these changes tested?

Local verification on macOS ARM64 with Go 1.25.2:

  • pre-commit run --all-files --show-diff-on-failure
  • ci/scripts/build.sh "$PWD"
  • ci/scripts/test.sh "$PWD" (race detector, assert,test, and noasm variants)
  • go test -race -short -tags=assert,test ./internal/...

The datafusion-go SQLLogicTest corpus also passes 24,958/24,958 assertions across 365 executable SQL files on macOS when an isolated Go module file replaces Arrow Go with this checkout. That includes all 62 records in arrow_typeof.slt; the released Arrow Go dependency fails two of those records. This downstream run used DataFusion 55.0.0 and datafusion-go commit 8fbd0218b6c5f3d9d64ec798ae466f66725be325.

Are there any user-facing changes?

Zero-size fixed-size lists can now be constructed, validated, and imported through the C data interface. Existing constructor and import checks for negative sizes are preserved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant