Skip to content

fix: allow empty V1 writes with non-nullable columns - #8394

Open
lance-gatefixer[bot] wants to merge 1 commit into
mainfrom
gatekeeper/fix-1914-1
Open

fix: allow empty V1 writes with non-nullable columns#8394
lance-gatefixer[bot] wants to merge 1 commit into
mainfrom
gatekeeper/fix-1914-1

Conversation

@lance-gatefixer

@lance-gatefixer lance-gatefixer Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Summary

  • keep V1 page-statistic bounds nullable when the source column is non-nullable
  • cover writing and reading an empty batch with a non-nullable string column

Root cause

The V1 statistics collector copied source-column nullability onto its min_value and max_value fields. Empty string pages legitimately have no bounds, so Arrow rejected their null statistics values while constructing the non-nullable stats struct.

Scope

This removes the panic when creating an empty V1 table. IVF_PQ index creation still requires vectors for centroid training and will continue to reject an empty dataset.

Validation

  • cargo fmt --all -- --check
  • cargo test -p lance-file (153 unit tests and 4 doctests)
  • cargo clippy --all --tests --benches -- -D warnings

Fixes #1914

@github-actions github-actions Bot added A-encoding Encoding, IO, file reader/writer bug Something isn't working labels Aug 7, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: approve.

The change fixes the V1 empty-batch panic at the statistics-schema boundary: min/max bounds may be absent even when the data column is non-nullable. Keeping those bounds nullable uses the existing V1 statistics grammar and preserves one statistics row per data batch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-encoding Encoding, IO, file reader/writer bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Panic when creating empty table

0 participants