Skip to content

test: pin parquet created_by in tests so expected output is independent of the version string - #26892

Merged
alamb merged 2 commits into
apache:mainfrom
alamb:alamb/pin-created-by-in-tests
Oct 29, 2026
Merged

alamb merged 2 commits into
apache:mainfrom
alamb:alamb/pin-created-by-in-tests

Conversation

@alamb

@alamb alamb commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

While testing a 56.0.0-dev version on main in #26845, CI failed because the default datafusion.execution.parquet.created_by value (datafusion version 55.1.0) is written into every Parquet file, so changing the version string changes the size of every Parquet file written during tests.

Since this happens any time the version string changes, let's just hard code it into the tests that don't matter

What changes are included in this PR?

Hard code a version in tests that don't need it

What is the testing strategy for this PR?

Existing CI.

Are there any user-facing changes?

No, test only.

…the version string

The default created_by embeds the DataFusion version, so a longer version
string (e.g. 56.0.0-dev) changes the size of every written parquet file,
which shifts byte based metrics (scan_efficiency_ratio, file_groups byte
ranges, file_size_bytes) in sqllogictest and datafusion-cli expectations.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 90d3b42)
@github-actions github-actions Bot added the sqllogictest SQL Logic Tests (.slt) label Oct 29, 2026
@alamb alamb added the development-process Related to development process of DataFusion label Oct 29, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@github-actions github-actions Bot removed the development-process Related to development process of DataFusion label Oct 29, 2026
@alamb
alamb marked this pull request as ready for review October 29, 2026 19:20

@mbutrovich mbutrovich left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense to me, thanks @alamb!

@alamb

alamb commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

Thank you for the quick review @mbutrovich

@alamb
alamb added this pull request to the merge queue Oct 29, 2026
Merged via the queue into apache:main with commit 6094f2e Oct 29, 2026
44 checks passed
@alamb
alamb deleted the alamb/pin-created-by-in-tests branch October 29, 2026 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sqllogictest SQL Logic Tests (.slt) v56.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants