Skip to content

fix: read parquet metadata after writing - #3668

Open
GayathriSrividya wants to merge 1 commit into
apache:mainfrom
GayathriSrividya:fix/parquet-metadata-read-metadata
Open

fix: read parquet metadata after writing#3668
GayathriSrividya wants to merge 1 commit into
apache:mainfrom
GayathriSrividya:fix/parquet-metadata-read-metadata

Conversation

@GayathriSrividya

Copy link
Copy Markdown
Contributor

closes #3108

Summary

  • reopen newly written parquet files and read metadata via pyarrow.parquet.read_metadata
  • use the read metadata consistently when computing DataFile statistics
  • add a regression test covering the write_file metadata path

Testing

  • /opt/homebrew/bin/ruff format pyiceberg/io/pyarrow.py tests/io/test_pyarrow.py
  • /opt/homebrew/bin/ruff check --fix pyiceberg/io/pyarrow.py tests/io/test_pyarrow.py
  • PYTHONPATH=. uv run --project . pytest tests/io/test_pyarrow.py -k write_file_reads_parquet_metadata -xvs

@github-actions

Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Inconsistency in arguments to parquet_metadata parameter of data_file_statistics_from_parquet_metadata() in iceberg-python/pyiceberg/io/pyarrow.py

1 participant