Repository navigation

[WriteFiles] Support direct writes of large streaming bundles to temp files (#40533) - #40543

Open

tejasiyer-dev wants to merge 11 commits into

apache:masterfrom

tejasiyer-dev:gcs-direct-write

Open

[WriteFiles] Support direct writes of large streaming bundles to temp files (#40533)#40543
tejasiyer-dev wants to merge 11 commits into
apache:masterfrom
tejasiyer-dev:gcs-direct-write

Commits on Oct 12, 2026
- [WriteFiles] Support direct writes of large streaming bundles to temp files
  tejasiyer-dev
  committed
  0a49326
- Remove experiment flag and configure direct write solely via withDirectWriteByteThreshold
  tejasiyer-dev
  committed
  d9d8af4
Commits on Oct 13, 2026
- Merge branch 'master' into gcs-direct-write
  tejasiyer-dev
  authored
  9485325
- Merge branch 'master' into gcs-direct-write
  tejasiyer-dev
  authored
  3e064fd
Commits on Nov 3, 2026
Commits on Nov 10, 2026
- Merge remote-tracking branch 'upstream/master' into gcs-direct-write
  tejasiyer-dev
  committed
  de6452b
- Stream elements to the large output once BundleLifter's threshold is reached. Evaluate the direct-write threshold per WriterKey instead of per bundle.
  tejasiyer-dev
  committed
  b1fe445
Commits on Nov 13, 2026
- Disabled Direct Write for merging windows. Fall back to default path with log warning if directWriteThreshold is set but direct write can't be used. Also evict BundleLifter keys in one pass when ov…
  tejasiyer-dev
  committed
  fec8a2e