All pull requests

Pull requests

• fix: count each reused exchange once in extended explain

  bugrun-all-spark-profiles
  #6379·andygrove opened Sep 29, 2026Member·

  ·
  1

• fix: build release jars from the same commit as the native libraries

  bug
  #6377·andygrove opened Sep 29, 2026Member·

  ·
  1

  4

• fix: make benchmark cases over ShortType columns exercise Comet

  bugrun-benchmark-check
  #6376·andygrove opened Sep 29, 2026Member·

  ·
  1

• fix: derive the native memory limit the way YARN and Kubernetes size executor containers

  bug
  #6375·andygrove opened Sep 29, 2026Member·

  ·
  1

• fix: warn about unreleased native memory only after a task's last native plan closes

  area:memorybug
  #6374·andygrove opened Sep 29, 2026Member·

  ·
  1

• docs: add a Celeborn integration guide

  documentation
  #6373·pingzh opened Sep 29, 2026Contributor·

  ·
  1

• fix: release input Arrow streams that native never takes

  bug
  #6372·andygrove opened Sep 29, 2026Member·

  ·
  1

• fix: serialize large-offset string and binary vectors with 32-bit offsets

  bug
  #6371·andygrove opened Sep 29, 2026Member·

  ·
  1

  2

• fix: count grouped integer SUM state in the aggregate's memory reservation

  area:aggregationarea:expressionsbug
  #6370·andygrove opened Sep 29, 2026Member·

  ·
  1

• fix: count every hash-based JVM shuffle batch as shuffle bytes written

  area:shufflebug
  #6369·andygrove opened Sep 29, 2026Member·

  ·
  1

• fix: size the native runtime for standalone executors without spark.executor.cores

  bug
  #6368·andygrove opened Sep 29, 2026Member·

  ·
  1

• fix: cap spark.comet.shuffle.jvm.batchSize at spark.comet.batchSize where it is read

  area:shufflebug
  #6366·andygrove opened Sep 29, 2026Member·Queued
  1

  1

• fix: check accelerated mapInArrow output against the declared schema

  area:udfbugrun-pyarrow-udf-tests
  #6365·andygrove opened Sep 29, 2026Member·

  ·
  1

• chore: remove AlignedArrowStreamReader and fix the Native to JVM FFI docs

  area:ffiarea:scanenhancementrun-spark-4.1-tests
  #6364·andygrove opened Sep 29, 2026Member·

  ·
  1

• fix: fail a native plan whose producer task is cancelled before its stream ends

  area:memorybugrun-spark-4.1-tests
  #6362·andygrove opened Sep 29, 2026Member·

  ·
  1

• fix: support JVM columnar shuffle with spark.shuffle.checksum.enabled=false

  area:shufflebug
  #6361·andygrove opened Sep 28, 2026Member·

  ·
  1

  2

• fix: install Comet's cache serializer only when Comet and native execution are enabled

  bug
  #6360·andygrove opened Sep 28, 2026Member·

  ·

• docs: add a backporting policy to the contributor guide

  documentation
  #6357·andygrove opened Sep 28, 2026Member·Queued

• fix: make Scala logging consistent and log exceptions with their stack traces

  area:expressionsarea:Icebergarea:joinsarea:scanarea:shufflearea:udfarea:writerbug
  #6356·andygrove opened Sep 28, 2026Member·

  ·

• feat: native Parquet writes to S3

  area:scanarea:writerenhancement
  #6355·comphead opened Sep 28, 2026Contributor·

  ·
  1

• fix: truncate timestamps around DST transitions the way Spark does

  area:expressionsbackport-1.0backport-1.1bug
  #6354·andygrove opened Sep 28, 2026Member·

  ·
  1

  2

• fix: warn when native and JVM timezone databases differ

  backport-1.0backport-1.1bug
  #6353·andygrove opened Sep 28, 2026Member·

  ·
  1

• docs: explain operator plan identity and exchange reuse

  documentation
  #6352·rich7420 opened Sep 28, 2026Contributor·Queued
  1

• fix: normalize session timezone IDs before passing them to native code

  area:expressionsarea:scanbackport-1.0backport-1.1bug
  #6351·andygrove opened Sep 28, 2026Member·

  ·
  1

• perf: optimize native CASE WHEN and IF (up to 12x faster)

  area:expressionsenhancementperformancerun-benchmark-checkrun-spark-4.1-tests
  #6350·andygrove opened Sep 28, 2026Member·

  ·
  1

  1