All pull requests

Pull requests

• [SPARK-60419][CORE] Fix file descriptor leak in FileSystemPersistenceEngine.deserializeFromFile when compression is enabled

  #59670·sarutak opened Oct 21, 2026Member·

  ·

• [SPARK-59151][SQL] Implement the CanonicalizeVariant Catalyst expression

  #59669·SavicStefan opened Oct 21, 2026Contributor·

  ·

• [SPARK-60418][SQL] Skip subexpression elimination for a conditional IN list element

  #59668·SEPURI-SAI-KRISHNA opened Oct 21, 2026Contributor·

  ·

• [SPARK-60417][ML][PYTHON] Remove type: ignore[no-untyped-def] in pyspark.ml.torch.distributor

  #59667·soaring0616 opened Oct 21, 2026Contributor·

  ·

• [MINOR][INFRA] Suggest Assisted-by instead of Generated-by in PR template

  #59666·nchammas opened Oct 21, 2026Contributor·

  ·
  3

• [SPARK-60416][SQL][SS] Allow Structured Streaming Coverage in SQL Tests

  #59665·siying opened Oct 21, 2026Contributor·

  ·
  5

• [SPARK-60333][SQL] Fix make_timestamp second overflow

  #59664·Huang-404-Q opened Oct 21, 2026·

  ·

• [SPARK-60222][CORE] Keep task completion listeners on the completing thread when another thread invokes listeners

  #59663·viirya opened Oct 21, 2026Member·

  ·

• [SQL] Expose subquery planning context to DataSource V2 scan builders

  #59662·yyanyy opened Oct 21, 2026Contributor·

  ·

• [SPARK-60413][SQL] Warn when evolving a DSv2 table that accepts any schema

  #59660·szehon-ho opened Oct 20, 2026Member·

  ·
  1

• [SPARK-60098][SQL] Fix memory leaks in ParquetColumnVector

  #59657·frroliveira opened Oct 20, 2026·

  ·
  1

• [SPARK-60406][SQL] Count CollapseProject's column references by expression ID

  #59655·AnishMahto opened Oct 20, 2026Contributor·

  ·

• [SPARK-60397][SQL] Fix ORC filter pushdown on nanosecond timestamp columns read with a lower precision

  #59654·dongjoon-hyun opened Oct 20, 2026Member·

  ·

• [SPARK-60399][DOCS][SQL] Add a Metric Views programming guide and a CREATE METRIC VIEW reference page

  #59651·lisancao opened Oct 20, 2026·

  ·

• [SPARK-60391][SQL] Add DataFrame API for BIN BY

  #59648·vranes opened Oct 20, 2026Contributor·

  ·
  4

• [WIP][SQL] Extract shared catalog identifier and DROP TABLE helpers

  #59646·vladanvasi-db opened Oct 20, 2026Contributor·

  ·
  3

• [SPARK-60386][SQL] Enforce FILE EXTERNAL in table columns and reject FILE where it would need an ordering

  #59640·dejankrak-db opened Oct 20, 2026Contributor·

  ·
  1

• [SPARK-60381][SQL] Account the late-materialization survivor buffer to the task's execution memory

  #59639·peter-toth opened Oct 20, 2026Contributor·

  ·
  24

• [ML][MLLIB][PYTHON] Say deprecated APIs will be removed in the future

  #59638·zhengruifeng opened Oct 20, 2026Contributor·

  ·

• [SPARK-59824][SQL][FOLLOWUP] Guard top-level geometry and geography inputs to pickle Python UDFs

  #59637·ivoson opened Oct 20, 2026Contributor·

  ·
  1

• [SPARK-60382][SQL] CoalesceBucketsInJoin should see only the streamed side of a broadcast join

  #59636·wangyum opened Oct 20, 2026Member·

  ·
  4

• [SPARK-60384][BUILD] Check binary compatibility against Spark 4.2.0

  #59635·zhengruifeng opened Oct 20, 2026Contributor·

  ·

• [SPARK-60383][CORE] Keep streaming shuffle implementation details out of the public API

  #59633·zhengruifeng opened Oct 20, 2026Contributor·

  ·
  2

• [MINOR][4.3][SQL][PYTHON] Document collect_union in the function reference

  #59631·zhengruifeng opened Oct 20, 2026Contributor·

  ·

• [SPARK-60379][SQL] Push down shuffle exchange through union

  #59630·ulysses-you opened Oct 20, 2026Contributor·

  ·
  3