Search before asking
Paimon version
master (1.5-SNAPSHOT)
Compute Engine
Any engine that spills to disk (external sort/merge, or ExternalBuffer under memory pressure).
Minimal reproduce step
- Set the table option
spill-compression = none.
- Trigger a disk spill (a large external sort or merge, or
ExternalBuffer spilling under memory pressure).
What doesn't meet your expectations?
The spill fails with a NullPointerException. spill-compression is a free-form string option (default zstd, no value validation), and none is an intended "no compression" value: BlockCompressionFactory.create deliberately returns null for it. But ChannelWriterOutputView dereferences that factory in its constructor (getCompressor()), and ChannelReaderInputView does the same on read-back (getDecompressor()), so selecting none crashes instead of writing uncompressed blocks.
Anything else?
The SPILL_COMPRESSION option description does not mention none among the supported values, which likely hid this from testing.
Are you willing to submit a PR?
Search before asking
Paimon version
master (1.5-SNAPSHOT)
Compute Engine
Any engine that spills to disk (external sort/merge, or
ExternalBufferunder memory pressure).Minimal reproduce step
spill-compression = none.ExternalBufferspilling under memory pressure).What doesn't meet your expectations?
The spill fails with a
NullPointerException.spill-compressionis a free-form string option (defaultzstd, no value validation), andnoneis an intended "no compression" value:BlockCompressionFactory.createdeliberately returnsnullfor it. ButChannelWriterOutputViewdereferences that factory in its constructor (getCompressor()), andChannelReaderInputViewdoes the same on read-back (getDecompressor()), so selectingnonecrashes instead of writing uncompressed blocks.Anything else?
The
SPILL_COMPRESSIONoption description does not mentionnoneamong the supported values, which likely hid this from testing.Are you willing to submit a PR?