Skip to content

spill-compression=none crashes with NullPointerException on disk spill #10275

Description

@LuciferYang

Search before asking

  • I searched in the issues and found no similar issues.

Paimon version

master (1.5-SNAPSHOT)

Compute Engine

Any engine that spills to disk (external sort/merge, or ExternalBuffer under memory pressure).

Minimal reproduce step

  1. Set the table option spill-compression = none.
  2. Trigger a disk spill (a large external sort or merge, or ExternalBuffer spilling under memory pressure).

What doesn't meet your expectations?

The spill fails with a NullPointerException. spill-compression is a free-form string option (default zstd, no value validation), and none is an intended "no compression" value: BlockCompressionFactory.create deliberately returns null for it. But ChannelWriterOutputView dereferences that factory in its constructor (getCompressor()), and ChannelReaderInputView does the same on read-back (getDecompressor()), so selecting none crashes instead of writing uncompressed blocks.

Anything else?

The SPILL_COMPRESSION option description does not mention none among the supported values, which likely hid this from testing.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions