Skip to content

GlobalIndexAssigner crashes on a late bootstrap key after an unaligned checkpoint #10294

Description

@LuciferYang

Search before asking

  • I searched in the issues and found no similar issues.

Paimon version

master (1.5-SNAPSHOT)

Compute Engine

Flink (cross-partition upsert / global dynamic-bucket sink) with unaligned checkpoints.

Minimal reproduce step

  1. Run a cross-partition upsert sink (global index assigner) with execution.checkpointing.unaligned.enabled = true.
  2. Have a checkpoint land during the bootstrap-drain window, so a KEY_PART record is still queued when the barrier triggers.

What doesn't meet your expectations?

The assigner crashes. GlobalIndexAssigner.bootstrapKey has checkArgument(inBoostrap()), which throws (and bootstrapKeys is already null, so it would NPE) when a KEY_PART record arrives after the bootstrap phase has ended. Under unaligned checkpoints the barrier overtakes buffered records, so prepareSnapshotPreBarrier runs endBoostrap (setting bootstrap=false, nulling bootstrapKeys) while KEY_PART records are still queued; those late records then hit bootstrapKey with bootstrap == false and fail the job, which can re-fail on restart in the same window. Aligned checkpoints never hit this, because barrier alignment processes every pre-barrier KEY_PART before endBoostrap.

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions