Skip to content

[lance] Close the native and Arrow writers even when the final flush fails - #10298

Open
PDGGK wants to merge 1 commit into
apache:masterfrom
PDGGK:fix-lance-close-after-flush-failure
Open

PDGGK wants to merge 1 commit into
apache:masterfrom
PDGGK:fix-lance-close-after-flush-failure

Conversation

@PDGGK

@PDGGK PDGGK commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Purpose

LanceRecordsWriter.close() flushes the final Arrow batch before closing the native Lance writer and the Arrow writer. If that final flush throws, the method returns immediately and both writer resources are skipped.

MosaicRecordsWriter, the other native Arrow-backed BundleFormatWriter implementation, already continues its cleanup after a flush failure while preserving the first failure and attaching later failures as suppressed exceptions.

What changes

Apply the same failure-path cleanup semantics to LanceRecordsWriter:

  • attempt the final flush first
  • still close the native Lance writer if the flush fails
  • still close the Arrow writer if an earlier cleanup step fails
  • rethrow the first failure and attach later cleanup failures as suppressed

The normal close order is unchanged.

Tests

LanceRecordsWriterTest#testCloseContinuesCleanupAfterFlushFailure verifies that a flush failure does not skip either close, and that later close failures are preserved as suppressed exceptions. The test fails with the production change reverted and passes with this fix.

The full LanceRecordsWriterTest suite passes 6/6. Spotless and checkstyle are clean, git diff --check passes, and the module compiles with maven.compiler.release=8.

API and Format

No change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant