fix(spark): keep the cause when a path cannot be loaded as a table - #9330
Merged
robert3005 merged 1 commit intoAug 11, 2026
Merged
Conversation
loadTable translates a failed schema inference into NoSuchTableException so SQL users see "table not found", but it discarded the original exception entirely. That threw away the message naming the offending path, and collapsed a credentials error or an unsupported Arrow type into the same "not found" as an empty directory. NoSuchTableException has no cause-accepting constructor common to the supported Spark versions, so carry the original as a suppressed exception and log it at warn. Signed-off-by: jackylee <qcsd2011@gmail.com>
robert3005
enabled auto-merge (squash)
August 11, 2026 12:50
robert3005
approved these changes
Aug 11, 2026
Merging this PR will improve performance by 13.04%
Performance Changes
Tip Curious why this is faster? Comment Comparing Footnotes
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
loadTabletranslates a failed schema inference intoNoSuchTableExceptionso SQL users see "table not found" rather than an internal error. But it discarded the original exception entirely — no cause, no suppressed, no log.That threw away the message #8858 added, which names the offending path, and collapsed a credentials failure or an unsupported Arrow type into the same "not found" as an empty directory.
What changes are included in this PR?
NoSuchTableExceptionhas no cause-accepting constructor common to Spark 3.5 and 4.1 (only(Identifier)is), so the original failure is carried as a suppressed exception and logged at warn../gradlew :vortex-spark_2.13:test --tests '…VortexCatalogTest' --tests '…VortexSqlTest'— 16 pass:vortex-spark_2.12:testspotlessCheck(both variants) andjavadoc— cleanThe existing
VortexCatalogTest.unreadablePathIsReportedAsMissingTableandVortexSqlTest.testDirectPathNotFoundstill pass: the exception type is unchanged.What APIs are changed? Are there any user-facing changes?
None. Same exception type; the cause is now recoverable from
getSuppressed()and appears in the log.AI assistance
Prepared with agentic AI assistance. I checked the
NoSuchTableExceptionconstructors available in both supported Spark versions before choosingaddSuppressedover a cause argument.