Skip to content

fix(spark): keep the cause when a path cannot be loaded as a table - #9330

Merged
robert3005 merged 1 commit into
vortex-data:developfrom
jackylee-ch:spark-catalog-preserve-cause
Aug 11, 2026
Merged

fix(spark): keep the cause when a path cannot be loaded as a table#9330
robert3005 merged 1 commit into
vortex-data:developfrom
jackylee-ch:spark-catalog-preserve-cause

Conversation

@jackylee-ch

Copy link
Copy Markdown
Contributor

Rationale for this change

loadTable translates a failed schema inference into NoSuchTableException so SQL users see "table not found" rather than an internal error. But it discarded the original exception entirely — no cause, no suppressed, no log.

That threw away the message #8858 added, which names the offending path, and collapsed a credentials failure or an unsupported Arrow type into the same "not found" as an empty directory.

What changes are included in this PR?

NoSuchTableException has no cause-accepting constructor common to Spark 3.5 and 4.1 (only (Identifier) is), so the original failure is carried as a suppressed exception and logged at warn.

  • ./gradlew :vortex-spark_2.13:test --tests '…VortexCatalogTest' --tests '…VortexSqlTest' — 16 pass
  • same on :vortex-spark_2.12:test
  • spotlessCheck (both variants) and javadoc — clean

The existing VortexCatalogTest.unreadablePathIsReportedAsMissingTable and VortexSqlTest.testDirectPathNotFound still pass: the exception type is unchanged.

What APIs are changed? Are there any user-facing changes?

None. Same exception type; the cause is now recoverable from getSuppressed() and appears in the log.

AI assistance

Prepared with agentic AI assistance. I checked the NoSuchTableException constructors available in both supported Spark versions before choosing addSuppressed over a cause argument.

loadTable translates a failed schema inference into NoSuchTableException
so SQL users see "table not found", but it discarded the original
exception entirely. That threw away the message naming the offending
path, and collapsed a credentials error or an unsupported Arrow type into
the same "not found" as an empty directory.

NoSuchTableException has no cause-accepting constructor common to the
supported Spark versions, so carry the original as a suppressed exception
and log it at warn.

Signed-off-by: jackylee <qcsd2011@gmail.com>
@robert3005 robert3005 added the changelog/chore A trivial change label Aug 11, 2026
@robert3005
robert3005 enabled auto-merge (squash) August 11, 2026 12:50
@codspeed-hq

codspeed-hq Bot commented Aug 11, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 13.04%

⚡ 1 improved benchmark
✅ 1943 untouched benchmarks
⏩ 85 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation decompress[u64, (1000, 16)] 73.3 µs 64.9 µs +13.04%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing jackylee-ch:spark-catalog-preserve-cause (1830b54) with develop (4b37237)

Open in CodSpeed

Footnotes

  1. 85 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@robert3005
robert3005 merged commit c4bb934 into vortex-data:develop Aug 11, 2026
95 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants