Skip to content

[spark] Prevent table deletion for self referencing RTAS - #9139

Open
ArnavBalyan wants to merge 1 commit into
apache:masterfrom
ArnavBalyan:arnav/fix-rtas
Open

[spark] Prevent table deletion for self referencing RTAS#9139
ArnavBalyan wants to merge 1 commit into
apache:masterfrom
ArnavBalyan:arnav/fix-rtas

Conversation

@ArnavBalyan

Copy link
Copy Markdown
Member

Purpose

  • A self referencing replace table on an existing table fails the query and accidentally performs a data drop.
  • This is because Spark drops the table before evaluating the query. The query always fails while reading the deleted manifest leading to original table being lost permanently.
  • Add guard to ensure this operation does not happen for partitioned table rtas and we can avoid data loss.

Tests

  • UT

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant