Skip to content

HBASE-30352: Tolerate stale recovered.edits below durable seqid in split/merge (#8599) - #8699

Merged
virajjasani merged 1 commit into
apache:branch-2from
nirdosh0110:HBASE-30352-branch-2
Sep 27, 2026
Merged

virajjasani merged 1 commit into
apache:branch-2from
nirdosh0110:HBASE-30352-branch-2

Conversation

@nirdosh0110

Copy link
Copy Markdown
Contributor

Backport of #8599 to branch-2 (JIRA: HBASE-30352).

Problem

On split/merge, AssignmentManagerUtil.checkClosedRegion aborts the procedure whenever any recovered.edits file exists for a parent region, to prevent data loss. In corner cases a region can be left with recovered.edits whose max seqid is already covered by the region's durable (last-flushed) seqid — those edits are provably redundant, yet their mere presence wedges the split/merge into a stuck RIT that requires manual cleanup.

Change

checkClosedRegion now tolerates such stale files: before aborting, it consults the master's last-flushed seqid for the region and, only when that watermark is authoritative (> 0), drops recovered.edits files whose max seqid is <= durableSeqId. If any file's seqid is above the watermark (or the watermark is unknown), it still aborts as before — no behavior change for the genuinely-unsafe case.

  • Scans the region WAL dir, region dir, and the "wrong" WAL region dir via WALSplitUtil.getSplitEditFilesSorted.
  • Deletes only files strictly at/below durableSeqId; a failed delete or an unparseable/higher seqid falls back to the safe abort.

Test

TestAssignmentManagerUtil covers the new path (3 tests, all green locally).

Verification (local, JDK17, -Dhadoop.profile=3.0)

  • mvn -pl hbase-server -am clean test -Dtest=TestAssignmentManagerUtil → Tests run: 3, Failures: 0, Errors: 0
  • mvn -pl hbase-server -am spotless:check → BUILD SUCCESS

Cherry-pick was clean apart from one import block (branch-2 uses CommonFSUtils/FSUtils; the master-only FutureUtils import was dropped as it has no usage on branch-2).

@nirdosh0110 nirdosh0110 changed the title HBASE-30352 Tolerate stale recovered.edits below durable seqid in split/merge HBASE-30352 Tolerate stale recovered.edits below durable seqid in split/merge (branch-2 backport) Sep 24, 2026
@virajjasani

Copy link
Copy Markdown
Contributor

Awaiting full github workflow results before merging master and branch-2 PRs

@virajjasani virajjasani changed the title HBASE-30352 Tolerate stale recovered.edits below durable seqid in split/merge (branch-2 backport) HBASE-30352: Tolerate stale recovered.edits below durable seqid in split/merge (#8599) Sep 27, 2026
@virajjasani
virajjasani merged commit f6408f1 into apache:branch-2 Sep 27, 2026
24 of 25 checks passed
virajjasani pushed a commit that referenced this pull request Sep 27, 2026
…lit/merge (#8699) (#8599)

Signed-off-by: Viraj Jasani <vjasani@apache.org>
virajjasani pushed a commit that referenced this pull request Sep 27, 2026
…lit/merge (#8699) (#8599)

Signed-off-by: Viraj Jasani <vjasani@apache.org>
@nirdosh0110
nirdosh0110 deleted the HBASE-30352-branch-2 branch September 27, 2026 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants