[fix](cloud) Exclude abandoned schema change shadow tablets from compaction score

[Yukang-Lian]

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

In cloud mode, a heavyweight schema change leaves NOT_READY shadow tablets in the BE tablet cache. When the job is cancelled (or removed), those tablets can never be converted or compacted, but they linger in the cache until the recycler permanently deletes their meta, because meta-service keeps returning their tablet meta before that.

Meanwhile the compaction scheduler records the max compaction score before filtering, so such an abandoned tablet keeps feeding tablet_cumulative_max_compaction_score with a large constant value, while /api/compaction_score only enumerates TABLET_RUNNING tablets and never shows it. The metric then reports a false backlog (observed: a flat 551 across many BEs for days in production) until the BE restarts, and the metric and the HTTP endpoint contradict each other, which makes the alarm impossible to diagnose.

Fix:

• MS reports whether a tablet still has an active schema change job via the existing get_rowset RPC: one extra point read of the tablet job KV, only when BE sets the new need_alter_job_info request field for its NOT_READY tablets. The job KV is written on job start and cleared synchronously on commit/abort, so it is an authoritative and timely liveness signal, independent of delayed recycling.
• BE refreshes CloudTablet::has_active_alter_job on each rowset sync. A NOT_READY tablet without an active alter job (a zombie tablet) is excluded from the max compaction score stats, from compaction candidates, and from /api/compaction_score, so the metric and the HTTP endpoint now describe the same set of tablets: RUNNING plus in-progress schema change new tablets. In-progress new tablets stay visible in both (they do get compacted via enable_new_tablet_do_compaction).
• /api/compaction_score entries now carry tablet_state and not_scheduled_reason (evaluated only for the returned top-n entries, so the cost is bounded by top_n), and /api/compaction/show reports the full schedulability breakdown including is_zombie, so a high score that nobody compacts explains itself. The scheduler log line also reports num_zombie.

Rolling upgrade is safe in both directions: an old MS simply leaves the response field unset and BE keeps today's behavior (the tablet is treated as alter-in-progress).

Manual test on a single-node cloud cluster (schema change conversion blocked via debug point, 60 double-written load txns, then CANCEL ALTER, recycler not running):

• While the job is alive: gauge = 11 vs top1 = 12 and the shadow tablets show up in top_n as NOTREADY, consistent for the first time.
• After cancel: within a few sync cycles the gauge falls back to the real backlog level, top_n no longer lists the shadows, the scheduler log shows num_zombie=2, and /api/compaction/show reports is zombie tablet: true / not scheduled reason: zombie. Before this fix the gauge stayed pinned until BE restart.

Release note

None

Check List (For Author)

• Test

  • Regression test
  • Unit Test
  • Manual test (add detailed scripts or steps below)
  • No need to test or manual test. Explain why:
    • This is a refactor/code format and no logic has been changed.
    • Previous test can cover this change.
    • No code files have been changed.
    • Other reason

• Behavior changed:

  • No.
  • Yes.
    • Abandoned (zombie) schema change shadow tablets no longer feed the max compaction score metrics, are no longer picked as compaction candidates (their prepare always fails against MS anyway), and are excluded from /api/compaction_score; /api/compaction_score now also lists in-progress schema change new tablets with a tablet_state field and a not_scheduled_reason field.

• Does this need documentation?

  • No.
  • Yes.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?