Docs: Minor/Major Release Watchers

[kiran1287]

Replaces #457, closed automatically when its source branch was renamed via the GitHub branch-rename API (which deletes the old ref rather than migrating the PR, contrary to what I expected). No commits were lost -- this PR is the exact same branch content under its new name, docs/release-watchers.

Summary

• minor-release-watcher.yml — twice weekly (Mon/Thu), scans only the current minor/patch line (2.0.0 -> 2.0.1 -> ...). Never touches main.
• major-release-watcher.yml — biweekly (every other Monday), scans only main for the next major (2.0 -> 2.1 -> ...). Classifies main-branch PRs as genuinely major-only vs. not-yet-backported minor work (backport search first, then deep scrutiny of "Type of change" if none found), and folds in a completeness check for already-tracked multi-PR features.

Both add a real dry_run mode (a genuine preview option, not just a testing artifact), a since override for the first catch-up run (see Testing below), and drop the redundant needs-docs label — every issue either workflow files already means that by existing.

Also includes .ai/release-watchers/PROCESS-OVERVIEW.md — full process documentation: overview, design principles, both workflows step by step with their actual prompt logic, the schedule, required secrets/variables, and known limitations. (This doc still describes the pre-fix lookback mechanism in a couple of places -- a correction pass is planned once everything below settles.)

Testing — what's actually been verified, and how

Everything below was checked on disposable branches, never a PR, per explicit request. Being specific here on purpose, since an earlier version of this description overstated what had been confirmed.

Confirmed working, via real live-API calls on disposable branches:

• The plain gh CLI commands this workflow depends on (label create, issue create/comment/edit/close) work against the live API with the default GITHUB_TOKEN
• The lookback mechanism itself: an earlier version used a repo Actions variable to remember the last run's timestamp -- confirmed by direct testing that the default GITHUB_TOKEN gets a 403 on that API regardless of declared permissions, so it never actually worked. Replaced with gh run list --event schedule, which only needs read access and was confirmed to filter correctly by trigger event
• The major watcher's extra wrinkle -- an off-week scheduled run also reports status: success while skipping its real work -- confirmed via the Jobs API that per-step skipped vs. success is reported correctly, and the lookback walk uses that to find the last real on-week run, not just the last non-erroring one
• The new since override input: confirmed it takes priority over the computed lookback when set, and falls through to the normal computation when blank
• The FORCE_DRY_RUN safety switch added for the first live rollout: confirmed the if: gate keyed on it actually skips the label-creation step, DRY_RUN resolves to true from it, and Claude's allowed tools genuinely omit gh issue create/edit/comment while it's on -- not just told not to use them, physically unable to
• De-duplication checks both marker prefixes (old daily-watcher:pr-X from before this workflow existed, and the new per-workflow prefix), confirmed live that searching by number alone finds either
• De-duplication also checks the "Backport #X" title pattern real backport PRs use, not just "Fixes #X" -- confirmed against real upstream backport PRs
• Major/minor classification logic verified by manually walking it against real PRs both ways: a since-backported PR (glossary pagination, #32223 -> #32473) correctly excluded, genuinely major-only PRs (dark mode switcher #32383, sourceConfig validation #29566) correctly included
• No orphaned test artifacts left behind -- every temporary label/issue/branch used in testing was cleaned up and verified closed/deleted

NOT yet verified -- and can't be, before this merges:
schedule, workflow_dispatch, and issue_comment triggers only fire from a repo's default branch -- this is a GitHub platform constraint, not something we chose to skip. That means the actual claude-code-action step (the part that reads real PRs, applies the filter/verify logic, and would create/update issues) has never executed successfully in a real run yet -- only the surrounding bash/gh logic above has been proven live. An earlier attempt to test it via a push-triggered disposable branch failed outright (claude-code-action doesn't support push as a trigger event) -- that failure was found only while re-verifying this PR's own claims, and is why this section is now this explicit.

Post-merge steps (do these before considering this fully live)

1. Run minor-release-watcher.yml and major-release-watcher.yml via workflow_dispatch from main. FORCE_DRY_RUN is hardcoded to "true" in both files, so this is safe regardless of what's passed in -- neither workflow can create, edit, or comment on anything yet.
2. Read the digest issue each produces ("Minor/Major Release Watcher — Scan Digest") and sanity-check it against what's actually open upstream right now. This is the first time the real prompt logic runs against live data end-to-end.
3. If that looks right, flip FORCE_DRY_RUN to "false" in its own small, separate, reviewable commit -- that's the actual go-live moment.
4. Separately, execution-test create-draft.yml's real git-push/PR-open path via an actual /create-draft comment -- also blocked pre-merge for the same platform reason above.
5. docs-collate is explicitly out of scope for this PR. It currently has its own, separate copy of the same broken lookback mechanism found and fixed here; that gets ported over in a dedicated follow-up pass after this merges.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
openmetadata	🟢 Ready	View Preview	Sep 17, 2026, 11:15 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

[kiran1287]

Addressed all 4 P1 findings in b8d03b7:

• Action auth: confirmed against the pinned action's actual source (src/github/token.ts) that setupGitHubToken() only reads OVERRIDE_GITHUB_TOKEN, wired exclusively from the github_token input -- an env var alone never reaches it, so every Claude step would have failed outright without id-token: write. Added github_token: ${{ github.token }} to all three invocations.
• Untrusted content isolation: split create-draft.yml into a read-only session (reads the issue + source PR, writes doc content and a title/body handoff to /tmp/, no git/PR/comment tools at all) and a write-capable session that only reads the handoff files -- it has no tool that can read the original source PR, so untrusted upstream content has no path into the session that can push and open a PR.
• Forced-dry lookback: the minor watcher's lookback took the most recent successful scheduled run at face value, but a FORCE_DRY_RUN'd run also completes with success while writing nothing. Ported the major watcher's own fix (walk back and check whether "Ensure tracking labels exist" actually ran, not just completed) -- the major watcher's existing off-week check already incidentally covered this, so it needed no change.
• Dry-run Write leak: Write was left unconditionally available in allowedTools even during FORCE_DRY_RUN, and the Slack-send step had no dry-run gate at all -- so an AI session that wrote slack-digest.txt anyway (confusion, or injected instructions) would have resulted in a real external Slack post despite the "safe by construction" design. Moved Write inside the same dry-run-gated branch as the issue-write commands, and gated the Slack-send step in both watchers.

[ayush-shah]

Changes requested for the new rollout-backlog P1 below and the residual shared-workspace boundary documented in the existing watcher thread. The older forced-dry watermark finding was not duplicated.

[ayush-shah]

[P1] Fail closed when there is no successful live baseline

Initial rollout performs only forced-dry previews, and an explicit since value used for a preview is not persisted. After FORCE_DRY_RUN is disabled, this 20-run lookup can find no live marker and silently falls back to four days (15 in the major watcher), permanently omitting older untracked changes. Require an explicit since value for the first live dispatch and whenever no real baseline is found, or scan from a deterministic exhaustive baseline and persist the successful live watermark. Do not let the 20-run cap age a valid baseline out.

[kiran1287]

Addressed both findings from the latest review round in bc37420:

• Fail closed when there is no successful live baseline: the cadence lookup in both watchers now fails the job outright on a live run when no successful non-dry baseline is found and no explicit since was given, instead of silently falling back to the 4-day (minor) / 15-day (major) default. Also raised the gh run list cap from 20 to 100 so a valid baseline further back can't get aged out and mistaken for "none found."
• Residual shared-workspace boundary: split both minor-release-watcher.yml and major-release-watcher.yml from a single job into prepare → scan → notify → slack, each on its own runner with its own checkout. The scan job's token now has no issues: write at all (minor) / issues: read only (major) -- an actual GitHub permissions boundary, not just allowedTools. The scan→notify handoff moves via upload-artifact/download-artifact and is schema-validated deterministically before the write-capable session reads it, including that every target_version matches this run's own computed value (not something the scan session could set) and, for major, that every completeness_checks.issue_number is a real, open, major-watcher-marked issue (checked live via gh issue view).