Fix stale changelog registry

[lcawl]

This PR must be merged after #3801 since it uses that PR as its base.

Context

changelog bundle CDN sourcing today does this:

flowchart LR
  bundleCmd["changelog bundle"] --> registry["GET changelog/org/repo/branch/registry.json"]
  registry --> listed["Download every listed YAML"]
  listed --> match["Filter by optional YAML prs/issues/products"]

Loading

After #3760, nothing writes changelog/{org}/{repo}/{branch}/registry.json. The scrubber only reconciles bundle/{product}/registry.json. Uploaded YAML is on S3/CDN; the pool index is frozen.

This means that changelogs that I've uploaded recently to the CDN are not findable by the docs-builder changelog bundle command because they're not reflected in the registry.

Implementation details

What changed in docs-builder

• The scrubber now owns both listings. YAML (and registry-key) events reconcile changelog/{org}/{repo}/{branch}/registry.json from the public YAML listing. target is always null for the pool. Client JSON is not copied.
• --prs (local and CDN) uses the same join as git-ref: leading filename digits or YAML prs:. Empty prs: still matches 12345.yaml.
• CDN --files / a path list GETs those basenames only (no registry.json). A missing name still fails the run.
• CDN --all and product-only filters error and point at --force-local. Local --all is unchanged. Git-ref infer is unchanged.

After merge, Kibana’s stale listing only heals once the scrubber Lambda is deployed. CLI-only is not enough. A --skip-etag-check upload (or the next YAML event) after that deploy rewrites changelog/elastic/kibana/main/registry.json.

Behavior to be aware of: a CDN monthly/product-only bundle (no PR/file list), like the cloud products: profile with only a date argument, now errors until phase 3. Use --force-local for those runs (this should be fine since we've now removed those profiles).
Kibana's serverless-release … prs.txt is --prs and is the path this fixes.

[Mpdreamz]

Thank you for the analysis. The problem is real. The only consumer of the pool index still reads the index. No process writes the index after #3760. Therefore the bundle command finds no entries when it reads from the CDN.

I do not want to add the pool index again. We removed it in the review of #3738, and the reason is still correct. Release-note discovery always starts with a known list of PRs. Therefore we can ask one question for each PR: does an entry exist? One conditional GET answers this question. A per-branch index has no maximum size. It becomes larger for the life of the branch. Each write also makes a new reconcile necessary.

I prefer a different method. Each changelog item gets an address that we can request directly.

Most items have a PR. For these items, the object key is the PR number. The name of the file in the contributing repository is not important. Therefore the bundle command can request each object directly. Each onboarded repository already sets PR-number file names. Therefore this change makes an existing practice into a rule. I examined the live layout. Only a few objects need a new name, and we can calculate each new name.

Some items have more than one PR. For these items we keep one entry that has the full text. For each other PR we write a small marker object. The marker refers to the entry. Therefore each PR in a release list has an object, and the result is one release-note item.

Some items have no PR and no issue. Many items on the known-issues pages are of this type. These items become notes. A note has a target, and the target is necessary. A Lambda function collects the notes for each target. Today this content goes directly into the bundle YAML and does not pass through the pool. Therefore this is a new function. It is not a function that we must keep without a change.

A release bundle then has two steps. First, the command requests one object for each PR. Second, the command reads the notes for the target. A per-target list has a maximum size, and it is complete when the release ships. A per-branch list does not have these properties.

Much of this PR is still useful. Please keep these parts:

• The direct GET for the path list. It is the basis for the new method. It also corrects a fault: a path-list run cannot see an object that the index does not list.
• The error for an unfiltered CDN request. Today the command downloads the full pool and gives no message.
• The removal of the pass-through for client-written pool JSON in the Lambda.
• The helper that reads the PR number from the start of a file name. The rename task and the path-list function both need this helper. It also removes duplicate code from the git-range resolver.

Please remove these parts:

• The pool reconcile in the reconciler.
• The pool-group schedule in the Lambda.

One more remark. The PR-number helper runs after the download of each entry. But the file-name part of the match is available before the body. If the helper runs earlier, the number of requests becomes much smaller. This is also true for the current code.

Can you divide the useful parts into separate PRs? I opened https://github.com/elastic/docs-eng-team/issues/789 for the address method and will link it here.