Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/cli/changelog/cmd-add.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@ For example:
- `"cloud-serverless 2025-08-05"`
- `"cloud-enterprise 4.0.3, cloud-hosted 2025-10-31"`

:::{note}
Specifying a version in the `--products` spec (the middle slot, for example `"elasticsearch 9.3.0 ga"`) is an error for `changelog add`. Entries derive their release line from their origin branch, not from a contributor-supplied version. To create an item that explicitly targets one or more versions — such as a known issue or a CVE — use [`changelog note`](/cli/changelog/note.md) instead, which accepts `versions` in place of a target.
Comment thread
Mpdreamz marked this conversation as resolved.
:::

The `changelog add` command resolves product values in the following order:

1. The `--products` CLI option always takes priority.
Expand Down
4 changes: 4 additions & 0 deletions docs/cli/changelog/cmd-bundle-amend.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
Amend a bundle with additional or excluded changelog entries without modifying the parent bundle file.
Amend bundles follow a specific naming convention: `{parent-bundle-name}.amend-{N}` plus the same `.yaml` or `.yml` extension as the parent, where `{N}` is a sequence number.

:::{note}
The suffix `.amend-notes` (for example `9.3.0.amend-notes.yaml`) is reserved for use by the changelog scrubber Lambda. The Lambda generates and manages these files automatically; you must not create, edit, or delete them manually.
:::

Specify at least one of `--add` or `--remove`.

To create a bundle, use [](/cli/changelog/bundle.md).
Expand Down
74 changes: 74 additions & 0 deletions docs/cli/changelog/cmd-note.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
## Description

Create a changelog note file for an item that applies to one or more specific release versions and has no associated pull request.
Comment thread
Mpdreamz marked this conversation as resolved.
Notes are used for known issues, security advisories, and other items that are not tied to a single PR.
For details and examples, go to [](/data/release-notes/create.md).

Note files are named `note-{slug}.yml` and are uploaded to the changelog pool like any other entry.
Each note declares `products[].versions` — the release versions it applies to — instead of deriving its release line from a branch.

## Options

: `--products`
Products and versions in the format `"product versions lifecycle, ..."` where `versions` is a `|`-separated list of release versions (for example, `"elasticsearch 9.3.0|9.4.0 ga"`).
Unlike `changelog add`, the middle slot is interpreted as a `|`-separated version list, not a single target.
The valid product identifiers are listed in [products.yml](https://github.com/elastic/docs-builder/blob/main/config/products.yml).

: `--title`
A short, user-facing headline for the note (max 80 characters). Required.

: `--type`
The type of change. For valid values, see [ChangelogEntryType.cs](https://github.com/elastic/docs-builder/blob/main/src/Elastic.Documentation/ChangelogEntryType.cs). Required.

: `--description`
Additional information about the note (max 600 characters). Optional.

: `--issues`
URLs of related issues. Optional citation field; does not determine note addressability.

## Product and version format

The `--products` option uses the same positional format as `changelog add`, but the middle slot is a version list:

- `"elasticsearch 9.3.0 ga"` — one version
- `"elasticsearch 9.3.0|9.4.0|9.5.0 ga"` — multiple versions
- `"cloud-serverless 2025-08-05"` — date-based release, one version

A note that spans products can declare each product separately:

```sh
docs-builder changelog note \
--title "Known issue with aggregations" \
--type known-issue \
--products "elasticsearch 9.3.0|9.4.0 ga" \
--products "kibana 9.3.0|9.4.0 ga"
```

## Output

The command writes a `note-{slug}.yml` file to the configured output directory.
The file contains `products[].versions` instead of `products[].target`:

```yaml
title: Known issue with aggregations
type: known-issue
products:
- product: elasticsearch
versions: [9.3.0, 9.4.0]
lifecycle: ga
```

## Lifecycle after creation

Notes are uploaded to `changelog/{org}/{repo}/{branch}/note-*.yml` in the private S3 bucket and go through the scrubber exactly like entries.
A Lambda-maintained index at `changelog/{org}/{repo}/notes-{version}.json` lists every note that applies to a given version.

If the release bundle for that version has already shipped when a note is uploaded, the scrubber Lambda automatically generates an amend sidecar (`{bundle}.amend-notes.yaml`) so the note reaches CDN consumers without a manual rerun.

## Configuration checks

The same configuration-file checks that apply to `changelog add` apply here:
valid `products`, `lifecycles`, and `type` values are validated against `docs/changelog.yml` when it exists.

Specifying a version target in `--products` for `changelog add` is an error; use `changelog note` instead.
Conversely, `--versions` has no meaning for `changelog add` — it is note-specific.
9 changes: 4 additions & 5 deletions docs/data/release-notes/_snippets/changelog-fields.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,11 @@ products:
# filters, and categorization.
# Refer to https://github.com/elastic/docs-builder/blob/main/config/products.yml for the acceptable values.

target:
versions:

# An optional string that facilitates pre-release doc previews.
# For products with version releases, it contains the target version number (V.R.M).
# For products with date releases, it contains the target release date
# or the date the PR was merged.
# Note files only — a required list of release versions this note applies to.
# This field is mandatory only when the changelog is a note (i.e. doesn't have a PR).
# Example: [9.3.0, 9.4.0]

lifecycle:

Expand Down
39 changes: 39 additions & 0 deletions docs/development/changelog-bundle-registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,16 @@ narrowed reconciliation to the bundle tree):
exclusively by the scrubber Lambda's `BundleRegistryReconciler`. This is the manifest the
`{changelog}` directive and external CDN consumers enumerate, and the subject of the rest of
this page.
- **Amend-notes sidecars** — `bundle/{product}/{parent}.amend-notes.yaml`, also **public bucket
only**, authored by the scrubber Lambda's `NoteAmendReconciler`. When a note is uploaded after
its release bundle has already shipped, the reconciler generates one aggregate sidecar per
published bundle that lists all such late notes. The Lambda rebuilds it from current state on
every reconcile, so redelivered events never produce duplicate amends. The `.amend-notes` suffix
is **reserved** — do not create files with that suffix manually; see
[](/cli/changelog/bundle-amend.md).
- **Notes index** — `changelog/{org}/{repo}/notes-{version}.json`, one per version, **public
bucket only**, produced by the scrubber Lambda's `NotesIndexReconciler`. See
[Notes-index format](#notes-index-format) below.
- **Changelog-entry index** — `changelog/{org}/{repo}/{branch}/registry.json`, a **legacy
client-authored pass-through**: the current `changelog upload` never writes one, but manifests
written by older CLI versions are still mirrored verbatim from the private bucket, because
Expand Down Expand Up @@ -110,6 +120,35 @@ for a product that was declared under `release_notes` but never published — th
remove the declaration), while a manifest with an empty `bundles` list would read as a valid
zero-bundle state. The reconciler deliberately restores the former.

## Notes-index format [notes-index-format]

For each release version that has at least one note, the scrubber Lambda writes a notes index at
`changelog/{org}/{repo}/notes-{version}.json`. Its schema (`schema_version: 1`):

```json
{
"schema_version": 1,
"notes": [
{ "path": "main/note-esql-oom.yml", "bundle_seq": 2 },
{ "path": "main/note-cve-2026-1234.yml", "bundle_seq": 1 }
]
}
```

| Field | Meaning |
|---|---|
| `schema_version` | Schema version. Currently `1`. |
| `notes[].path` | Pool-relative path of the note within `changelog/{org}/{repo}/`. The leading segment before the first `/` is the branch. |
| `notes[].bundle_seq` | Derived reporting field: `0` = no bundle published for this version yet, `1` = note shipped in the original bundle, `2` = note carried by the Lambda-generated `.amend-notes` sidecar. |

`bundle_seq` is derived — it is never authored and never a latch. The Lambda recomputes it on every
reconcile by comparing the notes index against the set of entries in the published bundle and its
amend sidecars.

A 404 on a notes index means "no notes published for this version". An empty `notes` array never
appears — the index is deleted rather than emptied, following the same
[absent ≠ empty](#absent-empty) rule as the bundle registry.

## Shallow per-tree change maps [shallow-maps]

Alongside the per-group manifests, the scrubber maintains one **shallow map per tree**, at the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ namespace Elastic.Documentation.Configuration.ReleaseNotes;
/// </summary>
public static partial class BundleAmendMerger
{
[GeneratedRegex(@"\.amend-(\d+)(\.ya?ml)$", RegexOptions.IgnoreCase)]
// Matches both numbered amends (.amend-1.yaml) and the reconciler-owned notes sidecar (.amend-notes.yaml).
[GeneratedRegex(@"\.amend-(\d+|notes)(\.ya?ml)$", RegexOptions.IgnoreCase)]
private static partial Regex AmendFileRegex();

/// <summary>Whether a path is an amend sidecar (<c>{name}.amend-{N}.yaml</c>).</summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -429,22 +429,22 @@ private static Uri CombineSegments(Uri baseUri, IReadOnlyList<string> segments)
/// <summary>
/// <summary>
/// Fetches all <c>note-*.yml</c> entries for <paramref name="org"/>/<paramref name="repo"/> at
/// <paramref name="target"/> from the CDN. Reads the <c>notes-{target}.json</c> index to enumerate
/// <paramref name="version"/> from the CDN. Reads the <c>notes-{version}.json</c> index to enumerate
/// the pool-relative note paths; a missing index means no notes (not an error). A listed note that
/// cannot be fetched is a hard error — the index is an authoritative promise that the note exists.
/// </summary>
/// <param name="baseUri">CDN base URI.</param>
/// <param name="org">Repository org (e.g. <c>elastic</c>).</param>
/// <param name="repo">Repository name (e.g. <c>kibana</c>).</param>
/// <param name="target">Target version string (e.g. <c>9.0.0</c>).</param>
/// <param name="version">Release version string (e.g. <c>9.0.0</c>).</param>
/// <param name="emitError">Called once per hard error; caller decides how to surface it.</param>
/// <param name="ctx">Cancellation token.</param>
/// <returns>The fetched note entries, keyed by pool-relative path (<c>main/note-foo.yml</c>).</returns>
public async Task<IReadOnlyList<CdnChangelogEntry>> FetchNotesAsync(
Uri baseUri,
string org,
string repo,
string target,
string version,
Action<string> emitError,
Cancel ctx)
{
Expand All @@ -454,21 +454,21 @@ public async Task<IReadOnlyList<CdnChangelogEntry>> FetchNotesAsync(
return [];
}

var indexUri = CombineSegments(baseUri, ["changelog", org, repo, $"notes-{target}.json"]);
var indexUri = CombineSegments(baseUri, ["changelog", org, repo, $"notes-{version}.json"]);
NotesIndex? index;
try
{
var (notFound, content) = await FetchTextOrNotFoundAsync(indexUri, 1, ctx).ConfigureAwait(false);
if (notFound)
{
_logger.LogDebug("Notes index for {Org}/{Repo}@{Target} not found at {Uri}; no notes to bundle", org, repo, target, indexUri);
_logger.LogDebug("Notes index for {Org}/{Repo}@{Version} not found at {Uri}; no notes to bundle", org, repo, version, indexUri);
return [];
}
index = JsonSerializer.Deserialize(content, NotesIndexJsonContext.Default.NotesIndex);
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
emitError($"Could not fetch notes index for {org}/{repo}@{target} from {indexUri}: {ex.Message}");
emitError($"Could not fetch notes index for {org}/{repo}@{version} from {indexUri}: {ex.Message}");
return [];
}

Expand All @@ -477,23 +477,24 @@ public async Task<IReadOnlyList<CdnChangelogEntry>> FetchNotesAsync(

var repoLabel = $"{org}/{repo}";
var entries = new List<CdnChangelogEntry>(index.Notes.Count);
foreach (var poolRelativePath in index.Notes)
foreach (var noteEntry in index.Notes)
{
ctx.ThrowIfCancellationRequested();

// Pool-relative path is "{branch}/note-{name}.yml"; split on first '/' only.
var poolRelativePath = noteEntry.Path;
var slash = poolRelativePath.IndexOf('/', StringComparison.Ordinal);
if (slash <= 0 || slash == poolRelativePath.Length - 1)
{
emitError($"Notes index for {repoLabel}@{target} lists an invalid pool-relative path '{poolRelativePath}'; expected {{branch}}/{{file}}.");
emitError($"Notes index for {repoLabel}@{version} lists an invalid pool-relative path '{poolRelativePath}'; expected {{branch}}/{{file}}.");
return [];
}
var branch = poolRelativePath[..slash];
var noteFileName = poolRelativePath[(slash + 1)..];

if (!ChangelogKeys.IsValidBranch(branch))
{
emitError($"Notes index for {repoLabel}@{target} lists path '{poolRelativePath}' with an invalid branch segment.");
emitError($"Notes index for {repoLabel}@{version} lists path '{poolRelativePath}' with an invalid branch segment.");
return [];
}

Expand All @@ -510,12 +511,12 @@ public async Task<IReadOnlyList<CdnChangelogEntry>> FetchNotesAsync(

// The notes index asserts this note exists — a miss is a real pipeline error.
emitError(
$"Note '{poolRelativePath}' for {repoLabel}@{target} is listed in the notes index but could not be fetched from {noteUri}: {lastError}. " +
$"Note '{poolRelativePath}' for {repoLabel}@{version} is listed in the notes index but could not be fetched from {noteUri}: {lastError}. " +
"Ensure the note was uploaded and scrubbed; if it persists check the changelog scrubber pipeline.");
return [];
}

_logger.LogInformation("Fetched {Count} note(s) for {Repo}@{Target} from {BaseUri}", entries.Count, repoLabel, target, baseUri);
_logger.LogInformation("Fetched {Count} note(s) for {Repo}@{Version} from {BaseUri}", entries.Count, repoLabel, version, baseUri);
return entries;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,26 @@ public record ChangelogEntryDto
public record ProductInfoDto
{
public string? Product { get; set; }

/// <summary>
/// Obsolete — entries derive applicability from their origin branch; notes use <see cref="Versions"/>.
/// Still deserialized for backward compatibility with already-published pool objects.
/// </summary>
[Obsolete("Entries derive applicability from their origin branch; notes use Versions.")]
public string? Target { get; set; }

/// <summary>
/// The releases this note applies to (note-only field). For entries this is always null or empty.
/// Expressed in the YAML as a sequence:
/// <code>
/// versions: [9.3.0, 9.4.0, 9.5.0]
/// </code>
/// or as a pipe-separated string in the <c>--products</c> CLI flag:
/// <code>
/// --products 'elasticsearch 9.3.0|9.4.0|9.5.0 ga'
/// </code>
/// </summary>
public List<string>? Versions { get; set; }

public string? Lifecycle { get; set; }
}
Original file line number Diff line number Diff line change
Expand Up @@ -99,11 +99,17 @@ public static string ChangelogRegistryKey(string poolGroup) =>
$"{ChangelogPrefix}{poolGroup}/{RegistryFileName}";

/// <summary>
/// The notes-index key for one target within a repo: <c>changelog/{org}/{repo}/notes-{target}.json</c>.
/// Repo-level and branch-agnostic — all notes for a target, regardless of which branch they were authored on.
/// The notes-index key for one release version within a repo: <c>changelog/{org}/{repo}/notes-{version}.json</c>.
/// Repo-level and branch-agnostic — all notes for a version, regardless of which branch they were authored on.
/// </summary>
public static string NotesIndexKey(string org, string repo, string target) =>
$"{ChangelogPrefix}{org}/{repo}/notes-{target}.json";
/// <remarks>
/// The slug in the key is the release version (e.g. <c>9.3.0</c> or <c>2026-05-15</c>).
/// Previously this parameter was named <c>target</c> to match the obsolete <c>target:</c> YAML field;
/// it was renamed to <c>version</c> when that field was replaced by <c>versions:</c> on notes.
/// The key layout (<c>notes-{slug}.json</c>) is unchanged — no migration is needed.
/// </remarks>
public static string NotesIndexKey(string org, string repo, string version) =>
$"{ChangelogPrefix}{org}/{repo}/notes-{version}.json";

/// <summary>
/// The S3 prefix that covers all branches and notes indexes of one repo: <c>changelog/{org}/{repo}/</c>.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,4 @@ public sealed record ChangelogRegistryBundle
[JsonSerializable(typeof(ChangelogRegistry))]
[JsonSerializable(typeof(ChangelogRegistryBundle))]
[JsonSerializable(typeof(Dictionary<string, string>))]
internal sealed partial class ChangelogRegistryJsonContext : JsonSerializerContext;
public sealed partial class ChangelogRegistryJsonContext : JsonSerializerContext;
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,62 @@
namespace Elastic.Documentation.Configuration.ReleaseNotes;

/// <summary>
/// Notes index published at <c>changelog/{org}/{repo}/notes-{target}.json</c>.
/// Lists pool-relative paths of all <c>note-*.yml</c> fragments for one target,
/// One entry in a <see cref="NotesIndex"/> — a pool-relative path to a <c>note-*.yml</c>
/// file and a derived sequence number that records how many published bundle files (original +
/// amends) already include this note.
/// </summary>
/// <remarks>
/// <para>
/// The origin branch is the leading segment(s) of <see cref="Path"/> before the last <c>/</c>
/// (e.g. <c>path[..path.LastIndexOf('/')]</c>) and is not stored separately to avoid a second
/// source of truth that can disagree with the path.
/// </para>
/// <para>
/// <b>bundle_seq values:</b>
/// <list type="bullet">
/// <item>0 — no bundle is published for this version yet; the note is unreleased.</item>
/// <item>1 — the note shipped in the original bundle.</item>
/// <item>2 — the note was picked up by the reconciler-owned <c>{parent}.amend-notes.yaml</c>.</item>
/// </list>
/// The field is derived and updated on every reconcile pass; it is never authored manually.
/// </para>
/// </remarks>
public sealed record NoteIndexEntry
{
/// <summary>Pool-relative path, e.g. <c>main/note-slow-rollover.yml</c>.</summary>
public required string Path { get; init; }

/// <summary>
/// How many published bundle files for this version already contain this note.
/// 0 = unreleased, 1 = in original bundle, 2 = in reconciler amend sidecar.
/// Derived on every reconcile; never authored.
/// </summary>
public int BundleSeq { get; init; }
}

/// <summary>
/// Notes index published at <c>changelog/{org}/{repo}/notes-{version}.json</c>.
/// Lists all <c>note-*.yml</c> fragments for one release version,
/// across every branch of the repo.
/// </summary>
/// <remarks>
/// Contents are paths, not bodies — the note files remain the single source of truth.
/// A stale index can only omit or over-list, never serve stale prose. Bundling a target
/// A stale index can only omit or over-list, never serve stale prose. Bundling a version
/// is therefore 1 GET for the index + one GET per listed note.
/// </remarks>
public sealed record NotesIndex
{
/// <summary>Pool-relative paths of notes for this target, e.g. <c>["main/note-slow-rollover.yml"]</c>.</summary>
public required IReadOnlyList<string> Notes { get; init; }
/// <summary>Schema version — bumped when consumers must change their parser.</summary>
public int SchemaVersion { get; init; } = CurrentSchemaVersion;

/// <summary>Current schema version constant.</summary>
public const int CurrentSchemaVersion = 1;

/// <summary>
/// Notes for this version. Each entry carries the pool-relative path, origin branch,
/// and a derived <c>bundle_seq</c>.
/// </summary>
public required IReadOnlyList<NoteIndexEntry> Notes { get; init; }
}

[JsonSourceGenerationOptions(
Expand All @@ -28,4 +71,5 @@ public sealed record NotesIndex
DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull
)]
[JsonSerializable(typeof(NotesIndex))]
[JsonSerializable(typeof(NoteIndexEntry))]
public sealed partial class NotesIndexJsonContext : JsonSerializerContext;
Loading
Loading