Skip to content

Publish GenDoc's own error catalog on every PR, and gate cli releases on breaking changes to it #167

Description

@Reefact

Context / Problem

#142 made FirstClassErrors.GenDoc document its own failures with FirstClassErrors (16 GENDOC_* codes). One of the three original motivations for that work was to "test the CI/CD integration for automatic documentation generation" — but the PR that closed #142 only proved the pipeline runs (a transient grep assertion in the floor CI job); it never actually published anything, and nothing tracks whether a later change to GenDoc's own errors is safe to ship.

Separately, #164/#165 reorganized doc/ and created doc/generated/gendoc/ specifically as "the CI/CD output target — living documentation produced by CI/CD (e.g. the error catalog emitted by GenDoc)". That folder exists today holding only a placeholder: nothing writes into it yet.

Two gaps follow from this:

  1. No living documentation. A developer integrating with fce or debugging a GENDOC_* code has no browsable reference for GenDoc's own error catalog — they would have to run fce generate themselves.
  2. No safety net for GenDoc's own compatibility contract. GenDoc ships bundled inside the fce tool (it has no NuGet package of its own — see tools/packaging/pack.sh), so a breaking change to its error catalog (a code renamed or removed, a context key dropped or retyped) is a breaking change of the cli train. Nothing today would stop that from shipping in a version bump that isn't major.

The library already ships exactly the CLI mechanics this needs — fce catalog update / fce catalog diff --fail-on breaking, a committed baseline snapshot, and a Breaking/Compatible/Informational classification (see Catalog Versioning Reference) — built for exactly this "track a catalog as a versioned contract" scenario, just never wired up for GenDoc's own catalog.

Goal

Close both gaps with two independent, closed-loop pieces of automation:

1. Keep the living documentation current, through normal review.

A dedicated workflow (gendoc-docs.yml) will run on every push to a pull-request branch. It will:

  • build fce and FirstClassErrors.GenDoc;
  • regenerate the human-readable catalog as Markdown (split layout — one page per error plus an index) into doc/generated/gendoc/catalog/;
  • run fce catalog diff --report markdown --fail-on none against a committed baseline into doc/generated/gendoc/errors-diff.mdinformational only, this never blocks a pull request;
  • commit both back onto the pull request's own branch (never main directly) when they changed, so the regenerated content becomes part of the normal diff a human reviews and merges — no bot-authored content ever lands on main without going through review.

2. Enforce the compatibility contract at the one point it actually matters: release.

release.yml, for the cli train only, will gain a check before packing: run fce catalog diff --fail-on breaking against the baseline. If it reports a breaking change, the version tag being published must bump the major component over the previous cli-v tag — otherwise the release fails, loudly, with a message telling the maintainer to either publish a major version or revert the breaking change. This runs on dry runs too, so a missed bump is caught by rehearsal, not by a real, irreversible publish attempt.

After a successful, non-dry-run cli publish, the workflow will accept the just-released catalog as the new baseline (fce catalog update) and push that refresh to main — the one direct-to-main write in this design, deliberately scoped to a moment that already required an explicit human action (pushing a release tag), never to an ordinary pull request.

Why this design

  • Never push generated content straight to main outside of a release. The repository's one existing precedent for "CI produces content and persists it" (changelog.yml) always opens a pull request for human review, because its content is model-inferred. GenDoc's catalog is a deterministic rendering with nothing to curate — but committing it still goes through the normal pull-request flow (onto the PR's own branch, merged by a human) rather than bypassing it, so there is exactly one exception to "nothing auto-commits to main": the post-release baseline refresh, and that one is justified by following a deliberate release action, not a routine merge.
  • Compare against the last release, not a rolling snapshot. The baseline only moves when a cli release actually publishes. Between releases it stays fixed, so every pull request's diff report — however many pull requests land in between — always reads as "what changed since the last thing that was actually shipped," which is the question that matters for the release gate.
  • Reuse the existing fce catalog primitives, add no new CLI behavior. fce catalog update/diff --fail-on breaking (exit code 2 on a breaking change) already implement the baseline-and-diff mechanics and the Breaking/Compatible/Informational classification; this work is wiring, not new product surface.
  • Gate the release, not the pull request. A breaking change to an error catalog is not itself wrong — an intentional one is exactly what a major version exists for. The problem is only a breaking change shipping silently, under a version number that promises compatibility. Blocking pull requests would fight normal incremental development; gating the release targets the one place the promise is actually made.

Scope

  • New workflow: .github/workflows/gendoc-docs.yml.
  • doc/generated/gendoc/: catalog/ (rendered Markdown), errors-baseline.json (the versioned contract), errors-diff.md (pending diff against it). Bootstrap the baseline once, from the current 16-error surface, since none exists yet.
  • .github/workflows/release.yml: the major-bump gate before Pack, and the baseline refresh after a successful cli publish.
  • doc/generated/README.md: describe the new gendoc/ sub-structure.

Non-goals

Verification

  • dotnet build FirstClassErrors.sln green (no source changes, so dotnet test is unaffected).
  • The exact commands both workflows run — fce generate --layout split, fce catalog update, fce catalog diff --fail-on none — run locally against the real FirstClassErrors.GenDoc.dll and produce the expected output.
  • fce catalog diff --fail-on breaking confirmed to exit 2 against a simulated breaking change (a removed code, on a scratch copy of the baseline — the real baseline untouched).
  • The release-gate's version-comparison logic exercised standalone against 6 scenarios: no breaking change; breaking change with a matching major bump; breaking change without one (correctly fails); breaking change on a train's first release (nothing to compare against, passes); a major bump alongside a pre-release suffix (2.0.0-beta.1); an unexpected fce exit code (fails loud rather than passing silently).

Note

This introduces a lasting policy — a breaking change to GenDoc's own error catalog mechanically forces a major version bump of the cli train — which a future maintainer would plausibly want to revisit deliberately. It may be worth an ADR of its own, or folding into ADR-0009 (which already covers GenDoc modeling its own errors); left to @reefact to judge.

Related

Refs #142 (the errors this documents), #164/#165 (the doc/generated/ target this fills in).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions