docs: refresh the CLI reference for stash 1.1.1, and stop it going stale - #103
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
The published CLI reached 1.1.1 on 2026-08-20. Production has been serving a reference generated from 1.0.0 ever since. It is in every build log: ⚠ Could not run stash@1.1.1; using cached fixture. ✓ Generated 14 CLI reference page(s) for stash v1.0.0 `npm view stash version` resolved 1.1.1 correctly; the next step, `npx --yes stash@1.1.1 manifest --json`, does not work in the Vercel build sandbox, and the generator fell back to the committed fixture and carried on. Eleven days of a version-old reference, and nothing failed, because a fallback that always works cannot fail. Two commands' worth of drift: 1.1.1 removed `db validate` and added `eql preflight` and `eql verify`, both of which carry substantial prose about managed-Postgres behaviour. The build no longer touches the network. `generate-docs:cli` reads the committed fixture and renders it — what deploys is exactly what is in the repo, and it is reviewable in the diff. Refreshing the fixture is now a separate, explicit step (`--refresh`, exposed as `generate-docs:cli:refresh`) that resolves the latest published CLI, runs it, and rewrites the fixture. It has no fallback: if it cannot run the CLI it fails, and it inherits stderr rather than discarding it, so the reason is visible. .github/workflows/cli-manifest.yml runs that refresh daily, and opens a PR when the published version has moved past the cached one — the same shape as the EQL pin workflow in #76. The refresh is not auto-merged: a release can remove commands as well as add them, and each removal is a section that disappears and an anchor that stops resolving, so the diff gets read. Also names the generic headings on the group pages, which #101 left alone because they come from a second render path. `eql.mdx` carried five identical `#### Examples` and eight identical `#### Flags` — eight duplicate anchors on one page, and eight chunks a retriever cannot tell apart. They now read `#### stash eql verify flags`, matching what #101 did for the single-command pages. Verified: `generate-docs:cli` with no network access reproduces the committed pages exactly; validate-links unchanged at its 9-failure baseline (all pre-existing, in generated integration api-reference dirs); types:check and biome clean.
Review question: what actually goes wrong with `npx stash@1.1.1 manifest
--json` on Vercel?
Checking older build logs answers a different and more useful question first.
The npx step has never once succeeded on Vercel:
2026-08-16 ⚠ Could not run stash@1.0.0; using cached fixture.
2026-08-18 ⚠ Could not run stash@1.1.0; using cached fixture.
2026-08-31 ⚠ Could not run stash@1.1.1; using cached fixture.
The first of those is a build where the fixture already matched the latest
release, so the output was correct and nothing looked wrong. The docs were
right by coincidence — the committed fixture happened to be current — and
1.1.x did not break anything, it made a pre-existing failure visible.
So the previous framing here ("from stash 1.1.1 every production build
logged…") understated it. Corrected in both the generator header and the
workflow.
The cause is still unknown, and honestly labelled as such rather than
asserted. stderr was discarded on every one of those runs
(`stdio: [..., "ignore"]`). What is known: it fails in a consistent ~8
seconds, and `npm view` in the same script succeeds, so the registry is
reachable.
The workflow is now also the diagnostic. It is the one step that runs the CLI,
it inherits stderr, and it fails loudly — so its first run either succeeds on
Ubuntu, which localises the fault to the Vercel builder, or prints the error
eight months of build logs never captured.
Review question: could stash be a dependency of the docs site itself?
Tried it, and it cannot be. `stash` depends on zod 3 and fumadocs on zod 4;
adding it hoists zod 3 to the root, the frontmatter schema in source.config.ts
stops inferring, and the docs stop type-checking:
src/lib/source.ts(62,43): error TS2345:
Argument of type '{}' is not assignable to parameter of type 'string'.
Which is the blast radius that argument had in the abstract, arriving on day
one instead of on some future CLI release. A docs build should not be
breakable by the CLI's dependency tree.
Everything else about the idea holds, so this takes it without the dependency:
- The version is PINNED in this script, exactly as EQL_RELEASE_TAG pins EQL in
generate-eql-docs.ts. What produced the fixture is recorded in the repo and
moves by reviewable commit, not by whatever npm serves that morning.
- Rendering asserts the fixture's version equals the pin, and fails otherwise.
Bumping one without regenerating the other is the exact drift that shipped a
version-old reference, and it is now caught with no network access at all.
- `--refresh` installs the pinned CLI into a temp directory and runs it from
there. No `npx`, which is the thing that never once worked on Vercel, so the
undiagnosed failure stops being load-bearing.
The workflow bumps the pin, regenerates, and opens a PR with both. It does the
two together deliberately: the gate means a PR that bumped only the pin would
be red on arrival.
Verified: both paths render byte-identical pages; the gate fires on a
simulated bump (`Cached manifest is stash v1.1.1 but the pin is v9.9.9`); the
workflow's pin-extraction and pin-rewrite steps both run correctly against the
real file; types:check and biome clean, with no new dependencies.
1c635bb to
08a5a04
Compare
There was a problem hiding this comment.
Pull request overview
Refreshes the generated stash CLI reference to match stash v1.1.1 and changes the generation pipeline to be deterministic/offline-by-default by rendering from a committed manifest fixture, with a separate explicit refresh step.
Changes:
- Pin the CLI version in
scripts/generate-cli-docs.ts, enforce “fixture version == pin”, and add a--refreshmode that installs/runs the pinned CLI to rewrite the fixture. - Update the committed CLI manifest fixture and regenerate the CLI reference MDX pages to reflect
stashv1.1.1 (including new/removed commands and heading de-duplication). - Add a scheduled GitHub Actions workflow to detect new
stashreleases and open a PR that bumps the pin + refreshes the fixture/pages.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/generate-cli-docs.ts | Pins CLI version, renders from committed fixture by default, adds refresh mode to rewrite fixture, and improves group-page heading uniqueness. |
| scripts/fixtures/stash-manifest.json | Updates cached CLI manifest from 1.0.0 → 1.1.1 with the new command surface and updated help text. |
| package.json | Adds generate-docs:cli:refresh script for explicit fixture refresh + regeneration. |
| content/docs/reference/cli/*.mdx | Regenerates CLI reference pages from the updated manifest (version stamps, new/removed commands, named section headings). |
| .github/workflows/cli-manifest.yml | Adds scheduled workflow to detect new CLI releases and open/update a refresh PR. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| const CLI_VERSION_PIN = process.env.STASH_VERSION ?? "1.1.1"; | ||
| // Refresh the fixture from npm instead of reading it. CI and humans only. | ||
| const REFRESH = process.argv.includes("--refresh"); | ||
| let CLI_VERSION = ""; // resolved to the latest published npm version at run time |
There was a problem hiding this comment.
Fixed in 52f0e70. The comment now says what the variable actually is — read back off the manifest in main() rather than assumed from CLI_VERSION_PIN, since the manifest is what the pages are built from and readFixture() already asserts the two agree.
Gave RUNNER the same treatment while there: it is how a reader invokes the CLI, not how this script obtains it, and those two meanings were easy to conflate — which is how the misleading provenance string in your next comment got written.
Three comments, all correct. The inline comment on CLI_VERSION still described it as "resolved to the latest published npm version at run time", which stopped being true in this PR. It now says what it is: read back off the manifest in main() rather than assumed from the pin, because the manifest is what the pages are built from and readFixture() already guarantees the two agree. RUNNER gets the same treatment — it is how a *reader* invokes the CLI, not how this script obtains it, and the two were easy to conflate. The generated marker and the on-page callout still told readers the pages came from `npx stash@<v> manifest --json` and that re-running `generate-docs:cli` would refresh from the latest published CLI. Neither is true now: rendering is offline from the committed fixture, and refreshing is a separate script behind a pin. Both strings corrected and the 14 pages regenerated. The callout keeps `npx stash@1.1.1 --help`, which is still exactly how a reader checks the live surface. The workflow used `bun-version: latest` while docs.yml pins 1.3.13, with a comment saying why. Pinned to match. A workflow that commits regenerated content is the last place to let a toolchain float. types:check clean; validate-links unchanged at its 9-failure baseline.
Stacked on #101 — it changes the same generator and the same generated pages. Merge #101 first and GitHub retargets this to
main.The docs are a CLI minor behind, and have been for eleven days
stash1.1.1 published on 2026-08-20. Production still serves a reference generated from 1.0.0. It is in every production build log:npm view stash versionresolves 1.1.1 correctly. The next step —npx --yes stash@1.1.1 manifest --json— does not work in the Vercel build sandbox, and the generator fell back to the committed fixture and carried on. Nothing failed, because a fallback that always works cannot fail.Why the
npxinvocation fails there is still unknown: the generator ran it withstdio: ["ignore", "pipe", "ignore"], so the reason was discarded. That is fixed below.What was actually stale
Not just a version stamp. 1.1.1:
db validate— documented today, gone from the CLIeql preflight— role/permission probe that names the statement each gap blockseql verify— catalog comparison that catches a partial EQL install which reports success and then fails at query timeBoth new pages carry real prose about managed-Postgres behaviour. Anyone reading the reference for the last eleven days could not find either command.
The build no longer touches the network
generate-docs:clireads the committed fixture and renders it. What deploys is exactly what is in the repo, and it is reviewable in the diff.Refreshing the fixture is a separate, explicit step:
which resolves the latest published CLI, runs it, and rewrites
scripts/fixtures/stash-manifest.json. It has no fallback — if it cannot run the CLI it fails — and it inherits stderr rather than discarding it, so a future failure explains itself.Automating the release follow-through
.github/workflows/cli-manifest.yml, daily, same shape as the EQL pin workflow in #76: comparesnpm view stash versionagainst the cached manifest, and when they differ, refreshes, regenerates, runs the links check, and opens a PR reporting whether links still resolve.Not auto-merged, deliberately. A release removes commands as well as adding them — this one dropped
db validate— and each removal is a page section that disappears and a URL anchor that stops resolving. The workflow removes the need to remember, not the need to review.Two things worth knowing:
DOCS_BRANCH: v2. That was the docs line when it was written; it ismainnow. It needs that changed before it will do anything useful.Also: duplicate headings on the group pages
#101 named the generic headings on single-command pages. The group pages come from a second render path it did not touch, and were worse:
eql.mdxcarried five identical#### Examplesand eight identical#### Flags. That is eight duplicate anchors on one page, and eight chunks a retriever cannot tell apart.Verified
generate-docs:cliwith no network reproduces the committed pages exactly — the determinism claim, tested rather than asserted.validate-linksunchanged at its 9-failure baseline (all pre-existing, in generated integrationapi-referencedirs absent from a local checkout).types:checkandbiome checkclean.Correction after review
The framing above ("stale for eleven days") understates it. Older build logs show the
npxstep has never succeeded on Vercel:The first is a build where the fixture already matched the latest release, so the output was correct and nothing looked wrong. The docs were right by coincidence. 1.1.x did not break anything — it made a pre-existing failure visible.
Why it fails is still unknown, and the code and workflow now say so rather than asserting a cause. stderr was discarded on every one of those runs (
stdio: [..., "ignore"]). What is known: it fails in a consistent ~8 seconds, andnpm viewin the same script succeeds, so the registry is reachable.The workflow doubles as the diagnostic. It is the one step that runs the CLI, it inherits stderr, and it fails loudly — so its first run either succeeds on Ubuntu, localising the fault to the Vercel builder, or prints the error the build logs never captured.
Update: version pinning
Tried making
stasha devDependency of the docs app. It cannot be one:stashneeds zod 3, fumadocs needs zod 4, and installing it hoists zod 3 to the root, at which point the frontmatter schema insource.config.tsstops inferring and the docs stop type-checking:That is the blast-radius objection arriving on day one rather than on some future CLI release. A docs build should not be breakable by the CLI dependency tree.
The rest of the idea holds and is now in:
generate-cli-docs.ts, exactly asEQL_RELEASE_TAGpins EQL ingenerate-eql-docs.ts. What produced the fixture is recorded in the repo and moves by reviewable commit.--refreshinstalls the pinned CLI into a temp directory and runs it from there. Nonpxanywhere, so the failure that never worked on Vercel stops being load-bearing whether or not it is ever diagnosed.The workflow bumps the pin and regenerates in one step, since the gate would make a pin-only PR red on arrival.
Verified: both render paths produce byte-identical pages, the gate fires on a simulated bump, the workflow pin-extract and pin-rewrite snippets run correctly against the real file, and
types:checkandbiome checkare clean with no new dependencies.