From dfd5caf53a3155e687080cd7eea98c6948641217 Mon Sep 17 00:00:00 2001 From: Oto Macenauer Date: Fri, 4 Sep 2026 15:03:11 +0200 Subject: [PATCH] refactor: make apps.json a source-only registry with one install path MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit apps.json duplicated everything the docs repo already declared — slug, name, description, icon, tags — and the duplication was not even resolved the way the docs claimed. fetch-apps.js merged the manifest over the registry entry ("manifest wins"), build-vite.js discarded that result for packaged apps, and Astro's loadRegistry read apps.json alone. So a catalog card showed whatever the onboarding PR typed rather than what the repo published, and the `pages` navigation manifest never reached getAppPages at all: the Array.isArray(app.pages) branch in apps.js was dead code and every packaged app was filesystem-crawled regardless of what its manifest said. A registry entry now names a source and nothing else: [{ "repo": "AbsaOSS/my-docs", "version": "latest" }] Display fields are rejected rather than ignored. An onboarding PR that sets one has a mistaken idea of where metadata lives, and silently dropping it would leave the author waiting for a card that never changes. One install path replaces two. stageEntry puts an artifact on disk — download, copy, or pack a local checkout — and is the only place the three sources differ. installArtifact then reads the manifest, validates it, copies one directory per declared app into apps/{slug}/ and checks the headless marker, identically for all of them. fetch-apps.js is now purely a downloader; src/utils/single-page.js becomes src/utils/registry.js and generalises from bundles to every artifact. Consequences worth naming: * `pages` is honoured, and is authoritative when present — the directory is not crawled, so nothing it happens to contain becomes a route the publisher did not ask for. * The headless-marker check runs for every source. It used to run only on the GitHub path, so the prebuilt fixtures CI actually uses were never checked — and the vendored one turned out to carry no marker at all. * The single-page type is gone as a registry concept. A markdown bundle and a docs site are both artifacts with a manifest; the reading-column layout is inferred from the artifact (one HTML file, no `pages`) rather than declared. * localPath runs a per-entry `pack` command (default `npm run pack:kb`) instead of a hard-coded `npm run build:headless`, which could not run in the Python example repo at all. * KB_REGISTRY selects the registry file, so a deployment repo can own its own list without forking this one. * The artifact size budget from contract/ARTIFACT.md is enforced on download, and archive members no app claims are reported rather than silently ignored. Fixtures rebuilt for the v1 layout. The vendored artifact is now one kb-docs.tar.gz whose manifest declares two apps: user-guide, crawled and complete, and guide-mirror, which carries a `pages` list of four and omits a fifth page that exists on disk. That asymmetry is what tests the feature, and it drops the artifact from 6.5 MB to 4.2 MB by not duplicating the vendored CMS bundle. Its HTML is now stamped with the headless marker, because a fixture that does not meet the contract cannot verify it. Suites green: 118 embedded (up 3 — the pages manifest had no coverage before), 22 standalone. Part of #73. Closes #75. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01PYtmxSeSXFg4wEXRcKCmAi --- CLAUDE.md | 21 +- README.md | 38 +- apps.json | 25 +- scripts/artifacts.js | 2 +- scripts/build-vite.js | 295 ++++++++------ scripts/fetch-apps.js | 284 ++++---------- scripts/setup-test-apps.mjs | 65 ++-- src/utils/apps.js | 39 +- src/utils/config.js | 14 + src/utils/registry.js | 366 ++++++++++++++++++ src/utils/single-page.js | 254 ------------ tests/artifact-safety.spec.js | 6 +- tests/build-integrity.spec.js | 90 ++++- tests/fixtures/docs-example.dist.tar.gz | Bin 2954072 -> 0 bytes tests/fixtures/docs-example.kb-docs.tar.gz | Bin 0 -> 4445669 bytes .../{bundle.json => kb-docs.json} | 9 +- .../release-process/index.html | 2 +- 17 files changed, 792 insertions(+), 718 deletions(-) create mode 100644 src/utils/registry.js delete mode 100644 src/utils/single-page.js delete mode 100644 tests/fixtures/docs-example.dist.tar.gz create mode 100644 tests/fixtures/docs-example.kb-docs.tar.gz rename tests/fixtures/single-page-bundle/{bundle.json => kb-docs.json} (78%) diff --git a/CLAUDE.md b/CLAUDE.md index a0ec676..2ee7a19 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -64,7 +64,7 @@ Orchestrator: `scripts/build-vite.js`. Flags: `--local`, `--headless`. ### Core Data Flow -`apps.json` → `scripts/fetch-apps.js` downloads `kb-docs.tar.gz` per app → `apps/{slug}/` → Astro's `src/pages/[...path].astro` catchall uses `getStaticPaths()` from `src/utils/apps.js` to enumerate every HTML file → `src/utils/transform.js` rewrites URLs and splits the document → `Base.astro` re-hosts the parts → static output in `dist/`. +`apps.json` (a source per entry) → `scripts/build-vite.js` stages each artifact (downloading via `scripts/fetch-apps.js` for a `repo`) → reads its `kb-docs.json` → copies one directory per declared app into `apps/{slug}/` → Astro's `src/pages/[...path].astro` catchall uses `getStaticPaths()` from `src/utils/apps.js` to enumerate every HTML file → `src/utils/transform.js` rewrites URLs and splits the document → `Base.astro` re-hosts the parts → static output in `dist/`. ### Key Source Files @@ -76,21 +76,23 @@ Orchestrator: `scripts/build-vite.js`. Flags: `--local`, `--headless`. - `src/components/Masthead.astro` — Persistent Knowledge base header + Library/current-app sub-nav (all pages, both modes) - `src/components/AppCard.astro`, `src/components/AppIcon.astro` — Catalog card and its icon - `src/templates/shadow-compat.js` — Shadow-DOM design-token styles, injected into the body by the layout -- `src/utils/config.js` — `PATH_PREFIX`/`BASE_PATH` and `isHeadlessBuild()` — the build-wide constants -- `src/utils/single-page.js` — Bundle manifest reading/validation + registry expansion, shared by both fetch paths and by Astro +- `src/utils/config.js` — `PATH_PREFIX`/`BASE_PATH`, `isHeadlessBuild()` and `REGISTRY_FILE` — the build-wide constants +- `src/utils/registry.js` — Registry validation, manifest reading/validation, expansion map. Shared by the build and by Astro so both resolve the same registry - `scripts/build-vite.js` — Build orchestrator (4 steps: prepare, hoist, copy assets, astro build) -- `scripts/fetch-apps.js` — GitHub Release artifact downloader +- `scripts/fetch-apps.js` — GitHub Release artifact downloader. Only *obtains* an artifact; installing it is one shared path in `build-vite.js` - `scripts/artifacts.js` — Safe tarball extraction + tree copy, shared by both fetch paths. Validates archive members (no traversal, no absolute paths, no symlinks) before anything is written, and replaces the old `cp -r`/`tar` shell-outs so the build runs on Windows - `scripts/hoist-inline-scripts.js` — Moves inline `