feat(cli): auto-detect embedded private packages [RED-862]

[sorccu]

Linear: RED-862

Stacked on #1435 — targets simo/red-855-embed-private-dep-tarballs and should be rebased/retargeted onto main once that PR merges. Note the test workflow only triggers on PRs targeting main/next/*, so CI checks will not run here until the retarget; the full unit suite (2013 tests) passes locally on this commit.

Adds automatic detection of private packages to embed in Playwright code bundles, complementing the explicit checks.embeddedPackages list from RED-855. Detection runs during deploy/test/pw-test, is enabled by default (checks.detectEmbeddedPackages, per-run --no-detect-embedded-packages, env CHECKLY_DETECT_EMBEDDED_PACKAGES), and fails soft: any detection problem degrades to a warning, never a failed run, while explicit embeddedPackages entries keep their strict, fatal guarantees.

Privacy model

Private package names never leave the machine unless explicitly opted in:

1. Zero-network proofs — a public recorded tarball URL, or a public effective registry, classifies a package without any traffic.
2. Scope mapping — @scope:registry pointing at a non-public registry ⇒ embed, no lookup (over-embedding is safe by the bundle contract).
3. Registry interrogation — remaining undecided packages are checked against the customer's own registry via the Sonatype Nexus REST API, using the .npmrc credentials: hosted ⇒ embed. A per-group source-repo visibility guard and a zero-hosted-repos guard prevent permission-filtered listings from minting unsound "public" verdicts; same-origin recorded sources that aren't Nexus-shaped are decided conservatively (hosted ⇒ embed, silence ⇒ still undecided).
4. Opt-in fallback — checks.detectEmbeddedPackagesFallback: "public-registry" allows an integrity diff against public npm for whatever remains undecided. This transmits those names, so it is off by default (skip + warning). Transmitted names yield durable cached verdicts, so a name never needs transmitting twice — including when a lookup fails mid-run (partial results are persisted).

Graph-frontier pruning (added after review of the initial version)

The fallback previously fetched one packument per unique undecided name — tens of thousands of requests for a large mirrored tree. Detection now extracts the dependency graph from the lockfile (pnpm v6/v9 snapshots/inline deps, npm v2/v3 nearest-node_modules resolution, aliases handled, git/file entries excluded) and runs the diff in frontier rounds: a package a provably public parent vouches for is assumed public without a lookup (its name is never transmitted — privacy strictly improves), while the exposed surface is always verified: workspace-direct dependencies, children of embedded packages, any version of a name with known private versions, parentless entries, and entries whose parents are still undecided (deferred to a later round). For a tree with ~100 direct dependencies, request count drops to roughly those roots plus the surface of private subtrees.

Soundness guardrails: assumed verdicts are refutable and never persisted to the immutable verdict cache; runs that assumed anything are not summary-cached (assumptions re-derive each run, and real registry verdicts take over once the registry becomes interrogable); the summary digest includes the fallback mode so flipping back to "skip" never serves an assumption-derived embed set; groups are processed inventory-capable-first so registry-proven embeds expose their children before any diff can assume them. The residual blind spot — a private artifact under a public name at an interior, fully-vouched tree position — cannot silently run wrong code (the runner install fails its lockfile integrity check loudly); the remedy (checks.embeddedPackages) is documented in the config JSDoc, the reference docs, and install troubleshooting.

Caching

Two levels in the CLI cache (node_modules/.cache/checkly, per-user dir as fallback): a detection summary keyed by lockfile bytes + registry config + credentials + explicit specs, and per-entry immutable integrity-proof verdicts. Degraded runs (skipped packages or a broken .npmrc, e.g. an unset ${VAR}) are never summary-cached, so problems keep warning until fixed. Warm runs make zero network requests.

Validated end-to-end against a real dockerized Nexus Repository CE instance: REST-based verdicts correct with zero public-registry traffic, scope tier fully offline, warm runs request-free.

Known follow-ups (minor, from review)

• No positive test pins the restAccessRemediable tags or the combined two-tier failure message.
• An unresolvable scope mapping produces one fail-soft download warning per package (plus the config-problem warning) instead of a single aggregate.
• A few small dedups (a thrice-spelled error predicate, the classifyEntries catch mirroring the tier decision below it).
• Graph pruning went through 5 review rounds (2 beyond the cap, user-authorized), converging at round 5 with zero confirmed findings. Rounds 4–5 restructured the algorithm materially: query priority is now frontier → assumption → minimal stall-breakers → last resort, so a dependency cycle no longer drags its descendant closure into the public registry (previously up to ~44% of transmitted names on a real large lockfile came from that path); assumption runs only when no query can add evidence; no version of a name is assumed while a sibling version is undecided anywhere in the run. Remaining accepted items: per-round putVerdicts calls interact with the 10k-entry verdict-cache truncation cap (pre-existing mechanism, extreme caches only); cross-group interplay is heuristically ordered (inventory-capable groups first) rather than guaranteed, and cross-group undecided parents do not defer assumption — both documented; propagation.publicKeys/assumedCount bookkeeping is spread across three write sites (hardening opportunity); the pnpm parser iterates packages twice (once for entries, once for v6 edges). A 6th round (substantive-findings-only) confirmed one issue, now fixed: the privacy docs under-stated what the fallback transmits (names resolved at multiple versions are also always verified — the docs now name that third category instead of claiming "other names are never transmitted"). It also flagged a performance regression for projects mixing an inventory-decided Nexus group with an assumption-using diff group: losing summary caching meant re-enumerating the Nexus inventory every run. Fixed in a follow-up commit by snapshotting the registry's RAW responses (never derived verdicts, which would be unsound for topology-dependent inventory 'public' results): repeat runs recompute verdicts from snapshotted data and make no requests. Privacy guardrails: listings persist as a name/format/type projection (proxy upstream URLs with embedded credentials never reach disk), inventories are restricted to lockfile-referenced keys, filenames hash the credentials, and clean runs write no snapshot. Failures are never snapshotted (retried live), and a snapshot whose listing fails a visibility guard triggers a single live listing re-fetch — the minimum that notices a registry-side permission grant — reusing the inventory while the listing is unchanged. This went through its own 3-round review (rounds found and fixed: a catalog-spill privacy issue, a dead-write class, stale-file resurrection, and structural simplifications); remaining accepted notes: InstanceState invariants span two methods, and the NexusRegistryApi handle is resolved twice per group.

🤖 Generated with Claude Code