build-perf regression blocking v3.16.0: full build +42%, incremental rebuild +53-83%

[carlos-alm]

Summary

The v3.16.0 release attempt (tag pushed at commit 2e0e94f, 2026-07-17) failed at the "Pre-publish benchmark gate" step in publish.yml because of genuine performance regressions against the v3.15.0 baseline:

Full build: 3521 → 5000 (+42%, threshold 25%)
1-file rebuild: 112 → 205 (+83%, threshold 75%)
1-file rebuild: 113 → 206 (+82%, threshold 75%)
1-file rebuild: 117 → 179 (+53%, threshold 50%)
1-file rebuild: 117 → 180 (+54%, threshold 50%)

(source: https://github.com/optave/ops-codegraph-tool/actions/runs/29568520693, job "Pre-publish benchmark gate")

Because that gate failed, publish/publish-dev were both skipped — v3.16.0 was never actually published to npm, and package.json on main is still at 3.15.0. The v3.16.0 git tag exists on origin but has no corresponding release. This is the release currently blocked.

Impact

This is a real regression (full + incremental build time), not a flaky benchmark — it needs to be bisected and fixed before v3.16.0 (or whatever the next version is) can ship.

Suspected area

git diff --name-only v3.15.0 2e0e94f -- crates/ shows ~50 changed extractor/builder files between the two, including several domain/graph/builder/ and domain/graph/builder/stages/ changes that are plausible candidates for a build-time regression:

• crates/codegraph-core/src/domain/graph/builder/pipeline.rs
• crates/codegraph-core/src/domain/graph/builder/stages/build_edges.rs
• crates/codegraph-core/src/domain/graph/builder/stages/detect_changes.rs
• crates/codegraph-core/src/domain/graph/builder/stages/import_edges.rs
• crates/codegraph-core/src/domain/graph/builder/stages/insert_nodes.rs
• crates/codegraph-core/src/domain/graph/resolve.rs
• crates/codegraph-core/src/domain/graph/builder/barrel_resolution.rs
• crates/codegraph-core/src/db/connection.rs

Bisecting v3.15.0..2e0e94f against the incremental/full build benchmarks (scripts/benchmark.ts, scripts/incremental-benchmark.ts) should narrow this down.

Related

Filed alongside a separate CI fix (detect-changes job in ci.yml was trusting the dangling v3.16.0 tag as "last published release," causing every PR's Engine-parity/Test jobs to fall back to the stale published 3.15.0 native binary instead of building fresh).

[carlos-alm]

Correction: this is not a genuine algorithmic regression, and no bisection is needed.

The numbers here match #2081's already-diagnosed growth-driven baseline drift almost exactly (repo self-benchmark corpus grew 629→1013+ files since the v3.15.0 baseline; full-build and 1-file-rebuild both scale with file count). #2107 already added KNOWN_REGRESSIONS entries ('3.15.0:Full build', '3.15.0:1-file rebuild') specifically to exempt exactly this.

The reason those entries didn't prevent this failure: the exemption's baseline fallback in assertNoRegressions only fired when latest.version === 'dev' (the per-PR gate's label). publish.yml's pre-publish-benchmark job labels its measurement with the real version being published (3.16.0), not 'dev' — so the fallback never applied on the actual release gate, and it failed on the exact regression it was supposed to be exempt from.

Fix: #2129 — drops the dev-only restriction so the baseline fallback applies to any comparison against a known-regressed baseline, including the real release-publish path.

No Rust builder bisection needed here. Will close this once the fix lands and a clean release confirms the gate passes.

[carlos-alm]

Direct empirical confirmation: I built the native addon from source for both v3.15.0 (cb42049) and the v3.16.0 release-candidate commit (2e0e94f) and ran scripts/incremental-benchmark.ts against each, same hardware, multiple runs, matched native binaries (not the stale prebuilt one pulled in via node_modules):

Metric	v3.15.0	v3.16.0-candidate
Full build (native)	5748ms, 5987ms	5823ms, 5021ms
Full build ms/file	9.33	7.86
1-file rebuild (native)	383ms, 567ms	559ms, 442ms
No-op rebuild	36ms	36ms, 70ms

Full build is flat-to-improved despite +17.8% more files (629→741 — the real growth between these two exact tags, notably smaller than the +61% figure from #2081, which was measured against a later dev snapshot). 1-file rebuild's v3.16.0 values fall entirely within v3.15.0's own run-to-run spread (383–567ms just from re-running the identical binary) — not statistically distinguishable from noise.

This directly contradicts the CI run's reported native Full build +42% / 1-file rebuild +53-83%. Given multiple metrics were simultaneously and uniformly elevated (never improved) in that one CI run, the likeliest explanation is a slow/contended runner instance for that specific job, not a code regression — consistent with this repo's well-documented history of CI jitter on these exact short-latency metrics.

No Rust bisection needed. Confirms the fix in #2129 is the right and complete resolution — recommend closing this once that PR merges (or now, if you'd rather not wait on the next release attempt).

Side finding, filing separately per repo convention: hit an intermittent native-addon crash (V8 "unreachable code" fatal error, non-deterministic — 1st or 2nd rebuild call within a process) on both engines, both versions, while running this comparison. Unrelated to the perf question but worth tracking.