Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 19 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,11 +198,11 @@ Call resolution the fork adds:

### Measured results

Upstream `main` at `69af19ac` against the fork at `645abbb8`, each run on Node.js 24.22.1 and on Bun 1.4.4. Index and sync figures were re-measured 2026-10-27 on the same 16-vCPU WSL2 host after the ambient-`declare module` scan fix and the resolve-pipeline changes below; MCP, command-line and explore figures are carried from the earlier 2026-10-27 run at `bc27fdb4` (unchanged by resolution-path changes). Seven corpora, arms in mirrored order; index figures are the median of two runs and sync figures the median of four. A comparison counts as a change only when the gap exceeds that comparison's own run-to-run spread — the larger of the two arms' (max − min) / median — and cells that cannot show that are marked inconclusive or no clear change rather than a fixed band. Upstream refuses to start on Bun, which reports itself as Node 26, because upstream blocks Node 25 and newer; it ran here with `CODEGRAPH_ALLOW_UNSAFE_NODE=1`. The host shared other work during the run, so spread-based cells carry more noise than a quiet host would; the numbers, the method, the graph sizes and the disclosures are in [`docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md`](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md). The scripts that produced them are in [`scripts/benchmarks/runtime/`](scripts/benchmarks/runtime/).
Upstream `main` at `69af19ac` against the fork at `de5a7b80`, each run on Node.js 24.22.1 and on Bun 1.4.4. Index and sync figures were re-measured 2026-10-28 on the same 16-vCPU WSL2 host after the ambient-`declare module` scan fix and the resolve-pipeline changes below; MCP, command-line and explore figures are carried from the earlier 2026-10-27 run at `bc27fdb4` (unchanged by resolution-path changes). Seven corpora, arms in mirrored order; index figures are the median of two runs and sync figures the median of four. A comparison counts as a change only when the gap exceeds that comparison's own run-to-run spread — the larger of the two arms' (max − min) / median — and cells that cannot show that are marked inconclusive or no clear change rather than a fixed band. Upstream refuses to start on Bun, which reports itself as Node 26, because upstream blocks Node 25 and newer; it ran here with `CODEGRAPH_ALLOW_UNSAFE_NODE=1`. The host shared other work during the run, so spread-based cells carry more noise than a quiet host would; the numbers, the method, the graph sizes and the disclosures are in [`docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md`](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md). The scripts that produced them are in [`scripts/benchmarks/runtime/`](scripts/benchmarks/runtime/).

![Fork against upstream CodeGraph on n8n and the command line: the fork indexes n8n 1.5 times faster, syncs an edited file 1.3 times faster, answers a warm question 1.8 times faster and uses 33% less memory while indexing. The fork is better in 57 of 76 comparisons, shows no clear change in 10 and is worse in 9.](assets/fork-vs-upstream-performance.svg?v=8)
![Fork against upstream CodeGraph on n8n and the command line: the fork indexes n8n 1.7 times faster, syncs an edited file 3.5 times faster, answers a warm question 1.8 times faster and uses 30% less memory while indexing. The fork is better in 57 of 76 comparisons, shows no clear change in 6 and is worse in 13.](assets/fork-vs-upstream-performance.svg?v=9)

The fork's one-file sync is faster than upstream's on six of the seven corpora on Node — 4% (pretix) to 56% (CPython), with supabase the one no-clear-change — and on Bun it now wins every clear comparison, pretix included (0.68 s against ~1.13 s after the retry fix below). Full indexing: the fork wins on n8n (1.5×), CPython (2.2×), discourse (1.4×), pretix-Node (8.14 s against 9.04 s) and the Bun arm of gin, Alamofire, discourse, supabase and n8n; gin-Node, Alamofire-Node and supabase-Node tie inside spread; pretix-Bun is within ~2% of upstream (the last residual). The speed costs some memory: indexing peaks higher on pretix (roughly +22% Node / +26% Bun after the worker-cache halving below) and supabase-Bun — the 12 in-process worker heaps are the disclosed trade for the index-time wins — while of the other five corpora on Node, four index at 4–32% less and Alamofire runs 1% higher (inside its spread). Cells where the fork still trails: pretix-Bun index (~+2%), the pretix and supabase-Bun index-memory cells, and a ~0% gin file-watcher cell inside noise. Three earlier losses — pretix-Bun sync, n8n busy-MCP memory on Node and the gin concurrent-explore wave — are wins in this tree; the post-merge measurements are in the benchmark doc's [follow-up section](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md#follow-up-merges). The benchmark doc has [every result](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md#every-result-at-a-glance) in one heatmap.
The fork's one-file sync is faster than upstream's on six of the seven corpora on Node — 45% (pretix) to 72% (n8n), with discourse the one loss (+47%) — and on Bun it wins six of seven the same way, discourse again the loss (+31%). Full indexing: the fork wins on n8n (1.7×), CPython (2.1×), discourse (1.6×), pretix-Node (7.48 s against 9.69 s), supabase-Node (15.6 s against 19.6 s) and Alamofire-Node (1.19 s against 1.42 s), and on the Bun arm of Alamofire, pretix (7.15 s against 7.20 s — a tie inside spread), CPython, discourse, supabase and n8n; gin index is the remaining loss (+6% Node / +18% Bun — ~0.05–0.14 s of fixed fork overhead on a 119-file index). The speed costs some memory: indexing peaks higher on pretix (+36% Node / +34% Bun) and supabase (+14% Node / +13% Bun) — the 12 in-process worker heaps are the disclosed trade for the index-time wins — while the fork now peaks lower than upstream on n8n (−30%), CPython (−9%), gin (−33%) and discourse-Bun (−28%). Cells where the fork still trails: gin index on both runtimes, discourse sync on both runtimes, and the pretix and supabase index-memory cells. The post-merge measurements are in the benchmark doc's [follow-up section](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md#follow-up-merges). The benchmark doc has [every result](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md#every-result-at-a-glance) in one heatmap.

The fork starts its resolver pool at or above 150,000 pending references. Below that, it can start the pool part-way through if at least 20,000 unresolved references remain and the estimated main-thread work is at least 0.4 s. An explicit `CODEGRAPH_PARALLEL_RESOLVE_MIN` replaces both gates. Resolution proceeds in pages of 40,000 references (`CODEGRAPH_RESOLVE_BATCH` overrides), across up to 12 workers (`CODEGRAPH_RESOLVE_WORKERS` sets the count, `0` disables the pool; `CODEGRAPH_NO_PARALLEL_RESOLVE=1` does the same). Against a pre-change baseline build, five tested corpora indexed 11-48% faster at 56-132% more peak memory with identical graphs ([measurements](docs/benchmarks/adaptive-pool-engage-2026-10-15.md)).

Expand All @@ -212,29 +212,29 @@ A [2026-10-15 mid-size comparison](docs/benchmarks/midsize-sweep-2026-10-15.md)

| Corpus | Upstream, Node | Fork, Node | Fork, Bun |
|---|---|---|---|
| gin (Go, 119 files) | 0.96 s, 558 MiB | 1.02 s, 399 MiB | 0.88 s, 308 MiB |
| Alamofire (Swift, 129 files) | 1.35 s, 879 MiB | 1.41 s, 891 MiB | 0.94 s, 881 MiB |
| pretix (Python + JS, 1,472 files) | 9.04 s, 3.29 GiB | 8.14 s, 4.17 GiB | 7.49 s, 3.79 GiB |
| CPython (C + Python, 3,708 files) | 55.0 s, 7.15 GiB | 25.2 s, 5.90 GiB | 25.1 s, 5.39 GiB |
| discourse (Ruby + JS, 20,277 files) | 19.6 s, 4.07 GiB | 13.7 s, 3.92 GiB | 12.1 s, 2.97 GiB |
| supabase (React + Next.js + TS, 10,716 files) | 18.2 s, 4.86 GiB | 15.5 s, 5.45 GiB | 14.6 s, 4.73 GiB |
| n8n (Vue + TS, 24,434 files) | 74.9 s, 12.39 GiB | 49.5 s, 8.26 GiB | 45.2 s, 8.28 GiB |
| gin (Go, 119 files) | 0.96 s, 567 MiB | 1.02 s, 381 MiB | 0.91 s, 310 MiB |
| Alamofire (Swift, 129 files) | 1.42 s, 817 MiB | 1.19 s, 1.05 GiB | 1.01 s, 887 MiB |
| pretix (Python + JS, 1,472 files) | 9.69 s, 2.92 GiB | 7.48 s, 3.97 GiB | 7.15 s, 3.85 GiB |
| CPython (C + Python, 3,708 files) | 53.7 s, 6.21 GiB | 25.5 s, 5.64 GiB | 25.6 s, 5.41 GiB |
| discourse (Ruby + JS, 20,277 files) | 21.7 s, 3.59 GiB | 13.2 s, 3.58 GiB | 12.4 s, 2.98 GiB |
| supabase (React + Next.js + TS, 10,716 files) | 19.6 s, 4.23 GiB | 15.6 s, 4.83 GiB | 14.5 s, 4.63 GiB |
| n8n (Vue + TS, 24,434 files) | 83.0 s, 11.00 GiB | 49.4 s, 7.65 GiB | 46.6 s, 8.43 GiB |

The fork's index was slower than upstream's on five of the seven corpora in the first pass of this run — the supabase and n8n regressions traced to upstream's ambient-`declare module` lookup (merged in fork #548): the Rust port scanned every declaration file per out-of-repo import. A per-file memo ([#554](https://github.com/bompus/codegraph/pull/554)) ended the reparse-every-reference quadratic, and the follow-up above lands the rest: a port of upstream's text tokenizer plus a `specifier → files` index, so each import checks only candidate declaration files instead of every `.d.ts`. The re-measured numbers in this table are the result: supabase went from 41.5 s to 15.5 s on Node and n8n from 178 s to 49.5 s — n8n now beats upstream 1.5× and CPython 2.2×. The remaining index losses from that run — pretix +33% and Alamofire +18% on Node, and the Bun arm of supabase and n8n — traced not to the ambient scan but to the resolve loop's batch boundaries: every 5,000-ref page drained the worker pipeline before the next fanned out. This run stacks three fixes for that — 40,000-ref pages with a 12-worker pool cap (same batch-boundary semantics; pool workers resolve against a fixed snapshot), then bounded work-stealing dispatch with earlier batch fan-out and non-blocking worker recycling, then a memoized near-duplicate pass that stopped re-tokenizing bodies it had just signed — and those losses are gone: pretix-Node now wins (8.14 s against 9.04 s), the Bun arms of supabase and n8n win, and pretix-Bun is within ~2% of upstream (the last residual). Later in the same stack, the deferred-decode path stopped turning off on framework projects: a framework `extract()` hook no longer forces every file onto decoded-object extraction — its output rides the kernel-buffer transport and merges post-decode ([#563](https://github.com/bompus/codegraph/pull/563); pretix went from 0 to 1,416 kernel-path files, n8n-Node index RSS −10%). The benchmark doc has the [full accounting](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md#the-supabase-resolution-regression). The trade is memory: indexing peaks higher on pretix and supabase now, lower on the other five, and its one-file sync is faster on six of seven (below).
The fork's index was slower than upstream's on five of the seven corpora in the first pass of this run — the supabase and n8n regressions traced to upstream's ambient-`declare module` lookup (merged in fork #548): the Rust port scanned every declaration file per out-of-repo import. A per-file memo ([#554](https://github.com/bompus/codegraph/pull/554)) ended the reparse-every-reference quadratic, and the follow-up above lands the rest: a port of upstream's text tokenizer plus a `specifier → files` index, so each import checks only candidate declaration files instead of every `.d.ts`. The re-measured numbers in this table are the result: supabase went from 41.5 s to 15.6 s on Node and n8n from 178 s to 49.4 s — n8n now beats upstream 1.7× and CPython 2.1×. The remaining index losses from that run — pretix +33% and Alamofire +18% on Node, and the Bun arm of supabase and n8n — traced not to the ambient scan but to the resolve loop's batch boundaries: every 5,000-ref page drained the worker pipeline before the next fanned out. This run stacks three fixes for that — 40,000-ref pages with a 12-worker pool cap (same batch-boundary semantics; pool workers resolve against a fixed snapshot), then bounded work-stealing dispatch with earlier batch fan-out and non-blocking worker recycling, then a memoized near-duplicate pass that stopped re-tokenizing bodies it had just signed — and those losses are gone: pretix-Node wins (7.48 s against 9.69 s), the Bun arms of supabase and n8n win, and pretix-Bun is a tie (7.15 s against 7.20 s). Later in the same stack, the deferred-decode path stopped turning off on framework projects: a framework `extract()` hook no longer forces every file onto decoded-object extraction — its output rides the kernel-buffer transport and merges post-decode ([#563](https://github.com/bompus/codegraph/pull/563); pretix went from 0 to 1,416 kernel-path files, n8n-Node index RSS −10%). The benchmark doc has the [full accounting](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-27.md#the-supabase-resolution-regression). The trade is memory: indexing peaks higher on pretix and supabase now, lower on the other five, and its one-file sync is faster on six of seven (below).

Upstream at `6560052` took 2.0 to 2.7 times as long to index Python as at `290e03f` (pretix 9.2 s to 18.8 s, CPython 35.6 s to 95.3 s; [a same-host check](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-03.md#since-2026-09-28) put the cause in upstream's code). At `69af19ac` upstream's CPython index is slower than at `986de95c` (55.0 s against 40.2 s on this host) while the fork's keeps improving, so the fork's CPython lead is 2.2× this run.
Upstream at `6560052` took 2.0 to 2.7 times as long to index Python as at `290e03f` (pretix 9.2 s to 18.8 s, CPython 35.6 s to 95.3 s; [a same-host check](docs/benchmarks/fork-vs-upstream-node-bun-2026-10-03.md#since-2026-09-28) put the cause in upstream's code). At `69af19ac` upstream's CPython index is slower than at `986de95c` (53.7 s against 40.2 s on this host) while the fork's keeps improving, so the fork's CPython lead is 2.1× this run.

**One-file sync** (edit one file, `codegraph sync`):

| Corpus | Upstream, Node | Fork, Node | Fork, Bun |
|---|---|---|---|
| gin | 0.47 s | 0.40 s | 0.29 s |
| Alamofire | 0.48 s | 0.29 s | 0.18 s |
| pretix | 1.62 s | 1.55 s | 0.68 s |
| CPython | 5.35 s | 2.38 s | 1.66 s |
| discourse | 4.78 s | 2.37 s | 1.77 s |
| supabase | 4.58 s | 2.76 s | 1.95 s |
| n8n | 11.6 s | 9.07 s | 8.10 s |
| gin | 0.34 s | 0.27 s | 0.19 s |
| Alamofire | 0.42 s | 0.27 s | 0.17 s |
| pretix | 1.56 s | 0.86 s | 0.60 s |
| CPython | 6.01 s | 2.65 s | 1.72 s |
| discourse | 1.19 s | 1.75 s | 1.20 s |
| supabase | 8.17 s | 3.05 s | 1.94 s |
| n8n | 11.6 s | 3.29 s | 2.45 s |

Upstream's supabase sync works in this run — the `FOREIGN KEY constraint failed` seen in every prior run ([colbymchenry/codegraph#2565](https://github.com/colbymchenry/codegraph/issues/2565)) was fixed upstream on 2026-10-19. Its two passes differed widely on Node (1.05 s and 8.1 s); the 4.58 s figure is the median of four, and the spread means that comparison counts as no clear change rather than a fork win.

Expand Down
Loading