Skip to content

Stream git log instead of buffering to fix ENOBUFS on large ranges - #122

Merged
RomainCscn merged 1 commit into
mainfrom
romain/lin-79789-stream-git-log
Jul 27, 2026
Merged

Stream git log instead of buffering to fix ENOBUFS on large ranges#122
RomainCscn merged 1 commit into
mainfrom
romain/lin-79789-stream-git-log

Conversation

@RomainCscn

@RomainCscn RomainCscn commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

To scan a release range, runLog ran git log through execSync, which collects the entire output in one in-memory buffer capped by maxBuffer — 1 MiB by Node default, 256 MiB since #103. When a range's output crosses the cap, Node kills git and the sync dies with spawnSync /bin/sh ENOBUFS. Crossing it depends on commit count × message size, so the same pipeline passes one day and crashes the next.

runLog now spawns git and parses the output as a stream: each commit record is parsed as soon as its %x1e separator arrives, and the raw bytes are discarded. No buffer ever holds the whole output, so there is no cap left to hit — memory scales with the parsed commits, not with whatever git prints.

Also folded in:

  • git is invoked with an argument array instead of a /bin/sh command string, so --include-paths patterns (CLI- or server-provided) are no longer interpolated into a shell command where metacharacters could break or inject into it.
  • getCurrentGitInfo read the HEAD message with the default 1 MiB cap inside one try/catch covering all three fields, so one oversized HEAD message failed the whole sync as "Could not get current commit". Each field now fails independently and the message read is bounded at 10 MiB.

Regression test: a git fast-import fixture drives a 270×1 MiB range past the old 256 MiB cap — it crashed before this change.

Fixes LIN-79789

@linear-code

linear-code Bot commented Jul 27, 2026

Copy link
Copy Markdown

LIN-79789

@RomainCscn
RomainCscn force-pushed the romain/lin-79789-stream-git-log branch from 9b59ab1 to 8906dbf Compare July 27, 2026 09:35
@RomainCscn
RomainCscn requested a review from mwolting July 27, 2026 10:00
@RomainCscn
RomainCscn merged commit 9537c01 into main Jul 27, 2026
10 checks passed
@RomainCscn
RomainCscn deleted the romain/lin-79789-stream-git-log branch July 27, 2026 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants