Skip to content

fix(box): listing projects runs git only when a repository changed - #195

Merged
sean-brydon merged 5 commits into
cosscom:mainfrom
Stener1:git-metadata-cache
Oct 29, 2026
Merged

sean-brydon merged 5 commits into
cosscom:mainfrom
Stener1:git-metadata-cache

Conversation

@Stener1

@Stener1 Stener1 commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Summary

berthd serve was the busiest process on an idle box: about 0.5 cores, and about 48 git execs a second (a box with 24 projects and about 100 worktrees). Nearly all of them were git worktree list, git remote get-url origin and git symbolic-ref, one set per project, every time anything described the projects.

Where they came from: describe() runs those 3–4 commands for a location, and every Locations.List and Locations.Get calls it, for all locations (List) or one (Get). About 70 call sites use those two. The steady callers:

  • the worktree watcher, which runs a full List every 10 s (watch.go);
  • GET /v1/locations, from the app (on every worktree.* event, plus a 60 s backstop), plugins and the CLI;
  • per-request Gets: presets, skills, services, sessions and the like;
  • placeAgents on /v1/stats (one worktree list per location, on every agent event).

The fix keeps each answer with a stamp of the .git files git reads to produce it, and runs git again only when the stamp changes (or after a minute):

 describe(location)
-  git worktree list --porcelain          # every call
-  git remote get-url origin              # every call
-  git symbolic-ref … origin/HEAD | HEAD  # every call
+  worktreeList(path)   cached until: HEAD, packed-refs, reftable, config,
+                                     worktrees/ and, for each worktree,
+                                     HEAD, its branch's loose ref, gitdir,
+                                     locked, its own reftable
+  remoteURL(path)      cached until: .git/config, config.worktree,
+                                     ~/.gitconfig, ~/.config/git/config
+  defaultBranch(path)  cached until: refs/remotes/origin/HEAD, HEAD,
+                                     packed-refs, reftable
  • A stamp is a stat (inode, size, mtime) of each file. Git replaces refs, HEAD and config by renaming a new file over the old one, so every change gives a new inode. It's taken before git runs, so a change that lands in between is caught on the next call.
  • A minute's max age covers what a stamp can't see: a global url.*.insteadOf, or an included config file.
  • Only answers git actually gave are kept. "No origin" counts as an answer; a git killed by its timeout, or one that didn't start, does not.
  • A folder with no .git of its own is asked every time, as before. parseWorktrees still runs on every call, so a worktree whose folder is gone drops at once.
  • No caller changes. internal/box/gitcache.go is new; describe, worktreePaths, remoteURL and defaultBranch go through it.

Evidence

Real box, same clients open, 40 s bpftrace of berthd's execve calls (tracepoint:syscalls:sys_enter_execve /curtask->real_parent->tgid == <berthd>/), with berthd's CPU over the same window read from /proc/<pid>/stat:

build git execs / 40 s berthd CPU
before (3 runs) 1,429 · 1,444 · 1,403 0.35 · 0.34 · 0.32 cores
after (3 runs) 4 · 16 · 11 0.13 · 0.09 · 0.07 cores

That's about 99% fewer git execs and about 70% less berthd CPU. An earlier build on the same box measured 1,800–1,913 per 40 s at 0.42–0.55 cores. "Before" is the build the box ran; "after" is the same tree plus this change.

Correctness on that box: every location in /v1/locations from the patched daemon matched git run directly (24 locations, 102 worktrees: worktrees, branches, HEADs, remotes, default branches; 0 mismatches). Each of the following showed up on the very next /v1/locations read: berthd worktree new, a commit in the new worktree, a branch switch, git worktree lock, berthd worktree rm, and git worktree add/remove done outside Shipyard. The MCP's berth_sessions still places every session in its worktree.

Tests (internal/box/gitcache_test.go). A git wrapper first on PATH counts every real exec:

TestListingLocationsAgainRunsNoGitUntilSomethingChanges
  List, then List/Get/worktreePaths ×3       → no git          (before: 4 execs per call)
  commit                                     → only worktree list, new Head
  git worktree add / commit in it / checkout / lock / remove   → each shows
  remote set-url, origin/HEAD → trunk        → slug and default branch follow
TestGitAnswersAreAskedAgainOnceTheyAreOld    → max age 0 re-runs git
TestAReftableRepositorysLinkedWorktreeIsAskedAgainWhenItSwitches
                                             → fails without the per-worktree reftable stamp, passes with it (git 2.54; skips on git without reftable)

With the cache turned off, the first test fails with the four git calls listed on every repeat.

App on a real dev box (verify-berthd, new scenario worktree-git-live, a dev box of this checkout with --agent). A worktree added with git shows in the sidebar, and once the box drops it after git worktree remove, it's gone after a reload:

added with git removed with git
added removed

An app that's already open keeps the removed worktree's row until it reloads, even though the box sends worktree.removed. That happens on main too (the same scenario fails the same way against a dev box built from main), so it's left for its own fix. The feature file says so.

Checks: go vet ./... passes. go test -race ./... passes except TestDetach (leadline/api, fails on main on this machine) and TestPaneDeadOfASignalIsItsShellStatus, a sleep 0.2; kill -INT timing test that also flakes on main under load (0/10 on main, 1/10 here at load ~50). The app build and the docs build pass. cargo check was not run: nothing in app/src-tauri changed.

Running in production

Since 2026-10-28 this change, together with #194, has run on one of my own boxes through my personal update channel (upstream's latest release plus my open PRs). The box is busy: 35–37 coding-agent sessions, 23 projects and about 91 worktrees, with load between 13 and 30 on 8 cores. All figures are from the same 40 s bpftrace trace and /proc CPU read as above.

build git execs / 40 s berthd CPU
release build, no fix (3 runs) 1,029 · 1,876 · 1,084 0.35 · 0.44 · 0.30 cores
same, during a burst of new worktrees 12,749 in 30 s (425/s) 1.5 cores
with this PR and #194 (4 runs) 22 · 23 · 13 · 29 0.19 · 0.16 · 0.18 · 0.18 cores
current channel build, settled, 37 sessions open 19 0.06 cores

Before the fix, berthd was the box's largest single CPU user, and its git exec rate grew with the number of worktrees and with how often clients listed locations. On the patched build:

  • /v1/locations matched git worktree list run directly for every project.
  • Creating, committing in, switching and removing a worktree each showed on the next read, within about 0.2 s.
  • Every scheduled flow run on the patched build succeeded.

Merge Danger

Door: two-way

Box-side only, with no stored state: a revert puts the git calls back as they were.

Blast Radius: worktree-list

The risk is a stale answer: a branch, HEAD, lock or remote that has changed but still shows the old value until the minute runs out. The stamp covers what git itself reads for each answer, including linked worktrees, packed refs and reftable. What it can't see (global or included git config) is bounded by the minute. Worktree creation and removal read the list through the same cache, but their own changes always change the stamp.

🤖 Generated with Claude Code

View guided diff

@vercel

vercel Bot commented Oct 27, 2026

Copy link
Copy Markdown

@Stener1 is attempting to deploy a commit to the seanbrydonio's projects Team on Vercel.

A member of the Team first needs to authorize it.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 9 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="docs/changelog.mdx">

<violation number="1" location="docs/changelog.mdx:25">
P3: `cachedGit` bypasses the cache for locations without their own `.git`, so those locations still run Git on every listing. Qualify this claim for repository-backed projects.</violation>
</file>

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread .claude/skills/verify-berthd/scripts/scenarios/worktree-git-live.mjs Outdated
Comment thread .claude/skills/verify-berthd/scripts/scenarios/worktree-git-live.mjs Outdated
Comment thread internal/box/gitcache.go Outdated
Comment thread .claude/skills/verify-berthd/features/worktree-git.md Outdated
Comment thread internal/box/stats.go Outdated
Comment thread internal/box/gitcache_test.go Outdated
Comment thread .claude/skills/verify-berthd/features/worktree-git.md Outdated
Comment thread docs/changelog.mdx Outdated
Comment on lines +25 to +26
branch, and asks git again only when the repository changes (or after a
minute).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: cachedGit bypasses the cache for locations without their own .git, so those locations still run Git on every listing. Qualify this claim for repository-backed projects.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At docs/changelog.mdx, line 25:

<comment>`cachedGit` bypasses the cache for locations without their own `.git`, so those locations still run Git on every listing. Qualify this claim for repository-backed projects.</comment>

<file context>
@@ -18,6 +18,12 @@ again.
+  git commands per project every time the app, a plugin, the MCP or its
+  own worktree watcher listed them, dozens a second on a box with a few
+  dozen projects. It now keeps each project's worktrees, remote and default
+  branch, and asks git again only when the repository changes (or after a
+  minute).
 - **Switching projects no longer opens a new task.** Clicking a project on
</file context>
Suggested change
branch, and asks git again only when the repository changes (or after a
minute).
branch, and usually reuses results for repositories with their own `.git` until a
relevant Git file changes (or after a minute); other locations are checked each time.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not changing this one. The changelog is plain words for users (AGENTS.md), and the exception is a project folder with no .git of its own, such as a folder that isn't a repository or one inside another repository. Those run git exactly as before, so nothing gets worse for them. The precise rule is in cachedGit's comment and the PR description. The bullet has moved to a new ## Unreleased section, because v0.3.31 shipped while this PR was open.

Stener1 and others added 5 commits October 28, 2026 21:46
Every Locations.List and Get ran git worktree list, remote get-url and
symbolic-ref for each location, and the app, plugins, the MCP and the
worktree watcher ask many times a second. Each answer is now kept with a
stamp of the .git files git read for it, and asked again only when one
changes, or after a minute.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… checks the sidebar follows

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… it switches branch

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eloads

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…st, and the live probe cleans up only what it made

From review:
- worktreeStamp also stamps the checkout's config.worktree, which can set
  core.bare for it.
- worktree-git-live removes the probe worktree and its branches only when
  this run created them, and with --force twice, so a failed step that
  left it locked still gets cleaned up.
- countGit quotes its paths with shellQuote.
- Comments and the feature file say a minute also expires an answer, that
  an open app needs a reload to drop a removed row, and which XDG config
  is watched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018aGVrvzCLsVZaL92TTA95t
@Stener1
Stener1 force-pushed the git-metadata-cache branch from b6d1844 to c7a6a4b Compare October 28, 2026 21:51
@sean-brydon
sean-brydon merged commit 1aebbc1 into cosscom:main Oct 29, 2026
1 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants