Skip to content

vmgr panics on start after a forced stop leaves an unreapable helper process #2662

Description

@recursiveforte

Describe the bug

The following is Claude output I reviewed


Summary

A guest OOM put the Linux VM's kernel into an RCU stall. A subsequent stop timed out and was forced. The vmgr helper never finished exiting — it is now stuck in kernel exit (STAT ?Es, RSS 0, reparented to launchd), so it can neither be killed nor reaped. Every start since then panics waiting for that PID to exit. The engine is down for all machines and for Docker, and quitting and relaunching the app does not clear it, because each new instance finds the same process and waits on it again.

Environment

  • OrbStack 2.2.3 (build 2020300, commit c83556b0ef8f1ba9a33abbb194622b6b7a1c0307)
  • macOS 26.5.1 (25F80), Apple M5 Pro, 64 GB
  • Machines: ubuntu (resolute, arm64), alpine, arch
  • ~/.orbstack/vmconfig.json: memory_mib: 16384

Timeline

  • Aug 17 13:25:08 — OrbStack starts; helper PID 2526 begins. Runs normally for two days.
  • Aug 19 ~16:1x — a build inside ubuntu with a ~13 GB peak RSS exhausts the 16 GB VM. Guest kernel log:

Out of memory: Killed process 1635965 (find) total-vm:14068kB, ... oom_score_adj:0
rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
rcu: rcu_sched kthread starved for 10017 jiffies! g985897 f0x0 RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=0
rcu: Unless rcu_sched kthread gets sufficient CPU time, OOM is now expected behavior.

  • Aug 19 16:18:16 — a stop is issued while the guest is in that state:

🌲 vmgr | time="08-19 16:18:16" level=error msg="graceful stop timed out, forcing"

  • Aug 19 16:18 onward — PID 2526 never completes exit. Every start panics.

The panic (verbatim, from ~/.orbstack/log/gui.log, exit status 6):

panic: wait for stopping vmgr to exit: context deadline exceeded

goroutine 17 [running, locked to thread]:
github.com/orbstack/macvirt/vmgr.check(...)
github.com/orbstack/macvirt/vmgr/main.go:138
github.com/orbstack/macvirt/vmgr.checkSpawnDaemon({0x1092bdc20, 0xa40d4d7e1a0})
github.com/orbstack/macvirt/vmgr/daemon.go:67 +0x74
github.com/orbstack/macvirt/vmgr.runSpawnDaemon()
github.com/orbstack/macvirt/vmgr/daemon.go:171 +0x80
github.com/orbstack/macvirt/vmgr.Main()
github.com/orbstack/macvirt/vmgr/main.go:1452 +0x100

Current state of the stuck process

$ ps -o pid,ppid,stat,lstart,comm -p 2526
PID PPID STAT STARTED COMM
2526 1 ?Es Mon Aug 17 13:25:08 2026 (OrbStack Helper)

E = trying to exit, RSS 0, parenthesised name, reparented to PID 1. It has been in this state since 16:18. (The two-day figure in etime is the process's total lifetime, not the fault duration.)

Why it doesn't self-clear

  • kill -9 is a no-op on a process already in exit.
  • The parent is launchd and there's no OrbStack launchd job to kickstart -k.
  • ~/.orbstack/run/ is empty and run.stale.bak was already rotated aside, so this isn't stale-socket state — checkSpawnDaemon is finding the live PID.
  • vmstate.json contains no PID, so there's no file to clear.

What I tried

Repeated quit-and-relaunch of the app, and orb start / orb list, all of which reproduce the panic above. I have not yet isolated a clean manual restart from those attempts, so I can't say with certainty that a reboot is the only recovery — only that programmatic restarts do not clear it.

Expected

A forced stop that strands a process should not permanently wedge the engine.
Summary

A guest OOM put the Linux VM's kernel into an RCU stall. A subsequent stop timed out and was forced. The vmgr helper never finished exiting — it is now stuck in kernel exit (STAT ?Es, RSS 0, reparented to launchd), so it can neither be killed nor reaped. Every start since then panics waiting for that PID to exit. The engine is down for all machines and for Docker, and quitting and relaunching the app does not clear it, because each new instance finds the same process and waits on it again.

Environment

  • OrbStack 2.2.3 (build 2020300, commit c83556b0ef8f1ba9a33abbb194622b6b7a1c0307)
  • macOS 26.5.1 (25F80), Apple M5 Pro, 64 GB
  • Machines: ubuntu (resolute, arm64), alpine, arch
  • ~/.orbstack/vmconfig.json: memory_mib: 16384

Timeline

  • Aug 17 13:25:08 — OrbStack starts; helper PID 2526 begins. Runs normally for two days.
  • Aug 19 ~16:1x — a build inside ubuntu with a ~13 GB peak RSS exhausts the 16 GB VM. Guest kernel log:

Out of memory: Killed process 1635965 (find) total-vm:14068kB, ... oom_score_adj:0
rcu: INFO: rcu_sched detected stalls on CPUs/ta

To Reproduce

No response

Expected behavior

No response

Diagnostic report (REQUIRED)

OrbStack info:
Version: 2.2.3
Commit: c83556b0ef8f1ba9a33abbb194622b6b7a1c0307 (v2.2.3)

System info:
macOS: 26.5.1 (25F80)
CPU: arm64, 18 cores
CPU model: Apple M5 Pro
Model: Mac17,9
Memory: 64 GiB

Full report: https://orbstack.dev/_admin/diag/orbstack-diagreport_2026-08-19T23-52-39.685567Z.zip

Screenshots and additional context (optional)

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    t/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions