Stop rbspy and its command when singed is terminated

[dduugg]

Closes #39.

Problem

singed -- <command> runs sudo rbspy record ... -- <command> with Kernel#system. When something kills singed, such as a script that backgrounds singed -- bin/rails server and later runs kill, only singed exits. sudo, rbspy and the server keep running without it, and no flamegraph is opened.

Passing the signal on unchanged wouldn't help. rbspy only handles SIGINT (ctrlc without its termination feature), so a SIGTERM kills rbspy straight away. It then writes no flamegraph and leaves the command running.

Change

While rbspy runs, singed:

• Passes the first SIGTERM on as SIGINT. rbspy then stops recording, kills the command and writes the flamegraph. After that, singed carries on as usual, fixing the file's ownership, cleaning it up and opening it.

  The SIGINT comes from a kill in a process group of its own. sudo before 1.9.13 (sudo-project/sudo@36742de) doesn't relay a signal sent from its own process group. singed is in that group, so that sudo stays in the terminal's foreground. That covers Ubuntu 22.04's sudo 1.9.9, macOS 13's and RHEL 8's. There, a SIGINT from singed itself was dropped, and singed then waited forever. If kill fails, as it does while sudo briefly runs entirely as root at startup, a later SIGTERM tries again.

• Ignores every SIGTERM after that, until it exits. rbspy treats a second SIGINT as "exit with haste" and writes nothing, and scripts can send SIGTERM more than once. The issue's script is an example: it runs cleanup from both its signal trap and its EXIT trap, a few milliseconds apart. rbspy can finish within those milliseconds, so the SIGTERMs stay ignored while singed opens the flamegraph too.

• Ignores SIGINT itself. Ctrl-C at a terminal already reaches rbspy, either directly or through sudo's pty, so passing it on too would interrupt rbspy twice. sudo filters terminal Ctrl-C for the same reason. As a side effect, when sudo runs without a pty, Ctrl-C no longer kills singed before it can open the flamegraph.

So to stop a backgrounded singed, send it SIGTERM, which is kill's default. A kill -INT sent to singed alone is now ignored. The README's Command Line section explains this, and two limits:

• Only singed's pid should get the signal. kill -- -<pgid>, or timeout without --foreground, sends SIGTERM to the whole process group. sudo 1.9.13 and later then relay it straight to rbspy, which exits without writing a flamegraph.
• Only the command itself is killed. Processes the command started may be left running.

"Leave --rate off rbspy's command line unless it's given" fixes a bug I found while testing. Without -r, singed passed rbspy a bare --rate, because a nil value in its options hash marks a flag. rbspy 0.53 rejects that outright (error: a value is required for '--rate <RATE>' but none was supplied), so singed -- <command> failed immediately unless given a rate.

"Keep sudo type-checked and harden the CLI spec" narrows #: self as untyped, which had switched off Sorbet for the rest of sudo. It also restores a trap handler that Ruby reports as nil, meaning one installed outside Ruby, as DEFAULT rather than ignoring the signal.

Release

"Release 0.4.0" bumps the version, so merging this releases it: once CI Test passes on main, Push Gem (#83) publishes 0.4.0 to RubyGems and creates its GitHub release. It's a minor bump because main has gained features since 0.3.0 (vernier, the Sidekiq middleware, explicit start and stop, vendored speedscope) and now requires Ruby 3.3. Gemfile.lock records singed's version too, so it's bumped with the gemspec; otherwise bundle install would change it and rake release would refuse to release from a dirty tree.

"Name Gusto Engineers as singed's authors" changes the gemspec's authors to Gusto Engineers and its email to dev@gusto.com, so 0.4.0 ships with them.

Testing

• spec/singed/cli_spec.rb runs exe/singed end to end in a subprocess, with stand-ins on PATH for:

  • sudo, a small Perl script that relays SIGINT and SIGTERM like sudo 1.9.12 did, dropping them when they come from its own process group. It's Perl because Ruby can't tell which process sent a signal;
  • rbspy, which mimics the real one: it stops at the first SIGINT, takes a moment to write, and exits without writing at a second SIGINT;
  • the commands that open flamegraphs.

  It checks that SIGTERM stops rbspy exactly once, the command is killed, and the flamegraph is written and opened. It also checks that:

  • a second SIGTERM isn't passed on;
  • a SIGTERM while the flamegraph opens is ignored;
  • SIGINT isn't passed on;
  • --rate is passed on;
  • a failing rbspy fails singed without opening anything.

  Mutating the code shows the examples catch each fix:

  • Against main's CLI, every example fails.
  • Sending the SIGINT from singed's own process group hangs the SIGTERM examples until they time out.
  • Without the once-only guard, the double-SIGTERM example fails.
  • Restoring the SIGTERM trap after interrupting rbspy fails the new opening example.

• The spec starts singed from a clean Bundler environment. Otherwise singed's Bundler.with_unbundled_env restores PATH from the inherited BUNDLER_ORIG_PATH, which drops the stand-ins, and the real sudo would run. GitHub's runners have passwordless sudo. Its waits allow 60 s, since 15 s timed out on a loaded machine.

• rspec passes on Ruby 3.3.11, 3.4.11 and 4.0.5, and 15 random-order runs of the CLI spec all passed. srb tc and rubocop are clean.

• With real sudo and rbspy 0.53.0, in Docker as a non-root user with passwordless sudo:

  Scenario	Ubuntu 22.04, sudo 1.9.9	Debian 13, sudo 1.9.16p2
  One SIGTERM (before these fixes)	hangs, and sudo, rbspy and the command keep running	works
  One SIGTERM	exit 0; file chowned, flamegraph opened, nothing left running	same
  Two SIGTERMs 0.1 s apart	same	not run
  Two SIGTERMs 3 or 20 ms apart, 8 runs each	16 of 16 pass	16 of 16 pass, and 8 of 8 at 40 ms
  SIGINT, then SIGTERM	SIGINT ignored, then exit 0	not run
  The issue's script, via bash and via setsid, with and without a SIGTERM to the script	all pass	all pass
  In a pty, with use_pty and !use_pty: the kill below, and a typed Ctrl-C	exit 0, one "Interrupted.", flamegraph opened	not run

  Before the second-SIGTERM fix, 3 of 8 runs failed at a 3 ms gap on Debian 13. singed was killed before it opened the flamegraph. I haven't run it on macOS. To try it there, this should stop after about 10 s and open a flamegraph:

  sudo -v && (bundle exec exe/singed -- ruby -e 'loop { 2**10 }' & pid=$!; sleep 10; kill $pid; wait $pid)

• I've left three narrow windows as they are. Fresh Eyes flagged the first two:

  • PID reuse: after Process.wait2 reaps sudo, but before the SIGTERM trap is restored, a SIGTERM would signal a PID that may have been reused. Operating systems allocate PIDs sequentially, so reuse within those microseconds is practically impossible, and the once-only guard limits it to a first SIGTERM.
  • SIGTERM during chown: a first SIGTERM that arrives during the few milliseconds of sudo chown interrupts it, so singed exits with an error and leaves the file owned by root. Either way it opens no flamegraph.
  • SIGTERM during startup: a SIGTERM in the first ~100 ms, before rbspy installs its SIGINT handler, kills rbspy without a flamegraph and leaves the command running. And if singed started with SIGINT ignored, as a non-interactive shell's & does, a SIGTERM in sudo's first few milliseconds is lost, so singed waits for the command to exit. main misbehaves at any time.