Error the test items when the controller cannot spawn a test process

[disberd]

I found this problem while I was trying out a customized version of the JuliaMCP.jl server. I used AI help to debug the problem to its root cause and to prepare the fix.

Problem

When the controller cannot spawn the program in juliaCmd, the test run does not complete. Its test items stay pending, and its test processes keep the status Launching. The client gets no error. The only record is this log entry:

┌ Error: Error in test process IO
│   testprocess_id = "…"
│   exception =
│    IOError: could not spawn setenv(`…/nonexistent/julia --startup-file=no …`,[…]): no such file or directory (ENOENT)
└ @ TestItemControllers src/testitemcontroller.jl:2620

To reproduce, start a test run with a juliaCmd that does not exist, for example /nonexistent/julia. A value such as "julia +1.12" gives the same result, because the controller uses the full string as one program name. A cancel ends the test run, but the test processes stay in the controller. At shutdown, the controller then waits the full grace period of 30 s for them.

Cause

start spawns the process in src/testprocess.jl:417-423, before the try at :430. The code after :430 sends TestProcessIOErrorMsg or TestProcessTerminatedMsg to the reactor. A spawn error skips this code. The error goes to the catch of the @async task in _launch_julia_process! (src/testitemcontroller.jl:2615-2621), and this catch only writes the log entry. So the reactor gets no message. The process stays in ProcessStarting, and execute_testrun waits for its test items with no limit.

Fix

The catch in _launch_julia_process! now also sends TestProcessIOErrorMsg(ps.id, :fatal). It uses the same try put!(…) catch end line as 7 other error paths in this file. The existing path for a fatal IO error does the rest:

1. handle!(::TestProcessIOErrorMsg) sends TestProcessTerminatedMsg.
2. The startup-crash branch of _handle_termination_during_run! errors the queued test items, and the test run completes.
3. handle!(::TestProcessTerminatedMsg) removes the process and calls on_process_terminated.

The handler ignores a message for an unknown or dead process, so a second message for the same process has no effect. The change also covers other errors that escape start before its error handler, for example from Sockets.listen.

Tests

The new test item A Julia command that cannot be spawned errors all test items in test/test_process_crash.jl runs 2 test items on 2 processes. Its juliaCmd is a path that does not exist.

• Without the fix, the test item errors with run_testrun timed out after 60s. Both processes log the IOError above. After the timeout, the controller also does not stop within the 10 s shutdown_timeout.
• With the fix, the test item passes in 4 s. The controller errors both test items, removes both processes, and stops at once.

The other test items in test_process_crash.jl also pass with the fix.

Possible follow-up

The message of an errored test item is "Test process crashed before running test item '…'". This message does not give the cause. A follow-up can keep a short cause on the process state and add it to this message. An example cause is "julia +1.12: no such file or directory (ENOENT)". The full IOError text is not suitable for this, because it contains the environment of the controller. This PR does not change the message.