Preserve diagnostics when wordpress.phpunit crashes in Playground

[chubes4]

Problem

The wordpress.phpunit recipe command can exit 1 through PHP.run() while returning HTTP 200 and no stdout, stderr, or response body. WP Codebox then reports only Comlink method call failed, making the workload failure impossible to diagnose or distinguish from a Playground transport/runtime failure.

This reproduced independently across two Automattic/intelligence PRs and multiple reruns:

• https://github.com/Automattic/intelligence/actions/runs/29786621757/job/88499427064
• https://github.com/Automattic/intelligence/actions/runs/29786621201/job/88499425656
• https://github.com/Automattic/intelligence/actions/runs/29790682083/job/88511604863

Representative diagnostic:

Recipe workflow steps[0] failed: wordpress.phpunit crashed before producing a structured response
PHP.run() failed with exit code 1.
httpStatusCode=200
exitCode=1
No Playground stdout, stderr, or response body was captured.
originalErrorClassName=PHPExecutionFailureError
cause=Comlink method call failed

The setup phase succeeds, including Composer autoloader installation, and the failure occurs in run_workloads. The same branches pass their selected PHP smoke tests directly.

Expected

• Capture PHP fatal/error output, PHPUnit bootstrap errors, and the response body before destroying the runtime.
• If the worker/Comlink boundary fails before PHP output is available, preserve worker error name, message, stack, request metadata, and runtime logs.
• Classify workload assertion/bootstrap failure separately from Playground transport/worker failure.
• Emit a structured failed command response whenever the runtime remains responsive enough to return HTTP 200.

Acceptance criteria

• A deterministic fixture makes wordpress.phpunit fail during bootstrap and proves stdout/stderr or a structured PHP error is retained.
• A deterministic fixture crashes the worker/Comlink method and proves worker diagnostics are retained.
• Homeboy/WP Codebox callers receive actionable raw_output instead of an empty-output wrapper.

AI assistance

• AI assistance: Yes
• Tool(s): OpenAI gpt-5.6-sol via OpenCode
• Used for: Correlated repeated CI failures and drafted the diagnostic-preservation report; Chris reviewed and owns the report.

[chubes4]

Post-merge verification against 76b901448c5d74d7d97752853fbf46cbe031e7d2 still fails the actionable raw_output acceptance criterion: https://github.com/Automattic/intelligence/actions/runs/29790682083/job/88519394803

The improved envelope now retains worker metadata and stack, but PHP stdout, stderr, and response body remain empty. The result references runtime.log, commands.log, events, and the Codebox artifact bundle only through ephemeral /tmp/... runner paths; those files are not present in the uploaded Homeboy observation artifact, and homeboy-ci-results is not emitted for the failed command. Reviewers therefore still cannot identify the underlying PHPUnit/bootstrap failure.

Observed after rerunning the exact failed check following #1917 merge:

wordpress.phpunit crashed before producing a structured response
failureClassification=runtime-worker-failure
PHP.run() failed with exit code 1
httpStatusCode=200
exitCode=1
No Playground stdout, stderr, or response body was captured.

The remaining fix needs to retain the runtime/command evidence in caller-consumable output or otherwise surface the underlying PHP failure before runtime teardown.

[chubes4]

Current-main downstream evidence shows #1918 covered only fatal termination, not the real empty bootstrap exit.

• Intelligence Filter runner workspace capture paths #887 job: https://github.com/Automattic/intelligence/actions/runs/29843396562/job/88677699796
• WP Codebox was built from exact SHA 795028fcda8acbe1e9e5d24c43d3b795d228f735, which contains Retain PHPUnit bootstrap failure diagnostics #1918 and Run bounded PHPUnit plans in one disposable runtime #1922.
• Provenance now correctly reports that SHA.
• wordpress.phpunit still exits 1 during the outer wp-load.php boundary with no stdout, stderr, response body, or structured diagnostic.

Root cause: phpFailureDiagnosticFilePhp() writes only when error_get_last() is one of the configured fatal types. A non-fatal die() / exit(1) before the generated PHPUnit body initializes its own stage file leaves the result empty. The #1918 integration used E_USER_ERROR, so it did not cover this actual path.

Follow-up acceptance: record bounded/redacted output or an explicit bootstrap-terminated marker whenever wp-load.php does not return, and prove it with a real Playground non-fatal bootstrap termination integration.

[chubes4]

#1932 merged the non-fatal exit() recorder as d804f12f, but downstream evidence showed the Intelligence payload terminates before any generated handler registers. The outer malformed-payload capture commit was pushed immediately after the PR merged and is therefore not in main. Opening a clean follow-up from current main; Intelligence verification pins already target source commit 58737c2a.

[chubes4]

Post-merge review finding from #1933: the new empty-suite success path is broader than changed-scope no-op handling. Missing configured test roots now exit 0, invalid/unreadable PHPUnit XML falls back to defaults, nonexistent discovered directories are only logged, and a resulting zero-file plan passes. This can mask test-root/XML/discovery configuration failures. Follow-up should permit success only when a valid, non-empty changed-test scope legitimately selects zero discovered tests; configuration and discovery failures must remain errors. The direct npx tsx tests/phpunit-project-autoload.test.ts assertion also needs updating for the new base64 bootstrap wrapper.