Repository navigation
Make the EditMode suite pass headlessly - #41
Merged
Merged
Conversation
Three EditMode tests failed on every headless run and were waived by hand at merge time. They are two unrelated problems, neither a production bug. EditorStateCache_IsChanging asserted activity.phase == "playmode_transition" while leaving TestRunStatus.IsRunning at whatever the ambient run set it to. "running_tests" outranks "playmode_transition", and IsRunning is true whenever the suite was started through the MCP run_tests tool — which is exactly what the local harness does. Drive that input by reflection alongside _isPlayModeChanging instead of assuming a quiescent editor. The two ReadConsoleTests failures were never MCP-specific: under -batchmode -nographics Unity does not record a log emitted inside a test into the console's LogEntries buffer, so they fail under the plain native runner too. Logs from ordinary editor code do land, so the buffer itself works. Split those tests in two. SplitMessageAndStackTrace becomes internal and gets five direct tests — that is what PreservesMultilineMessageBody was really about, and it runs identically everywhere. The live-console tests seed an entry and Assert.Ignore when it is not observable, keeping full strength in the Test Runner window without failing the headless gate. They now filter server-side rather than scanning a count=1000 window: GetConsoleEntries walks oldest-first and stops at count, so a busy console can push a fresh message out of an unfiltered window. Clear_Works dropped its "console should have logs" precondition, which was satisfied only by unrelated tests' logs — the class failed when run alone. Also opt the harness into discardUntitledScenes: the smoke leg dirties the untitled scene and run_tests fail-fasts on that, because an interactive editor would wedge on a native Save dialog. A headless gate has none. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three EditMode tests failed on every headless run and were waived by hand at merge time (most recently #38). They are two unrelated problems, and neither is a production bug — both are tests asserting on ambient editor state that headless Unity does not provide.
Correction to #38: its body says Unity's native runner passes all three. It does not. It passes only
EditorStateCache_IsChanging. The twoReadConsoleTestsfailures reproduce under the plain-batchmode -nographics -runTestsrunner with no MCP involvement at all — before this change, the native full suite was 1039 tests, 972 passed, 2 failed, exactly those two.EditorStateCache_IsChanging_TracksTransitionWindowNotWholeSessionThe test drove
_isPlayModeChangingby reflection but left the other input toComputeActivityPhaseat whatever the ambient run had set."running_tests"outranks"playmode_transition", andTestRunStatus.IsRunningis true whenever the suite was started through the MCPrun_teststool — which is exactly what the local harness does. So it passed natively and failed under the harness.Now drives
TestRunStatus._isRunningin the sametry/finallyinstead of assuming a quiescent editor. Safe to poke: a[Test]body runs synchronously inside one editor frame andTestRunStatusis only read from main-thread callbacks.ReadConsoleTestsRoot cause, established against a live headless editor:
read_consolereports 0 entries at rest.McpLog.Warningfrom an ordinary command handler → 1 entry, and it persists. SoLogEntriesworks headlessly.HandleCommand_Get_Worksalone and then reading the console filtered on"Test Log Message"→ 0 matches, while that exact string appears both inEditor.logand in UTF's captured per-test output.Unity does not record a log emitted inside a Test Framework test into the console buffer under
-batchmode -nographics. These tests only ever passed in a GUI editor.Split in two:
SplitMessageAndStackTracebecomesinternaland gets five direct tests (blank line inside the body preserved, CRLF normalization, stack frames split off, no-stack cases). That is whatPreservesMultilineMessageBodywas really about, and it runs identically in the GUI and headless. One visibility change is the whole production diff.Assert.Ignorewhen it is not observable — full strength in the Test Runner window, an explicit documented skip headlessly. They now filter server-side rather than scanning acount = 1000window:GetConsoleEntrieswalks oldest-first and stops atcount, so a busy console can push a freshly logged message out of an unfiltered window entirely.HandleCommand_Clear_Worksdrops its "console should have logs" precondition, which was satisfied only by unrelated tests' logs — the class failed the moment it was run alone. It now asserts what it owns: clear succeeds, console reads empty.Harness
_start_utfnow sendsdiscardUntitledScenes. The smoke leg dirties the untitled scene (it creates and deletes GameObjects) andrun_testsfail-fasts on that, because an interactive editor would wedge on a native Save dialog. A headless gate has none. This did not fire during #38's gate but fired immediately when probed, and would have surfaced as "run_tests start failed".Gate
--legs editmode: exit 0, 977/1026 passed, 0 failures.--legs smoke,editmode: exit 0, smoke 7/7.The gate is now green with no documented exception.
Not verified: the GUI Test Runner path. The two live-console tests should run there rather than skip; if they skip in the GUI too, the observability check needs revisiting.
🤖 Generated with Claude Code