Stop two AI tests depending on timing they do not control - #5078
Merged
elias-ba merged 0 commit intoAug 17, 2026
Conversation
Security Review ✅
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## timeout-keep-status-timeline #5078 +/- ##
==============================================================
+ Coverage 90.5% 90.6% +0.1%
==============================================================
Files 422 421 -1
Lines 20090 20012 -78
==============================================================
- Hits 18188 18138 -50
+ Misses 1902 1874 -28 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
elias-ba
force-pushed
the
timeout-keep-status-timeline
branch
from
August 17, 2026 05:34
be7f4bd to
25cad92
Compare
elias-ba
force-pushed
the
timeout-stable-message-order
branch
2 times, most recently
from
August 17, 2026 09:24
1894d0c to
dbe62b1
Compare
elias-ba
force-pushed
the
timeout-keep-status-timeline
branch
from
August 17, 2026 09:25
25cad92 to
88b5c6c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Two tests that passed alone and failed in a full run. That is the worst kind of red, because it lands on whoever pushes next rather than on whoever caused it. Both surfaced while working on the stack below this one, and neither is caused by it.
The AI chat tests picked a reply out of a session with
List.last/1. Messages come back ordered byinserted_atalone and the schema keeps that to the second, so a question and the answer to it written inside the same second tie and come back either way round. Against a real model the answer takes seconds and the order holds; in a test both writes land in the same millisecond, so it was a coin toss that usually landed the right way up. They now pick by role, which is what each of them meant.The ordering itself is deliberately left alone. It is only reachable when two messages share a second, which needs a reply to arrive instantly, and fixing it properly means a migration to microsecond timestamps. Worth doing, not worth doing here.
The credentials schema test asserted that nothing at all was logged.
with_log/1captures everything logged in the window, including from the async tests running beside it, so a warning anywhere else in the suite failed it. It now asserts the absence of the one message that would mean the types were not recognised, which is what the test is actually about.Closes #__
Validation steps
mix test test/lightning/ai_assistant/ai_assistant_test.exs test/lightning/credentials/schema_test.exsassistant_message/1back toList.last/1and run the full suite a few times; the global chat meta assertion atai_assistant_test.exs:3338fails intermittently.AI Usage
Pre-submission checklist