test(spark): reduce flakiness in TestHoodieClientMultiWriter insert-to-distinct-partitions test - #19758
Conversation
…o-distinct-partitions test testMultiWriterWithInsertsToDistinctPartitions submits its concurrent writer tasks in a tight loop. Under load the writers can race on acquiring the in-process lock in a way that intermittently fails the test. Add a small 100ms wait between task submissions to space them out and stabilize the test. Test-only change; no production behavior is affected. Co-authored-by: Y Ethan Guo <ethan@onehouse.ai>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #19758 +/- ##
============================================
- Coverage 77.37% 75.28% -2.10%
+ Complexity 33177 32313 -864
============================================
Files 2539 2539
Lines 140939 140939
Branches 17012 17012
============================================
- Hits 109056 106103 -2953
- Misses 24145 27016 +2871
- Partials 7738 7820 +82
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
hudi-agent
left a comment
hudi-agent
left a comment
There was a problem hiding this comment.
Thanks for working on this! This PR adds a 100ms stagger between concurrent writer-task submissions in testMultiWriterWithInsertsToDistinctPartitions to reduce CI flakiness, with no production behavior change. No correctness issues found. A few style/readability suggestions in the inline comments. Please take a look, and this should be ready for a Hudi committer or PMC member to take it from here. One comment on the inline note — it describes the fix but omits the why, which is the part future readers actually need.
cc @yihua
| @@ -753,6 +753,8 @@ public void testMultiWriterWithInsertsToDistinctPartitions(HoodieTableType table | |||
| throw new RuntimeException(e); | |||
| } | |||
There was a problem hiding this comment.
🤖 nit: the comment restates what the code does — could you replace it with a note on why the sleep reduces flakiness (e.g. which race or ordering issue it sidesteps)? That context is what a future reader actually needs.
5 participants
Describe the issue this Pull Request addresses
TestHoodieClientMultiWriter#testMultiWriterWithInsertsToDistinctPartitionscan fail intermittently. It starts several writers concurrently in a tight submission loop, all contending for the same in-process lock, and under CI load the lock-acquisition timing occasionally interleaves in a way that fails the assertions even though the multi-writer behavior itself is correct.Summary and Changelog
Add a small 100ms wait between concurrent writer-task submissions in the test so the writers are staggered slightly. This is test-only. The writers still overlap (each writer's Spark client setup plus OCC commit far exceeds 100ms, and completion is still gathered together afterwards), so the concurrency the test exercises is preserved.
Impact
No production behavior change. Reduces intermittent CI failures for this test.
Risk Level
low. Test-only timing change, covered by the existing test.
Documentation Update
none
Contributor's checklist