[CI] Restore Java 17 in the Spark Versions PreCommit so the Spark 4 module runs - #40093
Merged
Merged
Conversation
…odule runs apache#39205 replaced the two version Java setup with Java 11 only. The job still passes -Pjava17Home=$JAVA_HOME_17_X64, which is empty without the second JDK, and the build plugin disables every task of runners/spark/4 when its JDK is missing. Since then :runners:spark:4:build has been SKIPPED in every run and the module's tests have not executed in CI.
Contributor
|
Assigning reviewers: R: @Amar3tto for label build. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
Abacn
approved these changes
Sep 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
beam_PreCommit_Java_Spark_Versionsruns:runners:spark:3:sparkVersionsTest :runners:spark:4:build -Pjava17Home=$JAVA_HOME_17_X64. #39205 changed its Java setup from17plus11to11only, soJAVA_HOME_17_X64is empty,java17Homeresolves to nothing, andBeamModulePlugindisables every task ofrunners/spark/4because its required JDK is missing. Every run since then printsand reports green. The Spark 4 unit tests, including the streaming source tests from #39971 and the translator tests in #40090, have not executed in CI in that period.
beam_PreCommit_Javacompiles the module on its native JDK but does not run its tests.This restores the two version setup that #38453 introduced, same pattern as
beam_PreCommit_SQL_Java17. Java 11 stays the default JDK for the Spark 3 part of the job, Java 17 is handed to the Spark 4 module throughjava17Home.Verification after merge: the next master run of the workflow must show
:runners:spark:4:testexecuted. Local equivalent,./gradlew :runners:spark:4:build -Pjava17Home=/path/to/jdk17runs 234 tests.R: @Abacn