Skip to content

Use 20-minute JUnit timeout on non-DirectRunner in DeltaLakeToIcebergCdcIT - #41280

Draft
damccorm wants to merge 1 commit into
masterfrom
fix/delta-lake-to-iceberg-cdc-it-timeout
Draft

damccorm wants to merge 1 commit into
masterfrom
fix/delta-lake-to-iceberg-cdc-it-timeout

Conversation

@damccorm

Copy link
Copy Markdown
Contributor

Note: this PR was auto-generated by tooling. Before moving it out of draft, the author has done a full review of the PR, specifically evaluating it on the following criteria:

  • The PR is an appropriate fix for the issue (e.g. we're not changing a test to avoid a bug)
  • The PR is a simple fix to the issue (e.g. we're not adding infra when we could just modify a test)
  • The PR is not covering for other problems (e.g. if we're bumping timeouts, it is not because we're covering for a performance problem)

Full details of the methodology of generation and review can be found here - https://docs.google.com/document/d/14lj7ECVov4T4EdwmIzNlWrRGE3lLDMOwX5xUMGzaLoc/edit?usp=sharing

If you have concerns with the PR or with the underlying methodology used to generate this, please comment below - they will be addressed by the author not the tooling :)

Summary

  • Increases the JUnit @Rule Timeout in DeltaLakeToIcebergCdcIT from a hardcoded 10 minutes (10 * 60) to Timeout.seconds(OPTIONS.getRunner().equals(DirectRunner.class) ? 300 : 20 * 60), matching the runner-dependent timeout used in IcebergCatalogBaseIT and IcebergCdcWriteBaseIT.

Root Cause

In the IcebergIO Integration Tests on Dataflow workflow (run 43366381845), DeltaLakeToIcebergCdcIT > changeFeedAppliesToIcebergTableWithChangeTypeColumn failed with org.junit.runners.model.TestTimedOutException: test timed out after 600 seconds at DeltaLakeToIcebergCdcIT.applyChangeFeed(DeltaLakeToIcebergCdcIT.java:340) after the Dataflow Runner v2 job had already completed all operations (Finished operation ... at 05:45:51.531Z) and was in Cleaning up. Stopping worker pool... at 9m51s.

Commit ea3c5dfb210 (#40230) added both IcebergCdcWriteBaseIT (configured with Timeout.seconds(OPTIONS.getRunner().equals(DirectRunner.class) ? 300 : 20 * 60), matching IcebergCatalogBaseIT) and DeltaLakeToIcebergCdcIT (configured with a fixed 10-minute timeout), which leaves insufficient headroom for Dataflow Runner v2 worker startup and job teardown in :sdks:java:io:iceberg:dataflowIntegrationTest.

Verification

  • Verified locally with ./gradlew :sdks:java:io:iceberg:spotlessCheck :sdks:java:io:iceberg:compileTestJava :sdks:java:io:iceberg:checkstyleTest.

Verification

  • Triggered IO_Iceberg_Integration_Tests_Dataflow.yml workflow run on branch: 43379167610

@damccorm damccorm added the test-failure-cleanup Automated test health and flake cleanup PRs label Nov 13, 2026
@damccorm

Copy link
Copy Markdown
Contributor Author

/gemini review

@github-actions github-actions Bot added java io iceberg Iceberg IO connector size/xs XS: <10 lines changed labels Nov 13, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📋 Review Summary

This PR updates the JUnit @Rule Timeout in DeltaLakeToIcebergCdcIT from a fixed 10 minutes to 5 minutes on DirectRunner and 20 minutes on other runners (such as DataflowRunner). The change is clean, well-justified, and aligns DeltaLakeToIcebergCdcIT with the existing timeout conventions in IcebergCatalogBaseIT and IcebergCdcWriteBaseIT.

🔍 General Feedback

  • The runner-dependent timeout (300 seconds for DirectRunner, 20 * 60 seconds otherwise) provides adequate headroom for Dataflow Runner v2 worker startup and teardown while tightening timeouts for local/DirectRunner execution.
  • No issues found; the change is ready to merge.

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

@damccorm damccorm added the ready-for-human Draft PR with all checks passing and all review comments resolved, ready for human review label Nov 13, 2026
@codecov

codecov Bot commented Nov 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 57.74%. Comparing base (d7d31e9) to head (0e7feff).

Additional details and impacted files
@@              Coverage Diff              @@
##             master   #41280       +/-   ##
=============================================
+ Coverage     36.06%   57.74%   +21.68%     
  Complexity     2633     2633               
=============================================
  Files           716     1203      +487     
  Lines         88226   171888    +83662     
  Branches       2182     2182               
=============================================
+ Hits          31815    99257    +67442     
- Misses        53746    69966    +16220     
  Partials       2665     2665               

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

iceberg Iceberg IO connector io java ready-for-human Draft PR with all checks passing and all review comments resolved, ready for human review size/xs XS: <10 lines changed test-failure-cleanup Automated test health and flake cleanup PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant