[damccorm]

Note: this PR was auto-generated by tooling. Before moving it out of draft, the author has done a full review of the PR, specifically evaluating it on the following criteria:

• The PR is an appropriate fix for the issue (e.g. we're not changing a test to avoid a bug)
• The PR is a simple fix to the issue (e.g. we're not adding infra when we could just modify a test)
• The PR is not covering for other problems (e.g. if we're bumping timeouts, it is not because we're covering for a performance problem)

Full details of the methodology of generation and review can be found here - https://docs.google.com/document/d/14lj7ECVov4T4EdwmIzNlWrRGE3lLDMOwX5xUMGzaLoc/edit?usp=sharing

If you have concerns with the PR or with the underlying methodology used to generate this, please comment below - they will be addressed by the author not the tooling :)

Summary

Fixes flaky IcebergIO Integration Tests failures in BigQueryMetastoreCatalogIT.testReadWriteManagedPerOperationWithDefaultDrop (run 43154192604, run 42362603367):

org.apache.iceberg.exceptions.ForbiddenException: Exceeded rate limits: too many api_meta_requests for this user_method.
  at org.apache.iceberg.gcp.bqms.BigQueryMetastoreClientImpl.translate(BigQueryMetastoreClientImpl.java:214)
  at org.apache.iceberg.gcp.bqms.BigQueryMetastoreCatalog.dropNamespace(BigQueryMetastoreCatalog.java:427)
  at org.apache.beam.sdk.io.iceberg.catalog.IcebergCatalogBaseIT.catalogCleanup(IcebergCatalogBaseIT.java:195)

Root Cause

testReadWriteManagedPerOperationWithDefaultDrop creates and drops multiple tables in the BigQuery Metastore catalog, and then @After cleanUp() immediately calls catalogCleanup(namespaces) (listTables + dropTable + dropNamespace), which can exceed the per-second api_meta_requests rate limit on BigQueryMetastoreCatalog.dropNamespace and throw org.apache.iceberg.exceptions.ForbiddenException.

Fix

Retry each namespace cleanup in IcebergCatalogBaseIT.catalogCleanup up to 3 times with backoff (SETUP_TEARDOWN_SLEEP_MS * attempt) when ForbiddenException is thrown.

Verification

• ./gradlew :sdks:java:io:iceberg:spotlessCheck :sdks:java:io:iceberg:compileTestJava :sdks:java:io:iceberg:checkstyleTest
• Triggered IcebergIO Integration Tests (IO_Iceberg_Integration_Tests.yml) on branch fix/bqms-catalog-it-cleanup-ratelimit-retry (previous run 43166035024: success)

[damccorm]

/gemini review

[github-actions]

📋 Review Summary

This PR resolves transient ForbiddenException rate-limit failures during test teardown in BigQueryMetastoreCatalogIT by adding a retry mechanism with linear backoff in catalogCleanup. The implementation is clean, properly handles thread interrupts, and gives BigQuery's 10-second per-table metadata operation quota window sufficient time to reset.

🔍 General Feedback

• Because super.catalogCleanup(namespaces) iterates over catalog.listTables(namespace) and drops each table before dropping the namespace, retrying super.catalogCleanup is safe when a dropTable call hits a rate limit (already-dropped tables will no longer be returned by listTables on subsequent attempts).

---

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 57.75%. Comparing base (1878a5b) to head (7039747).
⚠️ Report is 20 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #41235      +/-   ##
============================================
- Coverage     59.27%   57.75%   -1.52%     
+ Complexity     4908     2617    -2291     
============================================
  Files          1461     1202     -259     
  Lines        186100   171679   -14421     
  Branches       3650     2163    -1487     
============================================
- Hits         110311    99159   -11152     
+ Misses        72440    69862    -2578     
+ Partials       3349     2658     -691     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[damccorm]

Instead of overriding, should we make this the default behavior in the superclass?

[damccorm]

/gemini review

[github-actions]

📋 Review Summary

This PR updates IcebergCatalogBaseIT.catalogCleanup to retry namespace and table teardown up to 3 times with linear backoff when encountering ForbiddenException (triggered by BigQuery Metastore rate limits). The change is clean, properly preserves interrupt status, and directly addresses the test flake.

🔍 General Feedback

• Moving the retry logic into the base IcebergCatalogBaseIT class is a good approach that benefits all catalog implementations sharing this test harness.
• Exception handling and backoff timing (SETUP_TEARDOWN_SLEEP_MS * attempt) are well-structured and appropriate for test cleanup.

---

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.