Repository navigation
Conversation
…partition combine_moments used `combined_n`, which is only assigned when two non-empty partitions are merged. When one partition holds all the data (for example a small input in one bundle), skew() and kurtosis() raised UnboundLocalError. The accumulated count `n` equals `combined_n` after every merge, so use it.
There was a problem hiding this comment.
📋 Review Summary
This PR resolves an UnboundLocalError in DeferredSeries.skew() and DeferredSeries.kurtosis() when all non-empty data resides within a single partition. Replacing combined_n with n in the final return expressions of combine_moments is mathematically sound and cleanly handles both single-partition and multi-partition inputs.
🔍 General Feedback
- The fix in
sdks/python/apache_beam/dataframe/frames.pyaccurately usesn, which holds the total sample count after iterating through all partitions regardless of whether the merge branch (else) was entered. - The added unit test in
sdks/python/apache_beam/dataframe/frames_test.pywithdistributed=Falsedirectly covers the single-partition execution path.
💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.
|
Assigning reviewers: R: @jrmccluskey for label python. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
Fixes #41283
Series.skew()andSeries.kurtosis()merge per-partition moments incombine_moments. The result usedcombined_n, which is only assigned when two non-empty partitions are merged, so when only one partition holds data (for example a small input processed in one bundle) the call raisedUnboundLocalError: cannot access local variable 'combined_n'. After the merge loopnalready holds the total count (it equalscombined_nafter every merge), so the result now usesn; results for several partitions are unchanged.Tests:
test_series_skew_kurtosis_single_partitionevaluates both on one partition (distributed=False); it fails on master with theUnboundLocalErrorand passes with the change.pytest apache_beam/dataframe/frames_test.py: 460 passed, 19 skipped. Withto_dataframeon abeam.Createof 6 floats, both calls failed on master on the default runner and on the FnApiRunner; after the change they return 0.9191843544212499 and -0.25414322711619963, the same as pandas.