Skip to content

count signed zeros as ties in KendallsCorrelation - #357

Open
insaf021 wants to merge 1 commit into
apache:masterfrom
insaf021:kendall-signed-zero-ties
Open

insaf021 wants to merge 1 commit into
apache:masterfrom
insaf021:kendall-signed-zero-ties

Conversation

@insaf021

Copy link
Copy Markdown
Contributor

On master:

KendallsCorrelationTest.testSignedZeroTies: expected:<0.8164965809277261> but was:<0.3333333333333333>

correlation({0.0, -0.0, 1.0}, {1, 2, 3})  = 0.3333  (tau-b: 2/sqrt(6) = 0.8165)
correlation({0.0, -0.0}, {1, 2})          = -1.0    (all x tied: NaN)

KendallsCorrelation.correlation boxes the inputs into Pair<Double, Double> and finds ties with Double.equals and orders with Double.compareTo. Both treat -0.0 and 0.0 as different values, so a pair whose x (or y) values are the two zeros is scored as concordant or discordant rather than tied, and the tau-b denominator loses that tie as well. Signed zeros turn up from ordinary arithmetic on the data (negation, products, small negatives rounded to zero), and the other correlations in the package treat them as equal.

The values are now normalised with + 0.0 when the pairs are built, which maps -0.0 to 0.0 and leaves every other double, including NaN, unchanged. The sort, the merge-sort swap count and the tie scans then all see one zero.

testSignedZeroTies covers ties in x, ties in y and the all-tied case; it fails on master as above and passes with the patch. mvn clean verify javadoc:javadoc -pl commons-math-legacy is green on JDK 21 (3273 tests, 0 Checkstyle violations).

correlation boxes the values into Pair<Double, Double> and detects ties
with Double.equals and orders them with Double.compareTo, which both
treat -0.0 and 0.0 as different values. A pair whose x or y values are
0.0 and -0.0 was counted as concordant or discordant instead of tied, so
correlation({0.0, -0.0, 1.0}, {1, 2, 3}) returned 1/3 instead of
2/sqrt(6).

Map -0.0 to 0.0 when the pairs are built.
@aherbert

Copy link
Copy Markdown
Contributor

Discussion issue: https://issues.apache.org/jira/browse/MATH-1698

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants