Sortino returns 0.0 when there is no losing period, and the report paints it "Suboptimal" - #600
Open
zorionarrillaga wants to merge 2 commits into
Conversation
get_sortino_ratio returned 0.0 when no period fell below the target. Since coding-kitties#599 the downside deviation is an RMS shortfall, so it is exactly 0.0 whenever nothing is negative and the guard fires on every strategy that never had a losing period. That is a verdict rather than a placeholder: highlight_sharpe_and_sortino paints 0 <= value < 1 orange, a band its own docstring calls "Suboptimal: Returns do not justify risk". get_profit_factor and get_omega_ratio resolve the same condition to float('inf'), and omega's docstring says it mirrors profit_factor's convention. Apply that convention here: inf when the excess return is positive, 0.0 when it is not, so a flat or below-risk-free run does not become unbounded. Empty input returned inf and now returns 0.0, matching get_omega_ratio. test_all_returns_above_risk_free is re-enabled unchanged; it already asserted inf.
analyze_backtest_windows computes its own Sortino inline and had the same `else 0.0` for a zero downside deviation. Left alone, the package would return two different Sortinos for the same input after this change.
Contributor
Author
|
Ran the full suite on the branch, in case it saves you a step: Both failures are The narrower run, the 18 test files that mention sortino, goes 241 to 243 with no failures. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-on from #599, and thanks for the fast turnaround on that one.
get_sortino_ratioreturns0.0when no period fell below the target. Since #599 the downsidedeviation is an RMS shortfall, so it's exactly
0.0whenever nothing is negative, and the guard atsortino_ratio.py:67fires.get_omega_ratioresolves that same condition on the same snapshots toinf, and its docstring saysit mirrors
get_profit_factor's division-by-zero convention. So the two disagree, and thedisagreement is visible:
Sortino RatioandProfit Factorare adjacent rows of the key-metricstable (
key_metrics_table.py:174-175), andhighlight_sharpe_and_sortinopaints0 <= value < 1orange, a band its own docstring calls "Suboptimal: Returns do not justify risk".
A 60-day backtest with three winning trades, flat in between and no losing period, on
maintoday:This applies the convention to Sortino:
infwhen the excess return is positive,0.0when itisn't, so a flat run or one that doesn't beat the risk-free rate doesn't become unbounded. Two other
things came with it.
get_sortino_ratio([])returnedinf, which made no data score the bestpossible value; it now returns
0.0, matchingget_omega_ratio([]). Andanalyze_backtest_windowscomputes its own Sortino inline, with the same
else 0.0— you'd flagged that file as one of thethree sites in your #599 review, so it's aligned here too rather than left to disagree with the
service.
One thing I'd flag rather than decide:
calmar_ratioalso returns0.0for a zero denominator, sothe tally across the package is three for
inf, onenaninget_sharpe_ratio, and Calmar withzero. I've only touched Sortino.
On tests: the empty-snapshot cases and
test_sortino_ratio_no_downsidenow assert the new values,there's a new case for no downside with a negative excess return, and
test_all_returns_above_risk_freeis re-enabled unchanged since it already asserted
inf. Across the 18 test files that mentionsortino, 241 passing before and 243 after, no failures.
Since you mentioned calling #599 out as a breaking metric fix in the CHANGELOG, this is a second
change to the same field in the same cycle. Happy to fold both into one note if that's easier.
One question, since you'd know better than me:
infmatchesprofit_factor,omega_ratioandrecovery, butget_sharpe_ratioreturnsnanfor its own zero-denominator case. If you'd ratherSortino match Sharpe than match the other three, I'm happy to switch it.