Skip to content

Sortino returns 0.0 when there is no losing period, and the report paints it "Suboptimal" - #600

Open
zorionarrillaga wants to merge 2 commits into
coding-kitties:mainfrom
zorionarrillaga:fix/sortino-no-downside-convention
Open

Sortino returns 0.0 when there is no losing period, and the report paints it "Suboptimal"#600
zorionarrillaga wants to merge 2 commits into
coding-kitties:mainfrom
zorionarrillaga:fix/sortino-no-downside-convention

Conversation

@zorionarrillaga

Copy link
Copy Markdown
Contributor

Follow-on from #599, and thanks for the fast turnaround on that one.

get_sortino_ratio returns 0.0 when no period fell below the target. Since #599 the downside
deviation is an RMS shortfall, so it's exactly 0.0 whenever nothing is negative, and the guard at
sortino_ratio.py:67 fires.

get_omega_ratio resolves that same condition on the same snapshots to inf, and its docstring says
it mirrors get_profit_factor's division-by-zero convention. So the two disagree, and the
disagreement is visible: Sortino Ratio and Profit Factor are adjacent rows of the key-metrics
table (key_metrics_table.py:174-175), and highlight_sharpe_and_sortino paints 0 <= value < 1
orange, a band its own docstring calls "Suboptimal: Returns do not justify risk".

A 60-day backtest with three winning trades, flat in between and no losing period, on main today:

  Sortino Ratio     0.00   color: #FFA500; font-weight: bold;
  Profit Factor      inf   (no colour)

This applies the convention to Sortino: inf when the excess return is positive, 0.0 when it
isn't, so a flat run or one that doesn't beat the risk-free rate doesn't become unbounded. Two other
things came with it. get_sortino_ratio([]) returned inf, which made no data score the best
possible value; it now returns 0.0, matching get_omega_ratio([]). And analyze_backtest_windows
computes its own Sortino inline, with the same else 0.0 — you'd flagged that file as one of the
three sites in your #599 review, so it's aligned here too rather than left to disagree with the
service.

One thing I'd flag rather than decide: calmar_ratio also returns 0.0 for a zero denominator, so
the tally across the package is three for inf, one nan in get_sharpe_ratio, and Calmar with
zero. I've only touched Sortino.

On tests: the empty-snapshot cases and test_sortino_ratio_no_downside now assert the new values,
there's a new case for no downside with a negative excess return, and test_all_returns_above_risk_free
is re-enabled unchanged since it already asserted inf. Across the 18 test files that mention
sortino, 241 passing before and 243 after, no failures.

Since you mentioned calling #599 out as a breaking metric fix in the CHANGELOG, this is a second
change to the same field in the same cycle. Happy to fold both into one note if that's easier.

One question, since you'd know better than me: inf matches profit_factor, omega_ratio and
recovery, but get_sharpe_ratio returns nan for its own zero-denominator case. If you'd rather
Sortino match Sharpe than match the other three, I'm happy to switch it.

get_sortino_ratio returned 0.0 when no period fell below the target. Since coding-kitties#599
the downside deviation is an RMS shortfall, so it is exactly 0.0 whenever nothing
is negative and the guard fires on every strategy that never had a losing period.

That is a verdict rather than a placeholder: highlight_sharpe_and_sortino paints
0 <= value < 1 orange, a band its own docstring calls "Suboptimal: Returns do not
justify risk". get_profit_factor and get_omega_ratio resolve the same condition to
float('inf'), and omega's docstring says it mirrors profit_factor's convention.

Apply that convention here: inf when the excess return is positive, 0.0 when it is
not, so a flat or below-risk-free run does not become unbounded. Empty input
returned inf and now returns 0.0, matching get_omega_ratio.

test_all_returns_above_risk_free is re-enabled unchanged; it already asserted inf.
analyze_backtest_windows computes its own Sortino inline and had the same
`else 0.0` for a zero downside deviation. Left alone, the package would return
two different Sortinos for the same input after this change.
@zorionarrillaga

Copy link
Copy Markdown
Contributor Author

Ran the full suite on the branch, in case it saves you a step:

2408 passed, 2 failed, 2 skipped, 21 errors in 624s

Both failures are tests/scenarios/test_readme_example.py
(test_readme_strategy_class_can_be_loaded and
test_readme_strategy_runs_fast_vector_backtest_with_offline_data). I re-ran that file alone on
cecda8b4 with a clean tree and got the same two, so they're not from this change. The 21 collection
errors are my environment being on Python 3.14, and they're identical on both trees.

The narrower run, the 18 test files that mention sortino, goes 241 to 243 with no failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant