Expected Behavior
Feature request, not a bug — mapping to the template below.
Lean's Report should offer built-in robustness checks that answer "how much can I trust this backtest result?" Specifically, three opt-in post-backtest sections:
- Monte Carlo — resample the trade sequence to produce confidence bands (5th/50th/95th percentile) for CAGR, max drawdown, and Sharpe.
- Parameter sensitivity — re-run the backtest with perturbed parameters (±5/10/20%) and report the sensitivity envelope.
- Dynamic regime classification — partition returns into bull/bear/sideways/high-vol/low-vol regimes and report per-regime performance.
Actual Behavior
Report currently provides descriptive statistics only (Sharpe, Sortino, drawdown, PSR, hardcoded crisis events, rolling metrics). There is no built-in way to test whether a result is robust to trade ordering, parameter perturbation, or market regime.
Potential Solution
Monte Carlo and regime classification run purely post-backtest on the existing trade log and equity curve — no engine changes, opt-in via config, new sections in Report/template.html. Parameter sensitivity is heavier (it re-runs backtests) and could reuse the existing optimization infrastructure (OptimizationParameter, ParameterSet) or a lighter standalone wrapper.
Asking for design direction before I start: would you accept these, and as one PR or landed separately? My suggestion is to ship Monte Carlo first as a standalone, lowest-risk PR, then discuss the other two.
I'll implement with full unit tests and a regression algorithm.
Prior art: quantstrat (R) has trade-return Monte Carlo; Backtrader has had community MC requests for years with no implementation; Zipline and Freqtrade have none. vectorbt offers resampling and parameter sweeps, so it isn't unprecedented in Python — but it isn't standard.
Checklist
Expected Behavior
Feature request, not a bug — mapping to the template below.
Lean's Report should offer built-in robustness checks that answer "how much can I trust this backtest result?" Specifically, three opt-in post-backtest sections:
Actual Behavior
Report currently provides descriptive statistics only (Sharpe, Sortino, drawdown, PSR, hardcoded crisis events, rolling metrics). There is no built-in way to test whether a result is robust to trade ordering, parameter perturbation, or market regime.
Potential Solution
Monte Carlo and regime classification run purely post-backtest on the existing trade log and equity curve — no engine changes, opt-in via config, new sections in Report/template.html. Parameter sensitivity is heavier (it re-runs backtests) and could reuse the existing optimization infrastructure (OptimizationParameter, ParameterSet) or a lighter standalone wrapper.
Asking for design direction before I start: would you accept these, and as one PR or landed separately? My suggestion is to ship Monte Carlo first as a standalone, lowest-risk PR, then discuss the other two.
I'll implement with full unit tests and a regression algorithm.
Prior art: quantstrat (R) has trade-return Monte Carlo; Backtrader has had community MC requests for years with no implementation; Zipline and Freqtrade have none. vectorbt offers resampling and parameter sweeps, so it isn't unprecedented in Python — but it isn't standard.
Checklist
masterbranch