fix(signals): a short series no longer returns an unqualified all-clear (#114) - #118
Merged
Merged
Conversation
…ar (#114) ## Summary - **What:** wire SignalConfig.min_observations into detection as an advisory threshold, and make SignalResult.summary report partial evaluation. - **Why:** on a four-point X chart six of eight rules cannot fire, yet the summary printed "✓ No signals detected" with no warning. The config's min_observations was never read. GHSA-hw63-2x95-fmpv / #114. - **Scope:** signals/detector.py, signals/result.py, docstrings, weco-rules docs, api.md, CHANGELOG, SECURITY.md version table, tests. ## Contract / Invariants (must remain true) - Which rules run, and what they flag, is unchanged: the per-rule table is now SignalDetector.RULE_MIN_OBSERVATIONS and still gates rule execution. - rules_skipped reason text unchanged. - SignalResult constructor additions are keyword-with-default; 4-arg positional construction still works and reads as complete. - to_json output format unchanged (records array). - No chart math touched; validation/e2e_bishop_report.py 280/280. ## Behavior Changes (explicit) - detect_signals() on fewer than min_observations rows emits one ProcessBehaviorWarning per chart naming both numbers (stacklevel reaches the caller of AnalysisResult.detect_signals). - is_partial / evaluation_status are also True/'partial' when the series is below min_observations, even if every runnable rule ran. - summary for a partial evaluation starts with "⚠ Partial evaluation in ..." and never contains "No signals detected"; the ✓ line is reserved for a complete evaluation. With violations present, a partial evaluation adds an "Evaluation: partial (...)" line. - New SignalResult attributes: rules_evaluated, rules_applicable, n_observations, min_observations, below_min_observations, evaluation_note. repr gains evaluation='partial'. Excel Summary sheet gains two rows. ## Tests - test_signals: partial test extended with denominator and summary asserts; below-threshold warns and is partial; at-threshold is complete and silent; threshold never changes which rules run; violations + partial shows the evaluation line. - test_signal_result_surface: partial summary is not an all-clear; positional construction reads complete. - test_documented_rule_coverage: docs minimum-observations table pinned to RULE_MIN_OBSERVATIONS; the advisory's four-point ramp reproducer. ## Manual Verification - pytest tests/: 2335 passed, 10 skipped - ruff check .: clean; mypy processbehavior/signals: clean - validation/e2e_bishop_report.py: exit 0, 280 assertions pass
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SignalConfig.min_observationsbecomes a real advisory threshold, andSignalResult.summaryreports partial evaluation instead of printing✓ No signals detectedon a series it could barely examine.min_observationswas never read by the detector. Reported in Series length (T) as an axis in design-state admissibility — and a saturation question #114 and privately as GHSA-hw63-2x95-fmpv.signals/detector.py,signals/result.py, two docstrings,docs/reference/weco-rules.md,docs/reference/api.md,CHANGELOG.md,SECURITY.mdversion table, tests.Contract / invariants
SignalDetector.RULE_MIN_OBSERVATIONSand still gates rule execution.validation/e2e_bishop_report.pyexits 0 with 280 of 280.to_jsonstill writes a records array.SignalResultconstructor additions are keyword-with-default; 4-arg positional construction still works and reads as complete.rules_skippedreason text is untouched. The clean-series✓ No signals detected in Xline is unchanged for a complete evaluation.Behaviour changes
detect_signals()on fewer thanmin_observationsrows emits oneProcessBehaviorWarningper chart naming both numbers. Silence withwarnings.simplefilter('ignore', ProcessBehaviorWarning).is_partial/evaluation_statusare also partial when the series is belowmin_observations, even if every runnable rule ran.⚠ Partial evaluation in X: 2 of 8 rules applicable at n=4 (below min_observations=20); skipped: rule_3 (needs 5), ... No signals from the rules evaluated.With violations present, the banner summary gains anEvaluation: partial (...)line.SignalResultattributes:rules_evaluated,rules_applicable,n_observations,min_observations,below_min_observations,evaluation_note.reprgainsevaluation='partial'. The Excel Summary sheet gains Evaluation Status and Rules Evaluated / Applicable rows.Methodology
Tests
test_signals::test_insufficient_data_partial_evaluation— extended: carries the3 of 8denominator and the summary is not an all-clear.test_signals::test_below_min_observations_is_partial_and_warns— every runnable rule ran, series under threshold: warns naming both numbers, result partial.test_signals::test_at_min_observations_is_complete_and_silent— adequate series: no warning, checkmark summary unchanged.test_signals::test_min_observations_does_not_change_which_rules_run— threshold 1 vs 1000 give identical rules and violations.test_signals::test_signals_present_and_partial_shows_evaluation_line.test_signal_result_surface::test_summary_partial_is_not_an_all_clear,::test_positional_construction_still_complete.test_documented_rule_coverage::test_docs_minimum_observations_table_matches_detector— the docs table is pinned toRULE_MIN_OBSERVATIONS.test_documented_rule_coverage::test_four_point_ramp_is_not_an_all_clear— the advisory's reproducer, end to end throughpb.formulate.pytest tests/— 2335 passed, 10 skippedruff check .— cleanmypy processbehavior/signals— cleanGolden masters untouched
Notes
RuleSet.custom(min_observations=)is still dropped byto_config(). Pre-existing, separate, not addressed here.run_rules_viz.pyearly-returns onhas_signals, so a partial result draws no annotation. Not addressed here.🤖 Generated with Claude Code
https://claude.ai/code/session_01BKphCi1LC1hjGYteWEK9JV