Report an undefined control-chart scale rather than collapsing the limits (#557, #558); remove TODO.md - #562
Open
kgdunn wants to merge 4 commits into
Open
Report an undefined control-chart scale rather than collapsing the limits (#557, #558); remove TODO.md#562kgdunn wants to merge 4 commits into
kgdunn wants to merge 4 commits into
Conversation
The file stated it would be deleted once those issues were triaged. That is now done: 20 of the 32 are closed and the 12 still open carry area and priority labels. Nothing in the repository referenced it. The one pointer it carried that is not already reproduced in an issue body, `git show 50815c8:TODO.txt` for the original free-form checklist, was moved to a comment on #199 before deleting. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0123j56Hk6jRz9zy91Doc1og
#558) A NaN in either vector makes every slope through that point undefined, so the inner np.nanmedian received an all-NaN list: it emitted "All-NaN slice encountered" and returned NaN, which the outer median then quietly discarded. The slope was therefore computed from whichever points happened to be clean, with nothing in the return value recording the omission. Measured before this change, x = [0,1,2,3] against y = [5, nan, 6, 72] returned 33.25 with a RuntimeWarning as the only signal. Non-finite pairs are now dropped pairwise and up front, so no all-NaN slice is ever built and no warning escapes. Fewer than three finite pairs raises ValueError naming the count, consistent with the existing len(x) <= 2 guard and with docs/development/error_handling.rst. The omission is documented, so it is a stated contract rather than a side effect of np.nanmedian. This function supplies the warm-up trend for ControlChart, which is where the leaked warning surfaced for callers who had never named it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0123j56Hk6jRz9zy91Doc1og
…its (#557) calculate_limits computed its scale as np.sqrt(max(0.0, resids)). That guard was written to stop a negative radicand reaching sqrt, but max(0.0, nan) returns 0.0: nan > 0.0 is False, so max keeps its first argument. Missing values at or near the start of a series propagate through the Holt-Winters recursion and leave every training-sample error NaN, so resids was NaN and the scale silently became zero. self.s and the plus/minus 3 sigma deltas taken from it then described a chart whose limits had no width, with no exception raised. Measured on 40 points of N(100, 2) with the first four set to NaN: s = 0.0 and both 3 sigma limits sat at 98.83071348008284, while sigma_0 was a healthy 1.092, so the existing zero-variance guard never fired. Two contributing paths are fixed alongside it, so no RuntimeWarning now escapes calculate_limits: the recursion's own fallback for a missing error (the median of the last ten absolute errors) is undefined when the gap is at the very start, and the lambda grid search evaluated cells carrying no finite error. Both record the undefined result directly rather than routing an all-NaN slice through np.nanmedian or np.nanmean. The radicand computation is shared by the grid search and the final estimate, so the two cannot drift: _training_error_radicand returns NaN for an unusable cell, and _tau_from_training_errors decides what to do about it. Series with no missing data are unaffected, to the last digit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0123j56Hk6jRz9zy91Doc1og
Both are user-facing bug fixes, so PATCH. CITATION.cff is kept in step in the same commit, and the changelog also records the removal of TODO.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0123j56Hk6jRz9zy91Doc1og
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ControlChart.calculate_limitscomputed its scale asnp.sqrt(max(0.0, resids)). That guard was written to stop a negative radicand reachingsqrt, butmax(0.0, nan)returns0.0:nan > 0.0isFalse, somaxkeeps its first argument. Missing values at or near the start propagate through the Holt-Winters recursion and leave every training-sample error NaN, soresidswas NaN and the scale silently became zero. Two contributing paths are fixed alongside it, so noRuntimeWarningescapescalculate_limitsany more.repeated_median_slopeno longer drops missing observations silently. A NaN made the innernp.nanmedianreceive an all-NaN list; it warned and returned NaN, which the outer median then discarded, so the slope came from whichever points happened to be clean. Non-finite pairs are now dropped pairwise and up front, the behaviour is documented, and fewer than three finite pairs raisesValueError.TODO.mdremoved. It was a migration index for issues Completed TODO.md items (MV plots, R2 attributes, VIP, jackknife CI, contribution plots) #188-219 and stated it would be deleted once those were triaged, which is now done.Measured on 40 points of N(100, 2), seed 42:
s = 2.305649s = 2.305649(unchanged)s = 2.302721+RuntimeWarnings = 2.302721, no warnings = 0.0, limits98.83071348008284to98.83071348008284ValueErrornaming the causeIn the failing case
sigma_0was a healthy 1.092, so the existing zero-variance guard at_holt_winters_warmup_fitnever fired. The collapse happened later, on a different quantity.This is the pattern
docs/development/error_handling.rstnames explicitly: "Never silently substitutenp.nanor1.0for an undefined statistic".Test plan
uv run pytestfull suite: 3219 passed, 41 skipped (all skips are remote-dataset downloads blocked by the sandbox proxy, pre-existing).uv run ruff check .anduv run ruff format --check .both pass.uv run mypy src/process_improve: no issues in 163 source files.s = 2.305649,target = 99.933).New coverage:
TestControlChartMissingValues(4 tests) andTestRepeatedMedianSlopeMissingValues(7 tests), replacing the four# TODOmarkers that stood in for them attests/test_monitoring.py:122-123andtests/test_regression.py:24-25.Checklist
pyproject.toml(PATCH: 1.85.1 to 1.85.2;CITATION.cffkept in step in the same commit)ruff check .passesCHANGELOG.mdupdatedNotes for review
Two things worth a second opinion:
error_istays NaN and poisonsalpha_hat/beta_hat/sigma_hatfor the rest of the series. A chart could instead carry its state forward unchanged across a missing observation, which would produce usable limits rather than an exception. That is a modelling decision about what the chart should do with missing data, closer to Batch missing-data handling and smoothing filters #200, so this PR only stops the silent wrong answer and leaves that choice open._holt_winters_parameter_fitover theC901/PLR0912thresholds. Rather than addnoqa([ENG-25] Many noqa: C901 / PLR0912 / PLR0915 / PLR0913 suppressions (69 instances) #307 already tracks 69 such suppressions) the radicand computation was extracted into_training_error_radicand, now shared by the lambda grid search and the final estimate so the two cannot drift, with_tau_from_training_errorsdeciding what to do about an undefined result. Net complexity of that function is lower than before this PR.Both issues were split out of #213, which listed them only as missing test cases. #197 and #198 were also reopened during this audit: they had been closed as completed while five of their listed items were still unimplemented.
🤖 Generated with Claude Code
https://claude.ai/code/session_0123j56Hk6jRz9zy91Doc1og
Generated by Claude Code