Skip to content

Report an undefined control-chart scale rather than collapsing the limits (#557, #558); remove TODO.md - #562

Open
kgdunn wants to merge 4 commits into
mainfrom
claude/audit-todo-fixme-items-gy9vj6
Open

Report an undefined control-chart scale rather than collapsing the limits (#557, #558); remove TODO.md#562
kgdunn wants to merge 4 commits into
mainfrom
claude/audit-todo-fixme-items-gy9vj6

Conversation

@kgdunn

@kgdunn kgdunn commented Sep 11, 2026

Copy link
Copy Markdown
Owner

Summary

  • ControlChart: NaN in the warm-up window silently yields zero-width control limits (s = 0.0) #557: a gap at the start of a series no longer collapses the control limits to zero width. ControlChart.calculate_limits computed its scale as np.sqrt(max(0.0, resids)). That guard was written to stop a negative radicand reaching sqrt, but max(0.0, nan) returns 0.0: nan > 0.0 is False, so max keeps its first argument. Missing values at or near the start propagate through the Holt-Winters recursion and leave every training-sample error NaN, so resids was NaN and the scale silently became zero. Two contributing paths are fixed alongside it, so no RuntimeWarning escapes calculate_limits any more.
  • repeated_median_slope silently drops NaN pairs and leaks RuntimeWarnings #558: repeated_median_slope no longer drops missing observations silently. A NaN made the inner np.nanmedian receive an all-NaN list; it warned and returned NaN, which the outer median then discarded, so the slope came from whichever points happened to be clean. Non-finite pairs are now dropped pairwise and up front, the behaviour is documented, and fewer than three finite pairs raises ValueError.
  • TODO.md removed. It was a migration index for issues Completed TODO.md items (MV plots, R2 attributes, VIP, jackknife CI, contribution plots) #188-219 and stated it would be deleted once those were triaged, which is now done.

Measured on 40 points of N(100, 2), seed 42:

Input before after
clean s = 2.305649 s = 2.305649 (unchanged)
NaN at index 0 s = 2.302721 + RuntimeWarning s = 2.302721, no warning
NaN in first 4 s = 0.0, limits 98.83071348008284 to 98.83071348008284 ValueError naming the cause

In the failing case sigma_0 was a healthy 1.092, so the existing zero-variance guard at _holt_winters_warmup_fit never fired. The collapse happened later, on a different quantity.

This is the pattern docs/development/error_handling.rst names explicitly: "Never silently substitute np.nan or 1.0 for an undefined statistic".

Test plan

  • uv run pytest full suite: 3219 passed, 41 skipped (all skips are remote-dataset downloads blocked by the sandbox proxy, pre-existing).
  • The 11 new tests were run against the unfixed source: 8 fail, 3 pass. The 3 that pass in both are the "clean input is unaffected" guards, which is what they are for. This rules out tests that cannot fail.
  • uv run ruff check . and uv run ruff format --check . both pass.
  • uv run mypy src/process_improve: no issues in 163 source files.
  • Clean-data results verified identical to the last digit before and after (s = 2.305649, target = 99.933).

New coverage: TestControlChartMissingValues (4 tests) and TestRepeatedMedianSlopeMissingValues (7 tests), replacing the four # TODO markers that stood in for them at tests/test_monitoring.py:122-123 and tests/test_regression.py:24-25.

Checklist

  • Version bumped in pyproject.toml (PATCH: 1.85.1 to 1.85.2; CITATION.cff kept in step in the same commit)
  • Tests added or updated where relevant
  • ruff check . passes
  • CHANGELOG.md updated

Notes for review

Two things worth a second opinion:

  1. Scope: fail loudly, not fix the modelling. The deeper cause is that the recursion's fallback for a missing error (the median of the last ten absolute errors) is undefined when the gap is at the very start, so error_i stays NaN and poisons alpha_hat / beta_hat / sigma_hat for the rest of the series. A chart could instead carry its state forward unchanged across a missing observation, which would produce usable limits rather than an exception. That is a modelling decision about what the chart should do with missing data, closer to Batch missing-data handling and smoothing filters #200, so this PR only stops the silent wrong answer and leaves that choice open.
  2. Complexity budget. The added guards pushed _holt_winters_parameter_fit over the C901 / PLR0912 thresholds. Rather than add noqa ([ENG-25] Many noqa: C901 / PLR0912 / PLR0915 / PLR0913 suppressions (69 instances) #307 already tracks 69 such suppressions) the radicand computation was extracted into _training_error_radicand, now shared by the lambda grid search and the final estimate so the two cannot drift, with _tau_from_training_errors deciding what to do about an undefined result. Net complexity of that function is lower than before this PR.

Both issues were split out of #213, which listed them only as missing test cases. #197 and #198 were also reopened during this audit: they had been closed as completed while five of their listed items were still unimplemented.

🤖 Generated with Claude Code

https://claude.ai/code/session_0123j56Hk6jRz9zy91Doc1og


Generated by Claude Code

The file stated it would be deleted once those issues were triaged. That is
now done: 20 of the 32 are closed and the 12 still open carry area and
priority labels. Nothing in the repository referenced it.

The one pointer it carried that is not already reproduced in an issue body,
`git show 50815c8:TODO.txt` for the original free-form checklist, was moved
to a comment on #199 before deleting.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0123j56Hk6jRz9zy91Doc1og
#558)

A NaN in either vector makes every slope through that point undefined, so the
inner np.nanmedian received an all-NaN list: it emitted "All-NaN slice
encountered" and returned NaN, which the outer median then quietly discarded.
The slope was therefore computed from whichever points happened to be clean,
with nothing in the return value recording the omission. Measured before this
change, x = [0,1,2,3] against y = [5, nan, 6, 72] returned 33.25 with a
RuntimeWarning as the only signal.

Non-finite pairs are now dropped pairwise and up front, so no all-NaN slice is
ever built and no warning escapes. Fewer than three finite pairs raises
ValueError naming the count, consistent with the existing len(x) <= 2 guard and
with docs/development/error_handling.rst. The omission is documented, so it is
a stated contract rather than a side effect of np.nanmedian.

This function supplies the warm-up trend for ControlChart, which is where the
leaked warning surfaced for callers who had never named it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0123j56Hk6jRz9zy91Doc1og
…its (#557)

calculate_limits computed its scale as np.sqrt(max(0.0, resids)). That guard
was written to stop a negative radicand reaching sqrt, but max(0.0, nan)
returns 0.0: nan > 0.0 is False, so max keeps its first argument. Missing
values at or near the start of a series propagate through the Holt-Winters
recursion and leave every training-sample error NaN, so resids was NaN and the
scale silently became zero. self.s and the plus/minus 3 sigma deltas taken from
it then described a chart whose limits had no width, with no exception raised.

Measured on 40 points of N(100, 2) with the first four set to NaN: s = 0.0 and
both 3 sigma limits sat at 98.83071348008284, while sigma_0 was a healthy 1.092,
so the existing zero-variance guard never fired.

Two contributing paths are fixed alongside it, so no RuntimeWarning now escapes
calculate_limits: the recursion's own fallback for a missing error (the median
of the last ten absolute errors) is undefined when the gap is at the very start,
and the lambda grid search evaluated cells carrying no finite error. Both record
the undefined result directly rather than routing an all-NaN slice through
np.nanmedian or np.nanmean.

The radicand computation is shared by the grid search and the final estimate, so
the two cannot drift: _training_error_radicand returns NaN for an unusable cell,
and _tau_from_training_errors decides what to do about it.

Series with no missing data are unaffected, to the last digit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0123j56Hk6jRz9zy91Doc1og
Both are user-facing bug fixes, so PATCH. CITATION.cff is kept in step in the
same commit, and the changelog also records the removal of TODO.md.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0123j56Hk6jRz9zy91Doc1og
@codecov

codecov Bot commented Sep 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 77.77778% with 8 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/process_improve/monitoring/control_charts.py 71.42% 3 Missing and 3 partials ⚠️
...c/process_improve/regression/_robust_regression.py 86.66% 1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants