fix(results): family-wide summary(alpha=) guard + honest dose-response bands (M-146 completion) - #811
Conversation
…e bands (M-146 completion)
summary(alpha=...) on the non-staggered results containers printed a
requested-alpha "{level}% CI" header over the FIT-TIME stored conf_int -
silent coverage mislabeling. Apply the shared
results_base._require_fit_alpha guard (M-146, previously staggered-only)
to DiDResults (SpilloverDiDResults by inheritance), MultiPeriodDiDResults,
SyntheticDiDResults, TripleDifferenceResults, TROPResults, and
ContinuousDiDResults; SyntheticControlResults joins too (its alpha was a
dead assignment - now an honest raise). alpha=0.0, previously swallowed by
the falsy `alpha or self.alpha` idiom, raises everywhere.
The helper gains an optional message= override (helper-formatted
{alpha}/{fit_alpha} placeholders; the staggered default message is
byte-unchanged, pinned by an exact-string test) because the default's
bootstrap-percentile rationale is false for the new sites: the five
generic sites share an accurate never-recompute message, TROP states the
uniform-contract rationale (its t interval WOULD be reconstructible), and
SyntheticControl states that it reports no alpha-based interval at all
(the confidence set is gamma-keyed).
plot_dose_response fixes in the same audit class:
- DataFrame-`se` rows with non-positive/non-finite se are masked from the
band with a UserWarning (previously a zero-SE row drew a finite
zero-width band); the resolved alpha is validated strictly inside (0,1)
(alpha=1.0 previously drew a zero-width band via z=0).
- Band legend was hard-coded "95% CI" in both renderers regardless of the
requested alpha: it is now derived via _coverage_label (exact
fractional coverage - alpha=0.025 reads "97.5% CI" - tolerant of
numpy-scalar fit alphas incl. float32) from the requested alpha on the
se branch and results.alpha on results= input, and the level-free "CI"
for bare-curve/explicit-CI input (no knowable level).
- alpha: Optional[float] = None; an explicitly passed alpha on any
non-`se` path warns instead of being silently ignored.
- The plotly toself band polygon filters non-finite-CI rows (a NaN vertex
split it into independently-closed sub-polygons) - CI values only, the
dose axis may be non-numeric; both renderers suppress an all-masked band
(no stray legend entry).
Docs: M-146 ledger note amended (code_refs extended; test_ref stays the
staggered anchor per the single-path schema), v4-design/migration-4.0/
REGISTRY wording extended family-wide with the SyntheticControl and TROP
posture Notes, and the plot_group_effects zero-SE Note extended to the
dose-response twin (including the deliberate alpha-semantics divergence:
plot_group_effects recomputes, plot_dose_response warns-and-ignores).
Behavior fragment in changelog.d; the audit TODO row is closed, and the
summaries' pre-existing truncating {level}% idiom (all ~19 sites incl.
the shipped staggered family) is deliberately unchanged - unifying it on
the exact formatter is recorded as its own TODO row.
Tests: TestSummaryAlphaContract per container (message-content pins so a
site that forgot message= fails), _require_fit_alpha override unit tests,
and matplotlib+plotly dose-response suites (mask contract incl. negative/
inf se, all-masked suppression, string-dose rendering, label matrix incl.
fractional and numpy-scalar alphas, explicit-alpha warnings on all four
no-op paths). Pre/post capture of all seven summaries verified
byte-identical on default calls.
Overall Assessment✅ Looks good — no unmitigated P0 or P1 findings. Executive Summary
Methodology
Code QualityNo findings. The shared guard and vectorized finite-SE masking are consistent and avoid inline inference anti-patterns. PerformanceNo findings. Added finite-value checks are linear in plotted rows and negligible relative to rendering. MaintainabilityNo findings. Centralizing the summary contract in Tech Debt
SecurityNo findings. No secrets, unsafe inputs, or sensitive-data handling changes were introduced. Documentation / TestsNo findings. Both plotting backends, invalid/all-masked SEs, alpha interactions, inherited summaries, and class-specific messages receive coverage. The changelog fragment passes its checker. Full tests could not be rerun because this environment lacks |
Summary
summary(alpha=)guard (M-146 completion): the non-staggered results containers printed a requested-alpha{level}%header over the FIT-TIME storedconf_int— silent coverage mislabeling. The sharedresults_base._require_fit_alphaguard now coversDiDResults(SpilloverDiDResultsby inheritance),MultiPeriodDiDResults,SyntheticDiDResults,TripleDifferenceResults,TROPResults,ContinuousDiDResults, andSyntheticControlResults(whosealphawas a dead assignment — now an honest raise).alpha=0.0, previously swallowed by the falsyalpha or self.alphaidiom, raises everywhere. Default-callsummary()output verified byte-identical pre/post on all seven containers.message=override (helper-formatted{alpha}/{fit_alpha}; the staggered default message is byte-unchanged, pinned by an exact-string test). The five analytic-default sites share an accurate generic message; TROP's states the uniform-contract rationale (its t interval WOULD be reconstructible — no false non-reconstructibility claim); SyntheticControl's states it reports no alpha-based interval (the confidence set isgamma-keyed).plot_dose_responsehonest bands/labels: non-positive/non-finiteserows are masked from the band with a warning (previously a zero-SE row drew a finite zero-width band);alphais validated strictly inside (0, 1) (alpha=1.0previously drew a zero-width band viaz=0); the band legend (previously hard-coded"95% CI") is derived via a_coverage_labelhelper — exact fractional coverage (97.5% CIforalpha=0.025), tolerant of numpy-scalar fit alphas — from the requested alpha on thesebranch andresults.alphaonresults=input, with the level-free"CI"where no level is knowable; an explicitly passedalphaon non-seinput warns instead of being silently ignored; the plotlytoselfband filters non-finite-CI rows (a NaN vertex split the polygon) and both renderers suppress an all-masked band.code_refsextended (note-prose only, no lifecycle change;test_refstays the staggered anchor per the single-path schema);docs/v4-design.md,docs/migration-4.0.md, and the REGISTRY M-146 note extended family-wide with SyntheticControl and TROP posture Notes; theplot_group_effectszero-SE Note extended to the dose-response twin (including the deliberate alpha-semantics divergence). The audit TODO row is closed; the summaries' pre-existing truncating{level}%idiom (~19 sites incl. the shipped staggered family) is deliberately unchanged and recorded as its own TODO row.Methodology references (required if estimator / math changes)
docs/methodology/REGISTRY.mdplot_group_effects)Validation
tests/test_estimators.py(family contract +_require_fit_alphaunit tests),tests/test_triple_diff.py,tests/test_trop.py,tests/test_continuous_did.py,tests/test_methodology_synthetic_control.py,tests/test_spillover.py,tests/test_visualization_new.py,tests/test_visualization_plotly.py(new dose-response class),tests/test_v4_matrix.py(commentary wording)diff_diff testsand mypy at zero errors; pre/post oracle capture of all seven summaries byte-identical on default callsSecurity / privacy