Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ Related tracking surfaces:

| Issue | Location | Origin | Effort | Priority |
|-------|----------|--------|--------|----------|
| Non-staggered `summary(alpha=...)` mislabel audit (the M-146 staggered-family guard's remaining siblings): `results.py:199` and `continuous_did_results.py:250` are VERIFIED mislabels (requested-alpha `{conf_level}%` header over stored fit-time `conf_int` — dual-review round 4); `results.py:792/:1290`, `synthetic_control_results.py:499`, `triple_diff.py:163`, `trop_results.py:207` carry the same `alpha = alpha or self.alpha` idiom and still need verification (a genuine recomputation from the SE at the requested alpha is legitimate on analytical fits). Apply the shared `results_base._require_fit_alpha` guard (or a real recomputation) per site. Include `plot_dose_response`'s raw-DataFrame CI reconstruction (`diff_diff/visualization/_continuous.py:105-108`): zero-SE rows draw a zero-width band — a weaker instance (user-supplied `se` column, no stored inference to contradict) of the M-146-adjacent zero-SE gate | `diff_diff/results.py`, `diff_diff/continuous_did_results.py`, `diff_diff/synthetic_control_results.py`, `diff_diff/triple_diff.py`, `diff_diff/trop_results.py`, `diff_diff/visualization/_continuous.py` | #794 follow-ups review | Mid | Medium |
| Consolidate the remaining estimator-entangled DR/logit score variants (`staggered.py::_doubly_robust` + RC twins, `triple_diff.py`, `lwdid.py`, `wooldridge.py`) onto the shared `_dr_scores.py` module, each migration with its own committed oracle capture (the ContinuousDiD lift's two-tier pattern in `tests/test_dr_scores.py`); and add a ridge vcov path to `solve_ridge` if an estimator ever needs analytical ridge inference | `diff_diff/_dr_scores.py` | dml-b0 | Mid | Low |
| hc2/hc2_bm floor `1 - h_ii` at 1e-10 in the shared leverage meat, fabricating finite (if inflated) variances for leverage-one observations - hc3 now fails closed there (LWDiD fix wave) but the pre-existing hc2 family behavior is released surface; decide fail-closed vs keep-floor for hc2/hc2_bm | `diff_diff/linalg.py` | #588 | Quick | Low |
| Numeric between-period cohorts (e.g. `first_treat=4.5` with integer times) are rejected by LWDiD while CallawaySantAnna estimates them and LWDiD's own datetime/Period cohorts map to the next observed period — close the dtype asymmetry by adopting the next-observed-period mapping for numeric cohorts too (contract documented in REGISTRY cohort-encodings Note + `docs/api/lwdid.rst` Input Contract). Lands only after PR #588 merges | `diff_diff/lwdid.py` | #588 | Quick | Low |
Expand Down Expand Up @@ -58,6 +57,7 @@ Related tracking surfaces:
| `WooldridgeDiD` does not apply the W2025 Sec 5.4 `D_{G_max} x X` covariate normalization, and three sibling covariate rank deficiencies are pre-existing. Measured with the period range pinned and only the never-treated units toggled: (1) time-invariant `exovar` is absorbed by the unit FE, 4 of 26 columns, IDENTICALLY with and without never-treated units; (2) `xgvar`'s cell x covariate block, 19 of 41, identical on both panels; (3) `xtvar` under `demean_covariates=False` does exhibit the `sum_g D_g x = x` dependency that the default demeaning removes; (4) the newly-reachable case -- time-VARYING data passed through `exovar`, which its own docstring reserves for time-invariant covariates -- where the paper's `dT_i` rule would give a deterministic `D_{G_max} x X` drop instead of QR's arbitrary pick (coefficients unaffected, `1.35e-14`; `rank_deficient_action="error"` raises). REGISTRY's narrowed Sec 5.4 note cross-references this row. **Trap for whoever takes it:** `xtvar` under the DEFAULT `demean_covariates=True` is FULL RANK -- the raw block carries demeaned values while `D_g x X` carries raw ones -- and forcing the drop there moves `overall_att` 1.11903 -> 1.46269. Pinned as-is by `TestComparisonSupportFiltering::test_cells_derived_groups_did_not_leak_into_the_design`. | `diff_diff/wooldridge.py` | #729-followup | Heavy | Medium |
| `WooldridgeDiD.n_control_units` counts never-treated UNITS on `control_group="never_treated"` regardless of method, but on the nonlinear paths (`logit`/`poisson`) treated units' pre-treatment rows ARE the identifying comparison -- only the OLS path absorbs them into their own cells. So the reported count under-states the comparison pool exactly where the REGISTRY control-pool asymmetry note applies. Widen to `not_yet_treated or (never_treated and method != "ols")`, or document the count as never-treated-units-by-definition. Behavior is PRE-EXISTING; documented for now in the REGISTRY control-pool Note rather than changed, because widening moves a public results field and wants its own ledger row and test matrix. | `diff_diff/wooldridge.py` | #729-followup | Mid | Low |
| `WooldridgeDiD` has no opt-out for comparison-support period filtering: a user who would rather see the refusal than a reduced sample cannot ask for it. Adding one means a constructor parameter (`get_params`/`set_params` propagation, transactional validation), a ledger row, and a test matrix across both predicate branches and all three `rank_deficient_action` modes -- deliberately out of scope for the change that introduced the filter. The always-on warning is the interim answer. | `diff_diff/wooldridge.py` | #729-followup | Mid | Low |
| Fractional confidence-level display unification: every `summary()` header computes `int((1 - alpha) * 100)` (~19 truncating sites incl. the shipped M-146 staggered family), so a fit `alpha=0.025` prints "97% Confidence Interval" over a 97.5% interval; `plot_dose_response`'s band label now renders exact fractional coverage via its `_coverage_label` helper - unify the summary surfaces on the same formatter (cross-family display change: moves every summary golden/doctest that pins a header, so it wants one sweep with its own test recapture, not a per-estimator drip) | `diff_diff/results_base.py`, `diff_diff/results.py` | alpha-guard review | Quick | Low |

### Performance

Expand Down
20 changes: 20 additions & 0 deletions changelog.d/20260830-alpha-mislabel-guard.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
### Behavioral Changes
- **`summary(alpha=...)` never recomputes or relabels stored inference,
family-wide** ([M-146] completion): a non-fit `alpha` now raises `ValueError`
at seven more sites - `DiDResults` (and `SpilloverDiDResults` by
inheritance), `MultiPeriodDiDResults`, `SyntheticDiDResults`,
`TripleDifferenceResults`, `TROPResults`, `ContinuousDiDResults` (all
previously printed a requested-alpha header over fit-time stored intervals),
and `SyntheticControlResults` (previously a silent no-op `alpha`); `alpha=0.0`,
previously swallowed by a falsy-`or` default, raises too. Re-fit at the
desired alpha instead.
- **`plot_dose_response` honest bands and labels**: DataFrame-`se` input masks
non-positive/non-finite `se` rows from the confidence band with a warning
(previously a zero-SE row drew a finite zero-width band) and validates
`alpha` strictly inside (0, 1); the band legend is alpha-derived on the `se`
branch, `results.alpha`-derived on `results=` input, and the level-free
"CI" for bare-curve/explicit-CI input (previously hard-coded "95% CI"
regardless of the requested alpha); an explicitly passed `alpha` on
non-`se` input warns instead of being silently ignored; the plotly band
polygon filters non-finite-CI rows (a NaN vertex previously mangled the
`toself` band) and both renderers suppress an all-masked band.
20 changes: 17 additions & 3 deletions diff_diff/continuous_did_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,12 @@
from diff_diff.aggregation import AggregationMixin, AggregationResult
from diff_diff.continuous_did_aggregation import _ContinuousDiDAggregationMixin
from diff_diff.results import _format_survey_block, _get_significance_stars
from diff_diff.results_base import BaseResults, build_event_study_surface
from diff_diff.results_base import (
_SUMMARY_ALPHA_MESSAGE,
BaseResults,
_require_fit_alpha,
build_event_study_surface,
)
from diff_diff.utils import safe_inference

__all__ = ["ContinuousDiDResults", "DoseResponseCurve"]
Expand Down Expand Up @@ -258,8 +263,17 @@ def coef_var(self) -> float:
return self.overall_att_se / abs(self.overall_att)

def summary(self, alpha: Optional[float] = None) -> str:
"""Generate formatted summary."""
alpha = alpha or self.alpha
"""Generate formatted summary.

Parameters
----------
alpha : float, optional
Accepted for signature uniformity. The stored intervals were
computed at fit time; a value different from the stored
``alpha`` raises ValueError rather than silently recomputing
or relabeling. Re-fit at the desired alpha instead.
"""
alpha = _require_fit_alpha(alpha, self.alpha, message=_SUMMARY_ALPHA_MESSAGE)
conf_level = int((1 - alpha) * 100)
w = 85

Expand Down
33 changes: 22 additions & 11 deletions diff_diff/results.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
import numpy as np
import pandas as pd

from diff_diff.results_base import BaseResults
from diff_diff.results_base import _SUMMARY_ALPHA_MESSAGE, BaseResults, _require_fit_alpha


def _format_survey_block(sm, width: int) -> list:
Expand Down Expand Up @@ -188,15 +188,17 @@ def summary(self, alpha: Optional[float] = None) -> str:
Parameters
----------
alpha : float, optional
Significance level for confidence intervals. Defaults to the
alpha used during estimation.
Accepted for signature uniformity. The stored intervals were
computed at fit time; a value different from the stored
``alpha`` raises ValueError rather than silently recomputing
or relabeling. Re-fit at the desired alpha instead.

Returns
-------
str
Formatted summary table.
"""
alpha = alpha or self.alpha
alpha = _require_fit_alpha(alpha, self.alpha, message=_SUMMARY_ALPHA_MESSAGE)
conf_level = int((1 - alpha) * 100)

lines = [
Expand Down Expand Up @@ -473,7 +475,12 @@ class SpilloverDiDResults(DiDResults):

def summary(self, alpha: Optional[float] = None) -> str:
"""Extended summary with ATT row, per-event-time direct block, and
per-(ring, event-time) spillover block."""
per-(ring, event-time) spillover block.

``alpha`` must equal the fit alpha (None = fit alpha): the inherited
guard raises on any other value rather than relabeling stored
intervals.
"""
base = super().summary(alpha=alpha)
insert_blocks: List[str] = []

Expand Down Expand Up @@ -781,15 +788,17 @@ def summary(self, alpha: Optional[float] = None) -> str:
Parameters
----------
alpha : float, optional
Significance level for confidence intervals. Defaults to the
alpha used during estimation.
Accepted for signature uniformity. The stored intervals were
computed at fit time; a value different from the stored
``alpha`` raises ValueError rather than silently recomputing
or relabeling. Re-fit at the desired alpha instead.

Returns
-------
str
Formatted summary table.
"""
alpha = alpha or self.alpha
alpha = _require_fit_alpha(alpha, self.alpha, message=_SUMMARY_ALPHA_MESSAGE)
conf_level = int((1 - alpha) * 100)

lines = [
Expand Down Expand Up @@ -1279,15 +1288,17 @@ def summary(self, alpha: Optional[float] = None) -> str:
Parameters
----------
alpha : float, optional
Significance level for confidence intervals. Defaults to the
alpha used during estimation.
Accepted for signature uniformity. The stored intervals were
computed at fit time; a value different from the stored
``alpha`` raises ValueError rather than silently recomputing
or relabeling. Re-fit at the desired alpha instead.

Returns
-------
str
Formatted summary table.
"""
alpha = alpha or self.alpha
alpha = _require_fit_alpha(alpha, self.alpha, message=_SUMMARY_ALPHA_MESSAGE)
conf_level = int((1 - alpha) * 100)

lines = [
Expand Down
29 changes: 24 additions & 5 deletions diff_diff/results_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,16 +92,35 @@ class BaseResults:
__slots__ = ()


def _require_fit_alpha(alpha: Optional[float], fit_alpha: float) -> float:
# Family-wide non-staggered override for _require_fit_alpha: the staggered
# default's rationale tail (bootstrap percentile intervals) is false for the
# analytic-default siblings, so they share this accurate generic wording.
# May use exactly the {alpha}/{fit_alpha} placeholders (helper-formatted).
_SUMMARY_ALPHA_MESSAGE = (
"This result stores intervals computed at fit time (alpha={fit_alpha}); "
"summary() never recomputes or relabels stored inference "
"(requested alpha={alpha}); re-fit with the desired alpha."
)


def _require_fit_alpha(
alpha: Optional[float], fit_alpha: float, message: Optional[str] = None
) -> float:
"""Reject a non-fit ``alpha``; summaries never recompute stored inference.

Shared by the staggered-family ``summary()`` methods (CallawaySantAnna
and siblings): stored intervals were computed at fit time, and bootstrap
percentile intervals cannot be reconstructed from the reported SE, so a
The family-wide ``summary()`` contract (staggered and non-staggered
results classes alike): stored intervals were computed at fit time, so a
requested alpha other than the fit alpha raises instead of silently
relabeling the confidence-interval header.
relabeling the confidence-interval header. ``message`` replaces the
ENTIRE default error text for classes where the default rationale would
be inaccurate; overrides may use exactly the named placeholders
``{alpha}`` and ``{fit_alpha}`` (this helper formats them — escape any
other literal brace as ``{{``/``}}``) and must keep the phrase
"never recomputes" (the family test pin).
"""
if alpha is not None and alpha != fit_alpha:
if message is not None:
raise ValueError(message.format(alpha=alpha, fit_alpha=fit_alpha))
raise ValueError(
f"This result stores intervals computed at alpha={fit_alpha}; "
f"summary() never recomputes or relabels stored inference "
Expand Down
5 changes: 4 additions & 1 deletion diff_diff/synthetic_control.py
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,10 @@ class SyntheticControl(BaseEstimator):
standard deviation across donors+treated (ddof=1), matching R ``Synth``.
``"none"`` is a deviation from R (see REGISTRY).
alpha : float, default 0.05
Significance level recorded for downstream (placebo) inference.
Significance level recorded on the results for signature
uniformity. Placebo and test-inversion entry points take their own
independent ``gamma``/``alpha`` arguments; the results
``summary(alpha=)`` accepts only this stored value.
seed : int, optional
Seed for the multistart random (Dirichlet) starting points.
v_cv_t0 : int, optional
Expand Down
29 changes: 25 additions & 4 deletions diff_diff/synthetic_control_results.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
import pandas as pd

from diff_diff.results import _format_survey_block, _get_significance_stars
from diff_diff.results_base import BaseResults
from diff_diff.results_base import BaseResults, _require_fit_alpha

__all__ = ["SyntheticControlResults"]

Expand Down Expand Up @@ -203,7 +203,11 @@ class SyntheticControlResults(BaseResults):
standardize : str
``"std"`` (per-row SD scaling) or ``"none"``.
alpha : float
Significance level recorded for downstream (placebo) inference.
Significance level recorded at fit time for signature uniformity.
Placebo and test-inversion entry points take their own independent
``gamma``/``alpha`` arguments; this field feeds only
``is_significant`` (always False here, since the Wald fields are
NaN by design).
rmspe_ratio : float
The treated unit's post/pre RMSPE ratio = ``sqrt(MSPE_post / MSPE_pre)`` —
the in-space placebo test statistic (ADH 2010 §2.4), computed at fit time.
Expand Down Expand Up @@ -489,14 +493,31 @@ def summary(self, alpha: Optional[float] = None) -> str:
Parameters
----------
alpha : float, optional
Significance level; defaults to the alpha used during estimation.
Accepted for signature uniformity; must equal the stored fit
``alpha`` (None = fit alpha) or ValueError is raised. This
summary reports no alpha-based interval (the Wald inference
fields are NaN by design); the displayed confidence set is
keyed on its own stored ``gamma``.

Returns
-------
str
Formatted summary table.
"""
alpha = alpha or self.alpha
# Tailored message: this class stores NO alpha-based interval (Wald
# fields NaN by design; the confidence set is gamma-keyed), so the
# shared messages' "stores intervals computed at alpha=..." would be
# factually wrong here.
alpha = _require_fit_alpha(
alpha,
self.alpha,
message=(
"This summary reports no alpha-based interval (Wald inference "
"fields are NaN by design; the displayed confidence set uses "
"its own stored gamma) and never recomputes stored inference "
"(requested alpha={alpha}, fit alpha={fit_alpha})."
),
)

n_top = min(5, len(self.donor_weights))
top_donors = sorted(self.donor_weights.items(), key=lambda kv: kv[1], reverse=True)[:n_top]
Expand Down
10 changes: 6 additions & 4 deletions diff_diff/triple_diff.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
from diff_diff._staggered_triple_diff_engine import _StaggeredTripleDiffEngineMixin
from diff_diff.linalg import _rank_guarded_inv, solve_logit, solve_ols
from diff_diff.results import _format_survey_block, _get_significance_stars
from diff_diff.results_base import BaseResults
from diff_diff.results_base import _SUMMARY_ALPHA_MESSAGE, BaseResults, _require_fit_alpha
from diff_diff.staggered_aggregation import CallawaySantAnnaAggregationMixin
from diff_diff.staggered_bootstrap import CallawaySantAnnaBootstrapMixin
from diff_diff.staggered_triple_diff_results import StaggeredTripleDiffResults
Expand Down Expand Up @@ -153,15 +153,17 @@ def summary(self, alpha: Optional[float] = None) -> str:
Parameters
----------
alpha : float, optional
Significance level for confidence intervals. Defaults to the
alpha used during estimation.
Accepted for signature uniformity. The stored intervals were
computed at fit time; a value different from the stored
``alpha`` raises ValueError rather than silently recomputing
or relabeling. Re-fit at the desired alpha instead.

Returns
-------
str
Formatted summary table.
"""
alpha = alpha or self.alpha
alpha = _require_fit_alpha(alpha, self.alpha, message=_SUMMARY_ALPHA_MESSAGE)
conf_level = int((1 - alpha) * 100)

lines = [
Expand Down
Loading
Loading