feat(v4): item 7a of the 4.0 cut - flip the df_convention default to "cluster" (M-004..M-006, M-128..M-131)

DEFERRED.md

@@ -8,10 +8,9 @@ provenance and AI-review deviation-documentation: a row here (or in
 - Shippable work lives in [TODO.md](TODO.md); the public feature roadmap in
   [ROADMAP.md](ROADMAP.md); monitoring and current-state notes in
   [docs/dev-status.md](docs/dev-status.md).
-- Version-gated lifecycle items (deprecated-kwarg removals, v4 default flips such as
-  `SyntheticDiD` `lambda_reg`/`zeta` and the clustered-df convention) are canonically
-  tracked in the CI-enforced ledger `docs/v4-deprecations.yaml`; rows here cross-link
-  their `M-xxx` ids where applicable.
+- Version-gated lifecycle items (deprecated-kwarg removals and default flips) are
+  canonically tracked in the CI-enforced ledger `docs/v4-deprecations.yaml`; rows here
+  cross-link their `M-xxx` ids where applicable.
 - The `PR` column points to the originating PR number or review tag.
 
 ---
@@ -130,7 +129,8 @@ here carry the remaining implementation work and cross-link their `M-xxx` ids.
 
 | Issue | Location | PR | Priority |
 |-------|----------|----|----------|
-| Clustered inference df **default flip at v4** (lifecycle + roster: docs/v4-deprecations.yaml M-004..M-006 + M-128..M-131; knob semantics: v4-design §7). Implementation work the ledger rows do not carry: the migration note, and flipping the test expectations that pin the current default (`TestDfConvention`, `test_moderate_t_pins_residual_df_convention`, the per-estimator knob suites). | `diff_diff/linalg.py::LinearRegression`, `diff_diff/estimators.py`, `diff_diff/twfe.py`, `diff_diff/sun_abraham.py`, `diff_diff/wooldridge.py`, `diff_diff/stacked_did.py`, `diff_diff/imputation.py` | — | Medium |
+
+No open rows: the clustered-df default flip shipped with the 4.0 cut.
 
 ## Decision record — won't-fix / waived
 

TODO.md

@@ -85,6 +85,7 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
 
 | Issue | Location | Origin | Effort | Priority |
 |-------|----------|--------|--------|----------|
+| Re-execute the nine tutorials whose committed outputs still show the 3.x residual-df p-values/CIs after the `df_convention` default moved to `"cluster"` (t(G-1)): `01_basic_did`, `02_staggered_did`, `04_parallel_trends`, `05_honest_did`, `07_pretrends_power`, `09_real_world_examples`, `11_imputation_did`, `13_stacked_did`, `16_wooldridge_etwfe` (not `16_survey_did`: the survey df wins). Run under `DIFF_DIFF_BACKEND=rust` with the per-worktree kernel, reconcile any prose that restates a moved number, and `--nbmake` each; scheduled for the 4.0 release PR (`docs/v4-design.md` section 9 item 10) together with any item-7b/8/9 movers (24 was refreshed in #879; 23 belongs to 7b). | `docs/tutorials/*.ipynb` | #879 | Mid | Medium |
 | TWFE event-study one-way HC2-BM absorbed-FE R parity against the `mpd_absorbed_fe_did` golden (`clubSandwich::vcovCR(cluster=1:n, type="CR2")`): the within spec with hc2_bm auto-clusters at unit, so the one-way lane needs an explicit singleton-cluster column (`cluster=<row id>` + hc2_bm = CR2 with G=n, PT2018 §3.3) once the G=n CR2 path is verified; the hc2 lane is pinned by `TestTWFEEventStudyAbsorbedFERParity` and the clustered hc2_bm per-period lane by `test_twfe_event_study_cluster_hc2_bm_per_period_df_matches_clubsandwich`; the MultiPeriodDiD lanes died at the 4.0 cut (PR 4c-ii) | `tests/test_estimators_vcov_type.py` | #v4-4c-i | Mid | Medium |
 | TWFE event-study power profile: `simulate_power`/`simulate_mde`/`simulate_sample_size` have no `_EstimatorProfile` for `TwoWayFixedEffects.fit(event_study=True)` since the MultiPeriodDiD registry entry went with the class (PR 4c-ii); `EventStudyResults` carries no scalar headline, so the profile needs a design (post-period average via the `tests/helpers/event_study_average.py` arithmetic, or a named period) plus its `_SURVEY_FIT_BUILDERS` twin | `diff_diff/power.py` | #v4-4c-ii | Mid | Low |
 | Clustered hc2_bm Bell-McCaffrey DOF guard is batch-relative (`linalg._cr2_bm_dof_inner`: a contrast is NaN'd when its scaled max\|B\| falls below 1e-10 of the batch maximum), so the event-study core keeps the otherwise-unused post-period-average contrast column as the anchor to hold the 3.x per-period NaN masks bit-identical (PR 4c-ii, user decision 2026-10-25). Reproduction: `x_nc = treated * 1[period >= 4] + 3e-6 * default_rng(seed).normal(size=n)` as a covariate on the oracle panel flips the mask between `[eye(k), avg]` and `eye(k)` on 20/20 seeds (cond(X) ~ 9e5; 1e-7 never flips, <= 3e-8 rank-drops) - too BLAS-sensitive for a committed oracle, hence the structural pin `test_hc2_bm_contrast_batch_keeps_post_average_anchor`. Decide whether the guard should use a batch-independent anchor (the treatment contrast, or the absolute floor only) and then drop the column | `diff_diff/estimators.py`, `diff_diff/linalg.py` | #v4-4c-ii | Mid | Medium |

changelog.d/20261028-v4-pr7a-df-convention-flip.md

@@ -0,0 +1,7 @@
+### Breaking Changes
+
+- **4.0 cut, `df_convention` default flip (`M-004`, `M-005`, `M-006`, `M-128`, `M-129`, `M-130`, `M-131`).** The analytical tail-df convention for CLUSTERED fits defaults to `"cluster"` (the Stata/fixest `t(G - 1)` reference) instead of `"residual"` (the fitted residual df `t(n - K_full)`) on `DifferenceInDifferences`, `TwoWayFixedEffects`, `LinearRegression`, `SunAbraham`, `WooldridgeDiD` (OLS arms), `StackedDiD` and `ImputationDiD` (pretrends leads); `LPDiD` already defaulted to `"cluster"` at 3.9. What moves: p-values and confidence intervals on clustered analytical fits - `TwoWayFixedEffects` (static, and event-study with `unit=`), `SunAbraham`, `WooldridgeDiD` OLS and `StackedDiD` cluster at `unit` by default, so their default fits move; `ImputationDiD` moves on the `pretrends=True` lead rows; `DifferenceInDifferences` / `LinearRegression` move only with an explicit `cluster=` / `cluster_ids=`; the analytical lanes inside bootstrapped fits (`WooldridgeDiD` per-cell inference and post-fit `aggregate()`, `ImputationDiD` bootstrapped leads) move too, as do the fake-timing / fake-group placebo diagnostics and the power simulators that relay those p-values. What does not: point estimates, standard errors and t-statistics; survey-design fits (survey df keeps precedence); `vcov_type="hc2_bm"` (Bell-McCaffrey DOF keep precedence); bootstrap percentile p-values / CIs; unclustered and Conley fits; the BJS overall/post inference of `ImputationDiD`; `WooldridgeDiD` logit/poisson arms; permutation and leave-one-out diagnostics. `df_convention="residual"` restores the 3.x inference bit-for-bit; on the surfaces where the knob is a documented no-op (non-`pretrends` `ImputationDiD`, `WooldridgeDiD` logit/poisson) the inert-config `UserWarning` now fires for an explicit `"residual"` rather than for an explicit `"cluster"`. `SyntheticDiD` inherits the flipped attribute from `DifferenceInDifferences` without reading it (no behaviour change).
+
+### Internal
+
+- Ledger rows `M-004`..`M-006` and `M-128`..`M-131` flip to `done` with `tests/test_v4_inference_defaults.py` as their `test_ref`; the spec's section 7 / item 7 SHIPPED notes and item 10 now record the deferred bulk re-execution of the nine tutorials whose clustered-default outputs moved (24 re-executed here if its outputs moved); the REGISTRY TwoWayFixedEffects clustered-CR1 inference-df note is relabelled a plain Note (the default now matches fixest) and the SunAbraham / ImputationDiD / StackedDiD / WooldridgeDiD / LPDiD tail-df notes read the 4.0 default; `TestDfConvention`, `test_moderate_t_pins_cluster_df_convention` and the per-estimator knob suites pin the G-1 default against explicit `"residual"` baselines (every default-vs-`"cluster"` comparison became default-vs-`"residual"` so none passes vacuously); `test_ci_matches_r_twfe` tightens to machine precision; the variance-conventions audit matrix keeps its residual-df rows as explicit `"residual"` arms (the K_full accounting pins) and gains eight G-1 `_default_df` twins (three in the fast set), with `docs/methodology/variance-conventions.md` regenerated; the committed TWFE event-study oracle lanes `pooled_cluster_unit` / `within_cluster_unit` and the ImputationDiD `pretrends_all` contract lanes name the `"residual"` convention they were captured under (metadata-only, no oracle number touched); the `DEFERRED.md` Version-gated (v4) row is retired.

diff_diff/estimators.py

@@ -158,22 +158,24 @@ class DifferenceInDifferences(BaseEstimator):
         path — absent ``cluster=``, pure Conley spatial HAC applies.
         ``survey_design=`` + Conley and ``inference='wild_bootstrap'`` +
         Conley both raise ``NotImplementedError``.
-    df_convention : {"residual", "cluster", "normal"}, default "residual"
+    df_convention : {"residual", "cluster", "normal"}, default "cluster"
         Degrees-of-freedom convention for analytical t-statistics, p-values,
-        and CIs. ``"residual"`` (default) uses the fitted residual df
-        (``n − K_full``); ``"cluster"`` uses the Stata/fixest cluster df
-        ``G − 1`` on clustered fits — it has no effect on unclustered fits
+        and CIs. ``"cluster"`` (default since 4.0) uses the Stata/fixest
+        cluster df ``G − 1`` on clustered fits — it has no effect on unclustered fits
         or on ``vcov_type="conley"`` (the combined Conley+cluster product
         kernel has no documented ``G − 1`` df reference and keeps the
-        residual df); ``"normal"`` deliberately uses normal-theory z
-        inference at the fallback level on every fit, clustered or not.
+        residual df); ``"residual"`` (the 3.x default) uses the fitted
+        residual df (``n − K_full``); ``"normal"`` deliberately uses
+        normal-theory z inference at the fallback level on every fit,
+        clustered or not.
         Applies only at the fallback level of the df resolution under every
         value: survey df and per-coefficient Bell-McCaffrey DOF
         (``vcov_type="hc2_bm"``) are more refined small-sample corrections
         and always take precedence. Point estimates, SEs, and t-statistics
         are unaffected — only the reference distribution changes. The
-        default flips to ``"cluster"`` at v4 (see the REGISTRY clustered-CR1
-        inference-df deviation note).
+        default flipped from ``"residual"`` to ``"cluster"`` at 4.0 (rows
+        M-004/M-005; pass ``df_convention="residual"`` to reproduce 3.x
+        inference; see the REGISTRY clustered-CR1 inference-df note).
 
     Attributes
     ----------
@@ -238,7 +240,7 @@ def __init__(
         conley_metric: str = "haversine",
         conley_kernel: str = "bartlett",
         conley_lag_cutoff: Optional[int] = None,
-        df_convention: str = "residual",
+        df_convention: str = "cluster",
     ):
         # Resolve/validate vcov_type via the shared helper so __init__ and
         # set_params use identical validation logic.
@@ -282,12 +284,11 @@ def __init__(
         # arrays are auto-derived from data[time].values + data[unit].values
         # at fit-time (panel estimators already take time/unit as column names).
         self.conley_lag_cutoff = conley_lag_cutoff
-        # Inference df convention for clustered analytical fits: "residual"
-        # (default; t/p/CI at the fitted residual df) or "cluster" (the
-        # Stata/fixest G-1 convention). Survey df and per-coefficient
-        # Bell-McCaffrey DOF always take precedence over either. The default
-        # flips to "cluster" at v4 (REGISTRY clustered-CR1 inference-df
-        # deviation note).
+        # Inference df convention for clustered analytical fits: "cluster"
+        # (default since 4.0; the Stata/fixest G-1 convention) or
+        # "residual" (the 3.x default; t/p/CI at the fitted residual df).
+        # Survey df and per-coefficient Bell-McCaffrey DOF always take
+        # precedence over either (REGISTRY clustered-CR1 inference-df note).
         self.df_convention = df_convention
 
         self.is_fitted_ = False
@@ -2009,7 +2010,8 @@ def _refit_mp_absorb(w_r):
             n_eff_df = int(round(np.sum(survey_weights)))
         df = n_eff_df - k_effective - n_absorbed_effects
         _df_cluster_knob_invalid = False
-        # Opt-in Stata/fixest cluster-df convention (df_convention="cluster"):
+        # Stata/fixest cluster-df convention (df_convention="cluster", the 4.0
+        # default):
         # the shared analytical df becomes G - 1 on a clustered fit. Placed
         # BEFORE the survey/replicate overrides below (which overwrite df, so
         # survey df keeps precedence) and upstream of the per-period BM-DOF

diff_diff/guides/llms-full.txt

@@ -60,7 +60,7 @@ DifferenceInDifferences(
     bootstrap_weights: str = "rademacher",  # "rademacher", "webb", or "mammen"
     seed: int | None = None,                # Random seed
     rank_deficient_action: str = "warn",    # "warn", "error", or "silent"
-    df_convention: str = "residual",        # Analytical t/p/CI df: "residual" (n-K, default), "cluster" (Stata/fixest G-1, clustered fits), or "normal" (deliberate z); survey df + hc2_bm BM-DOF keep precedence; default flips at v4
+    df_convention: str = "cluster",         # Analytical t/p/CI df: "cluster" (Stata/fixest G-1 on clustered fits; default since 4.0), "residual" (n-K; the 3.x default), or "normal" (deliberate z); survey df + hc2_bm BM-DOF keep precedence
 )
 ```
 
@@ -108,7 +108,7 @@ Two-Way Fixed Effects estimator for panel data. Inherits from DifferenceInDiffer
 TwoWayFixedEffects(
     cluster: str | None = None,   # Auto-clusters at unit level if None
     alpha: float = 0.05,
-    df_convention: str = "residual",  # Analytical t/p/CI df: "residual" (default), "cluster" (G-1), or "normal" (z); flips at v4
+    df_convention: str = "cluster",   # Analytical t/p/CI df: "cluster" (G-1; default since 4.0), "residual" (n-K; 3.x default), or "normal" (z)
 )
 ```
 
@@ -376,7 +376,7 @@ SunAbraham(
     conley_metric: str = "haversine",             # "haversine" | "euclidean"
     conley_kernel: str = "bartlett",              # "bartlett" | "uniform"
     conley_lag_cutoff: int | None = None,         # within-unit Bartlett max lag (0 = spatial-only)
-    df_convention: str = "residual",              # Analytical t/p/CI df for cells AND aggregates (3.9: aggregates share the cells' df — previously z): "residual" (default), "cluster" (G-1), "normal" (z); survey/BM DOF keep precedence; flips at v4
+    df_convention: str = "cluster",               # Analytical t/p/CI df for cells AND aggregates (3.9: aggregates share the cells' df — previously z): "cluster" (G-1; default since 4.0), "residual" (3.x default), "normal" (z); survey/BM DOF keep precedence
 )
 ```
 
@@ -424,7 +424,7 @@ ImputationDiD(
     aux_partition: str = "cohort_horizon",        # "cohort_horizon", "cohort", or "horizon"
     pretrends: bool = False,                     # Include pre-treatment horizons in event study
     leave_one_out: bool = False,                 # BJS 2024 App. A.9 leave-one-out finite-sample variance (larger, less-downward-biased SE)
-    df_convention: str = "residual",             # Pretrends lead-regression per-lead t/p/CI only (3.9: leads moved z -> t(residual)): "residual" (default), "cluster" (G-1), "normal" (z); BJS overall/post inference + Wald F knob-independent; flips at v4
+    df_convention: str = "cluster",              # Pretrends lead-regression per-lead t/p/CI only (3.9: leads moved z -> t(residual); 4.0: default G-1): "cluster" (default since 4.0), "residual" (3.x default), "normal" (z); BJS overall/post inference + Wald F knob-independent
 )
 ```
 
@@ -1104,7 +1104,7 @@ StackedDiD(
     rank_deficient_action: str = "warn",
     vcov_type: str = "hc1",                   # {"hc1","hc2_bm"}; classical/hc2 rejected (intrinsically clustered), conley deferred. survey_design=... requires hc1
     balance: str = "none",                    # {"none","entropy"}; "entropy" = CBWSDID covariate balancing (Ustyuzhanin 2026), requires fit(covariates=[...]) + weighting="aggregate", no survey_design
-    df_convention: str = "residual",          # Analytical t/p/CI df (3.9: hc1 lane moved z -> t(pooled residual df)): "residual" (default), "cluster" (G-1, positive-weight clusters), "normal" (z); BM/survey df keep precedence; flips at v4
+    df_convention: str = "cluster",           # Analytical t/p/CI df (3.9: hc1 lane moved z -> t(pooled residual df); 4.0: default G-1): "cluster" (G-1, positive-weight clusters; default since 4.0), "residual" (3.x default), "normal" (z); BM/survey df keep precedence
 )
 ```
 
@@ -1613,7 +1613,7 @@ WooldridgeDiD(
                                             # column is dropped per Sec 5.4, so cohort_trend_coefs
                                             # carries G-1 entries. Rejected with survey_design= and
                                             # with control_group="never_treated".
-    df_convention: str = "residual",        # OLS analytical t/p/CI df (3.9: default-hc1 arms moved z -> t(residual)): "residual" (default), "cluster" (G-1, hc1-clustered only), "normal" (z); survey/BM DOF keep precedence; GLM arms knob-independent (explicit non-default warns); flips at v4
+    df_convention: str = "cluster",         # OLS analytical t/p/CI df (3.9: hc1 arms moved z -> t(residual); 4.0: default G-1): "cluster" (G-1, hc1-clustered only; default since 4.0), "residual" (3.x default), "normal" (z); survey/BM DOF keep precedence; GLM arms knob-independent (explicit non-default warns)
     unsupported_period_action: str = "drop", # "drop" warns and filters; "error" refuses unsupported periods
 )
 ```
@@ -1834,7 +1834,7 @@ Returned by `SunAbraham.fit()`.
 | `cohort_effects` | `dict` | Cohort-level effects |
 | `df_convention` | `str | None` | The configured df convention echoed onto the results ("residual" | "cluster" | "normal"; 3.9) |
 | `inference_df` | `float | None` | The df the overall-ATT p/CI actually used (BM contrast df / survey df / knob-resolved fallback; None under bootstrap or "normal") |
-| `event_study_df` | `dict[int, float] | None` | Per-event-time df provenance (finite residual df on plain analytic fits since 3.9; None under bootstrap) |
+| `event_study_df` | `dict[int, float] | None` | Per-event-time df provenance (G-1 on plain clustered analytic fits under the 4.0 default; the finite residual df under `df_convention="residual"`; None under bootstrap) |
 
 **Methods:** `summary()`, `print_summary()`, `to_dataframe(level="event_study"|"cohort")`