Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions DEFERRED.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ provenance and AI-review deviation-documentation: a row here (or in

| Issue | Location | PR | Priority |
|-------|----------|----|----------|
| Chang (2020) Case 3 (multilevel treatment intensity) DML estimator: the paper prints no Case 3 regularity assumptions or variance theorem (Theorems 2/4 cover Cases 1-2 only; "same arguments" p. 13), the printed multilevel algorithm carries typos (review Gaps items 5-6), and no oracle exists (`DoubleMLDIDMulti` is staggered TIMING, not intensity) — needs the variance derivation + overlap assumptions before implementation | `docs/methodology/papers/chang-2020-review.md` | dml-b0 | Low |
| Chang (2020) Case 3 (multilevel treatment intensity) DML estimator: the paper prints no Case 3 regularity assumptions or variance theorem (Theorems 2/4 cover Cases 1-2 only; "same arguments" p. 13), the printed multilevel algorithm carries typos (review Gaps items 5-6), and no oracle exists (`DoubleMLDIDMulti` is staggered TIMING, not intensity) — needs the variance derivation + overlap assumptions before implementation. Published-version cross-check (2026-08-30) CONFIRMS the blocker in the version of record: printed Appendix A carries the same score/algorithm (same typos) with no Case 3 regularity/rate assumptions and no variance theorem | `docs/methodology/papers/chang-2020-review.md` | dml-b0 | Low |
| LWDiD PSM matching variance: implement the Abadie-Imbens (2006) matching variance (matched-control reuse + first-stage matching uncertainty) so `estimation_method='psm'` can report valid inference instead of the current fail-closed NaN tuple (point retained; naive var(diffs)/n was invalid under with-replacement reuse) | `diff_diff/lwdid.py` | #588 | Low |
| LWDiD cohort-relative sensitivity exclusions: `robustness_pre_periods` / `sensitivity_no_anticipation` currently reject multi-cohort staggered inputs because their exclusion windows are defined relative to the EARLIEST adoption (later cohorts' own pre periods fall inside the global post window and survive every restriction, mislabeling the specification). Supporting staggered inputs needs per-cohort window semantics (exclude the last k periods of each cohort's own `t < g` window before its transformation), which the current row-subset design cannot express — a per-cohort masking derivation + its aggregation contract | `diff_diff/lwdid_sensitivity.py` | #588 | Low |
| LWDiD estimation-time anticipation offset (CS-parity gap): no `anticipation=` parameter exists — no-anticipation is hard-assumed and the only surface is the single-cohort `sensitivity_no_anticipation` refit diagnostic. The LW device for anticipation is the anchor-window shift — drop the last k pre-treatment periods from the transformation/anchor window (`Ybar_{i,S0}` with `S0 < S-1`, per the REGISTRY LWDiD Anticipation edge case; NOT a cohort recode g → g − k) — and exposing an `anticipation=` parameter needs that derivation checked against LW 2025 NAC/CNAS before implementation | `diff_diff/lwdid.py` | LWDiD release audit | Low |
Expand Down Expand Up @@ -53,7 +53,6 @@ exists but parity can't be verified without a local toolchain.

| Issue | Location | PR | Priority |
|-------|----------|----|----------|
| Chang (2020) published-version cross-check of the arXiv `p_hat_k` typo: the library adopted the global treated-share convention on DoubleML parity evidence (REGISTRY DML-infrastructure Note; spike `benchmarks/doubleml/chang_case1_parity.py`), but the published *Econometrics Journal* text (paywalled) has not been checked against the arXiv printing to see whether it resolves the algorithm-vs-proof contradiction | `docs/methodology/papers/chang-2020-review.md` | dml-b0 | Low |
| `StaggeredTripleDifference` R cross-validation (the engine is shared with `TripleDifference`'s staggered mode since 3(b), so this covers both surfaces): CSV fixtures not committed (gitignored); tests skip without local R + `triplediff`. Commit fixtures or generate deterministically. | `tests/test_methodology_staggered_triple_diff.py` | #245 | Medium |
| Staggered DDD R parity (both surfaces - one shared engine since 3(b)): benchmark only tests the no-covariate path (`xformla=~1`). Add covariate-adjusted scenarios + aggregation-SE parity assertions. | `benchmarks/R/benchmark_staggered_triplediff.R` | #245 | Medium |
| Staggered DDD per-cohort group-effect SEs include WIF (both surfaces - one shared engine since 3(b)) (conservative vs R's `wif=NULL`); documented in REGISTRY. Could override the mixin for an exact R match (verification needs R `triplediff`). | `_staggered_triple_diff_engine.py` | #245 | Low |
Expand Down
35 changes: 31 additions & 4 deletions docs/methodology/REGISTRY.md
Original file line number Diff line number Diff line change
Expand Up @@ -779,7 +779,9 @@ DR score is self-normalized; the two are not interchangeable"):

- **Note:** p̂ convention (documented deviation). Chang's arXiv text is
internally contradictory about the fold-level scalar nuisance: the printed
algorithms show `p̂_k = (1/n)Σ_{i∈I_k^c} D_i` (an invalid normalizer) while
algorithms show `p̂_k = (1/n)Σ_{i∈I_k^c} D_i` — an invalid normalizer for
K > 2 (with n = N/K and |I_k^c| = N − n; at K = 2 the two sizes coincide,
so the printed formula IS a valid auxiliary-sample mean there) — while
the Theorem 1-2 proofs use fold means over `I_k` (see the review's Gaps).
The score functions take `p_hat` as a CALLER-supplied argument; the LIBRARY
convention — adopted for `DMLDiD` and used in all fixtures — is the
Expand All @@ -789,8 +791,33 @@ DR score is self-normalized; the two are not interchangeable"):
5.6e-17 vs `DoubleMLDID(score="observational",
in_sample_normalization=False)` under doubleml 0.11.4 with shared folds)
and asymptotic equivalence to both printed readings. The published
*Econometrics Journal* version has not yet been cross-checked against the
arXiv typo (tracked in DEFERRED.md).
*Econometrics Journal* version was cross-checked 2026-08-30 (doi:
10.1093/ectj/utaa001): Definition 3.1(b) prints the SAME
`p̂_k = (1/n)Σ_{i∈I_k^c} D_i` formula (and `λ̂_k` likewise) — the
printed-formula side of the contradiction persists in the version of
record, while the proofs are online-supplement-only and remain
unchecked (the arXiv-v3 proofs are the verified proof-side source). The
author's replication code (github.com/NengChiehChang/Diff-in-Diff,
commit `e93773d1c5ed`, inspected 2026-08-30; it fixes k = 2, so its own
printed-formula reading is self-consistent) uses MIXED conventions:
`p̂_k = mean(D)` over the propensity-TRIMMED evaluation fold — the
proofs' `E_{n,k}[D]` form, modulo trimming (`RO_Algorithm.R:70`,
`RCS_Algorithm.R:78`) — while `λ̂_k = mean(T)` over the AUXILIARY
sample (`RCS_Algorithm.R:58`). The √N-equivalence claim applies only to
the UNTRIMMED readings (main-fold, auxiliary-sample, global share). The
code's two extra operations are distinct and separately caveated:
(i) it DROPS rows with fitted ĝ outside (0.05, 0.95) before averaging,
which targets a propensity-trimmed-population ATT unless the trimming
is asymptotically inactive; (ii) the RCS score additionally applies a
one-sided REALIZED-SCORE-dependent deletion (`s = s[s < abs(min(s))]`,
`RCS_Algorithm.R:79`), an outcome-dependent truncation with NO
established estimand or covering variance theorem. The code is
consulted here ONLY to identify the scalar-normalizer placement, never
as an oracle for the complete estimator. The library
KEEPS the global full-sample share (exact DoubleML parity, documented
deviation — now from the author's code as well as the printed
algorithm; full cross-check log in
`docs/methodology/papers/chang-2020-review.md`).

**Cross-fitting (`_crossfit.py`)** — DML2-style unit-level K-fold machinery:
`assign_folds` captures the generator's bit-generator state BY VALUE before
Expand Down Expand Up @@ -864,7 +891,7 @@ double-weight).
`benchmarks/doubleml/chang_rcs_characterization.py` documents the
divergence for the shipped Case 2 lane), `DoubleMLDIDMulti` for Chang
Case 3.
- R: none for Chang's estimator; `DRDID::drdid_panel` for the SZ score.
- R: the author's replication code (github.com/NengChiehChang/Diff-in-Diff, commit `e93773d1c5ed`, k=2 scripts `RO_Algorithm.R`/`RCS_Algorithm.R`) — an EQUATION-LEVEL reference only, not a numerical oracle (mixed p̂/λ̂ normalization, propensity row-dropping, an RCS score-dependent deletion, and no packaged variance conventions; see the p̂ convention Note); `DRDID::drdid_panel` for the SZ score.

---

Expand Down
Loading
Loading