diff --git a/DEFERRED.md b/DEFERRED.md index 605120d0..795c8e25 100644 --- a/DEFERRED.md +++ b/DEFERRED.md @@ -20,7 +20,7 @@ provenance and AI-review deviation-documentation: a row here (or in | Issue | Location | PR | Priority | |-------|----------|----|----------| -| Chang (2020) Case 3 (multilevel treatment intensity) DML estimator: the paper prints no Case 3 regularity assumptions or variance theorem (Theorems 2/4 cover Cases 1-2 only; "same arguments" p. 13), the printed multilevel algorithm carries typos (review Gaps items 5-6), and no oracle exists (`DoubleMLDIDMulti` is staggered TIMING, not intensity) — needs the variance derivation + overlap assumptions before implementation | `docs/methodology/papers/chang-2020-review.md` | dml-b0 | Low | +| Chang (2020) Case 3 (multilevel treatment intensity) DML estimator: the paper prints no Case 3 regularity assumptions or variance theorem (Theorems 2/4 cover Cases 1-2 only; "same arguments" p. 13), the printed multilevel algorithm carries typos (review Gaps items 5-6), and no oracle exists (`DoubleMLDIDMulti` is staggered TIMING, not intensity) — needs the variance derivation + overlap assumptions before implementation. Published-version cross-check (2026-08-30) CONFIRMS the blocker in the version of record: printed Appendix A carries the same score/algorithm (same typos) with no Case 3 regularity/rate assumptions and no variance theorem | `docs/methodology/papers/chang-2020-review.md` | dml-b0 | Low | | LWDiD PSM matching variance: implement the Abadie-Imbens (2006) matching variance (matched-control reuse + first-stage matching uncertainty) so `estimation_method='psm'` can report valid inference instead of the current fail-closed NaN tuple (point retained; naive var(diffs)/n was invalid under with-replacement reuse) | `diff_diff/lwdid.py` | #588 | Low | | LWDiD cohort-relative sensitivity exclusions: `robustness_pre_periods` / `sensitivity_no_anticipation` currently reject multi-cohort staggered inputs because their exclusion windows are defined relative to the EARLIEST adoption (later cohorts' own pre periods fall inside the global post window and survive every restriction, mislabeling the specification). Supporting staggered inputs needs per-cohort window semantics (exclude the last k periods of each cohort's own `t < g` window before its transformation), which the current row-subset design cannot express — a per-cohort masking derivation + its aggregation contract | `diff_diff/lwdid_sensitivity.py` | #588 | Low | | LWDiD estimation-time anticipation offset (CS-parity gap): no `anticipation=` parameter exists — no-anticipation is hard-assumed and the only surface is the single-cohort `sensitivity_no_anticipation` refit diagnostic. The LW device for anticipation is the anchor-window shift — drop the last k pre-treatment periods from the transformation/anchor window (`Ybar_{i,S0}` with `S0 < S-1`, per the REGISTRY LWDiD Anticipation edge case; NOT a cohort recode g → g − k) — and exposing an `anticipation=` parameter needs that derivation checked against LW 2025 NAC/CNAS before implementation | `diff_diff/lwdid.py` | LWDiD release audit | Low | @@ -53,7 +53,6 @@ exists but parity can't be verified without a local toolchain. | Issue | Location | PR | Priority | |-------|----------|----|----------| -| Chang (2020) published-version cross-check of the arXiv `p_hat_k` typo: the library adopted the global treated-share convention on DoubleML parity evidence (REGISTRY DML-infrastructure Note; spike `benchmarks/doubleml/chang_case1_parity.py`), but the published *Econometrics Journal* text (paywalled) has not been checked against the arXiv printing to see whether it resolves the algorithm-vs-proof contradiction | `docs/methodology/papers/chang-2020-review.md` | dml-b0 | Low | | `StaggeredTripleDifference` R cross-validation (the engine is shared with `TripleDifference`'s staggered mode since 3(b), so this covers both surfaces): CSV fixtures not committed (gitignored); tests skip without local R + `triplediff`. Commit fixtures or generate deterministically. | `tests/test_methodology_staggered_triple_diff.py` | #245 | Medium | | Staggered DDD R parity (both surfaces - one shared engine since 3(b)): benchmark only tests the no-covariate path (`xformla=~1`). Add covariate-adjusted scenarios + aggregation-SE parity assertions. | `benchmarks/R/benchmark_staggered_triplediff.R` | #245 | Medium | | Staggered DDD per-cohort group-effect SEs include WIF (both surfaces - one shared engine since 3(b)) (conservative vs R's `wif=NULL`); documented in REGISTRY. Could override the mixin for an exact R match (verification needs R `triplediff`). | `_staggered_triple_diff_engine.py` | #245 | Low | diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index 4c245f2f..947ead41 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -779,7 +779,9 @@ DR score is self-normalized; the two are not interchangeable"): - **Note:** p̂ convention (documented deviation). Chang's arXiv text is internally contradictory about the fold-level scalar nuisance: the printed - algorithms show `p̂_k = (1/n)Σ_{i∈I_k^c} D_i` (an invalid normalizer) while + algorithms show `p̂_k = (1/n)Σ_{i∈I_k^c} D_i` — an invalid normalizer for + K > 2 (with n = N/K and |I_k^c| = N − n; at K = 2 the two sizes coincide, + so the printed formula IS a valid auxiliary-sample mean there) — while the Theorem 1-2 proofs use fold means over `I_k` (see the review's Gaps). The score functions take `p_hat` as a CALLER-supplied argument; the LIBRARY convention — adopted for `DMLDiD` and used in all fixtures — is the @@ -789,8 +791,33 @@ DR score is self-normalized; the two are not interchangeable"): 5.6e-17 vs `DoubleMLDID(score="observational", in_sample_normalization=False)` under doubleml 0.11.4 with shared folds) and asymptotic equivalence to both printed readings. The published - *Econometrics Journal* version has not yet been cross-checked against the - arXiv typo (tracked in DEFERRED.md). + *Econometrics Journal* version was cross-checked 2026-08-30 (doi: + 10.1093/ectj/utaa001): Definition 3.1(b) prints the SAME + `p̂_k = (1/n)Σ_{i∈I_k^c} D_i` formula (and `λ̂_k` likewise) — the + printed-formula side of the contradiction persists in the version of + record, while the proofs are online-supplement-only and remain + unchecked (the arXiv-v3 proofs are the verified proof-side source). The + author's replication code (github.com/NengChiehChang/Diff-in-Diff, + commit `e93773d1c5ed`, inspected 2026-08-30; it fixes k = 2, so its own + printed-formula reading is self-consistent) uses MIXED conventions: + `p̂_k = mean(D)` over the propensity-TRIMMED evaluation fold — the + proofs' `E_{n,k}[D]` form, modulo trimming (`RO_Algorithm.R:70`, + `RCS_Algorithm.R:78`) — while `λ̂_k = mean(T)` over the AUXILIARY + sample (`RCS_Algorithm.R:58`). The √N-equivalence claim applies only to + the UNTRIMMED readings (main-fold, auxiliary-sample, global share). The + code's two extra operations are distinct and separately caveated: + (i) it DROPS rows with fitted ĝ outside (0.05, 0.95) before averaging, + which targets a propensity-trimmed-population ATT unless the trimming + is asymptotically inactive; (ii) the RCS score additionally applies a + one-sided REALIZED-SCORE-dependent deletion (`s = s[s < abs(min(s))]`, + `RCS_Algorithm.R:79`), an outcome-dependent truncation with NO + established estimand or covering variance theorem. The code is + consulted here ONLY to identify the scalar-normalizer placement, never + as an oracle for the complete estimator. The library + KEEPS the global full-sample share (exact DoubleML parity, documented + deviation — now from the author's code as well as the printed + algorithm; full cross-check log in + `docs/methodology/papers/chang-2020-review.md`). **Cross-fitting (`_crossfit.py`)** — DML2-style unit-level K-fold machinery: `assign_folds` captures the generator's bit-generator state BY VALUE before @@ -864,7 +891,7 @@ double-weight). `benchmarks/doubleml/chang_rcs_characterization.py` documents the divergence for the shipped Case 2 lane), `DoubleMLDIDMulti` for Chang Case 3. -- R: none for Chang's estimator; `DRDID::drdid_panel` for the SZ score. +- R: the author's replication code (github.com/NengChiehChang/Diff-in-Diff, commit `e93773d1c5ed`, k=2 scripts `RO_Algorithm.R`/`RCS_Algorithm.R`) — an EQUATION-LEVEL reference only, not a numerical oracle (mixed p̂/λ̂ normalization, propensity row-dropping, an RCS score-dependent deletion, and no packaged variance conventions; see the p̂ convention Note); `DRDID::drdid_panel` for the SZ score. --- diff --git a/docs/methodology/papers/chang-2020-review.md b/docs/methodology/papers/chang-2020-review.md index 49d9c06c..16f9cd6a 100644 --- a/docs/methodology/papers/chang-2020-review.md +++ b/docs/methodology/papers/chang-2020-review.md @@ -12,6 +12,23 @@ > the arXiv v3 layout**, not the published article. Cross-check against the published version > before citing section/equation numbers in user-facing docs. +> **Published-version cross-check (2026-08-30, printed article pp. 177-191 + printed +> Appendix A pp. a1-a2).** The published text was checked against the suspected-typo list +> below; per-item verdicts are appended inline. Summary: every typo checkable from the +> printed text **persists unchanged** (items 1, 2, 5, 6, 7, 8 — the `p̂_k`/`λ̂_k` +> normalizer, the `Σ_20` `ψ_1` term, the multilevel `D_i`/`λ_0`, and both +> Lasso-penalty-recipe issues print identically); items 3, 4, and 9 live in material the +> published article relegates to the **online supplement** (Theorem proofs and the RCS / +> multilevel simulations) and are not checkable from the printed article. Structural +> mapping: arXiv Theorems 1-2 ↔ published Theorems 3.1-3.2 (Cases 1-2 only; multilevel +> still "can be proved by using the same arguments", published p. 184); the multilevel +> score/algorithm appear in printed Appendix A with **no regularity assumptions and no +> variance theorem** (the Case 3 blocker stands). The published empirical Table 2 numbers +> DIFFER from the arXiv Table 1 (see the empirical-application bullet). The published +> article ships a **replication package** (OUP Supporting Information) and footnote 1 +> links the author's R code (github.com/NengChiehChang/Diff-in-Diff) — superseding this +> review's earlier "no companion software package" statement. + --- ## Methodology Registry Entry @@ -172,7 +189,7 @@ the same variance estimators remain consistent under kernel first stages. ∂_λ∂_p ψ_2 = [(1-2λ)/(p²λ²(1-λ)²)] · [(D-g)/(1-g)] · ((T-λ)Y - ℓ_2) + [Y/(p²λ(1-λ))] · [(D-g)/(1-g)] **Reference implementation(s):** -- R: none — the paper ships no companion package. +- R: the author's replication code, github.com/NengChiehChang/Diff-in-Diff (commit `e93773d1c5ed`, k=2 scripts `RO_Algorithm.R` / `RCS_Algorithm.R`; linked from published footnote 1) — an EQUATION-LEVEL reference only, NOT a numerical oracle: mixed `p̂`/`λ̂` normalization, fixed-threshold propensity row-dropping, an RCS realized-score-dependent deletion, hard-coded seeds, and no packaged variance conventions (see the `p̂_k` contradiction block). - Stata: none. - Python (validation oracle — **panel lane only**): `doubleml.DoubleMLDID` implements a Chang/Zimmert-style orthogonal panel score and is the closest parity anchor for Case 1 — but only under a specific configuration (its `in_sample_normalization` option changes the score's normalization; pin the config and verify score equivalence at the equation level before treating any run as an oracle). **Version caveat:** `DoubleMLDID` is deprecated upstream ("will be removed with version 0.12.0. Please use DoubleMLDIDBinary instead", verified 2026-08-22) — pin the exact DoubleML version used for golden-fixture generation, archive the fixtures in-repo, and verify `DoubleMLDIDBinary`'s score/normalization equivalence separately before adopting it as the replacement anchor. **Scope caveat:** treat DoubleML as an **equation-level score oracle**, not a full-estimator finite-sample oracle, unless the fixture supplies identical fold assignments and scalar-nuisance normalization: upstream uses the global treated share and treatment-stratified sample splitting, while Chang specifies random folds and leaves the fold-level `p̂_k` convention ambiguous (see Gaps). Full-estimator comparisons under differing conventions are approximate/asymptotic — a finite-sample mismatch there does not falsify a paper-faithful implementation, and exact-parity fixtures must not silently import DoubleML's normalization as if it were Chang's. Add independent fixtures for whichever `p̂_k` convention is selected. - **Not oracles:** `doubleml.DoubleMLDIDCS` uses a Sant'Anna-Zhao-style repeated-cross-section score with four treatment-by-period outcome regressions and different normalizations — a *related* estimator, not an implementation of Chang's single-`ℓ_20` Equation 3.2 score or its `λ`-corrected variance. `DoubleMLDIDMulti` handles **staggered treatment timing**, not Chang's Case 3 multilevel treatment *intensity* — it is not a Case 3 anchor. For Cases 2-3, validation must rest on independent equation-level fixtures (Equations 3.2/3.3 and their variance corrections) plus recovery tests on the paper's simulation DGPs (Section 4). @@ -217,7 +234,7 @@ items remain open, tracked in DEFERRED.md): | Cross-fitting folds `K` | int ≥ 2, fixed (independent of N) | none prescribed in the paper | fixed by user; theory covers any fixed K ≥ 2 (pp. 10, 31-32) | | Propensity learner | model | none prescribed — "any ML methods or classical estimators such as kernel or series estimators" (p. 11); paper's example/simulation choice: Logit Lasso (Multi-Logit Lasso for multilevel) (Eq. 3.4) | penalty by K-fold CV with K = 10 (Van de Geer 2008) or Belloni, Chernozhukov, Chetverikov & Wei (2018) (Sec. 4, p. 17) | | Outcome learner (`ℓ` nuisances) | model | none prescribed; paper's example/simulation choices: modified Lasso (Belloni et al. 2012), random forest (500 trees in simulations) | Lasso penalty: iterated plug-in loadings, appendix p. 26 (below); RF: fixed 500 trees | -| Lasso penalty loadings `(λ_k, Υ̂_k)` | diagonal matrix + scalar | `λ_k = 2c·√M_k·Φ^{-1}(1 - γ/(2p))`, `c > 1`, `γ → 0` | **As printed (appendix "The Lasso Penalty", p. 26):** "Let `y_i` denote `Y_i(1) - Y_i(0)` or `(T_i - λ̂_k)`"; initial `γ̂_kj = sqrt((1/M_k) Σ_{i∈I_kz^c} q_ij²(y_i - ȳ_k)²)`; refined `γ̂_kj = sqrt((1/M_k) Σ_{i∈I_kz^c} q_ij² ε̂_i²)` with `ε̂_i = y_i - q_i'β*_k`, repeated B > 0 times; `ȳ_k = M^{-1} Σ_{i∈I_k^c} y_i`. `M = N - n` = full auxiliary-sample size; `M_k = |I_kz^c|` = its untreated subset. **Proposed implementation (corrections; see Suspected typos and Gaps):** use the same response the modified-Lasso objective (p. 12) minimizes — `y_i = Y_i(1) - Y_i(0)` (Case 1), `y_i = (T_i - λ̂_k)·Y_i` (Case 2) — pending comparison with the published article. (No defaults for c, γ, B printed; Belloni et al. 2012 conventions, e.g. c = 1.1, are the natural reference) | +| Lasso penalty loadings `(λ_k, Υ̂_k)` | diagonal matrix + scalar | `λ_k = 2c·√M_k·Φ^{-1}(1 - γ/(2p))`, `c > 1`, `γ → 0` | **As printed (appendix "The Lasso Penalty", p. 26):** "Let `y_i` denote `Y_i(1) - Y_i(0)` or `(T_i - λ̂_k)`"; initial `γ̂_kj = sqrt((1/M_k) Σ_{i∈I_kz^c} q_ij²(y_i - ȳ_k)²)`; refined `γ̂_kj = sqrt((1/M_k) Σ_{i∈I_kz^c} q_ij² ε̂_i²)` with `ε̂_i = y_i - q_i'β*_k`, repeated B > 0 times; `ȳ_k = M^{-1} Σ_{i∈I_k^c} y_i`. `M = N - n` = full auxiliary-sample size; `M_k = |I_kz^c|` = its untreated subset. **Proposed implementation (corrections; see Suspected typos and Gaps):** use the same response the modified-Lasso objective (p. 12) minimizes — `y_i = Y_i(1) - Y_i(0)` (Case 1), `y_i = (T_i - λ̂_k)·Y_i` (Case 2) — the published-version comparison is complete (2026-08-30, typo persists in print; see Suspected typos item 7). (No defaults for c, γ, B printed; Belloni et al. 2012 conventions, e.g. c = 1.1, are the natural reference) | | Kernel (alternative first stage) | kernel fn | none prescribed by theory — Assumption 3.3(1) requires a **compactly supported** order-`m` kernel; simulations use a standard Gaussian (violates 3.3's compact support) | Theorems 3-4 cover only 3.3-compatible kernels; using Gaussian must be documented as a deviation (Sec. 4, p. 17) | | Kernel bandwidth `h` | float | data-driven CV | CV/MSE-minimizing `h = N^{-1/(d+2s+2m)}` is VALID (no undersmoothing needed — SBP, Theorem 3); requires `2m > d + 2s` | | Overlap clip `κ` | float | not specified | implementation choice; theory requires fitted `g ∈ [κ, 1-κ]` | @@ -235,33 +252,33 @@ items remain open, tracked in DEFERRED.md): **Suspected typos in the arXiv v3 PDF** (carry to any implementation cross-check against the published *Econometrics Journal* version): -1. **`p̂_k` factor (Algorithm 1, main text, p. 11):** printed as `p̂_k = (1/n) Σ_{i∈I_k^c} D_i` — a `1/n` factor over the auxiliary sample `I_k^c` (whose size is `M = N - n`, not `n`) — likely intended as the auxiliary-sample average; verify against the published version. (Same issue for `λ̂_k`.) -2. **`ψ_1` inside `Σ_20` (Assumption 3.2, p. 14):** the definition prints `Σ_20 := E_P[(ψ_1(W,θ_0,p_0,η_10) + G_2p0(D - p_0) + G_2λ0(T - λ_0))²]` — the leading `ψ_1(W, θ_0, p_0, η_10)` appears to be a typo for `ψ_2(W, θ_0, p_0, λ_0, η_20)`; check the published version. -3. **`I_{4,k} = O_P(n^{1/2})` (proof of Theorem 2, p. 48):** the PDF text prints "O_P(n^{1/2})" — from the Chebyshev step and the final display `I_k = O_P(N^{-1/2}) + O_P(N^{-1/2} + ε_N)` this is evidently a typo for `n^{-1/2}`. -4. **(A.14) outer exponent (p. 51):** as printed the sup-norm display appears with `(...)² ≤ ε_N`; by parallel with (A.12) it should read as the `^{1/2}` L2 norm. -5. **Undefined `D_i` in the multilevel `p̂_w` (appendix algorithm, p. 25, step 2):** prints `p̂_w = (1/n) Σ_{i∈I_k^c} D_i`, but `D` is not defined in the multilevel notation (treatment is `W`); evidently intended as `I(W_i = w)` (verified against the PDF page 2026-08-22). -6. **Stray `λ_0` in the multilevel algorithm (p. 25, step 2):** the step says "construct the estimator of `p_0` and `λ_0`", but no `λ_0` exists in Case 3 (there is no time-sampling share in the repeated-outcomes multilevel setting); evidently copied from the Case 2 algorithm (verified against the PDF page 2026-08-22). -7. **Case 2 response in the Lasso penalty-loading recipe (appendix "The Lasso Penalty", p. 26):** the recipe literally prints "Let `y_i` denote `Y_i(1) - Y_i(0)` or `(T_i - λ̂_k)`" — the Case 2 response omits the `Y_i` multiplier (verified against the PDF page 2026-08-22), while the modified-Lasso objective it feeds (p. 12) minimizes over `(T_i - λ̂_k)·Y_i - q_i'β`. Loadings computed on a bare `(T_i - λ̂_k)` would produce a different first-stage estimator; implement loadings on the objective's response, `(T_i - λ̂_k)·Y_i`, pending comparison with the published article. -8. **`ȳ_k` population/denominator inconsistency (p. 26):** `ȳ_k = M^{-1} Σ_{i∈I_k^c} y_i` is defined as the mean over the FULL auxiliary sample (size `M`), while both loading sums run over the untreated subset `I_kz^c` with normalizer `1/M_k` (verified against the PDF page 2026-08-22). Whether the centering mean was intended over `I_k^c` or `I_kz^c` is not resolvable from the arXiv text; record the choice made at implementation time and compare with the published article. +1. **`p̂_k` factor (Algorithm 1, main text, p. 11):** printed as `p̂_k = (1/n) Σ_{i∈I_k^c} D_i` — a `1/n` factor over the auxiliary sample `I_k^c` (whose size is `M = N - n`, not `n`) — likely intended as the auxiliary-sample average. **Published-version verdict (2026-08-30): typo persists** — Definition 3.1(b) (published p. 183) prints the identical `(1/n)Σ_{i∈I_k^c}` formula for both `p̂_k` and `λ̂_k`. +2. **`ψ_1` inside `Σ_20` (Assumption 3.2, p. 14):** the definition prints `Σ_20 := E_P[(ψ_1(W,θ_0,p_0,η_10) + G_2p0(D - p_0) + G_2λ0(T - λ_0))²]` — the leading `ψ_1(W, θ_0, p_0, η_10)` appears to be a typo for `ψ_2(W, θ_0, p_0, λ_0, η_20)`. **Published-version verdict (2026-08-30): typo persists** — published Assumption 3.2 (p. 185) prints `ψ_1(W, θ_0, p_0, η_10)` identically. +3. **`I_{4,k} = O_P(n^{1/2})` (proof of Theorem 2, p. 48):** the PDF text prints "O_P(n^{1/2})" — from the Chebyshev step and the final display `I_k = O_P(N^{-1/2}) + O_P(N^{-1/2} + ε_N)` this is evidently a typo for `n^{-1/2}`. **Published-version check (2026-08-30): not checkable from the printed article** — the Theorem proofs are online-supplement-only. +4. **(A.14) outer exponent (p. 51):** as printed the sup-norm display appears with `(...)² ≤ ε_N`; by parallel with (A.12) it should read as the `^{1/2}` L2 norm. **Published-version check (2026-08-30): not checkable from the printed article** (online-supplement-only). +5. **Undefined `D_i` in the multilevel `p̂_w` (appendix algorithm, p. 25, step 2):** prints `p̂_w = (1/n) Σ_{i∈I_k^c} D_i`, but `D` is not defined in the multilevel notation (treatment is `W`); evidently intended as `I(W_i = w)` (verified against the PDF page 2026-08-22). **Published-version verdict (2026-08-30): typo persists** — printed Appendix A step 2 (p. a1) repeats the undefined `D_i`. +6. **Stray `λ_0` in the multilevel algorithm (p. 25, step 2):** the step says "construct the estimator of `p_0` and `λ_0`", but no `λ_0` exists in Case 3 (there is no time-sampling share in the repeated-outcomes multilevel setting); evidently copied from the Case 2 algorithm (verified against the PDF page 2026-08-22). **Published-version verdict (2026-08-30): typo persists** — printed Appendix A step 2 (p. a1) repeats the stray `λ_0`. +7. **Case 2 response in the Lasso penalty-loading recipe (appendix "The Lasso Penalty", p. 26):** the recipe literally prints "Let `y_i` denote `Y_i(1) - Y_i(0)` or `(T_i - λ̂_k)`" — the Case 2 response omits the `Y_i` multiplier (verified against the PDF page 2026-08-22), while the modified-Lasso objective it feeds (p. 12) minimizes over `(T_i - λ̂_k)·Y_i - q_i'β`. Loadings computed on a bare `(T_i - λ̂_k)` would produce a different first-stage estimator; implement loadings on the objective's response, `(T_i - λ̂_k)·Y_i`. **Published-version verdict (2026-08-30): typo persists** — the printed Appendix A Lasso-penalty recipe (p. a1) prints the same bare `(T_i - λ̂_k)` response. +8. **`ȳ_k` population/denominator inconsistency (p. 26):** `ȳ_k = M^{-1} Σ_{i∈I_k^c} y_i` is defined as the mean over the FULL auxiliary sample (size `M`), while both loading sums run over the untreated subset `I_kz^c` with normalizer `1/M_k` (verified against the PDF page 2026-08-22). Whether the centering mean was intended over `I_k^c` or `I_kz^c` is not resolvable from the arXiv text; record the choice made at implementation time. **Published-version verdict (2026-08-30): inconsistency persists** — printed Appendix A (p. a2) defines `ȳ_k = M^{-1}Σ_{i∈I_k^c}` beside `1/M_k` loading sums, identically. -9. **§4.2.1's printed-but-unused `β_0` (p. 19):** the RCS ML simulation section prints "where `β_0 = γ_0 + 0.5`" although `β_0` appears in none of its equations (`Y⁰(0) = 1 + ε_1` as printed), while §4.1.1 (p. 18) and §4.3.1 (p. 20) print `Y⁰(0) = X'β_0 + ε_1` in the parallel position — a suspected dropped `X'β_0` term; not resolvable from the arXiv text. Either reading leaves §4.2.1's parallel trends intact (the level term cancels in the recursive trend `Y⁰(1) = Y⁰(0) + 1 + ε_2`) and `ℓ_20` constant in X, so the library's §4.2.1 non-replicability conclusion (above) is robust to the ambiguity; check the published version. +9. **§4.2.1's printed-but-unused `β_0` (p. 19):** the RCS ML simulation section prints "where `β_0 = γ_0 + 0.5`" although `β_0` appears in none of its equations (`Y⁰(0) = 1 + ε_1` as printed), while §4.1.1 (p. 18) and §4.3.1 (p. 20) print `Y⁰(0) = X'β_0 + ε_1` in the parallel position — a suspected dropped `X'β_0` term; not resolvable from the arXiv text. Either reading leaves §4.2.1's parallel trends intact (the level term cancels in the recursive trend `Y⁰(1) = Y⁰(0) + 1 + ε_2`) and `ℓ_20` constant in X, so the library's §4.2.1 non-replicability conclusion (above) is robust to the ambiguity. **Published-version check (2026-08-30): not checkable from the printed article** — the printed §4 carries only the repeated-outcomes DGP (which DOES print `Y⁰(0) = X'β_0 + ε_1`, published p. 187, consistent with arXiv §4.1.1); the RCS and multilevel simulations are online-supplement-only. **Contradiction — where `p̂_k` (and `λ̂_k`) is computed (algorithms vs proofs):** -- Both printed algorithm statements agree with each other and were verified against the PDF pages directly (2026-08-22): main-text Algorithm 1 (p. 11) prints `p̂_k = (1/n) Σ_{i∈I_k^c} D_i` and the appendix multilevel algorithm (p. 25, step 2) prints `p̂_w = (1/n) Σ_{i∈I_k^c} D_i` — in both cases a `1/n` normalizer (n = |I_k|) over the **auxiliary sample `I_k^c`** (size `M = N - n`), which is not a valid mean of anything as printed (typo 1 above). +- Both printed algorithm statements agree with each other and were verified against the PDF pages directly (2026-08-22): main-text Algorithm 1 (p. 11) prints `p̂_k = (1/n) Σ_{i∈I_k^c} D_i` and the appendix multilevel algorithm (p. 25, step 2) prints `p̂_w = (1/n) Σ_{i∈I_k^c} D_i` — in both cases a `1/n` normalizer (n = |I_k|) over the **auxiliary sample `I_k^c`** (size `M = N - n`), which is not a valid mean of anything as printed for `K > 2` (typo 1 above; at `K = 2`, `|I_k^c| = n`, so the printed formula is exactly the auxiliary-sample mean — see the published-version verdict below). - The proofs of Theorems 1-2, however, use `p̂_k = E_{n,k}[D]` (fold mean over the **main fold `I_k`**, pp. 31, 33, 37), i.e. `p̂_k - p_0 = E_{n,k}[D - p_0]`, which is what generates the `G_1p0 (D - p_0)` influence-correction term in the variance algebra. -- So the two self-consistent readings of the printed formula are: `(1/M) Σ_{i∈I_k^c}` (auxiliary-sample mean — fix the normalizer, standard DML convention of fitting all nuisances on the complement) or `(1/n) Σ_{i∈I_k}` (main-fold mean — fix the index set, matching the proofs). Both are √N-consistent means of `D`, so the estimators are asymptotically equivalent; finite-sample values and the exact variance-correction algebra differ. Do not guess: consult the published *Econometrics Journal* version and DoubleML's implementation before fixing a convention. Pages: 11, 25, 31-37. -- **Library convention adopted (PR-B0):** DoubleML's implementation was consulted (it uses the GLOBAL full-sample treated share — verified to machine precision in the committed spike `benchmarks/doubleml/chang_case1_parity.py`), and the library adopts that third self-consistent reading; recorded as a `- **Note:**` deviation in REGISTRY's "Cross-fitting, DR-score, and ridge infrastructure (DML)" section. The published-version cross-check of the printed typo remains outstanding (DEFERRED.md, "Needs external reference"). +- So the two self-consistent readings of the printed formula are: `(1/M) Σ_{i∈I_k^c}` (auxiliary-sample mean — fix the normalizer, standard DML convention of fitting all nuisances on the complement) or `(1/n) Σ_{i∈I_k}` (main-fold mean — fix the index set, matching the proofs). Both are √N-consistent means of `D`, so the estimators are asymptotically equivalent; finite-sample values and the exact variance-correction algebra differ. Do not guess: consult the published *Econometrics Journal* version and DoubleML's implementation before fixing a convention. Pages: 11, 25, 31-37. **Published-version verdict (2026-08-30): the printed-formula side persists in the version of record** — published Definition 3.1(b) (p. 183) and printed Appendix A step 2 (p. a1) reprint the same `(1/n)Σ_{i∈I_k^c}` formulas; the proofs are online-supplement-only and were NOT re-checked (the proof-side of the contradiction rests on the verified arXiv-v3 proofs, pp. 31, 33, 37). The published text adjudicates nothing, but the **author's replication code gives evidence** (github.com/NengChiehChang/Diff-in-Diff, commit `e93773d1c5ed78d409d2ba6e1a87bec3d30c737c`, 2019-09-08, inspected 2026-08-30). Note first a K=2 qualification of typo 1: with `n = N/K` and `|I_k^c| = N − n`, the printed `(1/n)Σ_{i∈I_k^c}` normalizer is invalid for K > 2 but IS exactly the auxiliary-sample mean at K = 2 — and the author's scripts hard-code `k = 2`, so their own printed-formula reading is self-consistent. The code itself uses MIXED conventions: `p̂_k = mean(D[index1])` — the PROPENSITY-TRIMMED EVALUATION-FOLD mean (`RO_Algorithm.R:70,126`; `RCS_Algorithm.R:78,147`), the proofs' `E_{n,k}[D]` form modulo trimming; while `λ̂_k = mean(T[-K[q,]])` — the AUXILIARY-sample mean (`RCS_Algorithm.R:58,125`) — the other reading. Mixed conventions, so the code matches NEITHER printed algorithm line exactly and differs from the library's DoubleML-anchored global share; the three UNTRIMMED share readings (main-fold, auxiliary-sample, global) are √N-equivalent, but the author's code is NOT merely one of them modulo notation. Its two extra operations carry DISTINCT caveats: (i) fixed-threshold propensity deletion (rows with fitted ĝ outside (0.05, 0.95) dropped; (0.03, 0.97) in the variance sections) targets a propensity-trimmed-population ATT unless the trimming is asymptotically inactive; (ii) the RCS score additionally applies a one-sided REALIZED-SCORE-dependent deletion `s = s[which(s < abs(min(s)))]` before averaging (`RCS_Algorithm.R:79`) — an outcome-dependent truncation with no established estimand and no covering variance theorem (it cannot be described as a trimmed-population ATT). It is used here ONLY to identify the scalar-normalizer placement, not as an oracle for the complete estimator. Two further code-level conventions worth recording: rows with fitted ĝ outside (0.05, 0.95) (point estimates; (0.03, 0.97) in the variance sections) are DROPPED from the evaluation fold (`index1` subsetting), not clipped — unlike the library's clip-never-drop rule; and the variance section's `G = -finaltheta/mean(D[index1])` (`RO_Algorithm.R:124`) confirms the `Ĝ_1p = -θ̃/p̂` analogy the review inferred from the proofs. +- **Library convention adopted (PR-B0):** DoubleML's implementation was consulted (it uses the GLOBAL full-sample treated share — verified to machine precision in the committed spike `benchmarks/doubleml/chang_case1_parity.py`), and the library adopts that third self-consistent reading; recorded as a `- **Note:**` deviation in REGISTRY's "Cross-fitting, DR-score, and ridge infrastructure (DML)" section. The published-version cross-check completed 2026-08-30: the printed formula is identical in the published article, so the adopted convention stands unmodified (the former DEFERRED.md row is retired; the author's replication code was ALSO consulted 2026-08-30 — see the contradiction block above — and uses yet another mixed convention, leaving the DoubleML-parity choice as the best-anchored of the √N-equivalent readings). **Content not covered by the paper / open items:** -- No companion software package; no coverage/RMSE tables — simulation evidence is Monte Carlo histograms only (Figures 3-20, pp. 58-63; the "New" vs "HD" right-panel labeling is inferred to distinguish kernel-variant from Lasso-variant designs, but the figure section carries no prose confirming it). +- ~~No companion software package~~ **superseded by the published version (2026-08-30):** the article ships a replication package (OUP Supporting Information "Online Supplement / Replication package") and footnote 1 (p. 178) links the author's R code, github.com/NengChiehChang/Diff-in-Diff (R code inspected 2026-08-30 at commit `e93773d1c5ed` — see the p̂_k contradiction block for its conventions; the OUP-hosted supplement/replication archive itself remains unfetched). No coverage/RMSE tables — simulation evidence is Monte Carlo histograms only (Figures 3-20, pp. 58-63; the "New" vs "HD" right-panel labeling is inferred to distinguish kernel-variant from Lasso-variant designs, but the figure section carries no prose confirming it). - No recommended number of cross-fitting folds K anywhere in the paper. - No trimming/clipping rule for propensity scores (theory assumes the band; implementation must choose and document). - No explicit consistent formula for `Ĝ_2λ` is given in the theorem statements — the closed form for `G_2λ0` recovered from the proof (p. 55) is the natural sample analogue. -- Multilevel treatment: variance estimator and full theorem statements are not printed in the reviewed text (Theorems 2/4 cover Cases 1-2 only; multilevel results "can be proven using the same arguments", p. 13). The analogy `Ĝ_wp = -θ̃_w/p̂_w` is plausible but NOT stated in the paper. +- Multilevel treatment: variance estimator and full theorem statements are not printed in the reviewed text (Theorems 2/4 cover Cases 1-2 only; multilevel results "can be proven using the same arguments", p. 13). The analogy `Ĝ_wp = -θ̃_w/p̂_w` is plausible but NOT stated in the paper. **Published-version confirmation (2026-08-30):** printed Appendix A (pp. a1-a2) carries the same score/algorithm with no regularity assumptions and no variance theorem, and published p. 184 repeats "same arguments" — the Case 3 derivation blocker stands in the version of record. - Theorem 2 proof for the repeated-cross-sections case concludes within the reviewed range (pp. 51-56); no gaps remain in the proofs of Theorems 1-4 as reviewed, but the appendix's stated Lasso-penalty constants `c`, `γ`, `B` have no defaults (p. 26). - No clustering, no bootstrap, no missing-data handling, no aggregation across multilevel treatment arms. - The §4 simulation designs violate the paper's own regularity conditions in two symmetric ways: the Gaussian kernel used for the kernel-estimation variants violates Assumption 3.3's compact-support requirement (see the Tuning Parameters kernel row), and BOTH §4.2 propensities are unbounded-Gaussian-index logistics, so the a.s. strict-overlap Assumption 3.2(a) (`Pr(κ ≤ g_0(X) ≤ 1−κ) = 1` for fixed `κ > 0`) holds for NO fixed κ — §4.2.1's tail mass outside [0.01, 0.99] is ≈0.06% (`X'γ_0 ~ N(0.685, 1.21²)`), §4.2.2's ≈2.1e-5 (immaterial at the simulated Ns, but a theory/design inconsistency worth recording). -- Empirical application (Section 5, pp. 21-22, repeated cross sections via Eqs. 2.2/3.2): Sequeira (2016) tariff/bribery data, N = 1084. Table 1 exact estimate (SE) pairs (verified against the PDF 2026-08-22): Sequeira (2016) TWFE -2.928 (0.944); Abadie kernel -7.986 (3.028); orthogonal θ̃ kernel -8.670 (3.643); Abadie Lasso -7.499 (2.746); orthogonal θ̃ Lasso -9.191 (4.854) — usable as a rough replication target only if the Sequeira data is obtainable. +- Empirical application (Section 5, pp. 21-22, repeated cross sections via Eqs. 2.2/3.2): Sequeira (2016) tariff/bribery data, N = 1084. Table 1 exact estimate (SE) pairs (verified against the PDF 2026-08-22): Sequeira (2016) TWFE -2.928 (0.944); Abadie kernel -7.986 (3.028); orthogonal θ̃ kernel -8.670 (3.643); Abadie Lasso -7.499 (2.746); orthogonal θ̃ Lasso -9.191 (4.854) — usable as a rough replication target only if the Sequeira data is obtainable. **Published-version divergence (2026-08-30): the published Table 2 (p. 189) reports DIFFERENT numbers** — Sequeira (2016) -2.928 (0.944) unchanged; Abadie (kernel) -8.168 (3.072); DMLDiD (kernel) -6.998 (3.752); Abadie (Lasso) -6.432 (2.737); DMLDiD (Lasso) -5.222 (2.647) — the empirical results were evidently re-run for publication (bandwidth/penalty by 10-fold cross-validation, stated on p. 189; the published narrative states DMLDiD-Lasso is "only 60% higher" than Sequeira's estimate — a SOURCE quotation that conflicts with its own Table 2 arithmetic, |−5.222|/|−2.928| ≈ 1.78, i.e. ~78% higher). Any future replication targets the PUBLISHED Table 2, not the arXiv Table 1. - Simulation DGPs (Section 4, pp. 17-21) are fully specified and are the recommended validation fixtures: e.g. 4.1.1 repeated outcomes ML: `N ∈ {200, 500}`, `p ∈ {100, 300}`, `X ~ N(0, I_p)`, `γ_0 = (1, 1/2, 1/3, 1/4, 1/5, 0, ...)`, logistic PS, `β_0 = γ_0 + 0.5`, `θ_0 = 3`, errors N(0, 0.1) (variance/SD as printed "N(0,0.1)" — ambiguous as printed; the shipped §4.2.2 fixtures ADOPT the variance-0.1 reading, `σ = √0.1`, stated in the fixture comment); 4.3.1 multilevel: `W ∈ {0,1,2}`, shares (0.3, 0.3, 0.4), `θ_10 = 3`, `θ_20 = 6`. - §4.2 repeated-cross-section DGPs (pp. 18-20, extracted 2026-08-29 for the PR-B2 fixtures). **§4.2.1 (ML estimation):** `N ∈ {200, 500}`, `p ∈ {100, 300}`, `X_i ~ N(0.3·1, I_p)` (note the 0.3 mean, unlike §4.1.1), same `γ_0`, logistic PS `P(D=1|X) = sigmoid(X'γ_0)`; as printed `Y⁰(0) = 1 + ε_1`, `Y⁰(1) = Y⁰(0) + 1 + ε_2`, `Y¹(1) = θ_0 + Y⁰(1) + ε_3`, `θ_0 = 3`, `T ~ Bernoulli(0.5)`, observed `Y = Y(0) + T(Y(1) − Y(0))` (see suspected-typo 9 on the printed-but-unused `β_0`). Under our native unpenalized learners this design is NOT estimable: at (N=500, p=100) the logit+linear stack is pure noise across ~20 review seeds (att ~ −23..+16, SE ~4-10; `solve_logit`'s EPV guard fires every fold at EPV ~1.4-1.6 vs threshold 10; FITTED out-of-fold clipping ~5-29% of rows across seeds, mean ~14% — the estimated-propensity leg of strict overlap; TRUE-propensity mass outside [0.01, 0.99] is only ≈0.06%, `X'γ_0 ~ N(0.685, 1.21²)`), and at (N=200, p=100) `outcome_learner="linear"` fails closed on control-fold rank deficiency even under an oracle propensity (ridge/sieve fit there). Reproduce by generating the equations above at (500, 100) with `DMLDiD(panel=False)` and native learners over any batch of ~20 seeds — the qualitative behavior is seed-robust; the ranges are deliberately qualitative, not seed-pinned. **§4.2.2 (kernel estimation):** `N ∈ {200, 500}`, `D ~ Bernoulli(0.5)`, scalar `X | D ~ N(D, 1)`, `Y⁰(0) = ε_1`, `Y⁰(1) = Y⁰(0) + X + ε_2`, `Y¹(1) = θ_0 + Y⁰(1) + ε_3`, `θ_0 = 3`, `T ~ Bernoulli(0.5)`, same observation rule. True propensity `= sigmoid(X − 1/2)` by Bayes' rule; true `ℓ_20(X) = λ_0(1−λ_0)·X = 0.25·X`. The trend `X + ε_2` with group-imbalanced X violates UNCONDITIONAL parallel trends by construction (the unadjusted 2×2 contrast converges to `θ_0 + 1 = 4`) — the design's covariate-adjustment point. Replicated verbatim in `tests/test_methodology_dml_did.py`. - Finding to preserve for docs: in the repeated-cross-section simulations the orthogonal estimator appears well centered on the truth but visibly NOISIER than Abadie's plug-in at small N (Figures 9-14) — expect larger SEs. Orthogonality removes first-order nuisance-estimation bias (an asymptotic property under the Theorem 1 rates); it does NOT guarantee finite-sample unbiasedness, and histogram centering in simulations cannot establish it.