Scottish water and sewerage charges are zeroed in FRS 2024-25 (CSEWAMT retired)

[juaristi22]

Summary

create_frs computes water_and_sewerage_charges as

pe_household["water_and_sewerage_charges"] = (
    pd.Series(
        np.where(
            household.gvtregno == 12,
            household.csewamt + household.cwatamtd,
            household.watsewrt,
        )
    )
    .fillna(0)
    .values
    * WEEKS_IN_YEAR
)

(policyengine_uk_data/datasets/frs.py, around line 1265.)

In FRS 2024-25 (SN 9563) CSEWAMT carries no data at all — the header
survives but every one of the 16,288 households is blank. Because .fillna(0)
is applied to the result rather than per column, csewamt + cwatamtd
propagates NaN and the charge is zeroed for every Scottish household.

Measured on the raw househol.tab:

formula	nonzero households	nonzero share
current	12,644	0.776277
per-column fill	14,307	0.878377

The 1,663 differing households are all Scottish (England 11,483 and Wales
1,161 are identical either way; Northern Ireland is zero in both, correctly).

This is latent rather than new: in 2023-24, CSEWAMT was already missing for
378 Scottish households, a 0.0224 share gap (≡ 378/16,754). 2024-25 makes it
total.

Why the cell is empty

The FRS retired it. Per SN 9563 9563_dv_summary_2425.xlsx and
9563_frs202425_changes.xlsx:

• CWATAMT / CSEWAMT — "Wat./Sew. Charge: Final value after discount" —
  no longer populated.
• CWATAMT1 / CSEWAMT1 — "Weeklyised gross annual dom. water/sew. charge
  on bill", Scotland only, listed as "Added as DV in 2425", each carrying the
  note "DV created in 2024-25 as variable was removed from the dataset for
  2024-25".

So the replacements exist, but they are gross where the retired cells were
after discount, and there is no discounted sewerage counterpart to
CWATAMTD.

Consequences

1. water_and_sewerage_charges is 0 for all Scottish households, which flows
   into housing_costs and hbai_household_net_income_ahc — it depresses
   Scottish AHC housing costs, so it is AHC-poverty relevant rather than
   cosmetic.

2. The changelog note added with the council-tax netting change
   (changelog.d/scottish-ct-water-charges.md) states that the charges
   "remain captured in water_and_sewerage_charges". On this vintage that is
   not the case.

3. The netting path added in the same change is inconsistent with the charge
   path: it fills per column

   np.maximum(household.csewamt.fillna(0), 0) + np.maximum(household.cwatamtd.fillna(0), 0)

   so it survives the missing cell and still subtracts cwatamtd from
   ctannual, while the charge path zeroes. The bill is therefore netted by an
   amount that is not charged anywhere.

Magnitude

Weighted annual per Scottish household (2,564,102 grossed households), against
England and Wales at £489.67 on WATSEWRT:

basis	£/household
current behaviour (NaN-zeroed)	0.00
CWATAMTD alone	184.50

Note ORGWATAMT/ORGSEWAMT are annual, not weeklyised (their ratio to the
weeklyised DVs is ≈52.18), so they are not drop-in substitutes for the retired
cells.

Detection gap

A header-set comparison between vintages cannot see this: the column is still
present, just empty. A check that every consumed raw column carries at least
one non-missing value would have caught it at build time.

---

Found while building the microcosm UK spine against the 2024-25 line; the
same fix is applied there (PolicyEngine/microcosm#686), with the share
difference recorded as an incumbent-side signed difference.