CMORizer ESACCI-PERMAFROST

[axel-lauer]

Description

This PR adds a downloading and formatting script for the dataset ESACCI-PERMAFROST v3.0. The following three variables are supported:

• active layer thickness (alt)
• permafrost ground temperature (gtd)
• permafrost extent (pfr)

The CMORizer requires the custom CMOR tables defined in ESMValGroup/ESMValCore#2358.

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🛠 This pull request has a descriptive title
• 🛠 Code is written according to the code quality guidelines
• 🛠 Documentation is available
• 🛠 Tests run successfully
• 🛠 The list of authors is up to date
• 🛠 Any changed dependencies have been added or removed correctly
• 🛠 All checks below this pull request were successful

New or updated data reformatting script

• 🛠 Documentation is available
• 🛠 The dataset has been added to the CMOR check recipe
• 🛠 The dataset has been added to the shared data pools of DKRZ and Jasmin by the @ESMValGroup/OBS-maintainers team
• 🧪 Numbers and units of the data look physically meaningful

[LisaBock]

Thanks @axel-lauer !

I started the review and run in two issues:

• there were small modifications in the downloader necessary (see comment there)
• when starting the formatter I got the following error message:
  PermissionError: [Errno 13] Permission denied: '/work/bd0854/b380103/esacci-permafrost-weights'

Please update also the branch with the main branch.

[jlenh]

Looks good overall, just a few comments!

If you could also deal with the Codacy issues! It looks like mostly using the Path library instead of os, and some looping over dictionary keys. Let me know if you need assistance with that 😄 And there is a conflict to solve with the main branch.

The tests are failing because of the dependency on the newest ESMValCore version. I have also bumped the PR to the next version as it seems that the needed PR in ESMValCore will not be merged for v2.14.0.

[axel-lauer]

If you could also deal with the Codacy issues! It looks like mostly using the Path library instead of os, and some looping over dictionary keys. Let me know if you need assistance with that 😄 And there is a conflict to solve with the main branch.

I just updated with the latest main: 6f09965

For remaining Codacy issues, I would appreciate some help.

[valeriupredoi]

@axel-lauer please fix the two failing tests. also for Codacy stuff, datetime objects should ideally contain timezone info, see https://docs.astral.sh/ruff/rules/call-datetime-without-tzinfo/ - in fairness, that#s probably not needed for us, but I don't think it hurts adding an eg UTC timezone either 🍺

[axel-lauer]

@axel-lauer please fix the two failing tests. also for Codacy stuff, datetime objects should ideally contain timezone info, see https://docs.astral.sh/ruff/rules/call-datetime-without-tzinfo/ - in fairness, that#s probably not needed for us, but I don't think it hurts adding an eg UTC timezone either 🍺

Thanks @valeriupredoi for taking a look!

I added the time zone info and replaced calls to os with ones to Path. There are two remaining two Codacy issues: "too many branches" and "too many statements". I will ignore these. This is a simple CMORizer, no software for space rockets.

The tests of recipe_check_obs.yml are failing because PR ESMValGroup/ESMValCore#2358 containing the required custom CMOR tables is not merged yet.

[jlenh]

Looks good @axel-lauer, thanks for the modifications! I added two small comments.

Otherwise I tested the cmorizer on a few years of data and it produces correct outputs.

The remaining failing test is indeed due to the core PR that needs to be merged first, I tested it in dev mode and the recipe_check_obs works fine then (apart from the missing years in the files I have cmorised on my side) Maybe you can add the cmorised dataset in the data pool then to fully test :)

[axel-lauer]

Looks good @axel-lauer, thanks for the modifications! I added two small comments.

Otherwise I tested the cmorizer on a few years of data and it produces correct outputs.

The remaining failing test is indeed due to the core PR that needs to be merged first, I tested it in dev mode and the recipe_check_obs works fine then (apart from the missing years in the files I have cmorised on my side) Maybe you can add the cmorised dataset in the data pool then to fully test :)

Thanks @jlenh ! I just moved the CMORized files to our obs-pool on Levante.

[jlenh]

Looks good @axel-lauer, thanks for the modifications! I added two small comments.
Otherwise I tested the cmorizer on a few years of data and it produces correct outputs.
The remaining failing test is indeed due to the core PR that needs to be merged first, I tested it in dev mode and the recipe_check_obs works fine then (apart from the missing years in the files I have cmorised on my side) Maybe you can add the cmorised dataset in the data pool then to fully test :)

Thanks @jlenh ! I just moved the CMORized files to our obs-pool on Levante.

Great! Then we will come back to this once the Core PR is merged :)

[rbeucher]

Hi all — we’re getting ready for the next ESMValTool release, version 2.15, and are planning to freeze the code on 26 June 2026.

Could you please let us know whether you think you’ll be able to finalise the work in this PR by then? If not, no problem — it would just help us plan what can realistically make it into the release.

@flicj191 tagging Felicity for visibility.

[flicj191]

Hi all, as the associated PR ESMValGroup/ESMValCore#2358 containing the required custom CMOR tables is not merged yet and has some outstanding comments, this may need to be moved to the next milestone.
Please let me know if this will cause any issues? Thank you