regrid loses mask information as well as cell measures and ancillary variables

[rebeccaherman1]

ESMValCore/esmvalcore/preprocessor/_regrid.py

Line 781 in eb15017

def regrid(

When I regrid, the mask resets, and I lose my land mask! I'm using a target grid and area-weighted regridding, and it looks like I'm using an outdated version of iris (3.6.1 instead of 3.15). I'm a little bit lost when looking at the ESMValCore code; could someone give advice or guidance? Do we expect Iris to update the land mask when regridding? Would this be fixed by using a more recent version of Iris? Or should ESMValCore be taking care of it? In what step in the ESMValCore code was the land mask supposed to be regridded?

I have checked that in the canonical ordering, regridding comes after masking, so masking information should be saved. I have also now checked that ESMValCore accepts any version of Iris, so if updating iris would fix this, then the required version should be updated to reflect that.

Cell measures and ancillary variables get dropped as well!
We have seen such issues for other iris (SciTools/iris#5804) functions affecting preprocessor functions (#2166) before. Again, I'm not sure where the problem originates for this function.

I'm able to hack my way around losing the cell area in the context of a jupyter notebook because it can be re-calculated, but I don't know how to handle losing the land_area_fraction ancillary variable because I don't know how to regrid it (it isn't a cube) and don't know how to get it again on the regridded cube (I only see how to add it when first loading the cube). Furthermore, I don't know if I can preserve these hacks when trying to re-formulate my notebook as a recipe. Help appreciated.

@valeriupredoi ? 🍺

Thanks!

Before regridding:

After calling pp.regrid with area_weighted and a target grid:

[rebeccaherman1]

I have updated iris to 3.11.1 (seems to be the most recent version on DKRZ) and confirmed that the regridding behavior from ESMValCore works the same.

[bouweandela]

Could you share the input dataset, the target grid, and the regridding scheme? Perhaps a recipe?

• I would expect the mask to be preserved when regridding.
• How would you propose we regrid cell measures and ancillary variables? Which regridding scheme would produce meaningful results?
• If cell measures or ancillary variables are already present on the target grid cube, we could copy them over to the result cube.

[rebeccaherman1]

After some more investigating, it is possible that the mask was lost because mdtol defaults to 1, meaning that the regridded data is only masked if all input data was masked, which I think is not true in the above pictures. I'll need to check again with another example. I am trying to figure out how to adjust mdtol from within ESMValTool and I'm having a little trouble figuring it out -- the documentation isn't helping me much.

The problem of dropping the cell area and ancillary variables remains.

1. I so far am only using the cell measure cell_area, so I'm not prepared to comment about cell measures in general. However, I don't understand how regridding could be performed to a regular grid without understanding the resulting cell_area. Iris has a function for calculating the cell_area (iris.analysis.cartography.area_weights), and now I'm using that to bring the cell_area back. I don't know why this isn't done automatically.
2. For ancillary variables I'm using land_area_fraction. For this ancillary variable, it makes sense to me to use area-weighted regridding. That's what I'm doing now in my jupyter notebook -- I'm hackily making a duplicate cube where the data is the land_area_fraction and I'm calling regrid again. Again, I'm not familiar with the other ancillary variables, so I'm not sure if this is fully general or not. But it seems like a starting place. And I would hope that one of the following things could be true:

• some automatic regridding of cell measures and ancillary variables is applied
• it's easily possible to specify how these extra variables should be regridded as a user
• it's easily possible to retrieve the information after regridding (this appears to be somewhat true for cell_area, but I haven't understood a way to do it for land_area_fraction).

3. That's an interesting point about the target cube. However, it is also possible to use a target specification that is not a cube; that is what I am doing.

I haven't managed to turn my code into a recipe yet; partially because I have little practice with recipes, and partially because I don't know how to do that with all of my workarounds that require calling iris functions directly instead of through the preprocessor.

I'm using area_weighted. I'll try to get you the other things soon.

I am also wondering if it's possible to use cell_area*land_area_fraction as the area weights instead of cell_area; haven't managed to find anything about that yet on iris or ESMValTool documentation.

Thanks @bouweandela

[valeriupredoi]

@rebeccaherman1 if I remember correctly iris=3.6.1 had a number of serious issues, including this one SciTools/iris#5413 - which was raised by you (well, I opened the issue at Iris, but you had made us aware of an issue). Might be a good idea to use a newer version, if not the latest 🍺