diff --git a/doc/whats-new.rst b/doc/whats-new.rst index 6d44194f38f..c5e9d895c3b 100644 --- a/doc/whats-new.rst +++ b/doc/whats-new.rst @@ -407,6 +407,11 @@ Bug Fixes Documentation ~~~~~~~~~~~~~ +- Document how :py:meth:`Dataset.interp` and :py:meth:`DataArray.interp` handle + chunks of Dask arrays, and that loading chunked new coordinates first can + reduce the memory use for ``method="linear"`` and ``"nearest"`` + (:pull:`11786`). + By `Michael Niklas `_. - Clarified that :py:meth:`DataArray.item` and :py:meth:`Variable.item` are not implemented for lazy (e.g. dask) arrays, both in the method docstring and in the raised error message, which now suggests loading the data into memory first diff --git a/xarray/core/dataarray.py b/xarray/core/dataarray.py index f0912ef7c1e..c978a23b5e3 100644 --- a/xarray/core/dataarray.py +++ b/xarray/core/dataarray.py @@ -2429,6 +2429,12 @@ def interp( - When interpolating along multiple dimensions with methods `linear` and `nearest`, the process attempts to decompose the interpolation into independent interpolations along one dimension at a time. + - For Dask arrays, the chunks along the interpolated dimensions are merged + into a single chunk. With methods `linear` and `nearest`, the data is + first subset to the range of the new coordinates, so only the chunks that + are needed are read. This only happens if the new coordinates are in + memory, so loading chunked new coordinates first (e.g. with ``.compute()``) + can reduce the memory use considerably if they are cheap to compute. - The specific interpolation method and dimensionality determine which interpolant is used: diff --git a/xarray/core/dataset.py b/xarray/core/dataset.py index e49ca858a81..b1f241bcb52 100644 --- a/xarray/core/dataset.py +++ b/xarray/core/dataset.py @@ -3895,6 +3895,12 @@ def interp( - When interpolating along multiple dimensions with methods `linear` and `nearest`, the process attempts to decompose the interpolation into independent interpolations along one dimension at a time. + - For Dask arrays, the chunks along the interpolated dimensions are merged + into a single chunk. With methods `linear` and `nearest`, the data is + first subset to the range of the new coordinates, so only the chunks that + are needed are read. This only happens if the new coordinates are in + memory, so loading chunked new coordinates first (e.g. with ``.compute()``) + can reduce the memory use considerably if they are cheap to compute. - The specific interpolation method and dimensionality determine which interpolant is used: