Kerchunk and Xarray-Datatree

Overview

In this tutorial we are going to use a large collection of pre-generated Kerchunk reference files and open them with xarray-datatree. This chapter is heavily inspired by this blog post.

About the Dataset

This collection of reference files were generated from the NASA NEX-GDDP-CMIP6 (Global Daily Downscaled Projections) dataset. A version of this dataset is hosted on s3 as a collection of NetCDF files.

Prerequisites

Concepts	Importance	Notes
Kerchunk Basics	Required	Core
Multiple Files and Kerchunk	Required	Core
Kerchunk and Dask	Required	Core
Multi-File Datasets with Kerchunk	Required	IO/Visualization
Xarray-Datatree Overview	Required	IO

Time to learn: 30 minutes

Motivation

In total the dataset is roughly 12TB in compressed blob storage, with a single NetCDF file per yearly timestep, per variable. Downloading this entire dataset for analysis on a local machine would difficult to say the least. The collection of Kerchunk reference files for this entire dataset is only 272 Mb, which is about 42,000 times smaller!

Imports

import dask
import hvplot.xarray  # noqa
import pandas as pd
import xarray as xr
from datatree import DataTree
from distributed import Client
from fsspec.implementations.reference import ReferenceFileSystem

Read the reference catalog

The NASA NEX-GDDP-CMIP6 dataset is organized by GCM, Scenario and Ensemble Member. Each of these Scenario/GCM combinations is represented as a combined reference file, which was created by merging across variables and concatenating along time-steps. All of these references are organized into a simple .csv catalog in the schema:

GCM/Scenario	url

Organzing with Xarray-Datatree

Not all of the GCM/Scenario reference datasets have shared spatial coordinates and many of the have slight differences in their calendar and thus time dimension. Because of this, these cannot be combined into a single Xarray-Dataset. Fortunately Xarray-Datatree provides a higher level abstraction where related Xarray-Datasets are organized into a tree structure where each dataset corresponds to a leaf.

# Read the reference catalog into a Pandas DataFrame
cat_df = pd.read_csv(
    "s3://carbonplan-share/nasa-nex-reference/reference_catalog_nested.csv"
)
# Convert the DataFrame into a dictionary
catalog = cat_df.set_index("ID").T.to_dict("records")[0]

Load Reference Datasets into Xarray-DataTree

In the following cell we create a function load_ref_ds, which can be parallelized via Dask to load Kerchunk references into a dictionary of Xarray-Datasets.

def load_ref_ds(url: str):
    fs = ReferenceFileSystem(
        url,
        remote_protocol="s3",
        target_protocol="s3",
        target_options={"anon": True},
        lazy=True,
    )
    return xr.open_dataset(
        fs.get_mapper(),
        engine="zarr",
        backend_kwargs={"consolidated": False},
        chunks={"time": 300},
    )


tasks = {id: dask.delayed(load_ref_ds)(url) for id, url in catalog.items()}

Use Dask Distributed to load the Xarray-Datasets from Kerchunk reference files

Using Dask, we are loading 164 reference datasets into memory. Since they are are Xarray datasets the coordinates are loaded eagerly, but the underlying data is still lazy.

client = Client(n_workers=8)
client

Client

Client-ebc24433-f06f-11ee-8eac-000d3ae34a06

Connection method: Cluster object	Cluster type: distributed.LocalCluster
Dashboard: http://127.0.0.1:8787/status

Cluster Info

LocalCluster

d5e150f2

Dashboard: http://127.0.0.1:8787/status	Workers: 8
Total threads: 8	Total memory: 15.61 GiB
Status: running	Using processes: True

Scheduler Info

Scheduler

Scheduler-688ef8c8-c477-4552-a7f3-617165b34414

Comm: tcp://127.0.0.1:34187	Workers: 8
Dashboard: http://127.0.0.1:8787/status	Total threads: 8
Started: Just now	Total memory: 15.61 GiB

Workers

Worker: 0

Comm: tcp://127.0.0.1:33263	Total threads: 1
Dashboard: http://127.0.0.1:42899/status	Memory: 1.95 GiB
Nanny: tcp://127.0.0.1:44225
Local directory: /tmp/dask-scratch-space/worker-rog9mvpe

Worker: 1

Comm: tcp://127.0.0.1:42403	Total threads: 1
Dashboard: http://127.0.0.1:38961/status	Memory: 1.95 GiB
Nanny: tcp://127.0.0.1:44119
Local directory: /tmp/dask-scratch-space/worker-lwwf0dq4

Worker: 2

Comm: tcp://127.0.0.1:45273	Total threads: 1
Dashboard: http://127.0.0.1:43017/status	Memory: 1.95 GiB
Nanny: tcp://127.0.0.1:41461
Local directory: /tmp/dask-scratch-space/worker-u56g8xns

Worker: 3

Comm: tcp://127.0.0.1:43521	Total threads: 1
Dashboard: http://127.0.0.1:44097/status	Memory: 1.95 GiB
Nanny: tcp://127.0.0.1:40519
Local directory: /tmp/dask-scratch-space/worker-23u605pe

Worker: 4

Comm: tcp://127.0.0.1:36711	Total threads: 1
Dashboard: http://127.0.0.1:45967/status	Memory: 1.95 GiB
Nanny: tcp://127.0.0.1:34075
Local directory: /tmp/dask-scratch-space/worker-nnjp_buh

Worker: 5

Comm: tcp://127.0.0.1:43557	Total threads: 1
Dashboard: http://127.0.0.1:34165/status	Memory: 1.95 GiB
Nanny: tcp://127.0.0.1:41135
Local directory: /tmp/dask-scratch-space/worker-f7bh6td_

Worker: 6

Comm: tcp://127.0.0.1:39095	Total threads: 1
Dashboard: http://127.0.0.1:36869/status	Memory: 1.95 GiB
Nanny: tcp://127.0.0.1:36933
Local directory: /tmp/dask-scratch-space/worker-07x305fl

Worker: 7

Comm: tcp://127.0.0.1:33815	Total threads: 1
Dashboard: http://127.0.0.1:45835/status	Memory: 1.95 GiB
Nanny: tcp://127.0.0.1:39807
Local directory: /tmp/dask-scratch-space/worker-82407qql

catalog_computed = dask.compute(tasks)

Build an Xarray-Datatree from the dictionary of datasets

dt = DataTree.from_dict(catalog_computed[0])

Accessing the Datatree

A Datatree is a collection of related Xarray datasets. We can access individual datasets using UNIX syntax. In the cell below, we will access a single dataset from the datatree.

dt["ACCESS-CM2/ssp585"]

# or

dt["ACCESS-CM2"]["ssp585"]

<xarray.DatasetView> Size: 977GB
Dimensions:  (time: 31411, lat: 600, lon: 1440)
Coordinates:
  * lat      (lat) float64 5kB 6.934e-310 6.934e-310 6.934e-310 ... 0.0 0.0 0.0
  * lon      (lon) float64 12kB 6.934e-310 6.934e-310 6.934e-310 ... 0.0 0.0 0.0
  * time     (time) datetime64[ns] 251kB 2015-01-01T12:00:00 ... 2100-12-31T1...
Data variables:
    hurs     (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    huss     (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    pr       (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    rlds     (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    rsds     (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    sfcWind  (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    tas      (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    tasmax   (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    tasmin   (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
Attributes: (12/22)
    Conventions:           CF-1.7
    activity:              NEX-GDDP-CMIP6
    cmip6_institution_id:  CSIRO-ARCCSS
    cmip6_license:         CC-BY-SA 4.0
    cmip6_source_id:       ACCESS-CM2
    contact:               Dr. Rama Nemani: rama.nemani@nasa.gov, Dr. Bridget...
    ...                    ...
    scenario:              ssp585
    source:                BCSD
    title:                 ACCESS-CM2, r1i1p1f1, ssp585, global downscaled CM...
    tracking_id:           dcae752c-5de5-4170-8a98-e7043538d702
    variant_label:         r1i1p1f1
    version:               1.0

Convert a Datatree node to a Dataset

dt["ACCESS-CM2"]["ssp585"].to_dataset()

<xarray.Dataset> Size: 977GB
Dimensions:  (time: 31411, lat: 600, lon: 1440)
Coordinates:
  * lat      (lat) float64 5kB 6.934e-310 6.934e-310 6.934e-310 ... 0.0 0.0 0.0
  * lon      (lon) float64 12kB 6.934e-310 6.934e-310 6.934e-310 ... 0.0 0.0 0.0
  * time     (time) datetime64[ns] 251kB 2015-01-01T12:00:00 ... 2100-12-31T1...
Data variables:
    hurs     (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    huss     (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    pr       (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    rlds     (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    rsds     (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    sfcWind  (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    tas      (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    tasmax   (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
    tasmin   (time, lat, lon) float32 109GB dask.array<chunksize=(300, 600, 1440), meta=np.ndarray>
Attributes: (12/22)
    Conventions:           CF-1.7
    activity:              NEX-GDDP-CMIP6
    cmip6_institution_id:  CSIRO-ARCCSS
    cmip6_license:         CC-BY-SA 4.0
    cmip6_source_id:       ACCESS-CM2
    contact:               Dr. Rama Nemani: rama.nemani@nasa.gov, Dr. Bridget...
    ...                    ...
    scenario:              ssp585
    source:                BCSD
    title:                 ACCESS-CM2, r1i1p1f1, ssp585, global downscaled CM...
    tracking_id:           dcae752c-5de5-4170-8a98-e7043538d702
    variant_label:         r1i1p1f1
    version:               1.0

Operations across a Datatree

A Datatree contains a collection of datasets with related coordinates and variables. Using some in-built methods, we can analyze it as if it were a single dataset. Instead of looping through hundreds of Xarray datasets, we can apply operations across the Datatree. In the example below, we will lazily create a time-series.

ts = dt.mean(dim=["lat", "lon"])

Visualize a single dataset with HvPlot

display(  # noqa
    dt["ACCESS-CM2/ssp585"].to_dataset().pr.hvplot("lon", "lat", rasterize=True)
)

/home/runner/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/util.py:2127: RuntimeWarning: overflow encountered in scalar divide
  return length/diff
/home/runner/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/sheetcoords.py:232: RuntimeWarning: invalid value encountered in scalar multiply
  float_col = (x-self.lbrt[0]) * xdensity
WARNING:param.dynamic_operation: Callable raised "ValueError('cannot convert float NaN to integer')".
Invoked as dynamic_operation(numpy.datetime64('2015-01-01T12:00:00.000000000'))

WARNING:param.dynamic_operation: Callable raised "ValueError('cannot convert float NaN to integer')".
Invoked as dynamic_operation(numpy.datetime64('2015-01-01T12:00:00.000000000'), height=300, scale=1.0, width=700, x_range=None, y_range=None)

WARNING:param.dynamic_operation: Callable raised "ValueError('cannot convert float NaN to integer')".
Invoked as dynamic_operation(numpy.datetime64('2015-01-01T12:00:00.000000000'), height=300, scale=1.0, width=700, x_range=None, y_range=None)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/IPython/core/formatters.py:977, in MimeBundleFormatter.__call__(self, obj, include, exclude)
    974     method = get_real_method(obj, self.print_method)
    976     if method is not None:
--> 977         return method(include=include, exclude=exclude)
    978     return None
    979 else:

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/dimension.py:1286, in Dimensioned._repr_mimebundle_(self, include, exclude)
   1279 def _repr_mimebundle_(self, include=None, exclude=None):
   1280     """
   1281     Resolves the class hierarchy for the class rendering the
   1282     object using any display hooks registered on Store.display
   1283     hooks.  The output of all registered display_hooks is then
   1284     combined and returned.
   1285     """
-> 1286     return Store.render(self)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/options.py:1428, in Store.render(cls, obj)
   1426 data, metadata = {}, {}
   1427 for hook in hooks:
-> 1428     ret = hook(obj)
   1429     if ret is None:
   1430         continue

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/ipython/display_hooks.py:287, in pprint_display(obj)
    285 if not ip.display_formatter.formatters['text/plain'].pprint:
    286     return None
--> 287 return display(obj, raw_output=True)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/ipython/display_hooks.py:261, in display(obj, raw_output, **kwargs)
    259 elif isinstance(obj, (HoloMap, DynamicMap)):
    260     with option_state(obj):
--> 261         output = map_display(obj)
    262 elif isinstance(obj, Plot):
    263     output = render(obj)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/ipython/display_hooks.py:149, in display_hook.<locals>.wrapped(element)
    147 try:
    148     max_frames = OutputSettings.options['max_frames']
--> 149     mimebundle = fn(element, max_frames=max_frames)
    150     if mimebundle is None:
    151         return {}, {}

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/ipython/display_hooks.py:209, in map_display(vmap, max_frames)
    206     max_frame_warning(max_frames)
    207     return None
--> 209 return render(vmap)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/ipython/display_hooks.py:76, in render(obj, **kwargs)
     73 if renderer.fig == 'pdf':
     74     renderer = renderer.instance(fig='png')
---> 76 return renderer.components(obj, **kwargs)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/plotting/renderer.py:396, in Renderer.components(self, obj, fmt, comm, **kwargs)
    393 embed = (not (dynamic or streams or self.widget_mode == 'live') or config.embed)
    395 if embed or config.comms == 'default':
--> 396     return self._render_panel(plot, embed, comm)
    397 return self._render_ipywidget(plot)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/plotting/renderer.py:403, in Renderer._render_panel(self, plot, embed, comm)
    401 doc = Document()
    402 with config.set(embed=embed):
--> 403     model = plot.layout._render_model(doc, comm)
    404 if embed:
    405     return render_model(model, comm)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/panel/viewable.py:736, in Viewable._render_model(self, doc, comm)
    734 if comm is None:
    735     comm = state._comm_manager.get_server_comm()
--> 736 model = self.get_root(doc, comm)
    738 if self._design and self._design.theme.bokeh_theme:
    739     doc.theme = self._design.theme.bokeh_theme

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/panel/layout/base.py:320, in Panel.get_root(self, doc, comm, preprocess)
    316 def get_root(
    317     self, doc: Optional[Document] = None, comm: Optional[Comm] = None,
    318     preprocess: bool = True
    319 ) -> Model:
--> 320     root = super().get_root(doc, comm, preprocess)
    321     # ALERT: Find a better way to handle this
    322     if hasattr(root, 'styles') and 'overflow-x' in root.styles:

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/panel/viewable.py:667, in Renderable.get_root(self, doc, comm, preprocess)
    665 wrapper = self._design._wrapper(self)
    666 if wrapper is self:
--> 667     root = self._get_model(doc, comm=comm)
    668     if preprocess:
    669         self._preprocess(root)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/panel/layout/base.py:186, in Panel._get_model(self, doc, root, parent, comm)
    184 root = root or model
    185 self._models[root.ref['id']] = (model, parent)
--> 186 objects, _ = self._get_objects(model, [], doc, root, comm)
    187 props = self._get_properties(doc)
    188 props[self._property_mapping['objects']] = objects

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/panel/layout/base.py:168, in Panel._get_objects(self, model, old_objects, doc, root, comm)
    166 else:
    167     try:
--> 168         child = pane._get_model(doc, root, model, comm)
    169     except RerenderError as e:
    170         if e.layout is not None and e.layout is not self:

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/panel/pane/holoviews.py:429, in HoloViews._get_model(self, doc, root, parent, comm)
    427     plot = self.object
    428 else:
--> 429     plot = self._render(doc, comm, root)
    431 plot.pane = self
    432 backend = plot.renderer.backend

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/panel/pane/holoviews.py:525, in HoloViews._render(self, doc, comm, root)
    522     if comm:
    523         kwargs['comm'] = comm
--> 525 return renderer.get_plot(self.object, **kwargs)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/plotting/bokeh/renderer.py:68, in BokehRenderer.get_plot(self_or_cls, obj, doc, renderer, **kwargs)
     61 @bothmethod
     62 def get_plot(self_or_cls, obj, doc=None, renderer=None, **kwargs):
     63     """
     64     Given a HoloViews Viewable return a corresponding plot instance.
     65     Allows supplying a document attach the plot to, useful when
     66     combining the bokeh model with another plot.
     67     """
---> 68     plot = super().get_plot(obj, doc, renderer, **kwargs)
     69     if plot.document is None:
     70         plot.document = Document() if self_or_cls.notebook_context else curdoc()

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/plotting/renderer.py:217, in Renderer.get_plot(self_or_cls, obj, doc, renderer, comm, **kwargs)
    214     raise SkipRendering(msg.format(dims=dims))
    216 # Initialize DynamicMaps with first data item
--> 217 initialize_dynamic(obj)
    219 if not renderer:
    220     renderer = self_or_cls

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/plotting/util.py:270, in initialize_dynamic(obj)
    268     continue
    269 if not len(dmap):
--> 270     dmap[dmap._initial_key()]

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/spaces.py:1217, in DynamicMap.__getitem__(self, key)
   1215 # Not a cross product and nothing cached so compute element.
   1216 if cache is not None: return cache
-> 1217 val = self._execute_callback(*tuple_key)
   1218 if data_slice:
   1219     val = self._dataslice(val, data_slice)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/spaces.py:984, in DynamicMap._execute_callback(self, *args)
    981     kwargs['_memoization_hash_'] = hash_items
    983 with dynamicmap_memoization(self.callback, self.streams):
--> 984     retval = self.callback(*args, **kwargs)
    985 return self._style(retval)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/spaces.py:582, in Callable.__call__(self, *args, **kwargs)
    579     args, kwargs = (), dict(pos_kwargs, **kwargs)
    581 try:
--> 582     ret = self.callable(*args, **kwargs)
    583 except KeyError:
    584     # KeyError is caught separately because it is used to signal
    585     # invalid keys on DynamicMap and should not warn
    586     raise

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/util/__init__.py:1037, in Dynamic._dynamic_operation.<locals>.dynamic_operation(*key, **kwargs)
   1036 def dynamic_operation(*key, **kwargs):
-> 1037     key, obj = resolve(key, kwargs)
   1038     return apply(obj, *key, **kwargs)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/util/__init__.py:1026, in Dynamic._dynamic_operation.<locals>.resolve(key, kwargs)
   1024 elif isinstance(map_obj, DynamicMap) and map_obj._posarg_keys and not key:
   1025     key = tuple(kwargs[k] for k in map_obj._posarg_keys)
-> 1026 return key, map_obj[key]

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/spaces.py:1217, in DynamicMap.__getitem__(self, key)
   1215 # Not a cross product and nothing cached so compute element.
   1216 if cache is not None: return cache
-> 1217 val = self._execute_callback(*tuple_key)
   1218 if data_slice:
   1219     val = self._dataslice(val, data_slice)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/spaces.py:984, in DynamicMap._execute_callback(self, *args)
    981     kwargs['_memoization_hash_'] = hash_items
    983 with dynamicmap_memoization(self.callback, self.streams):
--> 984     retval = self.callback(*args, **kwargs)
    985 return self._style(retval)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/spaces.py:582, in Callable.__call__(self, *args, **kwargs)
    579     args, kwargs = (), dict(pos_kwargs, **kwargs)
    581 try:
--> 582     ret = self.callable(*args, **kwargs)
    583 except KeyError:
    584     # KeyError is caught separately because it is used to signal
    585     # invalid keys on DynamicMap and should not warn
    586     raise

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/util/__init__.py:1037, in Dynamic._dynamic_operation.<locals>.dynamic_operation(*key, **kwargs)
   1036 def dynamic_operation(*key, **kwargs):
-> 1037     key, obj = resolve(key, kwargs)
   1038     return apply(obj, *key, **kwargs)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/util/__init__.py:1026, in Dynamic._dynamic_operation.<locals>.resolve(key, kwargs)
   1024 elif isinstance(map_obj, DynamicMap) and map_obj._posarg_keys and not key:
   1025     key = tuple(kwargs[k] for k in map_obj._posarg_keys)
-> 1026 return key, map_obj[key]

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/spaces.py:1217, in DynamicMap.__getitem__(self, key)
   1215 # Not a cross product and nothing cached so compute element.
   1216 if cache is not None: return cache
-> 1217 val = self._execute_callback(*tuple_key)
   1218 if data_slice:
   1219     val = self._dataslice(val, data_slice)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/spaces.py:984, in DynamicMap._execute_callback(self, *args)
    981     kwargs['_memoization_hash_'] = hash_items
    983 with dynamicmap_memoization(self.callback, self.streams):
--> 984     retval = self.callback(*args, **kwargs)
    985 return self._style(retval)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/spaces.py:582, in Callable.__call__(self, *args, **kwargs)
    579     args, kwargs = (), dict(pos_kwargs, **kwargs)
    581 try:
--> 582     ret = self.callable(*args, **kwargs)
    583 except KeyError:
    584     # KeyError is caught separately because it is used to signal
    585     # invalid keys on DynamicMap and should not warn
    586     raise

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/util/__init__.py:1038, in Dynamic._dynamic_operation.<locals>.dynamic_operation(*key, **kwargs)
   1036 def dynamic_operation(*key, **kwargs):
   1037     key, obj = resolve(key, kwargs)
-> 1038     return apply(obj, *key, **kwargs)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/util/__init__.py:1030, in Dynamic._dynamic_operation.<locals>.apply(element, *key, **kwargs)
   1028 def apply(element, *key, **kwargs):
   1029     kwargs = dict(util.resolve_dependent_kwargs(self.p.kwargs), **kwargs)
-> 1030     processed = self._process(element, key, kwargs)
   1031     if (self.p.link_dataset and isinstance(element, Dataset) and
   1032         isinstance(processed, Dataset) and processed._dataset is None):
   1033         processed._dataset = element.dataset

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/util/__init__.py:1014, in Dynamic._process(self, element, key, kwargs)
   1012     return self.p.operation.process_element(element, key, **kwargs)
   1013 else:
-> 1014     return self.p.operation(element, **kwargs)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/hvplot/converter.py:1226, in HoloViewsConverter.__call__.<locals>.method_wrapper(ds, x, y)
   1225 def method_wrapper(ds, x, y):
-> 1226     el = method(x, y, data=ds.data)
   1227     el._transforms = dataset._transforms
   1228     el._dataset = ds

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/hvplot/converter.py:2188, in HoloViewsConverter.image(self, x, y, z, data)
   2186 element = self._get_element('image')
   2187 if self.geo: params['crs'] = self.crs
-> 2188 return (element(data, [x, y], z, **params).redim(**redim)
   2189         .apply(self._set_backends_opts, cur_opts=cur_opts, compat_opts=compat_opts))

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/element/raster.py:312, in Image.__init__(self, data, kdims, vdims, bounds, extents, xdensity, ydensity, rtol, **params)
    310     xdensity = xdensity if xdensity else util.compute_density(l, r, dim1, self._time_unit)
    311     ydensity = ydensity if ydensity else util.compute_density(b, t, dim2, self._time_unit)
--> 312 SheetCoordinateSystem.__init__(self, bounds, xdensity, ydensity)
    313 if non_finite:
    314    self.bounds = BoundingBox(points=((np.nan, np.nan), (np.nan, np.nan)))

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/sheetcoords.py:166, in SheetCoordinateSystem.__init__(self, bounds, xdensity, ydensity)
    162 self.__set_ydensity(ydensity or xdensity)
    164 self.lbrt = np.array(bounds.lbrt())
--> 166 r1,r2,c1,c2 = Slice._boundsspec2slicespec(self.lbrt,self)
    167 self.__shape = (r2-r1,c2-c1)

File ~/miniconda3/envs/kerchunk-cookbook/lib/python3.10/site-packages/holoviews/core/sheetcoords.py:518, in Slice._boundsspec2slicespec(boundsspec, scs)
    515 t_m,l_m = scs.sheet2matrix(l,t)
    516 b_m,r_m = scs.sheet2matrix(r,b)
--> 518 l_idx = int(np.ceil(l_m-0.5))
    519 t_idx = int(np.ceil(t_m-0.5))
    520 r_idx = int(np.floor(r_m+0.5))

ValueError: cannot convert float NaN to integer

:DynamicMap   [time]

Shut down the Dask cluster

client.shutdown()