Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

Calculate Surface Ocean Heat using CESM2 LENS data


Overview

  • This notebook is an adpation of a workflow in the NCAR gallery of the Pangeo collection

  • This notebook illustrates how to compute surface ocean heat content using potential temperature data from CESM2 Large Ensemble Dataset (Community Earth System Model 2) hosted on NCAR’s GDEX.

  • This data is open access and is accessed via OSDF using the pelicanFS package and demonstrates how you can stream data from NCAR’s GDEX

  • Please refer to the first chapter of this cookbook to learn more about OSDF, pelican or pelicanFS

Prerequisites

ConceptsImportanceNotes
Intro to Intake-ESMNecessaryUsed for searching CMIP6 data
Understanding of ZarrHelpfulFamiliarity with metadata structure
MatplotlibHelpfulPackage used for plotting
PelicanFSNecessaryThe python package used to stream data in this notebook
OSDFHelpfulOSDF is used to stream data in this notebook
  • Time to learn: 20 mins


Imports

Set up local dask cluster

Before we do any computation let us first set up a local cluster using dask

Loading...

Data Loading

Load CESM2 LENS data from NCAR’s GDEX

Loading...
Loading...
array(['osdf:///ncar-gdex/d010092/ocn/monthly/cesm2LE-historical-cmip6-TEMP.zarr', 'osdf:///ncar-gdex/d010092/ocn/monthly/cesm2LE-ssp370-cmip6-TEMP.zarr', 'osdf:///ncar-gdex/d010092/ocn/monthly/cesm2LE-ssp370-smbb-TEMP.zarr'], dtype=object)

--> The keys in the returned dictionary of datasets are constructed as follows:
	'component.experiment.frequency.forcing_variant'
Loading...
Loading...
dict_keys(['ocn.historical.monthly.cmip6', 'ocn.ssp370.monthly.smbb', 'ocn.ssp370.monthly.cmip6'])

Change units

Unit('centimeters')
Loading...
Loading...
  • Compute depth level deltas using the difference of z_t levels

Loading...

Compute Ocean Heat content for ocean surface

  • Ocean surface is considered to be the top 100m

  • The formula for this is:

    H=ρC∫0zT(z)dzH = \rho C \int_0^z T(z) dz

Where H is ocean heat content, the value we are trying to calculate,

ρ\rho is the density of sea water, 1026kg/m31026 kg/m^3 ,

CC is the specific heat of sea water, 3990J/(kgK)3990 J/(kg K) ,

zz is the depth limit of the calculation in meters,

and T(z)T(z) is the temperature at each depth in degrees Kelvin.

Loading...
Loading...
Loading...

Plot Ocean Heat

CPU times: user 443 ms, sys: 28.6 ms, total: 471 ms
Wall time: 466 ms
Loading...
2026-10-22 03:48:40,213 - distributed.worker - ERROR - Compute Failed
Key:       ('convert-convert_0-transpose-e6fb9c836fd2c42c20255265d80299c0', 27, 1, 0, 0, 0)
State:     executing
Task:  <Task ('convert-convert_0-transpose-e6fb9c836fd2c42c20255265d80299c0', 27, 1, 0, 0, 0) _execute_subgraph(...)>
Exception: 'ClientPayloadError("Response payload is not completed: <ContentLengthError: 400, message=\'Not enough data to satisfy content length header (received 14680064 of 85437122 bytes).\'>")'
Traceback: '  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/dask/array/core.py", line 140, in getter\n    c = np.asarray(c)\n        ^^^^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/core/indexing.py", line 691, in __array__\n    return np.asarray(to_numpy(self.get_duck_array()), dtype=dtype, copy=copy)\n                               ^^^^^^^^^^^^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/core/indexing.py", line 697, in get_duck_array\n    return self.array.get_duck_array()\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/core/indexing.py", line 930, in get_duck_array\n    return self.array.get_duck_array()\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/coding/common.py", line 80, in get_duck_array\n    return self.func(self.array.get_duck_array())\n                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/core/indexing.py", line 770, in get_duck_array\n    array = self.array[self.key]\n            ~~~~~~~~~~^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/backends/zarr.py", line 316, in __getitem__\n    return indexing.explicit_indexing_adapter(\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/core/indexing.py", line 1162, in explicit_indexing_adapter\n    result = raw_indexing_method(raw_key.tuple)\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/backends/zarr.py", line 279, in _getitem\n    return self._array[key]\n           ~~~~~~~~~~~^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/zarr/core.py", line 798, in __getitem__\n    result = self.get_orthogonal_selection(pure_selection, fields=fields)\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/zarr/core.py", line 1080, in get_orthogonal_selection\n    return self._get_selection(indexer=indexer, out=out, fields=fields)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/zarr/core.py", line 1343, in _get_selection\n    self._chunk_getitems(\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/zarr/core.py", line 2179, in _chunk_getitems\n    cdatas = self.chunk_store.getitems(ckeys, contexts=contexts)\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/zarr/storage.py", line 1435, in getitems\n    raise v\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/fsspec/asyn.py", line 359, in _run_coro\n    return await asyncio.wait_for(coro, timeout=timeout), i\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/asyncio/tasks.py", line 520, in wait_for\n    return await fut\n           ^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/fsspec/implementations/http.py", line 249, in _cat_file\n    out = await r.read()\n          ^^^^^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/aiohttp/client_reqrep.py", line 902, in read\n    self._body = await self.content.read()\n                 ^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/aiohttp/streams.py", line 448, in read\n    block = await self.readany()\n            ^^^^^^^^^^^^^^^^^^^^\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/aiohttp/streams.py", line 471, in readany\n    await self._wait("readany")\n  File "/home/runner/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/aiohttp/streams.py", line 365, in _wait\n    await waiter\n'

CPU times: user 11.7 s, sys: 2.11 s, total: 13.8 s
Wall time: 3min 8s
---------------------------------------------------------------------------
ContentLengthError                        Traceback (most recent call last)
File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/aiohttp/client_proto.py:145, in connection_lost()
    144 try:
--> 145     uncompleted = self._parser.feed_eof()
    146 except Exception as underlying_exc:

File aiohttp/_http_parser.pyx:625
--> 625     def feed_eof(self):
    626         cdef bytes desc

ContentLengthError: 400, message:
  Not enough data to satisfy content length header (received 14680064 of 85437122 bytes).

The above exception was the direct cause of the following exception:

ClientPayloadError                        Traceback (most recent call last)
Cell In[23], line 1
----> 1 get_ipython().run_cell_magic('time', '', 'hist_oceanheat_ano.plot()\n')

File <timed eval>:1
----> 1 'Could not get source, probably due dynamically evaluated source code.'

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/plot/accessor.py:48, in DataArrayPlotAccessor.__call__(self, **kwargs)
     46 @functools.wraps(dataarray_plot.plot, assigned=("__doc__", "__annotations__"))
     47 def __call__(self, **kwargs) -> Any:
---> 48     return dataarray_plot.plot(self._da, **kwargs)

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/plot/dataarray_plot.py:279, in plot(darray, row, col, col_wrap, ax, hue, subplot_kws, **kwargs)
    226 def plot(
    227     darray: DataArray,
    228     *,
   (...)    235     **kwargs: Any,
    236 ) -> Any:
    237     """
    238     Default plot of DataArray using :py:mod:`matplotlib:matplotlib.pyplot`.
    239 
   (...)    275     xarray.DataArray.squeeze
    276     """
    277     darray = darray.squeeze(
    278         d for d, s in darray.sizes.items() if s == 1 and d not in (row, col, hue)
--> 279     ).compute()
    281     plot_dims = set(darray.dims)
    282     plot_dims.discard(row)

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/core/dataarray.py:1254, in DataArray.compute(self, **kwargs)
   1224 """Trigger loading data into memory and return a new dataarray.
   1225 
   1226 Data will be computed and/or loaded from disk or a remote source.
   (...)   1251 Variable.compute
   1252 """
   1253 new = self.copy(deep=False)
-> 1254 return new.load(**kwargs)

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/core/dataarray.py:1180, in DataArray.load(self, **kwargs)
   1150 def load(self, **kwargs) -> Self:
   1151     """Trigger loading data into memory and return this dataarray.
   1152 
   1153     Data will be computed and/or loaded from disk or a remote source.
   (...)   1178     Variable.load
   1179     """
-> 1180     ds = self._to_temp_dataset().load(**kwargs)
   1181     new = self._from_temp_dataset(ds)
   1182     self._variable = new._variable

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/core/dataset.py:569, in Dataset.load(self, **kwargs)
    565         if chunked_data:
    566             chunkmanager = get_chunked_array_type(*chunked_data.values())
    567 
    568             # evaluate all the chunked arrays simultaneously
--> 569             evaluated_data: tuple[np.ndarray[Any, Any], ...] = chunkmanager.compute(
    570                 *chunked_data.values(), **kwargs
    571             )
    572 

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/namedarray/daskmanager.py:85, in DaskManager.compute(self, *data, **kwargs)
     80 def compute(
     81     self, *data: Any, **kwargs: Any
     82 ) -> tuple[np.ndarray[Any, _DType_co], ...]:
     83     from dask.array import compute
---> 85     return compute(*data, **kwargs)

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/dask/base.py:769, in compute(traverse, optimize_graph, scheduler, get, *args, **kwargs)
    766     expr = expr.optimize()
    767     keys = list(flatten(expr.__dask_keys__()))
--> 769     results = schedule(expr, keys, **kwargs)
    771 return repack(results)

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/core/indexing.py:691, in __array__()
    687 def __array__(
    688     self, dtype: DTypeLike | None = None, /, *, copy: bool | None = None
    689 ) -> np.ndarray:
    690     if Version(np.__version__) >= Version("2.0.0"):
--> 691         return np.asarray(to_numpy(self.get_duck_array()), dtype=dtype, copy=copy)
    693     else:
    694         return np.asarray(to_numpy(self.get_duck_array()), dtype=dtype)

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/core/indexing.py:697, in get_duck_array()
    696 def get_duck_array(self):
--> 697     return self.array.get_duck_array()

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/core/indexing.py:930, in get_duck_array()
    929 def get_duck_array(self):
--> 930     return self.array.get_duck_array()

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/coding/common.py:80, in get_duck_array()
     79 def get_duck_array(self):
---> 80     return self.func(self.array.get_duck_array())

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/core/indexing.py:770, in get_duck_array()
    767 from xarray.backends.common import BackendArray
    769 if isinstance(self.array, BackendArray):
--> 770     array = self.array[self.key]
    771 else:
    772     array = apply_indexer(self.array, self.key)

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/backends/zarr.py:316, in __getitem__()
    314 elif isinstance(key, indexing.OuterIndexer):
    315     method = self._oindex
--> 316 return indexing.explicit_indexing_adapter(
    317     key, array.shape, indexing.IndexingSupport.VECTORIZED, method
    318 )

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/core/indexing.py:1162, in explicit_indexing_adapter()
   1140 """Support explicit indexing by delegating to a raw indexing method.
   1141 
   1142 Outer and/or vectorized indexers are supported by indexing a second time
   (...)   1159 Indexing result, in the form of a duck numpy-array.
   1160 """
   1161 raw_key, numpy_indices = decompose_indexer(key, shape, indexing_support)
-> 1162 result = raw_indexing_method(raw_key.tuple)
   1163 if numpy_indices.tuple:
   1164     # index the loaded duck array
   1165     indexable = as_indexable(result)

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/xarray/backends/zarr.py:279, in _getitem()
    278 def _getitem(self, key):
--> 279     return self._array[key]

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/zarr/core.py:798, in __getitem__()
    796     result = self.vindex[selection]
    797 elif is_pure_orthogonal_indexing(pure_selection, self.ndim):
--> 798     result = self.get_orthogonal_selection(pure_selection, fields=fields)
    799 else:
    800     result = self.get_basic_selection(pure_selection, fields=fields)

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/zarr/core.py:1080, in get_orthogonal_selection()
   1077 # setup indexer
   1078 indexer = OrthogonalIndexer(selection, self)
-> 1080 return self._get_selection(indexer=indexer, out=out, fields=fields)

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/zarr/core.py:1343, in _get_selection()
   1340 if math.prod(out_shape) > 0:
   1341     # allow storage to get multiple items at once
   1342     lchunk_coords, lchunk_selection, lout_selection = zip(*indexer)
-> 1343     self._chunk_getitems(
   1344         lchunk_coords,
   1345         lchunk_selection,
   1346         out,
   1347         lout_selection,
   1348         drop_axes=indexer.drop_axes,
   1349         fields=fields,
   1350     )
   1351 if out.shape:
   1352     return out

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/zarr/core.py:2179, in _chunk_getitems()
   2177     if not isinstance(self._meta_array, np.ndarray):
   2178         contexts = ConstantMap(ckeys, constant=Context(meta_array=self._meta_array))
-> 2179     cdatas = self.chunk_store.getitems(ckeys, contexts=contexts)
   2181 for ckey, chunk_select, out_select in zip(ckeys, lchunk_selection, lout_selection):
   2182     if ckey in cdatas:

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/zarr/storage.py:1435, in getitems()
   1432     continue
   1433 elif isinstance(v, Exception):
   1434     # Raise any other exception
-> 1435     raise v
   1436 else:
   1437     # The function calling this method may not recognize the transformed
   1438     # keys, so we send the values returned by self.map.getitems back into
   1439     # the original key space.
   1440     results[keys_transformed[k]] = v

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/fsspec/asyn.py:359, in _run_coro()
    357 async def _run_coro(coro, i):
    358     try:
--> 359         return await asyncio.wait_for(coro, timeout=timeout), i
    360     except Exception as e:
    361         if not return_exceptions:

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/asyncio/tasks.py:520, in wait_for()
    517         raise TimeoutError from exc
    519 async with timeouts.timeout(timeout):
--> 520     return await fut

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/fsspec/implementations/http.py:249, in _cat_file()
    247 session = await self.set_session()
    248 async with session.get(self.encode_url(url), **kw) as r:
--> 249     out = await r.read()
    250     self._raise_not_found_for_status(r, url)
    251 return out

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/aiohttp/client_reqrep.py:902, in read()
    900 if self._body is None:
    901     try:
--> 902         self._body = await self.content.read()
    903         for trace in self._traces:
    904             await trace.send_response_chunk_received(
    905                 self.method, self.url, self._body
    906             )

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/aiohttp/streams.py:448, in read()
    446 blocks = []
    447 while True:
--> 448     block = await self.readany()
    449     if not block:
    450         break

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/aiohttp/streams.py:471, in readany()
    467 # TODO: should be `if` instead of `while`
    468 # because waiter maybe triggered on chunk end,
    469 # without feeding any data
    470 while not self._buffer and not self._eof:
--> 471     await self._wait("readany")
    473 return self._read_nowait(-1)

File ~/micromamba/envs/osdf-cookbook/lib/python3.12/site-packages/aiohttp/streams.py:365, in _wait()
    363 try:
    364     with self._timer:
--> 365         await waiter
    366 finally:
    367     self._waiter = None

ClientPayloadError: Response payload is not completed: <ContentLengthError: 400, message='Not enough data to satisfy content length header (received 14680064 of 85437122 bytes).'>

Indeed! The surface ocean is trapping heat as the globe warms!


Summary

In this notebook we used sea temperature data from the Community Earth System Model (CESM2) Large Ensemble dataset to compute surface ocean heat and convince ourselves that the surface ocean is trapping extra heat as the globe warms. We used an intake-ESM catalog backed by pelican links to stream data from NCAR’s Geoscience Data Exchange via NCAR’s OSDF origin!

What’s next?

In the next notebook, we will see how to load data from multiple OSDF origins into a workflow. We will stream CMIP6 model data from AWS and observational data from NCAR’s GDEX.

Resources and references