Load CMIP6 Data with Intake-ESM

---

Overview

Intake-ESM is an experimental new package that aims to provide a higher-level interface to searching and loading Earth System Model data archives, such as CMIP6. The package is under very active development, and features may be unstable. Please report any issues or suggestions on GitHub.

Prerequisites

Concepts	Importance	Notes
Intro to Xarray	Necessary
Understanding of NetCDF	Helpful	Familiarity with metadata structure

• Time to learn: 5 minutes

---

Imports

import xarray as xr
xr.set_options(display_style='html')
import intake
%matplotlib inline

Loading Data

Intake ESM works by parsing an ESM Collection Spec and converting it to an Intake catalog. The collection spec is stored in a .json file. Here we open it using Intake.

cat_url = "https://storage.googleapis.com/cmip6/pangeo-cmip6.json"
col = intake.open_esm_datastore(cat_url)
col

pangeo-cmip6 catalog with 7674 dataset(s) from 514818 asset(s):

	unique
activity_id	18
institution_id	36
source_id	88
experiment_id	170
member_id	657
table_id	37
variable_id	700
grid_label	10
zstore	514818
dcpp_init_year	60
version	736
derived_variable_id	0

We can now use Intake methods to search the collection, and, if desired, export a Pandas dataframe.

cat = col.search(experiment_id=['historical', 'ssp585'], table_id='Oyr', variable_id='o2',
                 grid_label='gn')
cat.df

	activity_id	institution_id	source_id	experiment_id	member_id	table_id	variable_id	grid_label	zstore	dcpp_init_year	version
0	CMIP	IPSL	IPSL-CM6A-LR	historical	r8i1p1f1	Oyr	o2	gn	gs://cmip6/CMIP6/CMIP/IPSL/IPSL-CM6A-LR/histor...	NaN	20180803
1	CMIP	IPSL	IPSL-CM6A-LR	historical	r5i1p1f1	Oyr	o2	gn	gs://cmip6/CMIP6/CMIP/IPSL/IPSL-CM6A-LR/histor...	NaN	20180803
2	CMIP	IPSL	IPSL-CM6A-LR	historical	r26i1p1f1	Oyr	o2	gn	gs://cmip6/CMIP6/CMIP/IPSL/IPSL-CM6A-LR/histor...	NaN	20180803
3	CMIP	IPSL	IPSL-CM6A-LR	historical	r2i1p1f1	Oyr	o2	gn	gs://cmip6/CMIP6/CMIP/IPSL/IPSL-CM6A-LR/histor...	NaN	20180803
4	CMIP	IPSL	IPSL-CM6A-LR	historical	r6i1p1f1	Oyr	o2	gn	gs://cmip6/CMIP6/CMIP/IPSL/IPSL-CM6A-LR/histor...	NaN	20180803
...	...	...	...	...	...	...	...	...	...	...	...
168	CMIP	CSIRO	ACCESS-ESM1-5	historical	r11i1p1f1	Oyr	o2	gn	gs://cmip6/CMIP6/CMIP/CSIRO/ACCESS-ESM1-5/hist...	NaN	20200803
169	CMIP	EC-Earth-Consortium	EC-Earth3-CC	historical	r1i1p1f1	Oyr	o2	gn	gs://cmip6/CMIP6/CMIP/EC-Earth-Consortium/EC-E...	NaN	20210113
170	ScenarioMIP	EC-Earth-Consortium	EC-Earth3-CC	ssp585	r1i1p1f1	Oyr	o2	gn	gs://cmip6/CMIP6/ScenarioMIP/EC-Earth-Consorti...	NaN	20210113
171	CMIP	CMCC	CMCC-ESM2	historical	r1i1p1f1	Oyr	o2	gn	gs://cmip6/CMIP6/CMIP/CMCC/CMCC-ESM2/historica...	NaN	20210114
172	ScenarioMIP	CMCC	CMCC-ESM2	ssp585	r1i1p1f1	Oyr	o2	gn	gs://cmip6/CMIP6/ScenarioMIP/CMCC/CMCC-ESM2/ss...	NaN	20210126

173 rows × 11 columns

Intake knows how to automatically open the Datasets using Xarray. Furthermore, Intake-ESM contains special logic to concatenate and merge the individual results of our query into larger, more high-level aggregated Xarray Datasets.

dset_dict = cat.to_dataset_dict(zarr_kwargs={'consolidated': True})
list(dset_dict.keys())

--> The keys in the returned dictionary of datasets are constructed as follows:
	'activity_id.institution_id.source_id.experiment_id.table_id.grid_label'

100.00% [27/27 01:16<00:00]

['CMIP.MRI.MRI-ESM2-0.historical.Oyr.gn',
 'CMIP.EC-Earth-Consortium.EC-Earth3-CC.historical.Oyr.gn',
 'ScenarioMIP.IPSL.IPSL-CM6A-LR.ssp585.Oyr.gn',
 'ScenarioMIP.NCAR.CESM2.ssp585.Oyr.gn',
 'CMIP.CCCma.CanESM5.historical.Oyr.gn',
 'CMIP.HAMMOZ-Consortium.MPI-ESM-1-2-HAM.historical.Oyr.gn',
 'ScenarioMIP.CCCma.CanESM5-CanOE.ssp585.Oyr.gn',
 'CMIP.IPSL.IPSL-CM5A2-INCA.historical.Oyr.gn',
 'CMIP.NCC.NorESM2-LM.historical.Oyr.gn',
 'ScenarioMIP.NCC.NorESM2-LM.ssp585.Oyr.gn',
 'ScenarioMIP.NCC.NorESM2-MM.ssp585.Oyr.gn',
 'CMIP.NCC.NorESM2-MM.historical.Oyr.gn',
 'CMIP.MIROC.MIROC-ES2L.historical.Oyr.gn',
 'ScenarioMIP.MPI-M.MPI-ESM1-2-LR.ssp585.Oyr.gn',
 'CMIP.CMCC.CMCC-ESM2.historical.Oyr.gn',
 'ScenarioMIP.DKRZ.MPI-ESM1-2-HR.ssp585.Oyr.gn',
 'ScenarioMIP.MRI.MRI-ESM2-0.ssp585.Oyr.gn',
 'ScenarioMIP.CMCC.CMCC-ESM2.ssp585.Oyr.gn',
 'ScenarioMIP.DWD.MPI-ESM1-2-HR.ssp585.Oyr.gn',
 'CMIP.CSIRO.ACCESS-ESM1-5.historical.Oyr.gn',
 'CMIP.MPI-M.MPI-ESM1-2-LR.historical.Oyr.gn',
 'ScenarioMIP.MIROC.MIROC-ES2L.ssp585.Oyr.gn',
 'ScenarioMIP.EC-Earth-Consortium.EC-Earth3-CC.ssp585.Oyr.gn',
 'CMIP.CCCma.CanESM5-CanOE.historical.Oyr.gn',
 'ScenarioMIP.CCCma.CanESM5.ssp585.Oyr.gn',
 'CMIP.IPSL.IPSL-CM6A-LR.historical.Oyr.gn',
 'CMIP.MPI-M.MPI-ESM1-2-HR.historical.Oyr.gn']

ds = dset_dict['CMIP.CCCma.CanESM5.historical.Oyr.gn']
ds

<xarray.Dataset> Size: 109GB
Dimensions:             (member_id: 35, dcpp_init_year: 1, time: 165, lev: 45,
                         j: 291, i: 360, bnds: 2, vertices: 4)
Coordinates:
  * i                   (i) int32 1kB 0 1 2 3 4 5 6 ... 354 355 356 357 358 359
  * j                   (j) int32 1kB 0 1 2 3 4 5 6 ... 285 286 287 288 289 290
    latitude            (j, i) float64 838kB dask.array<chunksize=(291, 360), meta=np.ndarray>
  * lev                 (lev) float64 360B 3.047 9.454 ... 5.375e+03 5.625e+03
    lev_bnds            (lev, bnds) float64 720B dask.array<chunksize=(45, 2), meta=np.ndarray>
    longitude           (j, i) float64 838kB dask.array<chunksize=(291, 360), meta=np.ndarray>
  * time                (time) object 1kB 1850-07-02 12:00:00 ... 2014-07-02 ...
    time_bnds           (time, bnds) object 3kB dask.array<chunksize=(165, 2), meta=np.ndarray>
    vertices_latitude   (j, i, vertices) float64 3MB dask.array<chunksize=(291, 360, 4), meta=np.ndarray>
    vertices_longitude  (j, i, vertices) float64 3MB dask.array<chunksize=(291, 360, 4), meta=np.ndarray>
  * member_id           (member_id) object 280B 'r10i1p1f1' ... 'r9i1p2f1'
  * dcpp_init_year      (dcpp_init_year) float64 8B nan
Dimensions without coordinates: bnds, vertices
Data variables:
    o2                  (member_id, dcpp_init_year, time, lev, j, i) float32 109GB dask.array<chunksize=(1, 1, 12, 45, 291, 360), meta=np.ndarray>
Attributes: (12/52)
    Conventions:                      CF-1.7 CMIP-6.2
    YMDH_branch_time_in_child:        1850:01:01:00
    activity_id:                      CMIP
    branch_method:                    Spin-up documentation
    branch_time_in_child:             0.0
    cmor_version:                     3.4.0
    ...                               ...
    intake_esm_attrs:table_id:        Oyr
    intake_esm_attrs:variable_id:     o2
    intake_esm_attrs:grid_label:      gn
    intake_esm_attrs:version:         20190429
    intake_esm_attrs:_data_format_:   zarr
    intake_esm_dataset_key:           CMIP.CCCma.CanESM5.historical.Oyr.gn

xarray.Dataset

• Dimensions:
  • member_id: 35
  • dcpp_init_year: 1
  • time: 165
  • lev: 45
  • j: 291
  • i: 360
  • bnds: 2
  • vertices: 4
• Coordinates: (12)
  • i
    (i)
    int32
    0 1 2 3 4 5 ... 355 356 357 358 359

    long_name :

    cell index along first dimension

    units :

    1

    array([  0,   1,   2, ..., 357, 358, 359], dtype=int32)

  • j
    (j)
    int32
    0 1 2 3 4 5 ... 286 287 288 289 290

    long_name :

    cell index along second dimension

    units :

    1

    array([  0,   1,   2, ..., 288, 289, 290], dtype=int32)

  • latitude
    (j, i)
    float64
    dask.array<chunksize=(291, 360), meta=np.ndarray>

    bounds :

    vertices_latitude

    long_name :

    latitude

    standard_name :

    latitude

    units :

    degrees_north

    Array Chunk Bytes 818.44 kiB 818.44 kiB Shape (291, 360) (291, 360) Dask graph 1 chunks in 170 graph layers Data type float64 numpy.ndarray

  • lev
    (lev)
    float64
    3.047 9.454 ... 5.375e+03 5.625e+03

    axis :

    Z

    bounds :

    lev_bnds

    long_name :

    ocean depth coordinate

    positive :

    down

    standard_name :

    depth

    units :

    m

    array([3.046773e+00, 9.454049e+00, 1.636397e+01, 2.389871e+01, 3.220929e+01,
           4.148185e+01, 5.194513e+01, 6.387905e+01, 7.762451e+01, 9.359412e+01,
           1.122835e+02, 1.342823e+02, 1.602840e+02, 1.910925e+02, 2.276233e+02,
           2.708962e+02, 3.220169e+02, 3.821444e+02, 4.524429e+02, 5.340197e+02,
           6.278525e+02, 7.347150e+02, 8.551112e+02, 9.892289e+02, 1.136922e+03,
           1.297724e+03, 1.470893e+03, 1.655472e+03, 1.850365e+03, 2.054414e+03,
           2.266454e+03, 2.485371e+03, 2.710133e+03, 2.939812e+03, 3.173588e+03,
           3.410756e+03, 3.650712e+03, 3.892950e+03, 4.137047e+03, 4.382654e+03,
           4.629485e+03, 4.877303e+03, 5.125919e+03, 5.375177e+03, 5.624952e+03])

  • lev_bnds
    (lev, bnds)
    float64
    dask.array<chunksize=(45, 2), meta=np.ndarray>

    Array Chunk Bytes 720 B 720 B Shape (45, 2) (45, 2) Dask graph 1 chunks in 170 graph layers Data type float64 numpy.ndarray

  • longitude
    (j, i)
    float64
    dask.array<chunksize=(291, 360), meta=np.ndarray>

    bounds :

    vertices_longitude

    long_name :

    longitude

    standard_name :

    longitude

    units :

    degrees_east

    Array Chunk Bytes 818.44 kiB 818.44 kiB Shape (291, 360) (291, 360) Dask graph 1 chunks in 170 graph layers Data type float64 numpy.ndarray

  • time
    (time)
    object
    1850-07-02 12:00:00 ... 2014-07-...

    axis :

    T

    bounds :

    time_bnds

    long_name :

    time

    standard_name :

    time

    array([cftime.DatetimeNoLeap(1850, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1851, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1852, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1853, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1854, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1855, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1856, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1857, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1858, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1859, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1860, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1861, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1862, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1863, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1864, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1865, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1866, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1867, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1868, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1869, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1870, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1871, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1872, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1873, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1874, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1875, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1876, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1877, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1878, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1879, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1880, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1881, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1882, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1883, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1884, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1885, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1886, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1887, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1888, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1889, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1890, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1891, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1892, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1893, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1894, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1895, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1896, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1897, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1898, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1899, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1900, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1901, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1902, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1903, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1904, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1905, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1906, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1907, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1908, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1909, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1910, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1911, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1912, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1913, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1914, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1915, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1916, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1917, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1918, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1919, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1920, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1921, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1922, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1923, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1924, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1925, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1926, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1927, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1928, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1929, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1930, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1931, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1932, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1933, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1934, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1935, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1936, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1937, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1938, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1939, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1940, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1941, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1942, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1943, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1944, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1945, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1946, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1947, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1948, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1949, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1950, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1951, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1952, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1953, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1954, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1955, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1956, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1957, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1958, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1959, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1960, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1961, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1962, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1963, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1964, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1965, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1966, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1967, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1968, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1969, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1970, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1971, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1972, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1973, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1974, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1975, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1976, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1977, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1978, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1979, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1980, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1981, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1982, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1983, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1984, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1985, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1986, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1987, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1988, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1989, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1990, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1991, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1992, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1993, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1994, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1995, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1996, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1997, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1998, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1999, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2000, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2001, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2002, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2003, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2004, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2005, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2006, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2007, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2008, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2009, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2010, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2011, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2012, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2013, 7, 2, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2014, 7, 2, 12, 0, 0, 0, has_year_zero=True)],
          dtype=object)

  • time_bnds
    (time, bnds)
    object
    dask.array<chunksize=(165, 2), meta=np.ndarray>

    Array Chunk Bytes 2.58 kiB 2.58 kiB Shape (165, 2) (165, 2) Dask graph 1 chunks in 170 graph layers Data type object numpy.ndarray

  • vertices_latitude
    (j, i, vertices)
    float64
    dask.array<chunksize=(291, 360, 4), meta=np.ndarray>

    units :

    degrees_north

    Array Chunk Bytes 3.20 MiB 3.20 MiB Shape (291, 360, 4) (291, 360, 4) Dask graph 1 chunks in 170 graph layers Data type float64 numpy.ndarray

  • vertices_longitude
    (j, i, vertices)
    float64
    dask.array<chunksize=(291, 360, 4), meta=np.ndarray>

    units :

    degrees_east

    Array Chunk Bytes 3.20 MiB 3.20 MiB Shape (291, 360, 4) (291, 360, 4) Dask graph 1 chunks in 170 graph layers Data type float64 numpy.ndarray

  • member_id
    (member_id)
    object
    'r10i1p1f1' ... 'r9i1p2f1'

    array(['r10i1p1f1', 'r10i1p2f1', 'r11i1p1f1', 'r12i1p1f1', 'r13i1p1f1',
           'r14i1p1f1', 'r15i1p1f1', 'r16i1p1f1', 'r17i1p1f1', 'r18i1p1f1',
           'r19i1p1f1', 'r1i1p1f1', 'r1i1p2f1', 'r20i1p1f1', 'r21i1p1f1',
           'r22i1p1f1', 'r23i1p1f1', 'r24i1p1f1', 'r25i1p1f1', 'r2i1p1f1',
           'r2i1p2f1', 'r3i1p1f1', 'r3i1p2f1', 'r4i1p1f1', 'r4i1p2f1', 'r5i1p1f1',
           'r5i1p2f1', 'r6i1p1f1', 'r6i1p2f1', 'r7i1p1f1', 'r7i1p2f1', 'r8i1p1f1',
           'r8i1p2f1', 'r9i1p1f1', 'r9i1p2f1'], dtype=object)

  • dcpp_init_year
    (dcpp_init_year)
    float64
    nan

    array([nan])

• Data variables: (1)
  • o2
    (member_id, dcpp_init_year, time, lev, j, i)
    float32
    dask.array<chunksize=(1, 1, 12, 45, 291, 360), meta=np.ndarray>

    cell_measures :

    area: areacello volume: volcello

    cell_methods :

    area: mean where sea time: mean

    comment :

    'Mole concentration' means number of moles per unit volume, also called 'molarity', and is used in the construction mole_concentration_of_X_in_Y, where X is a material constituent of Y. A chemical or biological species denoted by X may be described by a single term such as 'nitrogen' or a phrase such as 'nox_expressed_as_nitrogen'.

    history :

    meanm2y_mltby1em3

    long_name :

    Dissolved Oxygen Concentration

    original_name :

    O2

    standard_name :

    mole_concentration_of_dissolved_molecular_oxygen_in_sea_water

    units :

    mol m-3

    Array Chunk Bytes 101.42 GiB 215.80 MiB Shape (35, 1, 165, 45, 291, 360) (1, 1, 12, 45, 291, 360) Dask graph 490 chunks in 106 graph layers Data type float32 numpy.ndarray

• Indexes: (6)
  • i
    PandasIndex

    PandasIndex(Index([  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,
           ...
           350, 351, 352, 353, 354, 355, 356, 357, 358, 359],
          dtype='int32', name='i', length=360))

  • j
    PandasIndex

    PandasIndex(Index([  0,   1,   2,   3,   4,   5,   6,   7,   8,   9,
           ...
           281, 282, 283, 284, 285, 286, 287, 288, 289, 290],
          dtype='int32', name='j', length=291))

  • lev
    PandasIndex

    PandasIndex(Index([3.0467727184295654,  9.454049110412598,  16.36396598815918,
           23.898710250854492,  32.20928955078125,  41.48185348510742,
            51.94512939453125, 63.879051208496094,     77.62451171875,
            93.59412384033203, 112.28348541259766,  134.2822723388672,
           160.28399658203125, 191.09251403808594, 227.62332153320312,
            270.8962097167969, 322.01690673828125,  382.1443786621094,
            452.4429016113281,  534.0197143554688,  627.8524780273438,
            734.7150268554688,  855.1112060546875,  989.2288818359375,
           1136.9219970703125,  1297.724365234375, 1470.8929443359375,
              1655.4716796875,  1850.365478515625,  2054.413818359375,
             2266.45361328125,  2485.370849609375,   2710.13330078125,
            2939.811767578125,   3173.58837890625,     3410.755859375,
              3650.7119140625,   3892.94970703125,        4137.046875,
               4382.654296875,   4629.48486328125,   4877.30322265625,
              5125.9189453125,    5375.1767578125,   5624.95166015625],
          dtype='float64', name='lev'))

  • time
    PandasIndex

    PandasIndex(CFTimeIndex([1850-07-02 12:00:00, 1851-07-02 12:00:00, 1852-07-02 12:00:00,
                 1853-07-02 12:00:00, 1854-07-02 12:00:00, 1855-07-02 12:00:00,
                 1856-07-02 12:00:00, 1857-07-02 12:00:00, 1858-07-02 12:00:00,
                 1859-07-02 12:00:00,
                 ...
                 2005-07-02 12:00:00, 2006-07-02 12:00:00, 2007-07-02 12:00:00,
                 2008-07-02 12:00:00, 2009-07-02 12:00:00, 2010-07-02 12:00:00,
                 2011-07-02 12:00:00, 2012-07-02 12:00:00, 2013-07-02 12:00:00,
                 2014-07-02 12:00:00],
                dtype='object', length=165, calendar='noleap', freq='365D'))

  • member_id
    PandasIndex

    PandasIndex(Index(['r10i1p1f1', 'r10i1p2f1', 'r11i1p1f1', 'r12i1p1f1', 'r13i1p1f1',
           'r14i1p1f1', 'r15i1p1f1', 'r16i1p1f1', 'r17i1p1f1', 'r18i1p1f1',
           'r19i1p1f1', 'r1i1p1f1', 'r1i1p2f1', 'r20i1p1f1', 'r21i1p1f1',
           'r22i1p1f1', 'r23i1p1f1', 'r24i1p1f1', 'r25i1p1f1', 'r2i1p1f1',
           'r2i1p2f1', 'r3i1p1f1', 'r3i1p2f1', 'r4i1p1f1', 'r4i1p2f1', 'r5i1p1f1',
           'r5i1p2f1', 'r6i1p1f1', 'r6i1p2f1', 'r7i1p1f1', 'r7i1p2f1', 'r8i1p1f1',
           'r8i1p2f1', 'r9i1p1f1', 'r9i1p2f1'],
          dtype='object', name='member_id'))

  • dcpp_init_year
    PandasIndex

    PandasIndex(Index([nan], dtype='float64', name='dcpp_init_year'))

• Attributes: (52)

  Conventions :

  CF-1.7 CMIP-6.2

  YMDH_branch_time_in_child :

  1850:01:01:00

  activity_id :

  CMIP

  branch_method :

  Spin-up documentation

  branch_time_in_child :

  0.0

  cmor_version :

  3.4.0

  contact :

  ec.cccma.info-info.ccmac.ec@canada.ca

  data_specs_version :

  01.00.29

  experiment :

  all-forcing simulation of the recent past

  experiment_id :

  historical

  external_variables :

  areacello volcello

  forcing_index :

  1

  frequency :

  yr

  grid :

  ORCA1 tripolar grid, 1 deg with refinement to 1/3 deg within 20 degrees of the equator; 361 x 290 longitude/latitude; 45 vertical levels; top grid cell 0-6.19 m

  grid_label :

  gn

  initialization_index :

  1

  institution :

  Canadian Centre for Climate Modelling and Analysis, Environment and Climate Change Canada, Victoria, BC V8P 5C2, Canada

  institution_id :

  CCCma

  license :

  CMIP6 model data produced by The Government of Canada (Canadian Centre for Climate Modelling and Analysis, Environment and Climate Change Canada) is licensed under a Creative Commons Attribution ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and at https:///pcmdi.llnl.gov/. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law.

  mip_era :

  CMIP6

  nominal_resolution :

  100 km

  parent_activity_id :

  CMIP

  parent_experiment_id :

  piControl

  parent_mip_era :

  CMIP6

  parent_source_id :

  CanESM5

  parent_time_units :

  days since 1850-01-01 0:0:0.0

  product :

  model-output

  realm :

  ocnBgchem

  references :

  Geophysical Model Development Special issue on CanESM5 (https://www.geosci-model-dev.net/special_issues.html)

  source :

  CanESM5 (2019): aerosol: interactive atmos: CanAM5 (T63L49 native atmosphere, T63 Linear Gaussian Grid; 128 x 64 longitude/latitude; 49 levels; top level 1 hPa) atmosChem: specified oxidants for aerosols land: CLASS3.6/CTEM1.2 landIce: specified ice sheets ocean: NEMO3.4.1 (ORCA1 tripolar grid, 1 deg with refinement to 1/3 deg within 20 degrees of the equator; 361 x 290 longitude/latitude; 45 vertical levels; top grid cell 0-6.19 m) ocnBgchem: Canadian Model of Ocean Carbon (CMOC); NPZD ecosystem with OMIP prescribed carbonate chemistry seaIce: LIM2

  source_id :

  CanESM5

  source_type :

  AOGCM

  sub_experiment :

  none

  sub_experiment_id :

  none

  table_id :

  Oyr

  table_info :

  Creation Date:(20 February 2019) MD5:374fbe5a2bcca535c40f7f23da271e49

  title :

  CanESM5 output prepared for CMIP6

  variable_id :

  o2

  version :

  v20190429

  status :

  2019-10-25;created;by nhn2@columbia.edu

  version_id :

  v20190429

  intake_esm_vars :

  ['o2']

  intake_esm_attrs:activity_id :

  CMIP

  intake_esm_attrs:institution_id :

  CCCma

  intake_esm_attrs:source_id :

  CanESM5

  intake_esm_attrs:experiment_id :

  historical

  intake_esm_attrs:table_id :

  Oyr

  intake_esm_attrs:variable_id :

  o2

  intake_esm_attrs:grid_label :

  gn

  intake_esm_attrs:version :

  20190429

  intake_esm_attrs:_data_format_ :

  zarr

  intake_esm_dataset_key :

  CMIP.CCCma.CanESM5.historical.Oyr.gn

---

Summary

In this notebook, we used Intake-ESM to open an Xarray Dataset for one particular model and experiment.

What’s next?

We will see an example of downloading a dataset with fsspec and zarr.

Resources and references

• Original notebook in the Pangeo Gallery by Henri Drake and Ryan Abernathey