Search and Load CMIP6 Data via ESGF/OPeNDAP

---

Overview

This notebook shows how to search and load data via Earth System Grid Federation infrastructure. This infrastructure works great and is the foundation of the CMIP6 distribution system.

The main technologies used here are the ESGF search API, used to figure out what data we want, and OPeNDAP, a remote data access protocol over HTTP.

Prerequisites

Concepts	Importance	Notes
Intro to Xarray	Necessary
Understanding of NetCDF	Helpful	Familiarity with metadata structure

• Time to learn: 10 minutes

---

Imports

import warnings

from distributed import Client
import holoviews as hv
import hvplot.xarray
from matplotlib import pyplot as plt
import numpy as np
import pandas as pd
from pyesgf.search import SearchConnection
import xarray as xr

xr.set_options(display_style='html')
warnings.filterwarnings("ignore")
hv.extension('bokeh')

client = Client()
client

Client

Client-d6e5ce56-8ccb-11ef-8bdb-6045bdb5c525

Connection method: Cluster object	Cluster type: distributed.LocalCluster
Dashboard: http://127.0.0.1:8787/status

Cluster Info

LocalCluster

ffddc8e7

Dashboard: http://127.0.0.1:8787/status	Workers: 4
Total threads: 4	Total memory: 15.61 GiB
Status: running	Using processes: True

Scheduler Info

Scheduler

Scheduler-e844c40f-9e32-4ae3-946c-87a05fe22ba8

Comm: tcp://127.0.0.1:38861	Workers: 4
Dashboard: http://127.0.0.1:8787/status	Total threads: 4
Started: Just now	Total memory: 15.61 GiB

Workers

Worker: 0

Comm: tcp://127.0.0.1:34585	Total threads: 1
Dashboard: http://127.0.0.1:41513/status	Memory: 3.90 GiB
Nanny: tcp://127.0.0.1:44469
Local directory: /tmp/dask-scratch-space/worker-2z533llh

Worker: 1

Comm: tcp://127.0.0.1:39049	Total threads: 1
Dashboard: http://127.0.0.1:45033/status	Memory: 3.90 GiB
Nanny: tcp://127.0.0.1:45563
Local directory: /tmp/dask-scratch-space/worker-uryy7802

Worker: 2

Comm: tcp://127.0.0.1:40629	Total threads: 1
Dashboard: http://127.0.0.1:38329/status	Memory: 3.90 GiB
Nanny: tcp://127.0.0.1:34139
Local directory: /tmp/dask-scratch-space/worker-7xxpz9sk

Worker: 3

Comm: tcp://127.0.0.1:38561	Total threads: 1
Dashboard: http://127.0.0.1:46737/status	Memory: 3.90 GiB
Nanny: tcp://127.0.0.1:40793
Local directory: /tmp/dask-scratch-space/worker-h_lgcmew

Search using ESGF API

Fortunately, there is an ESGF API implemented in Python - pyesgf, which requires three major steps:

• Establish a search connection

• Query your data

• Extract the urls to your data

Once you have this information, you can load the data into an xarray.Dataset.

Configure the connection to a data server

First, we configure our connection to some server, using the distributed option (distrib=False). In this case, we are searching from the Lawerence Livermore National Lab (LLNL) data node.

conn = SearchConnection('https://esgf-node.llnl.gov/esg-search',
                        distrib=False)

Query our dataset

We are interested in a single experiment from CMIP6 - one of the Community Earth System Model version 2 (CESM2) runs, specifically the historical part of the simulation.

We are also interested in a single variable - temperature at the surface (tas), with a single ensemble member (r10i1p1f1)

ctx = conn.new_context(
    facets='project,experiment_id',
    project='CMIP6',
    table_id='Amon',
    institution_id="NCAR",
    experiment_id='historical',
    source_id='CESM2',
    variable='tas',
    variant_label='r10i1p1f1',
)

Extract the OpenDAP urls

In order to access the datasets, we need the urls to the data. Once we have these, we can read the data remotely!

result = ctx.search()[0]
files = result.file_context().search()
files

<pyesgf.search.results.ResultSet at 0x7f801b8d8ee0>

The files object is not immediately helpful - we need to extract the opendap_url method from this.

files[0].opendap_url

'http://aims3.llnl.gov/thredds/dodsC/css03_data/CMIP6/CMIP/NCAR/CESM2/historical/r10i1p1f1/Amon/tas/gn/v20190313/tas_Amon_CESM2_historical_r10i1p1f1_gn_185001-189912.nc'

We can use this for the whole list using list comprehension, as shown below.

opendap_urls = [file.opendap_url for file in files]
opendap_urls

['http://aims3.llnl.gov/thredds/dodsC/css03_data/CMIP6/CMIP/NCAR/CESM2/historical/r10i1p1f1/Amon/tas/gn/v20190313/tas_Amon_CESM2_historical_r10i1p1f1_gn_185001-189912.nc',
 'http://aims3.llnl.gov/thredds/dodsC/css03_data/CMIP6/CMIP/NCAR/CESM2/historical/r10i1p1f1/Amon/tas/gn/v20190313/tas_Amon_CESM2_historical_r10i1p1f1_gn_190001-194912.nc',
 'http://aims3.llnl.gov/thredds/dodsC/css03_data/CMIP6/CMIP/NCAR/CESM2/historical/r10i1p1f1/Amon/tas/gn/v20190313/tas_Amon_CESM2_historical_r10i1p1f1_gn_195001-199912.nc',
 'http://aims3.llnl.gov/thredds/dodsC/css03_data/CMIP6/CMIP/NCAR/CESM2/historical/r10i1p1f1/Amon/tas/gn/v20190313/tas_Amon_CESM2_historical_r10i1p1f1_gn_200001-201412.nc']

Read the data into an xarray.Dataset

Now that we have our urls to the data, we can use open multifile dataset (open_mfdataset) to read the data, combining the coordinates and chunking by time.

Xarray, together with the netCDF4 Python library, allow lazy loading.

ds = xr.open_mfdataset(opendap_urls,
                       combine='by_coords',
                       chunks={'time':480})
ds

<xarray.Dataset> Size: 453MB
Dimensions:    (time: 1980, nbnd: 2, lat: 192, lon: 288)
Coordinates:
  * lat        (lat) float64 2kB -90.0 -89.06 -88.12 -87.17 ... 88.12 89.06 90.0
  * lon        (lon) float64 2kB 0.0 1.25 2.5 3.75 ... 355.0 356.2 357.5 358.8
  * time       (time) object 16kB 1850-01-15 12:00:00 ... 2014-12-15 12:00:00
Dimensions without coordinates: nbnd
Data variables:
    time_bnds  (time, nbnd) object 32kB dask.array<chunksize=(480, 2), meta=np.ndarray>
    lat_bnds   (time, lat, nbnd) float64 6MB dask.array<chunksize=(600, 192, 2), meta=np.ndarray>
    lon_bnds   (time, lon, nbnd) float64 9MB dask.array<chunksize=(600, 288, 2), meta=np.ndarray>
    tas        (time, lat, lon) float32 438MB dask.array<chunksize=(480, 192, 288), meta=np.ndarray>
Attributes: (12/46)
    Conventions:                     CF-1.7 CMIP-6.2
    activity_id:                     CMIP
    branch_method:                   standard
    branch_time_in_child:            674885.0
    branch_time_in_parent:           306600.0
    case_id:                         24
    ...                              ...
    table_id:                        Amon
    tracking_id:                     hdl:21.14100/e47b79db-3925-45a7-9c0a-679...
    variable_id:                     tas
    variant_info:                    CMIP6 20th century experiments (1850-201...
    variant_label:                   r10i1p1f1
    DODS_EXTRA.Unlimited_Dimension:  time

xarray.Dataset

• Dimensions:
  • time: 1980
  • nbnd: 2
  • lat: 192
  • lon: 288
• Coordinates: (3)
  • lat
    (lat)
    float64
    -90.0 -89.06 -88.12 ... 89.06 90.0

    axis :

    Y

    bounds :

    lat_bnds

    standard_name :

    latitude

    title :

    Latitude

    type :

    double

    units :

    degrees_north

    valid_max :

    90.0

    valid_min :

    -90.0

    _ChunkSizes :

    192

    array([-90.      , -89.057592, -88.115183, -87.172775, -86.230366, -85.287958,
           -84.34555 , -83.403141, -82.460733, -81.518325, -80.575916, -79.633508,
           -78.691099, -77.748691, -76.806283, -75.863874, -74.921466, -73.979058,
           -73.036649, -72.094241, -71.151832, -70.209424, -69.267016, -68.324607,
           -67.382199, -66.439791, -65.497382, -64.554974, -63.612565, -62.670157,
           -61.727749, -60.78534 , -59.842932, -58.900524, -57.958115, -57.015707,
           -56.073298, -55.13089 , -54.188482, -53.246073, -52.303665, -51.361257,
           -50.418848, -49.47644 , -48.534031, -47.591623, -46.649215, -45.706806,
           -44.764398, -43.82199 , -42.879581, -41.937173, -40.994764, -40.052356,
           -39.109948, -38.167539, -37.225131, -36.282723, -35.340314, -34.397906,
           -33.455497, -32.513089, -31.570681, -30.628272, -29.685864, -28.743455,
           -27.801047, -26.858639, -25.91623 , -24.973822, -24.031414, -23.089005,
           -22.146597, -21.204188, -20.26178 , -19.319372, -18.376963, -17.434555,
           -16.492147, -15.549738, -14.60733 , -13.664921, -12.722513, -11.780105,
           -10.837696,  -9.895288,  -8.95288 ,  -8.010471,  -7.068063,  -6.125654,
            -5.183246,  -4.240838,  -3.298429,  -2.356021,  -1.413613,  -0.471204,
             0.471204,   1.413613,   2.356021,   3.298429,   4.240838,   5.183246,
             6.125654,   7.068063,   8.010471,   8.95288 ,   9.895288,  10.837696,
            11.780105,  12.722513,  13.664921,  14.60733 ,  15.549738,  16.492147,
            17.434555,  18.376963,  19.319372,  20.26178 ,  21.204188,  22.146597,
            23.089005,  24.031414,  24.973822,  25.91623 ,  26.858639,  27.801047,
            28.743455,  29.685864,  30.628272,  31.570681,  32.513089,  33.455497,
            34.397906,  35.340314,  36.282723,  37.225131,  38.167539,  39.109948,
            40.052356,  40.994764,  41.937173,  42.879581,  43.82199 ,  44.764398,
            45.706806,  46.649215,  47.591623,  48.534031,  49.47644 ,  50.418848,
            51.361257,  52.303665,  53.246073,  54.188482,  55.13089 ,  56.073298,
            57.015707,  57.958115,  58.900524,  59.842932,  60.78534 ,  61.727749,
            62.670157,  63.612565,  64.554974,  65.497382,  66.439791,  67.382199,
            68.324607,  69.267016,  70.209424,  71.151832,  72.094241,  73.036649,
            73.979058,  74.921466,  75.863874,  76.806283,  77.748691,  78.691099,
            79.633508,  80.575916,  81.518325,  82.460733,  83.403141,  84.34555 ,
            85.287958,  86.230366,  87.172775,  88.115183,  89.057592,  90.      ])

  • lon
    (lon)
    float64
    0.0 1.25 2.5 ... 356.2 357.5 358.8

    axis :

    X

    bounds :

    lon_bnds

    standard_name :

    longitude

    title :

    Longitude

    type :

    double

    units :

    degrees_east

    valid_max :

    360.0

    valid_min :

    0.0

    _ChunkSizes :

    288

    array([  0.  ,   1.25,   2.5 , ..., 356.25, 357.5 , 358.75])

  • time
    (time)
    object
    1850-01-15 12:00:00 ... 2014-12-...

    axis :

    T

    bounds :

    time_bnds

    standard_name :

    time

    title :

    time

    type :

    double

    _ChunkSizes :

    512

    array([cftime.DatetimeNoLeap(1850, 1, 15, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1850, 2, 14, 0, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1850, 3, 15, 12, 0, 0, 0, has_year_zero=True),
           ...,
           cftime.DatetimeNoLeap(2014, 10, 15, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2014, 11, 15, 0, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2014, 12, 15, 12, 0, 0, 0, has_year_zero=True)],
          dtype=object)

• Data variables: (4)
  • time_bnds
    (time, nbnd)
    object
    dask.array<chunksize=(480, 2), meta=np.ndarray>

    _ChunkSizes :

    [1 2]

    Array Chunk Bytes 30.94 kiB 7.50 kiB Shape (1980, 2) (480, 2) Dask graph 7 chunks in 9 graph layers Data type object numpy.ndarray

  • lat_bnds
    (time, lat, nbnd)
    float64
    dask.array<chunksize=(600, 192, 2), meta=np.ndarray>

    units :

    degrees_north

    _ChunkSizes :

    [192 2]

    Array Chunk Bytes 5.80 MiB 1.76 MiB Shape (1980, 192, 2) (600, 192, 2) Dask graph 4 chunks in 13 graph layers Data type float64 numpy.ndarray

  • lon_bnds
    (time, lon, nbnd)
    float64
    dask.array<chunksize=(600, 288, 2), meta=np.ndarray>

    units :

    degrees_east

    _ChunkSizes :

    [288 2]

    Array Chunk Bytes 8.70 MiB 2.64 MiB Shape (1980, 288, 2) (600, 288, 2) Dask graph 4 chunks in 13 graph layers Data type float64 numpy.ndarray

  • tas
    (time, lat, lon)
    float32
    dask.array<chunksize=(480, 192, 288), meta=np.ndarray>

    cell_measures :

    area: areacella

    cell_methods :

    area: time: mean

    comment :

    near-surface (usually, 2 meter) air temperature

    description :

    near-surface (usually, 2 meter) air temperature

    frequency :

    mon

    id :

    tas

    long_name :

    Near-Surface Air Temperature

    mipTable :

    Amon

    out_name :

    tas

    prov :

    Amon ((isd.003))

    realm :

    atmos

    standard_name :

    air_temperature

    time :

    time

    time_label :

    time-mean

    time_title :

    Temporal mean

    title :

    Near-Surface Air Temperature

    type :

    real

    units :

    K

    variable_id :

    tas

    _ChunkSizes :

    [ 1 192 288]

    Array Chunk Bytes 417.66 MiB 101.25 MiB Shape (1980, 192, 288) (480, 192, 288) Dask graph 7 chunks in 9 graph layers Data type float32 numpy.ndarray

• Indexes: (3)
  • lat
    PandasIndex

    PandasIndex(Index([             -90.0, -89.05759162303664,  -88.1151832460733,
           -87.17277486910994,  -86.2303664921466, -85.28795811518324,
            -84.3455497382199, -83.40314136125654, -82.46073298429319,
           -81.51832460732984,
           ...
            81.51832460732984,   82.4607329842932,  83.40314136125653,
            84.34554973821989,  85.28795811518324,   86.2303664921466,
            87.17277486910996,  88.11518324607329,  89.05759162303664,
                         90.0],
          dtype='float64', name='lat', length=192))

  • lon
    PandasIndex

    PandasIndex(Index([   0.0,   1.25,    2.5,   3.75,    5.0,   6.25,    7.5,   8.75,   10.0,
            11.25,
           ...
            347.5, 348.75,  350.0, 351.25,  352.5, 353.75,  355.0, 356.25,  357.5,
           358.75],
          dtype='float64', name='lon', length=288))

  • time
    PandasIndex

    PandasIndex(CFTimeIndex([1850-01-15 12:00:00, 1850-02-14 00:00:00, 1850-03-15 12:00:00,
                 1850-04-15 00:00:00, 1850-05-15 12:00:00, 1850-06-15 00:00:00,
                 1850-07-15 12:00:00, 1850-08-15 12:00:00, 1850-09-15 00:00:00,
                 1850-10-15 12:00:00,
                 ...
                 2014-03-15 12:00:00, 2014-04-15 00:00:00, 2014-05-15 12:00:00,
                 2014-06-15 00:00:00, 2014-07-15 12:00:00, 2014-08-15 12:00:00,
                 2014-09-15 00:00:00, 2014-10-15 12:00:00, 2014-11-15 00:00:00,
                 2014-12-15 12:00:00],
                dtype='object', length=1980, calendar='noleap', freq=None))

• Attributes: (46)

  Conventions :

  CF-1.7 CMIP-6.2

  activity_id :

  CMIP

  branch_method :

  standard

  branch_time_in_child :

  674885.0

  branch_time_in_parent :

  306600.0

  case_id :

  24

  cesm_casename :

  b.e21.BHIST.f09_g17.CMIP6-historical.010

  contact :

  cesm_cmip6@ucar.edu

  creation_date :

  2019-03-12T06:39:18Z

  data_specs_version :

  01.00.29

  experiment :

  Simulation of recent past (1850 to 2014). Impose changing conditions (consistent with observations). Should be initialised from a point early enough in the pre-industrial control run to ensure that the end of all the perturbed runs branching from the end of this historical run end before the end of the control. Only one ensemble member is requested but modelling groups are strongly encouraged to submit at least three ensemble members of their CMIP historical simulation.

  experiment_id :

  historical

  external_variables :

  areacella

  forcing_index :

  1

  frequency :

  mon

  further_info_url :

  https://furtherinfo.es-doc.org/CMIP6.NCAR.CESM2.historical.none.r10i1p1f1

  grid :

  native 0.9x1.25 finite volume grid (192x288 latxlon)

  grid_label :

  gn

  initialization_index :

  1

  institution :

  National Center for Atmospheric Research, Climate and Global Dynamics Laboratory, 1850 Table Mesa Drive, Boulder, CO 80305, USA

  institution_id :

  NCAR

  license :

  CMIP6 model data produced by <The National Center for Atmospheric Research> is licensed under a Creative Commons Attribution-[]ShareAlike 4.0 International License (https://creativecommons.org/licenses/). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file)[]. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law.

  mip_era :

  CMIP6

  model_doi_url :

  https://doi.org/10.5065/D67H1H0V

  nominal_resolution :

  100 km

  parent_activity_id :

  CMIP

  parent_experiment_id :

  piControl

  parent_mip_era :

  CMIP6

  parent_source_id :

  CESM2

  parent_time_units :

  days since 0001-01-01 00:00:00

  parent_variant_label :

  r1i1p1f1

  physics_index :

  1

  product :

  model-output

  realization_index :

  10

  realm :

  atmos

  source :

  CESM2 (2017): atmosphere: CAM6 (0.9x1.25 finite volume grid; 288 x 192 longitude/latitude; 32 levels; top level 2.25 mb); ocean: POP2 (320x384 longitude/latitude; 60 levels; top grid cell 0-10 m); sea_ice: CICE5.1 (same grid as ocean); land: CLM5 0.9x1.25 finite volume grid; 288 x 192 longitude/latitude; 32 levels; top level 2.25 mb); aerosol: MAM4 (0.9x1.25 finite volume grid; 288 x 192 longitude/latitude; 32 levels; top level 2.25 mb); atmoschem: MAM4 (0.9x1.25 finite volume grid; 288 x 192 longitude/latitude; 32 levels; top level 2.25 mb); landIce: CISM2.1; ocnBgchem: MARBL (320x384 longitude/latitude; 60 levels; top grid cell 0-10 m)

  source_id :

  CESM2

  source_type :

  AOGCM BGC

  sub_experiment :

  none

  sub_experiment_id :

  none

  table_id :

  Amon

  tracking_id :

  hdl:21.14100/e47b79db-3925-45a7-9c0a-6799c2f1e8ae

  variable_id :

  tas

  variant_info :

  CMIP6 20th century experiments (1850-2014) with CAM6, interactive land (CLM5), coupled ocean (POP2) with biogeochemistry (MARBL), interactive sea ice (CICE5.1), and non-evolving land ice (CISM2.1)

  variant_label :

  r10i1p1f1

  DODS_EXTRA.Unlimited_Dimension :

  time

Plot a quick look of the data

Now that we have the dataset, let’s plot a few quick looks of the data.

ds.tas.sel(time='1950-01').squeeze().plot(cmap='Spectral_r');

These are OPeNDAP endpoints. Xarray, together with the netCDF4 Python library, allow lazy loading.

Compute an area-weighted global average

Let’s apply some computation to this dataset. We would like to calculate the global average temperature. This requires weighting each of the grid cells properly, using the area.

Find the area of the cells

We can query the dataserver again, this time extracting the area of the cell (areacella).

ctx = conn.new_context(
    facets='project,experiment_id',
    project='CMIP6',
    institution_id="NCAR",
    experiment_id='historical',
    source_id='CESM2',
    variable='areacella',
)

As before, we extract the opendap urls.

result = ctx.search()[0]
files = result.file_context().search()
opendap_urls = [file.opendap_url for file in files]
opendap_urls

['http://aims3.llnl.gov/thredds/dodsC/css03_data/CMIP6/CMIP/NCAR/CESM2/historical/r11i1p1f1/fx/areacella/gn/v20190514/areacella_fx_CESM2_historical_r11i1p1f1_gn.nc']

And finally, we load our cell area file into an xarray.Dataset

ds_area = xr.open_dataset(opendap_urls[0])
ds_area

<xarray.Dataset> Size: 233kB
Dimensions:    (lat: 192, lon: 288, nbnd: 2)
Coordinates:
  * lat        (lat) float64 2kB -90.0 -89.06 -88.12 -87.17 ... 88.12 89.06 90.0
  * lon        (lon) float64 2kB 0.0 1.25 2.5 3.75 ... 355.0 356.2 357.5 358.8
Dimensions without coordinates: nbnd
Data variables:
    lat_bnds   (lat, nbnd) float64 3kB ...
    lon_bnds   (lon, nbnd) float64 5kB ...
    areacella  (lat, lon) float32 221kB ...
Attributes: (12/44)
    Conventions:            CF-1.7 CMIP-6.2
    activity_id:            CMIP
    branch_method:          standard
    branch_time_in_child:   674885.0
    branch_time_in_parent:  219000.0
    case_id:                972
    ...                     ...
    sub_experiment_id:      none
    table_id:               fx
    tracking_id:            hdl:21.14100/96455df2-979e-4cd4-8521-ddf307c6bc4a
    variable_id:            areacella
    variant_info:           CMIP6 20th century experiments (1850-2014) with C...
    variant_label:          r11i1p1f1

xarray.Dataset

• Dimensions:
  • lat: 192
  • lon: 288
  • nbnd: 2
• Coordinates: (2)
  • lat
    (lat)
    float64
    -90.0 -89.06 -88.12 ... 89.06 90.0

    axis :

    Y

    bounds :

    lat_bnds

    standard_name :

    latitude

    title :

    Latitude

    type :

    double

    units :

    degrees_north

    valid_max :

    90.0

    valid_min :

    -90.0

    _ChunkSizes :

    192

    array([-90.      , -89.057592, -88.115183, -87.172775, -86.230366, -85.287958,
           -84.34555 , -83.403141, -82.460733, -81.518325, -80.575916, -79.633508,
           -78.691099, -77.748691, -76.806283, -75.863874, -74.921466, -73.979058,
           -73.036649, -72.094241, -71.151832, -70.209424, -69.267016, -68.324607,
           -67.382199, -66.439791, -65.497382, -64.554974, -63.612565, -62.670157,
           -61.727749, -60.78534 , -59.842932, -58.900524, -57.958115, -57.015707,
           -56.073298, -55.13089 , -54.188482, -53.246073, -52.303665, -51.361257,
           -50.418848, -49.47644 , -48.534031, -47.591623, -46.649215, -45.706806,
           -44.764398, -43.82199 , -42.879581, -41.937173, -40.994764, -40.052356,
           -39.109948, -38.167539, -37.225131, -36.282723, -35.340314, -34.397906,
           -33.455497, -32.513089, -31.570681, -30.628272, -29.685864, -28.743455,
           -27.801047, -26.858639, -25.91623 , -24.973822, -24.031414, -23.089005,
           -22.146597, -21.204188, -20.26178 , -19.319372, -18.376963, -17.434555,
           -16.492147, -15.549738, -14.60733 , -13.664921, -12.722513, -11.780105,
           -10.837696,  -9.895288,  -8.95288 ,  -8.010471,  -7.068063,  -6.125654,
            -5.183246,  -4.240838,  -3.298429,  -2.356021,  -1.413613,  -0.471204,
             0.471204,   1.413613,   2.356021,   3.298429,   4.240838,   5.183246,
             6.125654,   7.068063,   8.010471,   8.95288 ,   9.895288,  10.837696,
            11.780105,  12.722513,  13.664921,  14.60733 ,  15.549738,  16.492147,
            17.434555,  18.376963,  19.319372,  20.26178 ,  21.204188,  22.146597,
            23.089005,  24.031414,  24.973822,  25.91623 ,  26.858639,  27.801047,
            28.743455,  29.685864,  30.628272,  31.570681,  32.513089,  33.455497,
            34.397906,  35.340314,  36.282723,  37.225131,  38.167539,  39.109948,
            40.052356,  40.994764,  41.937173,  42.879581,  43.82199 ,  44.764398,
            45.706806,  46.649215,  47.591623,  48.534031,  49.47644 ,  50.418848,
            51.361257,  52.303665,  53.246073,  54.188482,  55.13089 ,  56.073298,
            57.015707,  57.958115,  58.900524,  59.842932,  60.78534 ,  61.727749,
            62.670157,  63.612565,  64.554974,  65.497382,  66.439791,  67.382199,
            68.324607,  69.267016,  70.209424,  71.151832,  72.094241,  73.036649,
            73.979058,  74.921466,  75.863874,  76.806283,  77.748691,  78.691099,
            79.633508,  80.575916,  81.518325,  82.460733,  83.403141,  84.34555 ,
            85.287958,  86.230366,  87.172775,  88.115183,  89.057592,  90.      ])

  • lon
    (lon)
    float64
    0.0 1.25 2.5 ... 356.2 357.5 358.8

    axis :

    X

    bounds :

    lon_bnds

    standard_name :

    longitude

    title :

    Longitude

    type :

    double

    units :

    degrees_east

    valid_max :

    360.0

    valid_min :

    0.0

    _ChunkSizes :

    288

    array([  0.  ,   1.25,   2.5 , ..., 356.25, 357.5 , 358.75])

• Data variables: (3)
  • lat_bnds
    (lat, nbnd)
    float64
    ...

    units :

    degrees_north

    _ChunkSizes :

    [192 2]

    [384 values with dtype=float64]

  • lon_bnds
    (lon, nbnd)
    float64
    ...

    units :

    degrees_east

    _ChunkSizes :

    [288 2]

    [576 values with dtype=float64]

  • areacella
    (lat, lon)
    float32
    ...

    cell_methods :

    area: sum

    comment :

    AREA[0,:,:]

    description :

    Cell areas for any grid used to report atmospheric variables and any other variable using that grid (e.g., soil moisture content). These cell areas should be defined to enable exact calculation of global integrals (e.g., of vertical fluxes of energy at the surface and top of the atmosphere).

    frequency :

    fx

    id :

    areacella

    long_name :

    Grid-Cell Area for Atmospheric Grid Variables

    mipTable :

    fx

    out_name :

    areacella

    prov :

    fx ((isd.003))

    realm :

    atmos land

    standard_name :

    cell_area

    time_label :

    None

    time_title :

    No temporal dimensions ... fixed field

    title :

    Grid-Cell Area for Atmospheric Grid Variables

    type :

    real

    units :

    m2

    variable_id :

    areacella

    _ChunkSizes :

    [192 288]

    [55296 values with dtype=float32]

• Indexes: (2)
  • lat
    PandasIndex

    PandasIndex(Index([             -90.0, -89.05759162303664,  -88.1151832460733,
           -87.17277486910994,  -86.2303664921466, -85.28795811518324,
            -84.3455497382199, -83.40314136125654, -82.46073298429319,
           -81.51832460732984,
           ...
            81.51832460732984,   82.4607329842932,  83.40314136125653,
            84.34554973821989,  85.28795811518324,   86.2303664921466,
            87.17277486910996,  88.11518324607329,  89.05759162303664,
                         90.0],
          dtype='float64', name='lat', length=192))

  • lon
    PandasIndex

    PandasIndex(Index([   0.0,   1.25,    2.5,   3.75,    5.0,   6.25,    7.5,   8.75,   10.0,
            11.25,
           ...
            347.5, 348.75,  350.0, 351.25,  352.5, 353.75,  355.0, 356.25,  357.5,
           358.75],
          dtype='float64', name='lon', length=288))

• Attributes: (44)

  Conventions :

  CF-1.7 CMIP-6.2

  activity_id :

  CMIP

  branch_method :

  standard

  branch_time_in_child :

  674885.0

  branch_time_in_parent :

  219000.0

  case_id :

  972

  cesm_casename :

  b.e21.BHIST.f09_g17.CMIP6-historical.011

  contact :

  cesm_cmip6@ucar.edu

  creation_date :

  2019-04-02T04:17:01Z

  data_specs_version :

  01.00.29

  experiment :

  Simulation of recent past (1850 to 2014). Impose changing conditions (consistent with observations). Should be initialised from a point early enough in the pre-industrial control run to ensure that the end of all the perturbed runs branching from the end of this historical run end before the end of the control. Only one ensemble member is requested but modelling groups are strongly encouraged to submit at least three ensemble members of their CMIP historical simulation.

  experiment_id :

  historical

  forcing_index :

  1

  frequency :

  fx

  further_info_url :

  https://furtherinfo.es-doc.org/CMIP6.NCAR.CESM2.historical.none.r11i1p1f1

  grid :

  native 0.9x1.25 finite volume grid (192x288 latxlon)

  grid_label :

  gn

  initialization_index :

  1

  institution :

  National Center for Atmospheric Research, Climate and Global Dynamics Laboratory, 1850 Table Mesa Drive, Boulder, CO 80305, USA

  institution_id :

  NCAR

  license :

  CMIP6 model data produced by <The National Center for Atmospheric Research> is licensed under a Creative Commons Attribution-[]ShareAlike 4.0 International License (https://creativecommons.org/licenses/). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file)[]. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law.

  mip_era :

  CMIP6

  model_doi_url :

  https://doi.org/10.5065/D67H1H0V

  nominal_resolution :

  100 km

  parent_activity_id :

  CMIP

  parent_experiment_id :

  piControl

  parent_mip_era :

  CMIP6

  parent_source_id :

  CESM2

  parent_time_units :

  days since 0001-01-01 00:00:00

  parent_variant_label :

  r1i1p1f1

  physics_index :

  1

  product :

  model-output

  realization_index :

  11

  realm :

  atmos land

  source :

  CESM2 (2017): atmosphere: CAM6 (0.9x1.25 finite volume grid; 288 x 192 longitude/latitude; 32 levels; top level 2.25 mb); ocean: POP2 (320x384 longitude/latitude; 60 levels; top grid cell 0-10 m); sea_ice: CICE5.1 (same grid as ocean); land: CLM5 0.9x1.25 finite volume grid; 288 x 192 longitude/latitude; 32 levels; top level 2.25 mb); aerosol: MAM4 (0.9x1.25 finite volume grid; 288 x 192 longitude/latitude; 32 levels; top level 2.25 mb); atmoschem: MAM4 (0.9x1.25 finite volume grid; 288 x 192 longitude/latitude; 32 levels; top level 2.25 mb); landIce: CISM2.1; ocnBgchem: MARBL (320x384 longitude/latitude; 60 levels; top grid cell 0-10 m)

  source_id :

  CESM2

  source_type :

  AOGCM BGC

  sub_experiment :

  none

  sub_experiment_id :

  none

  table_id :

  fx

  tracking_id :

  hdl:21.14100/96455df2-979e-4cd4-8521-ddf307c6bc4a

  variable_id :

  areacella

  variant_info :

  CMIP6 20th century experiments (1850-2014) with CAM6, interactive land (CLM5), coupled ocean (POP2) with biogeochemistry (MARBL), interactive sea ice (CICE5.1), and non-evolving land ice (CISM2.1)

  variant_label :

  r11i1p1f1

Compute the global average

Now that we have the area of each cell, and the temperature at each point, we can compute the global average temperature.

total_area = ds_area.areacella.sum(dim=['lon', 'lat'])
ta_timeseries = (ds.tas * ds_area.areacella).sum(dim=['lon', 'lat']) / total_area
ta_timeseries

<xarray.DataArray (time: 1980)> Size: 8kB
dask.array<truediv, shape=(1980,), dtype=float32, chunksize=(480,), chunktype=numpy.ndarray>
Coordinates:
  * time     (time) object 16kB 1850-01-15 12:00:00 ... 2014-12-15 12:00:00

xarray.DataArray

• time: 1980

• dask.array<chunksize=(480,), meta=np.ndarray>

  Array Chunk Bytes 7.73 kiB 1.88 kiB Shape (1980,) (480,) Dask graph 7 chunks in 18 graph layers Data type float32 numpy.ndarray

• Coordinates: (1)
  • time
    (time)
    object
    1850-01-15 12:00:00 ... 2014-12-...

    axis :

    T

    bounds :

    time_bnds

    standard_name :

    time

    title :

    time

    type :

    double

    _ChunkSizes :

    512

    array([cftime.DatetimeNoLeap(1850, 1, 15, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1850, 2, 14, 0, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1850, 3, 15, 12, 0, 0, 0, has_year_zero=True),
           ...,
           cftime.DatetimeNoLeap(2014, 10, 15, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2014, 11, 15, 0, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2014, 12, 15, 12, 0, 0, 0, has_year_zero=True)],
          dtype=object)

• Indexes: (1)
  • time
    PandasIndex

    PandasIndex(CFTimeIndex([1850-01-15 12:00:00, 1850-02-14 00:00:00, 1850-03-15 12:00:00,
                 1850-04-15 00:00:00, 1850-05-15 12:00:00, 1850-06-15 00:00:00,
                 1850-07-15 12:00:00, 1850-08-15 12:00:00, 1850-09-15 00:00:00,
                 1850-10-15 12:00:00,
                 ...
                 2014-03-15 12:00:00, 2014-04-15 00:00:00, 2014-05-15 12:00:00,
                 2014-06-15 00:00:00, 2014-07-15 12:00:00, 2014-08-15 12:00:00,
                 2014-09-15 00:00:00, 2014-10-15 12:00:00, 2014-11-15 00:00:00,
                 2014-12-15 12:00:00],
                dtype='object', length=1980, calendar='noleap', freq=None))

• Attributes: (0)

By default the data are loaded lazily, as Dask arrays. Here we trigger computation explicitly.

%time ta_timeseries.load()

CPU times: user 409 ms, sys: 112 ms, total: 520 ms
Wall time: 10.9 s

<xarray.DataArray (time: 1980)> Size: 8kB
array([284.99948, 285.23215, 285.85364, ..., 288.54376, 287.61884,
       287.06284], dtype=float32)
Coordinates:
  * time     (time) object 16kB 1850-01-15 12:00:00 ... 2014-12-15 12:00:00

xarray.DataArray

• time: 1980

• 285.0 285.2 285.9 286.9 287.9 288.4 ... 290.0 289.3 288.5 287.6 287.1

  array([284.99948, 285.23215, 285.85364, ..., 288.54376, 287.61884,
         287.06284], dtype=float32)

• Coordinates: (1)
  • time
    (time)
    object
    1850-01-15 12:00:00 ... 2014-12-...

    axis :

    T

    bounds :

    time_bnds

    standard_name :

    time

    title :

    time

    type :

    double

    _ChunkSizes :

    512

    array([cftime.DatetimeNoLeap(1850, 1, 15, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1850, 2, 14, 0, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(1850, 3, 15, 12, 0, 0, 0, has_year_zero=True),
           ...,
           cftime.DatetimeNoLeap(2014, 10, 15, 12, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2014, 11, 15, 0, 0, 0, 0, has_year_zero=True),
           cftime.DatetimeNoLeap(2014, 12, 15, 12, 0, 0, 0, has_year_zero=True)],
          dtype=object)

• Indexes: (1)
  • time
    PandasIndex

    PandasIndex(CFTimeIndex([1850-01-15 12:00:00, 1850-02-14 00:00:00, 1850-03-15 12:00:00,
                 1850-04-15 00:00:00, 1850-05-15 12:00:00, 1850-06-15 00:00:00,
                 1850-07-15 12:00:00, 1850-08-15 12:00:00, 1850-09-15 00:00:00,
                 1850-10-15 12:00:00,
                 ...
                 2014-03-15 12:00:00, 2014-04-15 00:00:00, 2014-05-15 12:00:00,
                 2014-06-15 00:00:00, 2014-07-15 12:00:00, 2014-08-15 12:00:00,
                 2014-09-15 00:00:00, 2014-10-15 12:00:00, 2014-11-15 00:00:00,
                 2014-12-15 12:00:00],
                dtype='object', length=1980, calendar='noleap', freq=None))

• Attributes: (0)

Visualize our results

Now that we have our results, we can visualize using static and dynamic plots. Let’s start with static plots using matplotlib, then dynamic plots using hvPlot.

ta_timeseries['time'] = ta_timeseries.indexes['time'].to_datetimeindex()

fig = plt.figure(figsize=(12,8))
ta_timeseries.plot(label='monthly')
ta_timeseries.rolling(time=12).mean().plot(label='12 month rolling mean')
plt.legend()
plt.title('Global Mean Surface Air Temperature')

Text(0.5, 1.0, 'Global Mean Surface Air Temperature')

ta_timeseries.name = 'Temperature (K)'
monthly_average = ta_timeseries.hvplot(title = 'Global Mean Surface Air Temperature',
                                       label='monthly')
rolling_monthly_average = ta_timeseries.rolling(time=12).mean().hvplot(label='12 month rolling mean',)

(monthly_average * rolling_monthly_average).opts(legend_position='top_left')

---

Summary

In this notebook, we searched for and opened a CESM2 dataset using the ESGF API and OPeNDAP. We then plotted global average surface air temperature.

What’s next?

We will see some more advanced examples of using the CMIP6 data.

Resources and references

• Original notebook in the Pangeo Gallery by Henri Drake and Ryan Abernathey