Looking at NEXRAD Data from Moore, Oklahoma¶

Overview¶

Within this notebook, we will cover:

How to access NEXRAD data from AWS
How to read this data into Py-ART
How to customize your plots and maps

Prerequisites¶

Concepts	Importance	Notes
Intro to Cartopy	Required	Projections and Features
Matplotlib Basics	Required	Basic plotting
Py-ART Basics	Required	IO/Visualization

Time to learn: 45 minutes

Imports¶

import pyart
import fsspec
from metpy.plots import USCOUNTIES
import matplotlib.pyplot as plt
import cartopy.crs as ccrs
import cartopy.feature as cfeature
import warnings
warnings.filterwarnings("ignore")


## You are using the Python ARM Radar Toolkit (Py-ART), an open source
## library for working with weather radar data. Py-ART is partly supported
## by the U.S. Department of Energy Office of Science as part of
## the Atmospheric Radiation Measurement (ARM) User Facility.
##
## If you use this software to prepare a publication, please cite:
##
##     JJ Helmus and SM Collis, JORS 2016, doi: 10.5334/jors.119

/home/runner/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

How to Access NEXRAD Data from Amazon Web Services (AWS)¶

Let’s start first with NEXRAD Level 2 data, which is ground-based radar data collected by the National Oceanic and Atmospheric Administration (NOAA), as a part of the National Weather Service (NWS) observing network.

Level 2 Data¶

Level 2 data includes all of the fields in a single file - for example, a file may include:

Reflectivity
Velocity

Search for Data from the Moore, Oklahoma Tornado (May 20, 2013)¶

Data We will access data from the noaa-nexrad-level2 bucket, with the data organized as:

s3://noaa-nexrad-level2/year/month/date/radarsite/{radarsite}{year}{month}{date}_{hour}{minute}{second}_V06

We can use fsspec, a tool to work with filesystems in Python, to search through the bucket to find our files!

We start first by setting up our AWS S3 filesystem

fs = fsspec.filesystem("s3", anon=True)

Once we setup our filesystem, we can list files from May 20, 2013 from the NWS Oklahoma City, Oklahoma (KTLX) site, around 2000 UTC.

files = sorted(fs.glob("s3://noaa-nexrad-level2/2013/05/20/KTLX/KTLX20130520_20*"))
files

---------------------------------------------------------------------------
AccessDenied                              Traceback (most recent call last)
File ~/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/s3fs/core.py:169, in _error_wrapper(func, args, kwargs, retries)
    168 try:
--> 169     return await func(*args, **kwargs)
    170 except S3_RETRYABLE_ERRORS as e:

File ~/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/aiobotocore/context.py:36, in with_current_context.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
     35     await resolve_awaitable(hook())
---> 36 return await func(*args, **kwargs)

File ~/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/aiobotocore/client.py:449, in AioBaseClient._make_api_call(self, operation_name, api_params)
    448     error_class = self.exceptions.from_code(error_code)
--> 449     raise error_class(parsed_response, operation_name)
    450 else:

AccessDenied: An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied

The above exception was the direct cause of the following exception:

PermissionError                           Traceback (most recent call last)
Cell In[3], line 1
----> 1 files = sorted(fs.glob("s3://noaa-nexrad-level2/2013/05/20/KTLX/KTLX20130520_20*"))
      2 files

File ~/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/fsspec/asyn.py:217, in sync_wrapper.<locals>.wrapper(*args, **kwargs)
    214 @functools.wraps(func)
    215 def wrapper(*args, **kwargs):
    216     self = obj or args[0]
--> 217     return sync(self.loop, func, *args, **kwargs)

File ~/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/fsspec/asyn.py:106, in sync(loop, func, timeout, *args, **kwargs)
    104     raise FSTimeoutError from return_result
    105 elif isinstance(return_result, BaseException):
--> 106     raise return_result
    107 else:
    108     return return_result

File ~/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/fsspec/asyn.py:59, in _runner(event, coro, result, timeout)
     57     coro = asyncio.wait_for(coro, timeout=timeout)
     58 try:
---> 59     result[0] = await coro
     60 except Exception as ex:
     61     result[0] = ex

File ~/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/s3fs/core.py:969, in S3FileSystem._glob(self, path, **kwargs)
    967 if path.startswith("*"):
    968     raise ValueError("Cannot traverse all of S3")
--> 969 return await super()._glob(path, **kwargs)

File ~/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/fsspec/asyn.py:980, in AsyncFileSystem._glob(self, path, maxdepth, **kwargs)
    978 if prefix:
    979     kwargs["prefix"] = prefix
--> 980 allpaths = await self._find(
    981     root, maxdepth=depth, withdirs=withdirs, detail=True, **kwargs
    982 )
    984 pattern = glob_translate(path + ("/" if ends_with_sep else ""))
    985 pattern = re.compile(pattern)

File ~/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/s3fs/core.py:1006, in S3FileSystem._find(self, path, maxdepth, withdirs, detail, prefix, **kwargs)
    996     return await super()._find(
    997         bucket + "/" + key,
    998         maxdepth=maxdepth,
   (...)   1001         **kwargs,
   1002     )
   1003 # maxdepth + prefix: one delimiter-based listing for the first level
   1004 # (server-side prefix filter), then recurse into matching subdirs
   1005 # normally — avoids fetching all nested objects up front.
-> 1006 first_level = await self._lsdir(
   1007     path, delimiter="/", prefix=prefix, **kwargs
   1008 )
   1009 files = [o for o in first_level if o["type"] != "directory"]
   1010 dirs = [o for o in first_level if o["type"] == "directory"]

File ~/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/s3fs/core.py:877, in S3FileSystem._lsdir(self, path, refresh, max_items, delimiter, prefix, versions)
    875 dirs = []
    876 files = []
--> 877 async for c in self._iterdir(
    878     bucket,
    879     max_items=max_items,
    880     delimiter=delimiter,
    881     prefix=prefix,
    882     versions=versions,
    883 ):
    884     if c["type"] == "directory":
    885         dirs.append(c)

File ~/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/s3fs/core.py:919, in S3FileSystem._iterdir(self, bucket, max_items, delimiter, prefix, versions)
    917     kwargs["MaxKeys"] = 2 * max_items
    918 remaining = max_items
--> 919 async for i in self._list_pages(method, **kwargs):
    920     for l in i.get("CommonPrefixes", []):
    921         c = {
    922             "Key": l["Prefix"][:-1],
    923             "Size": 0,
    924             "StorageClass": "DIRECTORY",
    925             "type": "directory",
    926         }

File ~/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/s3fs/core.py:947, in S3FileSystem._list_pages(self, method, **kwargs)
    942 async def _list_pages(self, method, **kwargs):
    943     # Each page goes through _call_s3, so a transient error on one page is
    944     # retried like any other request instead of failing (or silently
    945     # truncating) the whole listing, as the paginator did (#982).
    946     while True:
--> 947         out = await self._call_s3(method, **kwargs)
    948         yield out
    949         if not out.get("IsTruncated"):

File ~/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/s3fs/core.py:466, in S3FileSystem._call_s3(self, method, *akwarglist, **kwargs)
    464 logger.debug("CALL: %s - %s - %s", method.__name__, akwarglist, kw2)
    465 additional_kwargs = self._get_s3_method_kwargs(method, *akwarglist, **kwargs)
--> 466 return await _error_wrapper(
    467     method, kwargs=additional_kwargs, retries=self.retries
    468 )

File ~/micromamba/envs/radar-cookbook-dev/lib/python3.12/site-packages/s3fs/core.py:216, in _error_wrapper(func, args, kwargs, retries)
    214             err = e
    215 err = translate_boto_error(err)
--> 216 raise err

PermissionError: Access Denied

We now have a list of files we can read in!

Read the Data into PyART¶

When reading into PyART, we can use the pyart.io.read_nexrad_archive or pyart.io.read module to read in our data.

radar = pyart.io.read_nexrad_archive(f's3://{files[3]}')

Notice how for the NEXRAD Level 2 data, we have several fields available

list(radar.fields)

Plot a quick-look of the dataset¶

Let’s get a quicklook of the reflectivity and velocity fields

display = pyart.graph.RadarMapDisplay(radar)
display.plot_ppi_map('reflectivity',
                     sweep=3,
                     vmin=-20,
                     vmax=60,
                     projection=ccrs.PlateCarree()
                    )

display.plot_ppi_map('velocity',
                     sweep=3,
                     projection=ccrs.PlateCarree(),
                    )

How to customize your plots and maps¶

Let’s add some more features to our map, and zoom in on our main storm

Combine into a single figure¶

Let’s start first by combining into a single figure, and zooming in a bit on our main domain.

# Create our figure
fig = plt.figure(figsize=[12, 4])

# Setup our first axis with reflectivity
ax1 = plt.subplot(121, projection=ccrs.PlateCarree())
display = pyart.graph.RadarMapDisplay(radar)
display.plot_ppi_map('reflectivity',
                     sweep=3,
                     vmin=-20,
                     vmax=60,
                     ax=ax1,)

# Zoom in by setting the xlim/ylim
plt.xlim(-99, -96)
plt.ylim(33.5, 36.5)

# Setup our second axis for velocity
ax2 = plt.subplot(122, projection=ccrs.PlateCarree())
display.plot_ppi_map('velocity',
                     sweep=3,
                     vmin=-40,
                     vmax=40,
                     projection=ccrs.PlateCarree(),
                     ax=ax2,)

# Zoom in by setting the xlim/ylim
plt.xlim(-99, -96)
plt.ylim(33.5, 36.5)
plt.show()

Add Counties¶

We can add counties onto our map by using the USCOUNTIES module from metpy.plots

# Create our figure
fig = plt.figure(figsize=[12, 4])

# Setup our first axis with reflectivity
ax1 = plt.subplot(121, projection=ccrs.PlateCarree())
display = pyart.graph.RadarMapDisplay(radar)
display.plot_ppi_map('reflectivity',
                     sweep=3,
                     vmin=-20,
                     vmax=60,
                     ax=ax1,)

# Zoom in by setting the xlim/ylim
plt.xlim(-99, -96)
plt.ylim(33.5, 36.5)

# Add counties
ax1.add_feature(USCOUNTIES,
                linewidth=0.5)

# Setup our second axis for velocity
ax2 = plt.subplot(122, projection=ccrs.PlateCarree())
display.plot_ppi_map('velocity',
                     sweep=3,
                     vmin=-40,
                     vmax=40,
                     projection=ccrs.PlateCarree(),
                     ax=ax2,)

# Zoom in by setting the xlim/ylim
plt.xlim(-99, -96)
plt.ylim(33.5, 36.5)

# Add counties
ax2.add_feature(USCOUNTIES,
                linewidth=0.5)

plt.show()

Zoom in even more¶

Let’s zoom in even more to our main feature - it looks like there is velocity couplet (where high positive and negative values of velcocity are close to one another, indicating rotation), near the center of our map.

# Create our figure
fig = plt.figure(figsize=[12, 4])

# Setup our first axis with reflectivity
ax1 = plt.subplot(121, projection=ccrs.PlateCarree())
display = pyart.graph.RadarMapDisplay(radar)
display.plot_ppi_map('reflectivity',
                     sweep=3,
                     vmin=-20,
                     vmax=60,
                     ax=ax1,)

# Zoom in by setting the xlim/ylim
plt.xlim(-98, -97)
plt.ylim(35, 36)

# Add counties
ax1.add_feature(USCOUNTIES,
                linewidth=0.5)

# Setup our second axis for velocity
ax2 = plt.subplot(122, projection=ccrs.PlateCarree())
display.plot_ppi_map('velocity',
                     sweep=3,
                     vmin=-40,
                     vmax=40,
                     projection=ccrs.PlateCarree(),
                     ax=ax2,)

# Zoom in by setting the xlim/ylim
plt.xlim(-98, -97)
plt.ylim(35, 36)

# Add counties
ax2.add_feature(USCOUNTIES,
                linewidth=0.5)

plt.show()

Customize our Labels and Add Finer Grid Labels¶

You’ll notice, by default, our colorbar label for the velocity field on the right extends across our entire figure, and the latitude/longitude labels on our axes are now gone. Let’s fix that!

# Create our figure
fig = plt.figure(figsize=[12, 4])

# Setup our first axis with reflectivity
ax1 = plt.subplot(121, projection=ccrs.PlateCarree())
display = pyart.graph.RadarMapDisplay(radar)
ref_map = display.plot_ppi_map('reflectivity',
                               sweep=3,
                               vmin=-20,
                               vmax=60,
                               ax=ax1,
                               colorbar_label='Equivalent Relectivity ($Z_{e}$) (dBZ)')


# Zoom in by setting the xlim/ylim
plt.xlim(-97.7, -97.1)
plt.ylim(35.2, 35.8)

# Add gridlines
gl = ax1.gridlines(crs=ccrs.PlateCarree(),
                  draw_labels=True,
                  linewidth=1, 
                  color='gray', 
                  alpha=0.3,
                  linestyle='--')

# Make sure labels are only plotted on the left and bottom
gl.xlabels_top = False
gl.ylabels_right = False

# Increase the fontsize of our gridline labels
gl.xlabel_style = {'fontsize':10}
gl.ylabel_style = {'fontsize':10}

# Add counties
ax1.add_feature(USCOUNTIES,
                linewidth=0.5)

# Setup our second axis for velocity
ax2 = plt.subplot(122, projection=ccrs.PlateCarree())
vel_plot = display.plot_ppi_map('velocity',
                                sweep=3,
                                vmin=-40,
                                vmax=40,
                                projection=ccrs.PlateCarree(),
                                ax=ax2,
                                colorbar_label='Radial Velocity (m/s)')

# Zoom in by setting the xlim/ylim
plt.xlim(-97.7, -97.1)
plt.ylim(35.2, 35.8)

# Add gridlines
gl = ax2.gridlines(crs=ccrs.PlateCarree(),
                   draw_labels=True,
                   linewidth=1, 
                   color='gray', 
                   alpha=0.3,
                   linestyle='--')

# Make sure labels are only plotted on the left and bottom
gl.xlabels_top = False
gl.ylabels_right = False

# Increase the fontsize of our gridline labels
gl.xlabel_style = {'fontsize':10}
gl.ylabel_style = {'fontsize':10}

# Add counties
ax2.add_feature(USCOUNTIES,
                linewidth=0.5)
plt.show()

Summary¶

Within this example, we walked through how to use MetPy and PyART to read in NEXRAD Level 2 data from the Moore Oklahoma tornado in 2013, create some quick looks, and customize the plots to analyze the tornadic supercell closest to the radar.

What’s next?¶

Other examples will look at additional data sources and radar types, including data from the Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Facility, and work through more advanced workflows such as completing a dual-Doppler analysis.

Looking at NEXRAD Data from Moore, Oklahoma