Data access via earthaccess library and vizualization with cartopy¶

Overview¶

Within this notebook, we will cover:

1. How to search data via Earthdata Search web application
2. How to access NASA Earth Science data via earthaccess
3. How to subset, set attributes and modify coordinates for xarray Datasets
4. How to visualize data with hvplot and cartopy
5. How to download data

Prerequisites¶

• Time to learn: 30 minutes

Imports¶

Sea surface height anomaly¶

Let’s look for some sea surface height anomaly data for the Western Tropical Atlantic. In order to access it, we will use the earthaccess python library. It is used to make it easier for the user to find, stream and download NASA Earth Science data. More information about this library can be found in their documentation page and Github repository.

Registering for an Earthdata account and authenticating¶

Before searching for data, it is necessary to register for an Earthdata login profile, which can be done easily and quickly this way.

After registering, earthaccess has to authenticate you as a user. There are a few ways to do that, as stated here. The easiest way of doing is just executing auth = earthaccess.login() in your jupyter notebook, and that would prompt for input of the username and password for the user.

For the purposes of this demonstration we have environment variables that will be used for authentication. So you just need to execute the cell below:

After it is authenticated, we are ready to start our search for data!

Searching and accessing the data¶

earthaccess allows us to look for datasets (called DataCollections) and specific data files (called DataGranules). To look for them, we need some criteria in order to perform the search. Here we will use

• a shortname, which is a dataset identifier;
• a temporal window: we want data between those dates; and
• a bounding box: we want data within that area.

One good way to get a better understanding of what to look for is visiting the Earthdata Search website. There you can search by keywords and select filters to see which data could be helpful in your research. Once you find a dataset (Collection) that interests you, you click on its name. Below we show an example in which the “MEaSUREs Gridded Sea Surface Height Anomalies Version 2205” is the dataset of interest.

After clicking on the dataset name, you click on Collection Details, which will send you to a page that has more information about the dataset.

Feel free to read the details about the dataset, to make sure that’s what would help you in your work. If so, copy the name, usually all in caps, that is shown inside a light gray box on the top of the page. That is the identifier for that specific dataset. That’s the main information we’ll need to provide earthaccess library to search for it.

Now that we have an identifier for the dataset we want to access, let’s use earthaccess to find sea surface height anomaly data in September and October, 2020, in the Western Tropical Atlantic:

The function search_data returns a list with the Granules found. Here we can think of Granules as files, or time-steps. Great, we found some that match our criteria. Now, let’s take a look at one of them:

So here we have all the metadata information about this first Granule. You can also check all the Granules from the list, if you’d like to make sure they are all of interest to you:

OK, we found some results that interest us. Now let’s access them!

Two possibly important limiting factors for research are limited computing capacity and limited data storage. Even if data storage is not very limited, unecessarily working with large files is not best practice. So here we’ll not download the data; we’ll stream it instead, using xarray. Through xarray we can take a look at the data, subset, perform some analyses and, only if we are certain that we want to look further into it, download the data.

So let’s start by opening the data Granules as a dataset with xarray. This can take some time, depending on your system.

Subsetting the data, adjusting the coordinates and assigning attributes¶

If we take a look at the loaded dataset, we notice a few things:

• The search yielded 12 granules, so we have 12 time-steps in the resulting dataset
• The Latitude and Longitude span a much greater area than requested in the search. The dataset loaded contains the area that we requested, plus much more.
• we have a few data variables besides SLA (Sea Level Anomaly Estimate)
• the Longitude coordinate goes from 0 to 360 instead of from -180 to 180.
• there are multiple attributes. Those are very important, since they contain dataset metadata information.

That being said, we need to do a few things to make this dataset ready for analysis, visualization, and/or download.

The first thing to do is to adjust the Longitude coordinates and sort the data accordingly:

Now we can see that the dataset has Longitude values between -180 and 180. Great!

The next step is to subset the data - we don’t need SLA for the entire globe, just for the Western Tropical Atlantic, so there’s no need to be messing with large files. To subset the data, we leverage the power of slicing in xarray:

This looks better, but in order for this to be the main object of analysis and for us to be able to delete the original dataset (spanning the entire global ocean), we need to make sure it has metadata information. In order to do that, we gather some attributes from the original dataset, remove the ones that don’t make sense after subsetting, and assign these attributes to the sla_subset DataArray:

Great, now that we have data and metadata. Now, let’s take a quick look at the data for the few time-steps that we have loaded and see if they show any interesting features for us to investigate further. We can do that easily and interactively with hvplot:

After inspection, we decide that the data for Sept 23rd, 2020 looks promising. So let’s subset further, to have just this time-step in our DataArray:

Note that the subsetted DataArray has all the attributes that we have assigned from the original dataset.

Sea Surface height anomaly visualization¶

Now let’s make a more ellaborate visualization of the data to analyze the features better. For that, we will use the library cartopy.

The colorbar centered in zero helps us understand the data better. The main features are in the eastern part, where close to the coast we can see strong gradients, but altimetry data too close to the coast might have some issues. So we focus on the features further from the coast, and the main ones we see are circular contour lines between 5°N and 10°N, and 45°W and 55°W. Some knowledge of the ocean circulation patterns in the area suggest that these features are associated with the North Brazil Current retroflection and a North Brazil Current Ring. Can we see any signature of that in salinity? Let’s check!

Sea surface salinity data¶

Searching and accessing the data¶

Similarly to what was done for sea surface height anomaly, we’ll visit the Earthdata Search website to see what salinity data is available for the region and the time-period we’re interested. After some search, we find that the dataset with the shortname “SMAP_RSS_L3_SSS_SMI_8DAY-RUNNINGMEAN_V5” seems applicable, even though it’s an 8-day mean. Let’s set the same bounding box, the time span for around the same day we analyzed for sea surface height anomaly, and see what data is available:

We notice that there were 9 granules for the time span between 2020-09-23 and 2020-09-24, which seems excessive. Let’s take a look at some of the granules:

Looking at the EndingDateTime and BeginningDateTime for each granule, we conclude that earthaccess found all the granules in which the date 2020-09-23 and/or 2020-09-24 were used to calculate the 8-day mean, and that’s why there are so many granules.

Let’s load the dataset:

In this dataset we see many of the same issues we saw in the sea surface height anomaly dataset:

• large area, much bigger than the bouding box
• other data variables besides sea surface salinity
• longitude between 0 and 360 instead of -180 and 180
• multiple attributes - that’s a good thing!

Subsetting the data, adjusting the coordinates and assigning attributes¶

Now we need to adjust this dataset, very similarly to what we did for the sea surface height anomaly one. First, we convert the longitude coordinates and sort the data accordingly:

Then we subset just for the variable we want, for the area within the bounding box, and make sure to assign most attributes from the original dataset:

Now we need to choose a time-step. If we look at the coordinates “time”, we’ll notice that the days in it are the center days of the running mean. So we’ll just pick the same day used for the sea surface height anomaly data, ‘2020-09-23’:

Sea surface salinity visualization¶

Great, we got a DataArray with just one time-step; now let’s plot a map of it:

The sea surface salinity map seems good, but if we put the plots next to each other, we could have a better understanding of what’s going on.

Sea surface height anomaly and sea surface salinity combined visualization¶

Great; with the maps side-by-side, it’s easier to understand the features. In the salinity map, we can see the freshwater discharge from the Orinoco and Amazon Rivers, close to the coast, at about 61°W,9°N and -48°W,0°, respectively. In addition, we can see a somewhat circular fresher water feature between about 50°W-55°W and 5°N-10°N. The location of this feature coincides with the location of the high circular sea surface height anomaly contours, strenghtening our hypothesis that it’s a North Brazil Current Ring transporting freshwater from the Amazon River to the Caribbean. To the right, one can see the higher salinity values in a concave configuration, coinciding with the location of other circular sea surface height anomaly contours, suggesting this is a signature of the North Brazil Current retroflection, bringing salty water from the Equatorial Atlantic. For more information on this process, see this article.

Saving the data¶

Remember: we have done all this visualization without downloading one single data file! That’s one of the best functionalities of xarray, the ability of looking at the data with “no strings attached”, so you can only download once you know that dataset will work for you.

Now that we’ve seen that the subset DataArrays have interesting features, we may want to download them to our local machines, so we can analyze them further. That’s very easy to do with xarray, by saving it in netCDF format. Here we build some filenames to use to save the DataArrays, but of course we could use any filename.

One very important thing to notice is that the saved files will contain the attributes we assigned from the original Dataset. This is necessary for reproducibility, so other people can build on your work, or even future you can have access to details of the data.

Summary¶

In this notebook we were able to leverage the earthaccess library to access large datasets with xarrray and visualize them with matplotlib and cartopy. We learned the advantages of accessing data programatically, including promoting science reproducibility. We were able to subset data and then, only when we identified data that would be of interest, download it. That is specially advantageous for cases with limited available storage or limited computing capacity.

What’s next?¶

Nasa Earthdata Search has various types of data, in different formats and from different platforms. The user is certainly encouraged to play around with other types of data, to become more comfortable with this tool.

The earthaccess library is a wrap around the NASA Earth Science APIs. With the knowledge from this notebook, the user may understand bettter the structure of metadata in APIs and feel more comfortable accessing data via APIs from other sources, such as NCEI and USGS.

Resources and references¶

• Earthdata Search web application
• info on earthaccess
• earthaccess Github repository
• earthaccess documentation page
• Earthdata cloud clinic
• USGS Science Data Catalog API Documentation
• NCEI API user documentation
• Intro to cartopy
• Matplotlib basics
• netCDF and CF: the basics
• Introduction to Xarray
• Article on North Brazil Current Rings