Visualizing High-Resolution Grids

In this tutorial, you’ll learn about:

• Techniques for visualizing high-resolution grids, including best practices for performance optimization and rendering

• The importance of rasterization in managing computational overhead when working with detailed global datasets

• Strategic use of subsetting functionality to enhance visualization performance for regional analyses

• Key differences between point-based and polygon-based visualization approaches at varying grid resolutions

• Effective use of map projections to improve spatial data representation and address point distribution challenges

• Control and optimization of visualization quality through pixel ratio parameters for both point and polygon plots

Time to learn: 15 minutes

---

Introduction

This notebook explores techniques for working with high-resolution unstructured grids, building upon the foundational concepts previously covered. While earlier examples focused on coarse-resolution grids to introduce core principles, many real-world applications require handling more detailed grid structures.

For this demonstration, we utilize output from the 30km MPAS Atmosphere model, representing a significant increase in resolution compared to our previous examples. While this resolution allows us to effectively illustrate key visualization concepts, it’s important to note that atmospheric and climate models often employ even higher resolutions. The concepts covered in this notebook are applicable to these higher resolutions.

import cartopy.crs as ccrs
import holoviews as hv
import uxarray as ux

hv.extension("matplotlib")

grid_path = "../../meshfiles/x1.655362.grid.nc"
data_path = "../../meshfiles/x1.655362.data.nc"

uxds = ux.open_dataset(grid_path, data_path)
uxds["relhum_200hPa"]

<xarray.UxDataArray 'relhum_200hPa' (Time: 1, n_face: 655362)> Size: 3MB
[655362 values with dtype=float32]
Dimensions without coordinates: Time, n_face
Attributes:
    units:      percent
    long_name:  Relative humidity vertically interpolated to 200 hPa

<xarray.UxDataArray 'relhum_200hPa' (Time: 1, n_face: 655362)> Size: 3MB
[655362 values with dtype=float32]
Dimensions without coordinates: Time, n_face
Attributes:
    units:      percent
    long_name:  Relative humidity vertically interpolated to 200 hPa

uxarray.UxDataArray

'relhum_200hPa'

• Time: 1
• n_face: 655362

• ...

  [655362 values with dtype=float32]

• Coordinates: (0)
• Indexes: (0)
• Attributes: (2)

  units :

  percent

  long_name :

  Relative humidity vertically interpolated to 200 hPa

Show Grid Information

<uxarray.Grid>
Original Grid Type: MPAS
Grid Dimensions:
  * n_node: 1310720
  * n_face: 655362
  * n_max_face_nodes: 10
  * n_max_node_faces: 3
  * n_nodes_per_face: (655362,)
Grid Coordinates (Spherical):
  * node_lon: (1310720,)
  * node_lat: (1310720,)
  * face_lon: (655362,)
  * face_lat: (655362,)
Grid Coordinates (Cartesian):
Grid Connectivity Variables:
  * face_node_connectivity: (655362, 10)
  * node_face_connectivity: (1310720, 3)
Grid Descriptor Variables:
  * n_nodes_per_face: (655362,)

uxarray.UxDataArray.uxgrid

• Dimensions:
  • n_node: 1310720
  • n_face: 655362
  • n_max_face_nodes: 10
  • n_max_node_faces: 3
• Spherical Coordinates: (4)
  • node_lon
    (n_node)
    float32
    -175.0 -174.8 ... -103.0 -175.0

    standard_name :

    longitude

    long name :

    Longitude of the corner nodes of each face

    units :

    degrees_east

    array([-174.95294, -174.81796, -174.86942, ...,  113.04705, -102.95294,
           -174.95294], dtype=float32)

  • node_lat
    (n_node)
    float32
    26.44 26.53 26.67 ... -52.62 -52.62

    standard_name :

    latitude

    long name :

    Latitude of the corner nodes of each face

    units :

    degrees_north

    array([ 26.43807 ,  26.525747,  26.667755, ..., -10.812317, -52.62263 ,
           -52.62263 ], dtype=float32)

  • face_lon
    (n_face)
    float32
    -175.0 -103.0 ... -175.2 -174.7

    standard_name :

    longitude

    long name :

    Longitude of the center of each face

    units :

    degrees_east

    array([-174.95294, -102.95294,  115.01709, ..., -174.95294, -175.1781 ,
           -174.72784], dtype=float32)

  • face_lat
    (n_face)
    float32
    26.57 26.57 90.0 ... -52.54 -52.54

    standard_name :

    latitude

    long name :

    Latitude of the center of each face

    units :

    degrees_north

    array([ 26.56505 ,  26.56505 ,  90.      , ..., -52.780724, -52.54337 ,
           -52.54337 ], dtype=float32)

• Cartesian Coordinates: (0)
• Connectivity: (2)
  • node_face_connectivity
    (n_node, n_max_node_faces)
    int64
    0 163843 163844 ... 655360 655361

    cf_role :

    node_face_connectivity

    long name :

    Faces that neighbor each node

    start_index :

    0

    _FillValue :

    -9223372036854775808

    array([[     0, 163843, 163844],
           [     0, 163844, 163845],
           [     0, 163845, 163846],
           ...,
           [655353, 655354, 655355],
           [655356, 655357, 655358],
           [655359, 655360, 655361]])

  • face_node_connectivity
    (n_face, n_max_face_nodes)
    int64
    0 1 ... -9223372036854775808

    cf_role :

    face_node_connectivity

    long name :

    Maps every face to its corner nodes.

    start_index :

    0

    _FillValue :

    -9223372036854775808

    array([[                   0,                    1,                    2,
            ..., -9223372036854775808, -9223372036854775808,
            -9223372036854775808],
           [                   5,                    6,                    7,
            ..., -9223372036854775808, -9223372036854775808,
            -9223372036854775808],
           [                  10,                   11,                   12,
            ..., -9223372036854775808, -9223372036854775808,
            -9223372036854775808],
           ...,
           [              983022,              1228798,               983034,
            ..., -9223372036854775808, -9223372036854775808,
            -9223372036854775808],
           [             1310719,               983028,               983033,
            ..., -9223372036854775808, -9223372036854775808,
            -9223372036854775808],
           [             1310719,               983039,               983038,
            ..., -9223372036854775808, -9223372036854775808,
            -9223372036854775808]])

• Descriptors: (0)
• Attributes: (54)

  model_name :

  mpas

  core_name :

  init_atmosphere

  source :

  MPAS

  Conventions :

  MPAS

  git_version :

  v6.0-dirty

  on_a_sphere :

  YES

  sphere_radius :

  6371229.0

  is_periodic :

  NO

  x_period :

  0.0

  y_period :

  0.0

  history :

  Mon Aug 16 12:09:52 2021: ncks -v latVertex,lonVertex,latCell,lonCell,latEdge,lonEdge,cellsOnVertex,verticesOnCell,verticesOnEdge,edgesOnVertex,edgesOnCell,nEdgesOnCell,angleEdge x1.655362.init.nc foo.nc mpirun -n 18 ./init_atmosphere_model

  parent_id :

  0hhmjs26t0

  mesh_spec :

  0.0

  config_init_case :

  7

  config_calendar_type :

  gregorian

  config_start_time :

  2016-08-01_00:00:00

  config_stop_time :

  2016-09-10_00:00:00

  config_theta_adv_order :

  3

  config_coef_3rd_order :

  0.25

  config_num_halos :

  2

  config_nvertlevels :

  75

  config_nsoillevels :

  4

  config_nfglevels :

  38

  config_nfgsoillevels :

  4

  config_months :

  12

  config_geog_data_path :

  /glade/p/work/wrfhelp/WPS_GEOG/

  config_met_prefix :

  FILE

  config_sfc_prefix :

  SST

  config_fg_interval :

  86400

  config_landuse_data :

  USGS

  config_topo_data :

  GMTED2010

  config_use_spechumd :

  NO

  config_ztop :

  40000.0

  config_nsmterrain :

  1

  config_smooth_surfaces :

  YES

  config_dzmin :

  0.3

  config_nsm :

  30

  config_tc_vertical_grid :

  YES

  config_extrap_airtemp :

  linear

  config_static_interp :

  NO

  config_native_gwd_static :

  NO

  config_gwd_cell_scaling :

  1.0

  config_vertical_grid :

  YES

  config_met_interp :

  YES

  config_input_sst :

  NO

  config_frac_seaice :

  YES

  config_pio_num_iotasks :

  0

  config_pio_stride :

  1

  config_block_decomp_file_prefix :

  x1.655362.graph.info.part.

  config_number_of_blocks :

  0

  config_explicit_proc_decomp :

  NO

  config_proc_decomp_file_prefix :

  graph.info.part.

  file_id :

  ytj0fh0qmd

  NCO :

  netCDF Operators version 4.9.5 (Homepage = http://nco.sf.net, Code = http://github.com/nco/nco)

Polygon Plots

When working with high-resolution grids at global scales, performance optimization becomes crucial. As detailed in the Data Visualization notebook, setting rasterize=True is strongly recommended for these scenarios. This parameter significantly improves rendering performance by converting vector-based polygons into raster images during visualization.

%%time
uxds["relhum_200hPa"][0].plot.polygons(rasterize=True)

CPU times: user 5.11 s, sys: 104 ms, total: 5.21 s
Wall time: 5.21 s

The initial rendering of polygon plots requires significant processing time due to the conversion of each grid face into a Polygon object and its subsequent storage in a GeoDataFrame. However, this computational investment yields long-term benefits: once these geometries are processed and stored, they can be efficiently reused in subsequent visualizations. This caching mechanism ensures that follow-up polygon plots render substantially faster, making the initial processing time a worthwhile trade-off for improved ongoing performance. Below we can observe the faster rendering time after the initial execution.

%%time
res = uxds["relhum_200hPa"][0].plot.polygons(rasterize=True)

CPU times: user 291 ms, sys: 32 μs, total: 291 ms
Wall time: 290 ms

Subsetting

To enhance performance when visualizing specific geographic regions, you can combine polygon plotting with subsetting functionality. This optimization strategy significantly reduces computational overhead by rendering only the polygons within your region of interest, rather than processing the entire grid. For example, when analyzing weather patterns over North America, subsetting the global grid to that continent will dramatically improve rendering speed and resource efficiency.

relhum_subset = uxds["relhum_200hPa"][0].subset.bounding_box(
    lon_bounds=[-5, 5], lat_bounds=[-2.5, 2.5]
)
relhum_subset.plot.polygons(rasterize=True)

We can restrict our region even further to clearly see the original polygons.

relhum_subset = uxds["relhum_200hPa"][0].subset.bounding_box(
    lon_bounds=[-0.5, 0.5], lat_bounds=[-0.25, 0.25]
)
relhum_subset.plot.polygons(rasterize=True)

Controlling Resolution

The pixel_ratio parameter offers precise control over resolution in polygon plots. This parameter determines how closely the rasterized output approximates the original vector-based polygon representation.

Setting a larger pixel ratio produces higher-fidelity visualizations that more accurately represent the underlying data structure and polygon geometries. These high-resolution outputs closely resemble traditional vector polygon plots, preserving intricate spatial relationships and boundaries.

Conversely, specifying a smaller pixel ratio generates lower-resolution visualizations. While this approach may sacrifice some visual detail, it can significantly improve rendering performance for large datasets or when exact polygon boundaries are less critical for the analysis at hand.

This resolution control mechanism enables you to balance visual accuracy with computational efficiency based on your specific visualization requirements and performance constraints.

(
    uxds["relhum_200hPa"][0].plot.polygons(
        rasterize=True, pixel_ratio=0.1, title="0.1 Pixel Ratio"
    )
    + uxds["relhum_200hPa"][0].plot.polygons(
        rasterize=True, pixel_ratio=4.0, title="4.0 Pixel Ratio"
    )
).cols(1)

Point-Based Visualization Strategies

Previous examination of rasterized point plots in the Data Visualization section revealed their limitations for coarse-resolution grids. However, as grid resolution increases, this visualization method becomes increasingly advantageous, delivering superior rendering performance compared to polygon-based approaches while maintaining high visual quality.

Our 30km global grid demonstration showcases this enhanced effectiveness. At this resolution, rasterized point plots achieve visual fidelity approaching that of traditional polygon plots, though with two important technical considerations to address during implementation:

The visualization output exhibits missing values, denoted as NaN in the resulting display. Additionally, unprojected visualizations reveal an inherent characteristic of global coordinate systems: point density increases notably near the equator relative to the poles. This creates a distinctive variation in visual density—an expected artifact when representing spherical data in a planar format.

%%time
uxds["relhum_200hPa"][0].plot.points(rasterize=True)

CPU times: user 26.1 ms, sys: 67 μs, total: 26.2 ms
Wall time: 25.9 ms

Optimizing Point Distribution Using Map Projections

Using appropriate map projections significantly enhances point distribution offering marked improvements over unprojected representations. Map projections help normalize the spatial distribution of data points, substantially reducing the clustering effects observed in standard latitude-longitude visualizations.

For example, an Orthographic projection provides one effective approach, though various other projections can also help. While projections address many distribution challenges, some missing values may persist in the output. However, these remaining gaps typically represent a considerable improvement over unprojected versions, where equatorial clustering creates more pronounced visualization artifacts.

%%time
uxds["relhum_200hPa"][0].plot.points(rasterize=True, projection=ccrs.Orthographic())

CPU times: user 26.4 ms, sys: 0 ns, total: 26.4 ms
Wall time: 26.2 ms

Controlling Resolution & Binning

The pixel_ratio parameter provides precise control over the rasterization process by determining the size of individual raster pixels in your visualization. This parameter directly influences both the visual resolution and the data aggregation characteristics of your plot.

When you specify a larger pixel ratio, the visualization creates smaller raster pixels. This higher-resolution approach means fewer data points are aggregated within each pixel, potentially leading to gaps in the visualization where pixels contain no data points. These empty pixels appear as NaN values in the final output.

Conversely, setting a smaller pixel ratio creates larger pixels that capture and aggregate more data points. This approach typically produces a more continuous visualization by reducing the likelihood of empty pixels, though it may sacrifice some fine detail in the process.

The pixel ratio effectively serves as a resolution control mechanism, allowing you to balance between granular detail and visual continuity based on your specific visualization requirements. This flexibility becomes particularly valuable when working with datasets of varying densities or when focusing on specific geographic regions that require different levels of detail.

(
    uxds["relhum_200hPa"][0].plot.points(
        rasterize=True,
        projection=ccrs.Orthographic(),
        pixel_ratio=0.5,
        title="0.5 Pixel Ratio",
    )
    + uxds["relhum_200hPa"][0].plot.points(
        rasterize=True,
        projection=ccrs.Orthographic(),
        pixel_ratio=3.0,
        title="3.0 Pixel Ratio",
    )
).cols(1)