Newsletters - MATLAB Digest

Reading and Visualizing Data from the Earth Observing System (EOS)

(Part 1 of 4)

MATLAB can read and write a wide variety of scientific and image data files. It includes capabilities to process and visualize these images and data sets. This article illustrates how to use MATLAB to explore, extract, and display satellite remote sensing data sets distributed by the National Aeronautical and Space Administration (NASA) in Hierarchical Data Format (HDF).

The examples included use both plain HDF and HDF-EOS files. HDF-EOS is a higher-level format that is used to manage and distribute satellite imagery and data products from the Earth Observing System (EOS), a series of satellites designed to study global climate change. This article demonstrates how to use the georeferencing information in HDF-EOS to display the data in its correct geographic location.

The article also shows how to use direct indexed color to create multiple quantitative color scales in MATLAB graphics. The techniques shown here can be applied to a broad range of scientific and engineering data.

Specifically, we demonstrate how to:

Examine HDF and HDF-EOS data sets and extract data (using hdftool)
Apply different color scales to different data sets within the same MATLAB figure (using colormapeditor)
Create colorscales with tick labels in physical units (using colorscale)
Display georeferenced data sets in latitude-longitude coordinates
Display georeferenced data sets in a projected coordinate system using the Mapping Toolbox

hdftool and colormapeditor are new GUIs that were introduced in MATLAB 6.5 (R13), and colorscale is a new function that we have contributed to MATLAB Central. You can download the full set of files required to run the demonstration at: www.mathworks.com/matlabcentral/fileexchange/Files.jsp?fileId=2611.

Color Scaling Preview

Managing your own color scaling is not difficult, given a little planning. Consider the following comparison based on data to be discussed more completely later in this article.

We want to display a 180-by-360 global grid of one-degree-by-one-degree cells as an indexed color image. Cells covering land areas represent a quantity known as an NDVI, or normalized difference vegetation index. NDVI starts near -0.25 in barren areas and ranges close to 1.0 for areas with high vegetation coverage and chlorophyll content. In our data set, NDVI has been scaled and shifted, then encoded into unsigned 8-bit integers according to the formula

encodedValue = ndviValue/ndviScaleFactor + ndviAddOffset,with ndviScaleFactor = 0.008 and ndviAddOffset = 128. The encoded values range from approximately 90 to just under 255. However, some land areas have no data and are designated with the value 1 in the encoded data set. Water areas are designated with the value 0.

We select a colormap with a deep blue in the first row (for water), a dark gray in the second row (for "no data"), then a smooth gradation from white through brown, yellow-green, and dark green. Next, we attempt to apply this colormap to the data using imagesc, along with colorbar, with the following result:

Click on image to see enlarged view.

View the MATLAB code used to create this figure.

With reasonable effort we have positioned and labeled our axes nicely, but we are left with the following problems:

The null data land areas (parts of the southern continents, mainly) fall outside of the axes CLim limits, so they use the first row of the color map and hence appear blue, just like the water.
The colorbar is labeled in terms of colormap row subscripts instead of physically meaningful NDVI values. Also, the two special colors appear at its left end.
We are unable to overlay another indexed color data set with a colormap of its own.

Solving problem 1 requires using direct color data mapping and replacing imagesc with image. The next step is to either match the data to the colormap or vice versa. colorscale solves problem 2. The solution to problem 3 is to concatenate the colormaps for two or more data sets and remap the data values to match the new composite colormap. By combining these approaches, we obtain the following:

Click on image to see enlarged view.

View the MATLAB code used to create this figure.

It is clear that we have overcome problems 1 and 2, because null data land areas are now distinct from water and the NDVI color scale bar is in natural NDVI units. Our approach overcame problem 3, also. We used a composite colormap that included a section for sea-surface temperature (SST) calculated from thermal infrared emissions, which were recorded by a satellite sensor. We included a SST swath covering parts of the eastern Mediterranean Sea and the northern Red Sea. SST has a color scale of its own in natural units of degrees Celsius.

Next, we examine the details behind this example.

HDF and HDF-EOS

The National Center for Supercomputing Applications (NCSA) introduced the Hierarchical Data Format (HDF) in 1987 to manage a wide variety of scientific data. An HDF file has a self-documenting structure. It consists of hierarchically arranged groups of objects, such as images, tables, text, data arrays, and lower-level groups. NASA and NCSA created HDF-EOS to manage and distribute data from the Earth Observing System. HDF-EOS format builds on the standard HDF data objects to add high-level grid, point, and swath data objects. HDF-EOS objects reference their data to Earth coordinates (latitude-longitude or projected map coordinates) and to time (see http://earthobservatory.nasa.gov/Study/HDFEOS/).

MATLAB enables you to work with HDF and HDF-EOS at multiple levels. The most basic interfaces consist of low-level and API-level functions. (See hdf*.) These interfaces provide functionality that is equivalent to the C and Fortran libraries available from NCSA and NASA. In MATLAB, you can also work at a high level, using hdfinfo and hdfread. And, with MATLAB 6.5, you can graphically explore and extract data from an HDF or HDF-EOS file with the new HDF Import Tool (hdftool). For further information on HDF/HDF-EOS support in MATLAB see Working with HDF Data in the MATLAB online documentation.

There are actually two distinct HDF formats: HDF 4.x (and earlier) and HDF5. Most HDF-based scientific data products being created today are in HDF 4, and the NCSA plans to continue support for HDF 4.x indefinitely. The new HDF5 represents a major change in architecture. It is really a completely new (and incompatible) format. HDF5-based data products are in the planning phase or are just beginning to be developed. HDF-EOS was based on HDF 4.x, but data products from some future sensors and EOS missions will use a new version based on HDF5, called HDF-EOS5. MATLAB 6.5 (and earlier) supports only HDF 4 and HDF-EOS 2 (the version based on HDF 4), but HDF5 capabilities are under active development for inclusion in a future release.

The examples in this article illustrate the use of hdfinfo, hdfread, and hdftool. We use these high-level tools to extract a subset of an HDF-EOS swath data set and to read a global grid of data from an HDF data file. We also use these functions to read important metadata and georeferencing information. We visualize the data in MATLAB as images and surfaces, displayed in their proper locations on latitude-longitude and projected map coordinate axes.

Part 2