Working with Landsat Surface Reflectance High Level Data Products in ENVI

For the past several months, I have been hearing from ENVI users about a new set of provisional Landsat products being provided by the USGS. These products, known as Surface Reflectance High Level Data Products,  are generated by specialized software that was originally developed by NASA. The software applies radiometric calibration and atmospheric correction algorithms to Level-1 Landsat data products. What this means for users of Landsat data, is that you can access data that has been pre-processed to top-of-atmosphere (TOA) reflectance, surface reflectance, or (in the case of thermal bands) brightness temperature. The data products also include masks for clouds, cloud shadows, adjacent clouds, land, and water. Users can also order several spectral index products for their Landsat scenes. This is an interesting development because although this sort of pre-processing and analysis is bread and butter for ENVI, the ability to order a product that has been pre-processed using proven radiometric calibration and atmospheric correction algorithms ensures analysis will be as accurate as possible and can ultimately save users a lot of time.

This past week, I was finally able to dig into these data products to see what we could do with them in ENVI. The good news is that for the most part, there's not much a user needs to do to start working with the data products in ENVI. The first step is simply ordering the products that you would like. To order the products you must have an account with the USGS (if you don't have one already, it's easy to get one). You can order the products from the USGS Earth Resources Observation and Science (EROS) Center Science Processing Architecture (ESPA) On Demand Interface. This interface is pretty nice. The only real trick is that you have to put the Landsat scene IDs for the scenes you would like to order into a text file that you will upload to the interface. I typically order Landsat scenes through GloVis, so I used GloVis to search for some Landsat 8 scenes that I wanted to order and copied the scene IDs into a text file once I found the ones I wanted (here's an example of a Landsat 8 scene ID: LC80330342013269LGN00). The screenshot below shows an example of the ordering interface.

Once I uploaded the text file to the interface, there were a lot of options to choose from. Since I wanted to check out all of the possibilities, I ordered quite a few products. For starters, I ordered the original Level-1 input products and metadata. This was nice because I got the raw data for each band and the *MTL.txt file that ENVI can read to open the original Level-1 product with all of the metadata and band information you would expect. I also ordered TOA reflectance products, surface reflectance products, brightness temperature products, and a few of the spectral indices that are available. Users also have the ability to reproject all of the images in their order to a common coordinate system, modify the image extents, and resize pixels. I chose not to do this, but it is really nice that this is available. The last choice is the output format that you would like to receive your products in. There are three formats to choose from: GeoTiff, ENVI, and HDF-EOS2. I have not tested the HDF-EOS2 data format, but I have confirmed that both the GeoTiff and ENVI formats are supported in ENVI. Once I received my order, I used 7-zip to untar and unzip my files. The following screenshot shows an example of the files I received from my GeoTiff order.

So, what can we do with all of this in ENVI? For starters, if you ordered the original input images, you can drag-n-drop the *MTL.txt file into the ENVI display (or do File > Open and select the *MTL.txt file). This would open all of the GeoTiff files associated with the original Level-1 product in ENVI with all of the appropriate metadata and band combinations you would expect, allowing you to do all of the pre-processing and analysis that ENVI is capable of yourself.

But, let's be honest, if we wanted to do this ourselves, we wouldn't have ordered the higher level data products to begin with! While I can't touch on all of the things that we can do with these products, I'd like to talk about what I felt was the most important thing to be sure to support, which is working with the surface reflectance images for the multispectral bands. This is what I felt would hold the most value to our users because they are the radiometrically calibrated and atmospherically corrected products that would be highly useful for calculating spectral indices, performing land use classification and determining land change from multitemporal datasets. The problem I encountered is that the metadata for these bands seems to be contained in a .xml file that contains information for all of the files for the scene. ENVI does not natively read this file (though it is possible we will support this in the future if customers show enough interest). For now, users can manually load the individual surface reflectance images for each band into ENVI and use the Layer Stacking tool to combine the individual files into one multispectral image containing all the bands, as shown below.

Once the bands have been layer stacked into one file, the Edit ENVI Header tool can be used to add appropriate metadata, such as appropriate band names, band wavelengths, acquisition time, sensor information, etc... This information can be found in the *MTL.txt file, in the *.xml file, or can be gleaned from the names of the individual filenames. While this approach will work, it's awfully manual and manual approaches to doing things take time. So, what I have come up with this week is a new set of extensions that will automatically stack the multispectral surface reflectance bands and add the appropriate metadata. So far, I have developed extensions that work with Landsat 8 in ENVI and GeoTiff formats, but soon I plan to have extensions that will do this for all Landsat sensors. Once complete, I will make these tools available in the ENVI extensions library.

If people show enough interest, I can develop similar tools that will automatically stack TOA reflectance bands and add the appropriate metadata. Once the multispectral surface reflectance images are stacked into a single file, users can carry on with whatever analysis they choose, such as performing Tasseled Cap, Principle Components Analysis, choosing from one of ENVI's 65 built-in spectral indices, analyzing multitemporal changes, or creating classification images. I welcome feedback since I would like to know what other uses people might have for these datasets and how we can best support them in ENVI.