New in ENVI 5.4: ENVITask Returning a Virtual Raster

Since the creation of custom ENVITasks in ENVI 5.2 SP1, there has been the requirement that your procedure must commit all output objects to disk. There was the rule that the procedure wrapped by the task must have an input keyword telling it the filename to use to write the object to. In the task template, you would have an output parameter that mapped to that keyword, and then during task loading the framework would magically create an input parameter for you, mapped to the same keyword with TYPE set to “ENVIURI”.

The automatic creation of the input parameter and the internal correlation of the two parameters were done with the best of intentions, to simplify the process of creating custom tasks. Alas the user feedback on this feature wasn’t always as rosy as we hoped.

So in ENVI 5.4 we’re shaking things up and giving you the task developer more control. If you still use the “version” property in the task template, and have it set to “5.2.1”, “5.3”, or “5.3.1”, then you’ll get the old behavior. But if you switch to using “schema” set to the value “envitask_3.0”, then a new set of rules apply to the procedure, and what you can do inside it. In the new paradigm, your procedure will have separate keywords for the input filename and output object reference. If you like, you can skip the filename keyword completely and return an object that hasn’t been tethered to disk at all. This makes life much easier for types like ENVIGCPSet and ENVITiePointSet, but also allows for a procedure that constructs a complex virtual raster chain based on the other input parameters.

You might be asking why you would want a task to return a virtual raster. With the new Classification Framework that is part of ENVI 5.4, you need to make sure that you prepare the data you want to run through a trained classifier the same way you prepared your training data. One way to do this is to create a task that returned the preprocessed data in a consistent way. If you can get away with not having to save that preprocessed attribute raster to disk, why not take advantage of the time and space advantages of using a virtual raster.

The following example is a modified version of the new Custom Task Tutorial in the ENVI 5.4 release. That task wraps a procedure that uses a number of other tasks internally to perform all the preprocessing.  Here I’ve modified it to use virtual rasters and other ENVI API function calls to avoid ever writing a file to disk.

The code goes through a number of steps, which I will describe after showing it:

pro SentinelVegIndices_blog, INPUT_RASTER_10M=raster10m, $
                             INPUT_RASTER_20M=raster20m, $
                             OUTPUT_RASTER=outputRaster
  compile_opt idl2, hidden
 
  ; Get the spatial reference of the 10-meter raster
  spatialRef = raster10m.SPATIALREF
  coordSys = ENVICoordSys(COORD_SYS_CODE=spatialRef.COORD_SYS_CODE)
 
  ; Create a spatial grid definition
  grid = ENVIGridDefinition(coordSys, $
                            PIXEL_SIZE=spatialRef.PIXEL_SIZE, $
                            NCOLUMNS = raster10m.NCOLUMNS, $
                            NROWS = raster10m.NROWS, $
                            TIE_POINT_MAP=spatialRef.TIE_POINT_MAP, $
                            TIE_POINT_PIXEL = spatialRef.TIE_POINT_PIXEL)
 
  ; Regrid the 20-meter bands to 10 meters
  regriddedRaster = ENVISpatialGridRaster(raster20m, GRID_DEFINITION=grid)
 
  ; Create a layer stack
  layerStack = ENVIMetaspectralRaster([raster10m, regriddedRaster], $
                                      SPATIALREF=spatialRef)
 
  ; Compute image statistics
  stats = ENVIRasterStatistics(layerStack)
 
  ; Perform dark subtraction as an alternative to atmospheric correction
  bandRasters = ObjArr(layerStack.nBands)
  for i = 1, layerStack.nBands do begin
    expression = 'b' + i.ToString() + ' - ' + stats.Min[i-1].ToString()
    bandRasters[i-1] = ENVIPixelwiseBandMathRaster(layerStack, expression)
  endfor
  bandStackRaster = ENVIMetaspectralRaster(bandRasters, $
                                           SPATIALREF=spatialRef)
 
  ; we need to put the wavelengths back into the band stack,
  ; they were removed by the band math
  metadata = ENVIRasterMetadata()
  metadata['wavelength'] = layerStack.Metadata['wavelength']
  metadata['wavelength units'] = layerStack.Metadata['wavelength units']
  correctedRaster = ENVIMetadataOverrideRaster(bandStackRaster, $
                                               METADATA_OVERRIDE=metadata)
 
  ; Scale pixel values from 0 to 1
  gains = MAKE_ARRAY(layerStack.NBANDS, /FLOAT, VALUE=0.0001)
  offsets = FLTARR(layerStack.NBANDS)
  scaledRaster = ENVIGainOffsetRaster(correctedRaster, gains, offsets)
 
 
  ; Create vegetation indices
  indices = [ 'Enhanced Vegetation Index', $
              'Global Environmental Monitoring Index', $
              'Leaf Area Index', $
              'Plant Senescence Reflectance Index', $
              'Red Edge Normalized Difference Vegetation Index' ]
  outputRaster = ENVISpectralIndexRaster(scaledRaster, indices)
end

The first step is to upsample the 20m raster to 10m pixels, which in the tutorial is performed using the ENVIRegridRasterTask. This can be done with the ENVISpatialGridRaster virtual raster, once we have constructed an ENVIGridDefinition with the appropriate mixing of properties form the 10m and 20m rasters.

Next the tutorial uses the ENVIBuildBandStackTask to build a metaspectral stack of all the 10m bands. Here we use the ENVIMetaspectralRaster virtual raster, though we have to pass in the original 10m raster’s spatial reference to keep this raster registered on the map.

Next is the dark subtraction. The tutorial uses the ENVIRasterStatisticsTask, but here we just use the API function ENVIRasterStatistics() to accomplish the same thing.

The band minima values are used to perform dark object subtraction. The tutorial uses the ENVIDarkSubtractionCorrectionTask, which handles this as a single raster.  Here I had to build separate band math equations for each band, as the ENVIPixelwiseBandMathRaster virtual raster always returns a single band output, so I have to build an array of band math expressions and then metaspectrally stack the results, again passing in the spatial reference.

One pitfall of the band math is that it removes most of the spectral metadata, so I have to put the wavelength metadata back into the raster so the spectral index calculations select the correct bands. I do this with ENVIMetadataOverrideRaster(), using a copy of the original metaspectral raster’s metadata values.

The raster is then scaled down by a factor of 10000.0 with ENVIGainOffsetRaster, to simulate the atmospheric correction better and produce spectral index values that are more accurate. Lastly, the scaled raster is passed into ENVISpectralIndexRaster to calculate 5 different spectral indices.

Once this has been written, we can use an almost identical .task file to wrap the procedure. The main difference is that we no longer need to specify the OUTPUT_URI parameter, and I had a slightly different procedure name. This task’s output raster is never commited to disk, but it can be used as input to another task by calling ENVIRaster::Dehydrate() on it, which yields a 25KB JSON representation.

Here is the updated .task file:

{
  "name": "SentinelVegIndices_blog",
  "base_class": "ENVITaskFromProcedure",
  "routine": "SentinelVegIndices_blog",
  "display_name": "Compute Sentinel-2A Vegetation Indices",
  "description": "This task regrids the visible and near-infrared bands of a Sentinel-2A Level-1C 20-meter image to 10 meters. It creates a layer stack of all bands. It applies dark-object subtraction as a simple alternative to atmospheric correction. It scales the reflectance pixel values from 0 to 1, then computes a select group of vegetation indices.",
  "schema": "envitask_3.0",
  "parameters": [
    {
    "name": "INPUT_RASTER_10M",
    "display_name": "Select a 10-meter image",
    "type": "ENVIRASTER",
    "direction": "input",
    "required": true,
    "description": "Select a Sentinel Level-1C 10-meter image."
    },
    {
    "name": "INPUT_RASTER_20M",
    "display_name": "Select a 20-meter image",
    "type": "ENVIRASTER",
    "direction": "input",
    "required": true,
    "description": "Select a Sentinel Level-1C 20-meter image."
    },
    {
    "name": "OUTPUT_RASTER",
    "display_name": "Output image",
    "type": "ENVIRASTER",
    "direction": "output",
    "required": true,
    "description": "This is a reference to the output raster."
    }
  ]
}