Making ENVITasks with Unknown Output Cardinality Work

As I’ve blogged about before, wrapping your algorithms in ENVITasks can be a relatively straightforward process.  I especially enjoy using the option of not specifying my output product URIs and letting the system generate temporary filenames for me.  But there is a big caveat with this feature, which is that it won’t behave correctly when your output parameter is a dynamic array (ENVIRaster[*], for example), which makes the input URI parameter also a dynamic array of type ENVIURI[*].  In this case the system will not know how many temporary filenames are needed, so it will only generate 1, which will likely make your task fail.

Now you might be asking how a task could be designed that doesn’t know how many output item it generates, but it isn’t as far-fetched as it sounds.  We might want to build a simple task that takes in a single multi-band ENVIRaster and then exports separate single-band rasters, or chops it up into a specified number of tiles. If you wanted to test another task for its sensitivity to global vs local statistics, you could have a task that outputs a set of different random spatial subsets, so you can see if you get similar outputs for the same inputs with different surroundings.  In each of these cases the number of output ENVIRasters can vary depending on the input ENVIRaster and other parameter values.  The key factor is that if the output parameter is not a statically defined array size.

Let’s look at the simplest and most deterministic case – slicing a multi-band raster into a collection of single-band rasters.  Here is the task template:

{
    "name": "RasterBandSlicer",
    "baseClass": "ENVITaskFromProcedure",
    "routine": "RasterBandSlicer",
    "displayName": "ENVIRaster Band Slicer",
    "description": "This task takes an input raster and exports each band as a separate output raster.",
    "version": "5.3",
    "invocationType": "keywords",
    "parameters": [
        {
            "name": "INPUT_RASTER",
            "keyword": "INPUT_RASTER",
            "displayName": "Input Raster",
            "dataType": "ENVIRASTER",
            "direction": "input",
            "parameterType": "required",
                "description": "Specify the raster to slice into separate bands."
        },
        {
            "name": "OUTPUT_RASTER",
            "keyword": "OUT_FILENAMES",
            "displayName": "Output Rasters",
            "dataType": "ENVIRASTER[*]",
            "direction": "output",
            "parameterType": "required",
            "description": "This is an array of output rasters of filetype ENVI."
        }
    ]
}

And the PRO code for the procedure:

pro RasterBandSlicer, INPUT_RASTER=inputRaster, OUT_FILENAMES=outFilenames
  compile_opt idl2
 
  nBands = inputRaster.nBands
 
  if (N_Elements(outFilenames) ne nBands) then begin
    Message, 'Invalid OUT_FILENAMES, must have ' + StrTrim(nBands,2) + ' elements'
  endif
 
  for i = 0, nBands-1 do begin
    subRaster = ENVISubsetRaster(inputRaster, BANDS=i)
    subRaster.Export, outFilenames[i], 'ENVI'
  endfor
end

If we try to run this task without specifying any output filenames, it will fail because the ENVITask framework will only generate one filename when it encounters the ENVIURI[*] input parameter:

IDL> nv = ENVI(/HEADLESS)
IDL> file = FilePath('qb_boulder_msi', ROOT_DIR=nv.ROOT_DIR, SUBDIR='data')
IDL> oRaster = nv.OpenRaster(file)
IDL> oTask = ENVITask('RasterBandSlicer')
IDL> oTask.Input_Raster = oRaster
IDL> oTask.Execute
% RASTERBANDSLICER: Invalid OUT_FILENAMES, must have 4 elements
% Execution halted at: $MAIN$     

So we need to manually generate the appropriate number of filenames, 4 in this case:

IDL> nv = ENVI(/HEADLESS)
IDL> file = FilePath('qb_boulder_msi', ROOT_DIR=nv.ROOT_DIR, SUBDIR='data')
IDL> oRaster = nv.OpenRaster(file)
IDL> oTask = ENVITask('RasterBandSlicer')
IDL> oTask.Input_Raster = oRaster
IDL> tmp = nv.GetTemporaryFilename('')
IDL> oTask.Output_Raster_URI = [ tmp+'-band0.dat', tmp+'-band1.dat', $
                                 tmp+'-band2.dat', tmp+'-band3.dat']
IDL> oTask.Execute
ENVI> print, oTask.Output_Raster
<ObjHeapVar223720(ENVIRASTER)><ObjHeapVar223723(ENVIRASTER)><ObjHeapVar223726(ENVIRASTER)><ObjHeapVar223729(ENVIRASTER)>

The trick is that we need the task to know the cardinality of the output parameters before Execute is called on it - it’s the only way the system will know how many filenames to generate.  We can’t know this until the INPUT_RASTER parameter has its value set to an ENVIRaster, so we have a small window of opportunity – after SetProperty and before Execute.  The only option is to take advantage of the fact that the task is an object and use polymorphism to override SetProperty and update the OUTPUT_RASTER_URI parameter when we can.

If we look at the task template, we see that it is using the ENVITaskFromProcedure class, so that is the class we will inherit from.  The framework uses the task name to define the class of the task object, so we will follow that pattern and call the subclass ENVIRasterBandSliceTask.  Here is a barebones version of this class:

function enviRasterBandSlicerTask::Init, _REF_EXTRA=refExtra
  compile_opt idl2, hidden
 
  return, self.enviTaskFromProcedure::Init(_EXTRA=refExtra)
end
 
pro enviRasterBandSlicerTask::Cleanup
  compile_opt idl2, hidden
 
  self.enviTaskFromProcedure::Cleanup
end
 
pro enviRasterBandSlicerTask::SetProperty, _REF_EXTRA=refExtra
  compile_opt idl2, hidden
  ; call base class implementation to make sure everything's okay
  self.enviTaskFromProcedure::SetProperty, _EXTRA=refExtra
 
  ; get the input_raster parameter object to see if it has a valid value
  inputRasterParam = self.Parameter('INPUT_RASTER')
  ; if value is not a valid objRef, then return, since we need to query it's properties
  if (~Obj_Valid(inputRasterParam.Value)) then return
 
  ; get output_raster_uri parameter, so we can update its TYPE property
  outURIParam = self.Parameter('OUTPUT_RASTER_URI')
  ; define new type as statically sized array of ENVIURI
  newType = 'ENVIURI[' + StrTrim(inputRasterParam.Value.nBands,2) + ']'
  ; use undocumented _SetProperty to update the TYPE property
  outURIParam._SetProperty, TYPE=newType
end
 
pro enviRasterBandSlicerTask__define
  compile_opt idl2, hidden
 
  void = {enviRasterBandSlicerTask,       $
          inherits enviTaskFromProcedure  $
         }
end

The class is pretty boilerplate – inherit from ENVITaskFromProcedure and have passthrough Init() and Cleanup methods.  The only special code is in SetProperty, where we first call the base class implementation and then check if the INPUT_RASTER parameter has had its value set to a valid ENVIRaster yet or not.  If there is a valid ENVIRaster value, then we use its NBANDS property to create a new ENVIURI array type definition and set it on the OUTPUT_RASTER_URI parameter.  As I commented in the code, we have to use the ENVITaskParameter::_SetProperty method to update the TYPE property.  This is an undocumented method that is used internally to set what are normally immutable property values, but in this case we need to use it.

The only other change needed is to update the task template to specify ENVIRasterBandSlicerTask as the “baseClass” value instead of ENVITaskFromProcedure.  We don’t have to make any changes to the RasterBandSlicer procedure, since we didn’t change the parameter definitions at all.  But now the task will work in the desktop API without setting OUTPUT_RASTER_URI, and it works in ESE.