TrainTensorFlowMaskModel Task

This task has been depricated and renamed to TrainTensorFlowPixelModel.

This task trains a mask-based TensorFlow model. The model can be untrained or previously trained.

This task is part of ENVI Deep Learning, which requires a separate license and installation.

Example

Sample data files are available on our ENVI Tutorials web page. Click the "Deep Learning" link in the ENVI Tutorial Data section to download a .zip file containing the data. Extract the contents to a local directory. Files are located in the tornado directory.

This example builds label rasters from four input rasters and ROIs. It trains a deep learning model using three of the label rasters as training rasters and one as a validation raster. Follow these steps:

Copy and paste the code below into a new file in the IDL Editor.
Change the Path variable to the directory on your computer that contains the tutorial data.
Save the file as TrainTensorFlowMaskModelExample.pro.
Compile and run the program.

; Start the application

e = ENVI()

LabelRasterList = List()

; Change the path to the directory

; containing the tutorial data

Path = 'C:\MyTutorialFiles\'

; Open training rasters and ROIs

FOR i = 1,4 DO BEGIN

  ; Open training raster

  File = Path + 'TrainingRaster' + StrTrim(i,2) + '.dat'

  Raster = e.OpenRaster(File)

  ; Open ROI file

  ROIFile = Path + 'TrainingRaster' + StrTrim(i,2) + 'ROIs.xml'

  ROI = e.OpenROI(ROIFile)

  ; Create label raster

  LabelTask = ENVITask('BuildLabelRasterFromROI')

  LabelTask.INPUT_RASTER = Raster

  LabelTask.INPUT_ROI = ROI

  LabelTask.CLASS_NAMES = ['Roof Damage', $

    'Structural Damage', 'Rubble', 'Blue Tarps']

  LabelTask.Execute

  LabelRasterList.Add, LabelTask.OUTPUT_RASTER

ENDFOR

LabelRasters = LabelRasterList.ToArray()

TrainingRasters = LabelRasters[0:2]

ValidationRasters = LabelRasters[3]

; Get the task from the catalog of ENVITasks

Task = ENVITask('TrainTensorFlowMaskModel')

; Define inputs

Task.TRAINING_RASTERS = TrainingRasters

Task.VALIDATION_RASTERS = ValidationRasters

Task.BLUR_DISTANCE = [1,10]

Task.CLASS_WEIGHT = [0,2]

Task.EPOCHS = 20

Task.PATCH_SAMPLING_RATE = 16

Task.SOLID_DISTANCE = [0]

Task.LOSS_WEIGHT = 0.8

; Run the task

Task.Execute

Print, Task.OUTPUT_MODEL, /IMPLIED_PRINT

END

Syntax

Result = ENVITask('TrainTensorFlowMaskModel')

Input properties (Set, Get): AUGMENT_ROTATION, AUGMENT_SCALE, BLUR_DISTANCE, CLASS_NAMES, CLASS_WEIGHT, EPOCHS, INPUT_MODEL, LOSS_WEIGHT, OUTPUT_LAST_MODEL_URI, OUTPUT_MODEL_URI, PATCH_SAMPLING_RATE, PATCHES_PER_BATCH, PATCHES_PER_EPOCH, SOLID_DISTANCE, TRAINING_RASTERS, VALIDATION_RASTERS

Output properties (Get only): COMPLETED_EPOCHS, OUTPUT_LAST_MODEL, OUTPUT_MODEL, VALIDATION_F1, VALIDATION_LOSS, VALIDATION_PRECISION, VALIDATION_RECALL

Properties marked as "Set" are those that you can set to specific values. You can also retrieve their current values any time. Properties marked as "Get" are those whose values you can retrieve but not set.

Methods

This task inherits the following methods from ENVITask. See the ENVITask topic in ENVI Help.

AddParameter
Execute
Parameter
ParameterNames
RemoveParameters

Properties

This task inherits the following properties from ENVITask:

COMMUTE_ON_DOWNSAMPLE

COMMUTE_ON_SUBSET

DESCRIPTION

DISPLAY_NAME

NAME

REVISION

See the ENVITask topic in ENVI Help for details.

This task also contains the following properties:

AUGMENT_ROTATION (optional)

Specify whether to rotate training inputs during data augmentation. See Data Augmentation.

AUGMENT_SCALE (optional)

Specify whether to scale training inputs during data augmentation. See Data Augmentation.

BLUR_DISTANCE (optional)

Specify a [2, *] double-precision array with the minimum and maximum distance in pixels to expand feature (non-background) areas beyond the solid distance expansion. The area expanded is blurred, diminishing from the edges of the feature areas to the blur distance. The maximum blur distance is used at the beginning of training and decreased to the minimum blur distance at the end of training. If not set, the value will be read from INPUT_MODEL if it was previously trained; otherwise, the value will default to [10.0,0.0]. The number of elements in the array should be twice the number of classes in CLASS_NAMES. If you specify only one set of minimum and maximum distance values, those values will be used for all classes associated with CLASS_NAMES.

CLASS_NAMES (optional)

Specify a string array with the list of class names the model will be trained to find. Training and validation rasters will be checked for consistency against this list.

CLASS_WEIGHT (optional)

Specify a double-precision array with the minimum and maximum weights for having a more even balance of classes (including background) during patch selection. Balancing of patch selection is weighted by the maximum value at the beginning of training and decreased to the minimum value at the end of training. In general, set the maximum higher for sparser training sets. The useful range for the maximum value is between 0.0 and 3.0. If not set, the value will be read from INPUT_MODEL if it was previously trained; otherwise, the value will default to [2.0,0.0].

COMPLETED_EPOCHS

This is the number of training epochs completed by this task.

EPOCHS (optional)

An epoch is a full pass of the entire training dataset through the algorithm's learning process. Specify the number of Epochs to run. Training inputs are adjusted at the end of each epoch.

INPUT_MODEL (required)

Specify the mask-based ENVITensorFlowModel to be trained. The model can be untrained or previously trained.

LOSS_WEIGHT (optional)

Specify the weight of feature (non-background) pixels when calculating how well the model is fitting the training data. The useful range is between 0.0 and 3.0. If not set, the value will be read from INPUT_MODEL if it was previously trained; otherwise, the value will default to 1.0.

OUTPUT_LAST_MODEL

This is a reference to the last trained ENVITensorFlowModel.

OUTPUT_LAST_MODEL_URI (optional)

Specify a string with the fully qualified filename and path of the output HDF5 file that will contain the trained model from the last epoch.

OUTPUT_MODEL

This is the ENVITensorFlowModel extracted from the URI.

OUTPUT_MODEL_URI (optional)

Specify a string with the fully qualified filename and path of the output HDF5 file that will contain the best trained model according to validation loss.

PATCH_SAMPLING_RATE (optional)

Specify the average number of patches that each pixel in TRAINING_RASTERS and VALIDATION_RASTERS will belong to. The default value is 16.

PATCHES_PER_BATCH (optional)

Specify the number of patches to run per batch. A batch comprises one iteration of training; model parameters are adjusted at the end of each iteration. Batches are run in an epoch until the number of patches per epoch is met or exceeded. If you do not specify a value, ENVI will automatically determine a suitable value.

PATCHES_PER_EPOCH (optional)

Specify the number of training patches to use per epoch. If you do not specify a value, ENVI will automatically determine a suitable value.

SOLID_DISTANCE (optional)

Specify a double-precision array with the distance in pixels to expand feature (non-background) areas. SOLID_DISTANCE is typically used for point or line inputs. If not set, the value will be read from INPUT_MODEL if it was previously trained; otherwise, the value will default to 0.0. The number of elements in the array should match the number of classes in CLASS_NAMES. If you only specify one distance value, that value will be used for all classes associated with CLASS_NAMES.

TRAINING_RASTERS (required)

Specify one or more rasters to use to train INPUT_MODEL.

VALIDATION_F1

This is the validation F1 of the output model.

VALIDATION_LOSS

This is an array containing the validation loss of the output model for each epoch.

VALIDATION_PRECISION

This is the validation precision of the output model.

VALIDATION_RECALL

This is the validation recall of the output model.

VALIDATION_RASTERS (required)

Specify one or more rasters to use to validate INPUT_MODEL after each epoch.

Version History

Deep Learning 1.0	Introduced
Deep Learning 1.1	Added CLASS_NAMES, OUTPUT_LAST_MODEL, and OUTPUT_LAST_MODEL_URI properties. Modified BLUR_DISTANCE and SOLID_DISTANCE properties to accept multi-dimensional arrays. Modified PATCHES_PER_EPOCH property to accept no input, in which case ENVI will automatically determine a suitable value.
Deep Learning 1.2	Added AUGMENT_ROTATION and AUGMENT_SCALE properties.
Deep Learning 3.0	Depricated