CalculateConfusionMatrixFromRaster Task

This task returns a confusion matrix as well as a number of accuracy measurements computed from a classification raster and ground truth regions of interest (ROIs).

The classification raster contains predicted class values from a classification, which are accompanied by class names. The truth ROIs contain the actual, or expected, class names of a particular region of the raster. A confusion matrix is created by comparing the predicted names to the truth names.

Example

The code example below evaluates classifications using a confusion matrix.

PRO EvaluateClassificationUsingConfusionMatrix

    COMPILE_OPT IDL2

    ; Start the application

    e = ENVI()

    ; Open an input file

    File = Filepath('qb_boulder_msi', Subdir=['data'], $

    Root_Dir=e.Root_Dir)

    Raster = e.OpenRaster(File)

    File2 = Filepath('qb_boulder_roi.xml', Subdir=['data'], $

    Root_Dir=e.Root_Dir)

    Rois = envi.OpenROI(roiFile)

    ; Get training statistics

    StatTask = ENVITask('ROIStatistics')

    StatTask.INPUT_RASTER = Raster

    StatTask.INPUT_ROI = Rois

    StatTask.Execute

    ; Get the task from the catalog of ENVITasks

    Task = ENVITask('MahalanobisDistanceClassification')

    ; Define inputs

    Task.INPUT_RASTER = Raster

    Task.COVARIANCE = StatTask.Covariance

    Task.MEAN = StatTask.Mean

    Task.CLASS_PIXEL_COUNT = StatTask.Roi_Pixel_Count

    Task.CLASS_NAMES = [Rois[0].name, Rois[1].name, Rois[2].name]

    Task.CLASS_COLORS = [[0,0,255], [0,255,0], [255,0,0]]

    ; Run the task and display the result

    Task.Execute

    ClassRaster = Task.OUTPUT_RASTER

    View = e.GetView()

    Layer = View.CreateLayer(ClassRaster)

    ; Add the output to the Data Manager

    envi.Data.Add, ClassRaster

    ; Calculate the confusion matrix

    ConfusionMatrix = ENVICalculateConfusionMatrixFromRaster(ClassRaster, Rois)

    ; Print results

    Print, 'Confusion Matrix:'

    Print, ConfusionMatrix.Confusion_Matrix

    Print, 'Errors of commission: '

    Print, Transpose([[ConfusionMatrix.Column_Names+': '], [(ConfusionMatrix.CommissionError()).ToString()]])

    Print, 'Errors of omission: '

    Print, Transpose([[ConfusionMatrix.Column_Names+': '], [(ConfusionMatrix.OmissionError()).ToString()]])

    Print, 'Overall accuracy: ', ConfusionMatrix.Accuracy()

END

Syntax

Result = ENVITask('CalculateConfusionMatrixFromRaster')

Input parameters (Set, Get): INPUT_RASTER, INPUT_ROIS

Output parameters (Get only): ACCURACY, CLASS_NAMES, COMISSION_ERROR, CONFUSION_MATRIX, F1, KAPPA_COEFFICIENT, OMISSION_ERROR, PRODUCER_ACCURACY, USER_ACCURACY

Parameters marked as "Set" are those that you can set to specific values. You can also retrieve their current values any time. Parameters marked as "Get" are those whose values you can retrieve but not set.

Input Parameters

INPUT_RASTER (required)

Specify a classification raster.

INPUT_ROIS (required)

Specify an array of ENVIROIs containing truth data.

ENVIConfusionMatrix is a persistable object that can be saved and restored.

Output Parameters

ACCURACY

This is the overall accuracy of the confusion matrix, calculated by summing the number of correctly classified values and dividing by the total number of values.

CLASS_NAMES

This is the list of classes in the order the confusion matrix and the accuracy measurements are generated.

COMISSION_ERROR

Errors of commission represent the fraction of values that were predicted to be in a class but do not belong to that class. They are a measure of false positives. The result is an array with one value per class.

CONFUSION_MATRIX

This is the resulting confusion matrix.

F1

This is the F1 score, which is the harmonic mean of the User Accuracy and Producer Accuracy values. The result is an array with one value per class.

KAPPA_COEFFICIENT

The kappa coefficient measures the agreement between classification and truth values. A kappa value of 1 represents perfect agreement, while a value of 0 represents no agreement.

OMISSION_ERROR

Errors of omission represent the fraction of values that belong to a class but were predicted to be in a different class. They are a measure of false negatives. The result is an array with one value per class.