7590
ENVI Statistics File (.sta) Format Specification
This section describes the structure of the ENVI statistics file (.sta) for ENVI and ENVI Classic.
The values contained in the header give a reference of where the statistics came from (either the full file, a spatial subset of the file, or a region of interest - ROI).
The byte order for the ENVI .sta file is always big endian (IEEE MSB). If you are on a little-endian machine (for example, Intel x86), you will need to appropriately byte swap all the values from the file when reading them in.
|
Bytes
|
Data Type
|
Description
|
|
0:3
|
32-bit long integer
|
Identifies the file as an ENVI Classic .sta file. Possible values are:
- 1095584078 for newer stats files [swap = 1313426753]
- 1111838282 for older stats files [swap = 1246643522]
The swap value is the little endian version of this 32-bit long integer value. So if the first four bytes matches this swap value, you will need to byte swap all the values when reading from disk.
|
|
4:7
|
32-bit integer
|
Number of samples of source file
|
|
8:11
|
32-bit integer
|
Number of lines of source file
|
|
12:15
|
32-bit integer
|
Number of bands of source file
|
|
16:19
|
32-bit integer
|
IDL data type of source file
|
|
20:23
|
32-bit integer
|
ROI index of source file. The value is -1 for a “standard” statistics file or a value >= 0 for an “roi” statistics file.
|
|
24:27
|
32-bit integer
|
Starting sample of spatial subset 1
|
|
28:31
|
32-bit integer
|
Ending sample of spatial subset 1
|
|
32:35
|
32-bit integer
|
Starting line of spatial subset 1
|
|
36:39
|
32-bit integer
|
Ending line of spatial subset 1
|
1 The values of the spatial subset are all zeros when the ROI index has a value greater than or equal to 0.
Next is a set of 32-bit file offsets: one for each band and one additional offset to the "end" where the covariance and eigenvalues are located, if they are present. So the number of these values is dependent on the value of bytes 12:15.
|
Bytes
|
Data Type
|
Description
|
|
40:40+(nb+1)*4-1
|
32-bit integer
|
File offset for band n histogram. If this value is 0, then the band does not contain a histogram. There are nb+1 of these offsets (where nb is the number of bands), and the last of the offsets points to the location in the file of the advanced covariance statistics.
|
|
x:x+3
|
32-bit integer
|
Number of bytes (string length) of upcoming file or ROI name string.
|
|
x+4:x+4+length-1
|
Byte
|
Array of filename characters. The string will have a file or ROI name separated brackets, and the two brackets are separated by a caret symbol (^). For example:
"[<filename>]^[<roi name>]"
New files contain a third bracketed string with the letter “b”; for example:
"[<filename>]^[<roi name>]^[b]"
If this [b] string exists, it indicates that the histogram information for the file will contain a “binsize” property. Also, the histogram min/max values will be double-precision floating-point instead of single-precision floating-point as with older versions.
The following shows an example string:
"[/data/qb_boulder_msi]^[ ]^[b]"
|
|
x:x+(nb*4)-1
|
32-bit floating point
|
Band wavelengths (if available). If there are no wavelengths, then these values will be 1.0, 2.0, etc. There are no wavelength units associated with these values.
|
|
Bytes
|
Data Type
|
Description
|
|
x:x+nb-1
|
Byte array
|
0: No statistics are available for that band.
1: Statistics are available for that band.
|
Next is a set of statistics for each band. The first four bytes of the file indicates whether these values are single-precision or double-precision floating-point values. The size of the statistics arrays is equal to the number of bands in the statistics file, even if some of these bands do not have actual statistics (in which case the min, max, mean, and standard deviation are set to 0 for these bands, if there are any bands).
The following table applies to older, single-precision, floating-point files:
|
Bytes
|
Data Type
|
Description
|
|
x:x+(4*(nb*4))-1
|
32-bit floating point
|
Statistics, ordered as follows:
Data min array(nb)
Data max array(nb)
Data mean array(nb)
Data stdev array(nb)
|
The following table applies to newer, double-precision, floating-point files:
|
Bytes
|
Data Type
|
Description
|
|
x:x+(4*(nb*8))-1
|
64-bit floating point
|
Statistics, ordered as follows:
Data min array(nb)
Data max array(nb)
Data mean array(nb)
Data stdev array(nb)
|
All of the other information in the statistics file is accessed using the given file offset values. This information can include a histogram for each band as well as a covariance and eigenvector matrix (along with the eigenvalues) for each band.
To access the covariance matrices, go to the file location contained in the file offset array offset [nb]. If this offset has a value of zero, then the file does not contain advanced statistics.
|
Bytes
|
Data Type
|
Description
|
|
offset:offset+3
|
32-bit integer
|
The number of bands contained in the covariance matrices. Because ENVI can calculate statistics at different types by different processes for different reasons, the bands used to calculate the covariance can be completely different than those used to calculate the basic statistics.
|
|
offset+4:offset+4+(covNb*4)-1
|
32-bit integer
|
Array of which bands were used. To have a covariance matrix, this array must contain at least two bands.
|
Next is the set of matrix and array values.
The following table applies to older, single-precision, floating-point files:
|
Bytes
|
Data Type
|
Description
|
|
x:x+(covNb*4+(2*(covNb*covNb*4)))
|
32-bit floating point
|
Covariance statistics, ordered as follows:
Covariance array(covNb,covNb)Eigenvectors array(covNb,covNb)
Eigenvalues array(covNb)
|
The following table applies to newer, double-precision, floating-point files:
|
Bytes
|
Data Type
|
Description
|
|
x:x+(covNb*8+(2*(covNb*covNb*8)))
|
64-bit floating point
|
Covariance statistics, ordered as follows:
Covariance array(covNb,covNb)
Eigenvectors array(covNb,covNb)
Eigenvalues array(covNb)
|
To access the histogram for a given band, go to the file location contained in the file offset array of offset [band-1]. If this offset has a value of 0, then no histogram is available for that band.
|
Bytes
|
Data Type
|
Description
|
|
offset:offset+3
|
32-bit integer
|
Run length encoding flag 1
|
|
offset+4:offset+7
|
32-bit floating point
|
Histogram data min value (single precision)
|
|
offset+8:offset+11
|
32-bit floating point
|
Histogram data max value (single precision)
|
|
offset+12:offset+15
|
32-bit integer
|
Number of bins in histogram array
|
1 If the run length encoding flag is set to 1:
- Starting at bytes offset+16, read the following data:
- 32-bit integer length of run length encoding vector
- 32-bit integer array(nBins) of run length encoding vector positions
- 32-bit integer array(nBins) of run length encoding histogram values
Then set up the histogram vector as follows:
vec = Lonarr(vectorLength)
vec[vectorPositions] = histogramValues
If the run length encoding flag is set to 0:
- Starting at bytes offset+16, read the following data:
- 32-bit integer array(nBins) of histogram vector
Next, if "^[b]" is included in the name string, there will also be three 64-bit, double-precision, floating-point values after the histogram vector:
- Histogram min
- Histogram max
- Histogram bin size
Otherwise, ENVI Classic computes the bin size as follows:
nBins = N_Elements(vec)
binsize = (histMax - histMin) / (nBins-1d)
This process used the single-precision histogram min/max values read in at the start of the band histogram information.
This example reads the basic statistics from a .sta file.
function ReadBasicStats, staName, dataMin, dataMax, dataMean, dataStdv, $
BANDS=bands
compile_opt idl2
Openr, unit, staName, /GET_LUN
header = Lonarr(10)
Readu, unit, header
byteSwap = (header[0] eq 1246643522 || $
header[0] eq 1313426753)
if (~byteSwap && $
header[0] ne 1095584078 && $
header[0] ne 1111838282) then return,0 ; not a .sta file
if (byteSwap) then Swap_Endian_Inplace, header
; flag whether to use Float() or Double() arrays
oldFormat = (header[0] eq 1111838282)
numBands = header[3]
pointers = Lonarr(numBands+1)
strLength = 0L
Readu, unit, pointers, strLength
if (byteSwap) then begin
Swap_Endian_Inplace, pointers
Swap_Endian_Inplace, strLength
endif
; read and parse the file name string
text = Bytarr(strLength)
Readu, unit, text
strings = Strtok(String(text), '^', /EXTRACT)
fileName = Strmid(strings[0], 1, Strlen(strings[0])-2)
roiName = Strmid(strings[1], 1, Strlen(strings[1])-2)
newestFormat = (N_Elements(strings) eq 3) ; "^[b]" is present
; read the wavelengths
wl = Fltarr(numBands)
Readu, unit, wl
if (byteSwap) then Swap_Endian_Inplace, wl
; read the data
bandPos = Bytarr(numBands)
if (oldFormat) then begin
dataMin = Fltarr(numBands)
dataMax = Fltarr(numBands)
dataMean = Fltarr(numBands)
dataStdv = Fltarr(numBands)
endif else begin
dataMin = Dblarr(numBands)
dataMax = Dblarr(numBands)
dataMean = Dblarr(numBands)
dataStdv = Dblarr(numBands)
endelse
Readu, unit, bandPos, dataMin, dataMax, dataMean, dataStdv
Free_Lun, unit
if (byteSwap) then begin
Swap_Endian_Inplace, dataMin
Swap_Endian_Inplace, dataMax
Swap_Endian_Inplace, dataMean
Swap_Endian_Inplace, dataStdv
endif
; which bands have valid stats
bands = Where(bandPos eq 1)
return, 1
end
This example reads the histogram for a given band.
function ReadHistogram, staName, band, histMin, histMax, histVec, binsize
compile_opt idl2
Openr, unit, staName, /GET_LUN
header = Lonarr(10)
Readu, unit, header
byteSwap = (header[0] eq 1246643522 || $
header[0] eq 1313426753)
if (~byteSwap && $
header[0] ne 1095584078 && $
header[0] ne 1111838282) then begin
; not a .sta file
Free_Lun, unit
return,0
endif
if (byteSwap) then Swap_Endian_Inplace, header
; flag whether to use Float() or Double() arrays
oldFormat = (header[0] eq 1111838282)
numBands = header[3]
dataType = header[4]
if (band lt 0 || band ge numBands) then begin
; invalid band request
Free_Lun, unit
return, 0
endif
pointers = Lonarr(numBands+1)
strLength = 0L
Readu, unit, pointers, strLength
if (byteSwap) then begin
Swap_Endian_Inplace, pointers
Swap_Endian_Inplace, strLength
endif
; read and parse the file name string
text = Bytarr(strLength)
Readu, unit, text
strings = Strtok(String(text), '^', /EXTRACT)
fileName = Strmid(strings[0], 1, Strlen(strings[0])-2)
roiName = Strmid(strings[1], 1, Strlen(strings[1])-2)
newestFormat = (N_Elements(strings) eq 3) ; "^[b]" is present
if (pointers[band] eq 0) then begin
; histogram not present
Free_Lun, unit
return, 0
endif
; use the file offset array to go to the correct position in the
; file containing the requested histogram
Point_Lun, unit, pointers[band]
rle = 0L
histMin = 0.
histMax = 0.
nBins = 0L
Readu, unit, rle, histMin, histMax, nBins
if (byteSwap) then begin
Swap_Endian_Inplace, rle
Swap_Endian_Inplace, histMin
Swap_Endian_Inplace, histMax
Swap_Endian_Inplace, nBins
endif
if (rle) then begin
; run length encoded path
vecSize = 0L
vecPtr = Lonarr(nBins)
vecData = Lonarr(nBins)
Readu, unit, vecSize, vecPtr, vecData
if (byteSwap) then begin
Swap_Endian_Inplace, vecSize
Swap_Endian_Inplace, vecPtr
Swap_Endian_Inplace, vecData
endif
histVec = Lonarr(vecSize)
histVec[vecPtr] = vecData
endif else begin
; full histogram vector
histVec = Lonarr(nBins)
Readu, unit, histVec
if (byteSwap) then Swap_Endian_Inplace, histVec
endelse
if (newestFormat) then begin
; new format: binsize is written to the file
histMin = 0d
histMax = 0d
binsize = 0d
Readu, unit, histMin, histMax, binsize
if (byteSwap) then begin
Swap_Endian_Inplace, histMin
Swap_Endian_Inplace, histMax
Swap_Endian_Inplace, binsize
endif
endif else begin
; old format: binsize is derived from other values
if (N_Elements(histVec) gt 1) then begin
nBins = N_Elements(histVec) > Envi_Hist_Number_Of_Bins()
binSize = (histMax - Double(histMin)) / (nBins-1d)
if (Envi_Data_Type_Is_Integer(dataType)) then $
binSize = Ceil(binsize) > 1d
endif else $
binsize = 1d
endelse
Free_Lun, unit
return, 1
end